Part 3: Annotated Specification
Beacon Chain State Transition Function
Block processing
def process_block(state: BeaconState, block: BeaconBlock) -> None:
process_block_header(state, block)
process_randao(state, block.body)
process_eth1_data(state, block.body)
process_operations(state, block.body) # [Modified in Altair]
process_sync_aggregate(state, block.body.sync_aggregate) # [New in Altair]
These are the tasks that the beacon node performs in order to process a block and update the state. If any of the called functions triggers an assert
failure or any other kind of exception, then the entire block is invalid, and any state changes must be rolled back.
process_operations()
covers the processing of any slashing reports (proposer and attester) in the block, any attestations, any deposits, and any voluntary exits.
Used by | state_transition() |
Uses | process_block_header() , process_randao() , process_eth1_data() , process_operations() , process_sync_aggregate() |
Block header
def process_block_header(state: BeaconState, block: BeaconBlock) -> None:
# Verify that the slots match
assert block.slot == state.slot
# Verify that the block is newer than latest block header
assert block.slot > state.latest_block_header.slot
# Verify that proposer index is the correct index
assert block.proposer_index == get_beacon_proposer_index(state)
# Verify that the parent matches
assert block.parent_root == hash_tree_root(state.latest_block_header)
# Cache current block as the new latest block
state.latest_block_header = BeaconBlockHeader(
slot=block.slot,
proposer_index=block.proposer_index,
parent_root=block.parent_root,
state_root=Bytes32(), # Overwritten in the next process_slot call
body_root=hash_tree_root(block.body),
)
# Verify proposer is not slashed
proposer = state.validators[block.proposer_index]
assert not proposer.slashed
A straightforward set of validity conditions for the block header data.
The version of the block header object that this routine stores in the state is a duplicate of the incoming block's header, but with its state_root
set to its default empty Bytes32()
value. See process_slot()
for the explanation of this.
Used by | process_block() |
Uses | get_beacon_proposer_index() , hash_tree_root() |
See also | BeaconBlockHeader, process_slot() |
RANDAO
def process_randao(state: BeaconState, body: BeaconBlockBody) -> None:
epoch = get_current_epoch(state)
# Verify RANDAO reveal
proposer = state.validators[get_beacon_proposer_index(state)]
signing_root = compute_signing_root(epoch, get_domain(state, DOMAIN_RANDAO))
assert bls.Verify(proposer.pubkey, signing_root, body.randao_reveal)
# Mix in RANDAO reveal
mix = xor(get_randao_mix(state, epoch), hash(body.randao_reveal))
state.randao_mixes[epoch % EPOCHS_PER_HISTORICAL_VECTOR] = mix
A good source of randomness is foundational to the operation of the beacon chain. Security of the protocol depends significantly on being able to unpredictably and uniformly select block proposers and committee members. In fact, the very name "beacon chain" was inspired by Dfinity's concept of a randomness beacon.
The current mechanism for providing randomness is a RANDAO, in which each block proposer provides some randomness and all the contributions are mixed together over the course of an epoch. This is not unbiasable (a malicious proposer may choose to skip a block if it is to its advantage to do so), but is good enough. In future, Ethereum might use a verifiable delay function (VDF) to provide unbiasable randomness.
Early designs had the validators pre-committing to "hash onions", peeling off one layer of hashing at each block proposal. This was changed to using a BLS signature over the epoch number as the entropy source. Using signatures is both a simplification, and an enabler for multi-party (distributed) validators. The (reasonable) assumption is that sufficient numbers of validators generated their secret keys with good entropy to ensure that the RANDAO's entropy is adequate.
The process_randao()
function simply uses the proposer's public key to verify that the RANDAO reveal in the block is indeed the epoch number signed with the proposer's private key. It then mixes the hash of the reveal into the current epoch's RANDAO accumulator. The hash is used in order to reduce the signature down from 96 to 32 bytes, and to make it uniform. EPOCHS_PER_HISTORICAL_VECTOR
past values of the RANDAO accumulator at the ends of epochs are stored in the state.
From Justin Drake's notes:
Using
xor
inprocess_randao
is (slightly) more secure than usinghash
. To illustrate why, imagine an attacker can grind randomness in the current epoch such that two of his validators are the last proposers, in a different order, in two resulting samplings of the next epochs. The commutativity ofxor
makes those two samplings equivalent, hence reducing the attacker's grinding opportunity for the next epoch versushash
(which is not commutative). The strict security improvement may simplify the derivation of RANDAO security formal lower bounds.
Note that the assert
statement means that the whole block is invalid if the RANDAO reveal is incorrectly formed.
Used by | process_block() |
Uses | get_beacon_proposer_index() , compute_signing_root() , get_domain() , bls.Verify() , hash() , xor() , get_randao_mix() |
See also | EPOCHS_PER_HISTORICAL_VECTOR |
Eth1 data
def process_eth1_data(state: BeaconState, body: BeaconBlockBody) -> None:
state.eth1_data_votes.append(body.eth1_data)
if state.eth1_data_votes.count(body.eth1_data) * 2 > EPOCHS_PER_ETH1_VOTING_PERIOD * SLOTS_PER_EPOCH:
state.eth1_data = body.eth1_data
Blocks may contain Eth1Data
which is supposed to be the proposer's best view of the Eth1 chain and the deposit contract at the time. There is no incentive to get this data correct, or penalty for it being incorrect.
If there is a simple majority of the same vote being cast by proposers during each voting period of EPOCHS_PER_ETH1_VOTING_PERIOD
epochs (6.8 hours) then the Eth1 data is committed to the beacon state. This updates the chain's view of the deposit contract, and new deposits since the last update will start being processed.
This mechanism has proved to be fragile in the past, but appears to be workable if not perfect.
Used by | process_block() |
See also | Eth1Data , EPOCHS_PER_ETH1_VOTING_PERIOD |
Operations
def process_operations(state: BeaconState, body: BeaconBlockBody) -> None:
# Verify that outstanding deposits are processed up to the maximum number of deposits
assert len(body.deposits) == min(MAX_DEPOSITS, state.eth1_data.deposit_count - state.eth1_deposit_index)
def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None:
for operation in operations:
fn(state, operation)
for_ops(body.proposer_slashings, process_proposer_slashing)
for_ops(body.attester_slashings, process_attester_slashing)
for_ops(body.attestations, process_attestation)
for_ops(body.deposits, process_deposit)
for_ops(body.voluntary_exits, process_voluntary_exit)
Just a dispatcher for handling the various optional contents in a block.
Deposits are optional only in the sense that some blocks have them and some don't. However, as per the assert
statement, if, according to the beacon chain's view of the Eth1 chain, there are deposits pending, then the block must include them, otherwise the block is invalid. On the face of it, this suggests that it is important for a block proposer to have access to an Eth1 node, so as to be able to obtain the deposit data. In practice, this turns out to be not so important, although. with Altair, the proposer reward was increased by a factor of four, which increases the importance of the Eth1 node.
Block proposers are explicitly rewarded for including any available attestations and slashing reports. There is a validity condition, and thus an implicit reward, related to including deposit messages. The incentive for including voluntary exits is that a smaller validator set means higher rewards for the remaining validators.
Used by | process_block() |
Uses | process_proposer_slashing() , process_attester_slashing() , process_attestation() , process_deposit() , process_voluntary_exit() |
See also | BeaconBlockBody |
Proposer slashings
def process_proposer_slashing(state: BeaconState, proposer_slashing: ProposerSlashing) -> None:
header_1 = proposer_slashing.signed_header_1.message
header_2 = proposer_slashing.signed_header_2.message
# Verify header slots match
assert header_1.slot == header_2.slot
# Verify header proposer indices match
assert header_1.proposer_index == header_2.proposer_index
# Verify the headers are different
assert header_1 != header_2
# Verify the proposer is slashable
proposer = state.validators[header_1.proposer_index]
assert is_slashable_validator(proposer, get_current_epoch(state))
# Verify signatures
for signed_header in (proposer_slashing.signed_header_1, proposer_slashing.signed_header_2):
domain = get_domain(state, DOMAIN_BEACON_PROPOSER, compute_epoch_at_slot(signed_header.message.slot))
signing_root = compute_signing_root(signed_header.message, domain)
assert bls.Verify(proposer.pubkey, signing_root, signed_header.signature)
slash_validator(state, header_1.proposer_index)
A ProposerSlashing
is a proof that a proposer has signed two blocks at the same height. Up to MAX_PROPOSER_SLASHINGS
of them may be included in a block. It contains the evidence in the form of a pair of SignedBeaconBlockHeader
s.
The proof is simple: the two proposals come from the same slot, have the same proposer, but differ in one or more of parent_root
, state_root
, or body_root
. In addition, they were both signed by the proposer. The conflicting blocks do not need to be valid: any pair of headers that meet the criteria, irrespective of the blocks' contents, are liable to be slashed.
As ever, the assert
statements ensure that the containing block is invalid if it contains any invalid slashing claims.
Fun fact: the first slashing to occur on the beacon chain was a proposer slashing. Two clients running side-by-side with the same keys will often produce the same attestations since the protocol is designed to encourage that. Independently producing the same block is very unlikely as blocks contain much more data.
Used by | process_block() |
Uses | is_slashable_validator() , get_domain() , compute_signing_root() , bls.Verify() , slash_validator() |
See also | ProposerSlashing |
Attester slashings
def process_attester_slashing(state: BeaconState, attester_slashing: AttesterSlashing) -> None:
attestation_1 = attester_slashing.attestation_1
attestation_2 = attester_slashing.attestation_2
assert is_slashable_attestation_data(attestation_1.data, attestation_2.data)
assert is_valid_indexed_attestation(state, attestation_1)
assert is_valid_indexed_attestation(state, attestation_2)
slashed_any = False
indices = set(attestation_1.attesting_indices).intersection(attestation_2.attesting_indices)
for index in sorted(indices):
if is_slashable_validator(state.validators[index], get_current_epoch(state)):
slash_validator(state, index)
slashed_any = True
assert slashed_any
AttesterSlashing
s are similar to proposer slashings in that they just provide the evidence of the two aggregate IndexedAttestation
s that conflict with each other. Up to MAX_ATTESTER_SLASHINGS
of them may be included in a block.
The validity checking is done by is_slashable_attestation_data()
, which checks the double vote and surround vote conditions, and by is_valid_indexed_attestation()
which verifies the signatures on the attestations.
Any validators that appear in both attestations are slashed. If no validator is slashed, then the attester slashing claim was not valid after all, and therefore its containing block is invalid.
Examples: a double vote attester slashing; surround vote attester slashings.
Used by | process_block() |
Uses | is_slashable_attestation_data() , is_valid_indexed_attestation() , is_slashable_validator() , slash_validator() |
See also | AttesterSlashing |
Attestations
def process_attestation(state: BeaconState, attestation: Attestation) -> None:
data = attestation.data
assert data.target.epoch in (get_previous_epoch(state), get_current_epoch(state))
assert data.target.epoch == compute_epoch_at_slot(data.slot)
assert data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot <= data.slot + SLOTS_PER_EPOCH
assert data.index < get_committee_count_per_slot(state, data.target.epoch)
committee = get_beacon_committee(state, data.slot, data.index)
assert len(attestation.aggregation_bits) == len(committee)
# Participation flag indices
participation_flag_indices = get_attestation_participation_flag_indices(state, data, state.slot - data.slot)
# Verify signature
assert is_valid_indexed_attestation(state, get_indexed_attestation(state, attestation))
# Update epoch participation flags
if data.target.epoch == get_current_epoch(state):
epoch_participation = state.current_epoch_participation
else:
epoch_participation = state.previous_epoch_participation
proposer_reward_numerator = 0
for index in get_attesting_indices(state, data, attestation.aggregation_bits):
for flag_index, weight in enumerate(PARTICIPATION_FLAG_WEIGHTS):
if flag_index in participation_flag_indices and not has_flag(epoch_participation[index], flag_index):
epoch_participation[index] = add_flag(epoch_participation[index], flag_index)
proposer_reward_numerator += get_base_reward(state, index) * weight
# Reward proposer
proposer_reward_denominator = (WEIGHT_DENOMINATOR - PROPOSER_WEIGHT) * WEIGHT_DENOMINATOR // PROPOSER_WEIGHT
proposer_reward = Gwei(proposer_reward_numerator // proposer_reward_denominator)
increase_balance(state, get_beacon_proposer_index(state), proposer_reward)
Block proposers are rewarded here for including attestations during block processing, while attesting validators receive their rewards and penalties during epoch processing.
This routine processes each attestation included in the block. First a bunch of validity checks are performed. If any of these fails, then the whole block is invalid (it is most likely from a proposer on a different fork, and so useless to us):
- The target vote of the attestation must be either the previous epoch's checkpoint or the current epoch's checkpoint.
- The target checkpoint and the attestation's slot must belong to the same epoch.
- The attestation must be no newer than
MIN_ATTESTATION_INCLUSION_DELAY
slots, which is one. So this condition rules out attestations from the current or future slots. - The attestation must be no older than
SLOTS_PER_EPOCH
slots, which is 32. - The attestation must come from a committee that existed when the attestation was created.
- The size of the committee and the size of the aggregate must match (
aggregation_bits
). - The (aggregate) signature on the attestation must be valid and must correspond to the aggregated public keys of the validators that it claims to be signed by. This (and other criteria) is checked by
is_valid_indexed_attestation()
.
Once the attestation has passed the checks it is processed by converting the votes from validators that it contains into flags in the state.
It's easy to skip over amidst all the checking, but the actual attestation processing is done by get_attestation_participation_flag_indices()
. This takes the source, target, and head votes of the attestation, along with its inclusion delay (how many slots late was it included in a block) and returns a list of up to three flags corresponding to the votes that were both correct and timely, in participation_flag_indices
.
For each validator that signed the attestation, if each flag in participation_flag_indices
is not already set for it in its epoch_participation
record, then the flag is set, and the proposer is rewarded. Recall that the validator making the attestation is not rewarded until the end of the epoch. If the flag is already set in the corresponding epoch for a validator, no proposer reward is accumulated: the attestation for this validator was included in an earlier block.
The proposer reward is accumulated, and weighted according to the weight assigned to each of the flags (timely source, timely target, timely head).
If a proposer includes all the attestations only for one slot, and all the relevant validators vote, then its reward will be, in the notation established earlier,
Where is the total maximum reward per epoch for attesters, calculated in get_flag_index_deltas()
. The total available reward in an epoch for proposers including attestations is 32 times this.
Deposits
def get_validator_from_deposit(state: BeaconState, deposit: Deposit) -> Validator:
amount = deposit.data.amount
effective_balance = min(amount - amount % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE)
return Validator(
pubkey=deposit.data.pubkey,
withdrawal_credentials=deposit.data.withdrawal_credentials,
activation_eligibility_epoch=FAR_FUTURE_EPOCH,
activation_epoch=FAR_FUTURE_EPOCH,
exit_epoch=FAR_FUTURE_EPOCH,
withdrawable_epoch=FAR_FUTURE_EPOCH,
effective_balance=effective_balance,
)
Create a newly initialised validator object from a deposit. This was factored out of process_deposit()
for better code reuse between the Phase 0 spec and the sharding spec.
The state
parameter in the input argument list is an oversight: it is not used or required.
Used by | process_deposit() |
See also | Validator , Deposit , FAR_FUTURE_EPOCH , MAX_EFFECTIVE_BALANCE |
def process_deposit(state: BeaconState, deposit: Deposit) -> None:
# Verify the Merkle branch
assert is_valid_merkle_branch(
leaf=hash_tree_root(deposit.data),
branch=deposit.proof,
depth=DEPOSIT_CONTRACT_TREE_DEPTH + 1, # Add 1 for the List length mix-in
index=state.eth1_deposit_index,
root=state.eth1_data.deposit_root,
)
# Deposits must be processed in order
state.eth1_deposit_index += 1
pubkey = deposit.data.pubkey
amount = deposit.data.amount
validator_pubkeys = [validator.pubkey for validator in state.validators]
if pubkey not in validator_pubkeys:
# Verify the deposit signature (proof of possession) which is not checked by the deposit contract
deposit_message = DepositMessage(
pubkey=deposit.data.pubkey,
withdrawal_credentials=deposit.data.withdrawal_credentials,
amount=deposit.data.amount,
)
domain = compute_domain(DOMAIN_DEPOSIT) # Fork-agnostic domain since deposits are valid across forks
signing_root = compute_signing_root(deposit_message, domain)
# Initialize validator if the deposit signature is valid
if bls.Verify(pubkey, signing_root, deposit.data.signature):
state.validators.append(get_validator_from_deposit(state, deposit))
state.balances.append(amount)
state.previous_epoch_participation.append(ParticipationFlags(0b0000_0000))
state.current_epoch_participation.append(ParticipationFlags(0b0000_0000))
state.inactivity_scores.append(uint64(0))
else:
# Increase balance by deposit amount
index = ValidatorIndex(validator_pubkeys.index(pubkey))
increase_balance(state, index, amount)
Here, we process a deposit from a block. If the deposit is valid, either a new validator is created or the deposit amount is added to an existing validator.
The call to is_valid_merkle_branch()
ensures that it is not possible to fake a deposit. The eth1data.deposit_root
from the deposit contract has been agreed by the beacon chain and includes all pending deposits visible to the beacon chain. The deposit itself contains a Merkle proof that it is included in that root. The state.eth1_deposit_index
counter ensures that deposits are processed in order. In short, the proposer provides leaf
and branch
, but neither index
nor root
.
Deposits are signed with the private key of the depositing validator, and the corresponding public key is included in the deposit data. This constitutes a "proof of possession" of the private key, and prevents nastiness like the rogue key attack. Note that compute_domain()
is used directly here when validating the deposit's signature, rather than the more usual get_domain()
wrapper. This is because deposit messages are valid across beacon chain forks (such as Phase 0 and Altair), so we don't want to mix the fork version into the domain. In addition, deposits can be made before genesis_validators_root
is known.
If the Merkle branch check fails, then the whole block is invalid. However, individual deposits can fail the signature check without invalidating the block. This allows incorrectly signed deposits to be de-queued (via state.eth1_deposit_index += 1
) without blocking further progress (that increment would have to be reverted if the block were invalid).
Note that it is not possible to change a validator's withdrawal credentials after the initial deposit: the withdrawal credentials of subsequent deposits for the same validator are ignored; only the credentials appearing on the initial deposit are stored on the beacon chain. This is an important security measure. If an attacker steals a validator's signing key, we don't want them to be able to change the withdrawal credentials in order to steal the stake for themselves. However, it works both ways, and a vulnerability was identified for staking pools in which a malicious operator could potentially front-run a deposit transaction with a 1 ETH deposit to set the withdrawal credentials to their own.
Used by | process_operations() |
Uses | is_valid_merkle_branch() , hash_tree_root() , compute_domain() , compute_signing_root() , bls.Verify() , get_validator_from_deposit() |
See also | Deposit |
Voluntary exits
def process_voluntary_exit(state: BeaconState, signed_voluntary_exit: SignedVoluntaryExit) -> None:
voluntary_exit = signed_voluntary_exit.message
validator = state.validators[voluntary_exit.validator_index]
# Verify the validator is active
assert is_active_validator(validator, get_current_epoch(state))
# Verify exit has not been initiated
assert validator.exit_epoch == FAR_FUTURE_EPOCH
# Exits must specify an epoch when they become valid; they are not valid before then
assert get_current_epoch(state) >= voluntary_exit.epoch
# Verify the validator has been active long enough
assert get_current_epoch(state) >= validator.activation_epoch + SHARD_COMMITTEE_PERIOD
# Verify signature
domain = get_domain(state, DOMAIN_VOLUNTARY_EXIT, voluntary_exit.epoch)
signing_root = compute_signing_root(voluntary_exit, domain)
assert bls.Verify(validator.pubkey, signing_root, signed_voluntary_exit.signature)
# Initiate exit
initiate_validator_exit(state, voluntary_exit.validator_index)
A voluntary exit message is submitted by a validator to indicate that it wishes to cease being an active validator. A proposer receives voluntary exit messages via gossip or via its own API and then includes the message in a block so that it can be processed by the network.
Most of the checks are straightforward, as per the comments in the code. Note the following.
- Voluntary exits are ignored if they are included in blocks before the given
epoch
, so nodes might buffer any future-dated exits they see before putting them in a block. - A validator must have been active for at least
SHARD_COMMITTEE_PERIOD
epochs (27 hours). See there for the rationale. - Voluntary exits are signed with the validator's usual signing key. There is some discussion about changing this to also allow signing of a voluntary exit with the validator's withdrawal key.
If the voluntary exit message is valid then the validator is added to the exit queue by calling initiate_validator_exit()
.
At present it is not possible for a validator to exit and re-enter, but this functionality may be introduced in future.
Used by | process_operations() |
Uses | is_active_validator() , get_domain() , compute_signing_root() , bls.Verify() , initiate_validator_exit() |
See also | VoluntaryExit , SHARD_COMMITTEE_PERIOD |
Sync aggregate processing
def process_sync_aggregate(state: BeaconState, sync_aggregate: SyncAggregate) -> None:
# Verify sync committee aggregate signature signing over the previous slot block root
committee_pubkeys = state.current_sync_committee.pubkeys
participant_pubkeys = [pubkey for pubkey, bit in zip(committee_pubkeys, sync_aggregate.sync_committee_bits) if bit]
previous_slot = max(state.slot, Slot(1)) - Slot(1)
domain = get_domain(state, DOMAIN_SYNC_COMMITTEE, compute_epoch_at_slot(previous_slot))
signing_root = compute_signing_root(get_block_root_at_slot(state, previous_slot), domain)
assert eth_fast_aggregate_verify(participant_pubkeys, signing_root, sync_aggregate.sync_committee_signature)
# Compute participant and proposer rewards
total_active_increments = get_total_active_balance(state) // EFFECTIVE_BALANCE_INCREMENT
total_base_rewards = Gwei(get_base_reward_per_increment(state) * total_active_increments)
max_participant_rewards = Gwei(total_base_rewards * SYNC_REWARD_WEIGHT // WEIGHT_DENOMINATOR // SLOTS_PER_EPOCH)
participant_reward = Gwei(max_participant_rewards // SYNC_COMMITTEE_SIZE)
proposer_reward = Gwei(participant_reward * PROPOSER_WEIGHT // (WEIGHT_DENOMINATOR - PROPOSER_WEIGHT))
# Apply participant and proposer rewards
all_pubkeys = [v.pubkey for v in state.validators]
committee_indices = [ValidatorIndex(all_pubkeys.index(pubkey)) for pubkey in state.current_sync_committee.pubkeys]
for participant_index, participation_bit in zip(committee_indices, sync_aggregate.sync_committee_bits):
if participation_bit:
increase_balance(state, participant_index, participant_reward)
increase_balance(state, get_beacon_proposer_index(state), proposer_reward)
else:
decrease_balance(state, participant_index, participant_reward)
Similarly to how attestations are handled, the beacon block proposer includes in its block an aggregation of sync committee votes that agree with its local view of the chain. Specifically, the sync committee votes are for the head block that the proposer saw in the previous slot. (If the previous slot is empty, then the head block will be from an earlier slot.)
We validate these votes against our local view of the chain, and if they agree then we reward the participants that voted. If they do not agree with our local view, then the entire block is invalid: it is on another branch.
To perform the validation, we form the signing root of the block at the previous slot, with DOMAIN_SYNC_COMMITTEE
mixed in. Then we check if the aggregate signature received in the SyncAggregate
verifies against it, using the aggregate public key of the validators who claimed to have signed it. If either the signing root (that is, the head block) is wrong, or the list of participants is wrong, then the verification will fail and the block is invalid.
Like proposer rewards, but unlike attestation rewards, sync committee rewards are not weighted with the participants' effective balances. This is already taken care of by the committee selection process that weights the probability of selection with the effective balance of the validator.
Running through the calculations:
total_active_increments
: the sum of the effective balances of the entire active validator set normalised with theEFFECTIVE_BALANCE_INCREMENT
to give the total number of increments.total_base_rewards
: the maximum rewards that will be awarded to all validators for all duties this epoch. It is at most in the notation established earlier.max_participant_rewards
: the amount of the total reward to be given to the entire sync committee in this slot.participant_reward
: the reward per participating validator, and the penalty per non-participating validator.proposer_reward
: one seventh of the participant reward.
Each committee member that voted receives a reward of participant_reward
, and the proposer receives one seventh of this in addition.
Each committee member that failed to vote receives a penalty of participant_reward
, and the proposer receives nothing.
In our notation the maximum issuance (reward) due to sync committees per slot is as follows.
The per-epoch reward is thirty-two times this. The maximum reward for the proposer in respect of sync aggregates: