Part 3: Annotated Specification
Beacon Chain State Transition Function
def process_block(state: BeaconState, block: BeaconBlock) -> None: process_block_header(state, block) process_randao(state, block.body) process_eth1_data(state, block.body) process_operations(state, block.body) # [Modified in Altair] process_sync_aggregate(state, block.body.sync_aggregate) # [New in Altair]
These are the tasks that the beacon node performs in order to process a block and update the state. If any of the called functions triggers an
assert failure or any other kind of exception, then the entire block is invalid, and any state changes must be rolled back.
process_operations() covers the processing of any slashing reports (proposer and attester) in the block, any attestations, any deposits, and any voluntary exits.
def process_block_header(state: BeaconState, block: BeaconBlock) -> None: # Verify that the slots match assert block.slot == state.slot # Verify that the block is newer than latest block header assert block.slot > state.latest_block_header.slot # Verify that proposer index is the correct index assert block.proposer_index == get_beacon_proposer_index(state) # Verify that the parent matches assert block.parent_root == hash_tree_root(state.latest_block_header) # Cache current block as the new latest block state.latest_block_header = BeaconBlockHeader( slot=block.slot, proposer_index=block.proposer_index, parent_root=block.parent_root, state_root=Bytes32(), # Overwritten in the next process_slot call body_root=hash_tree_root(block.body), ) # Verify proposer is not slashed proposer = state.validators[block.proposer_index] assert not proposer.slashed
A straightforward set of validity conditions for the block header data.
The version of the block header object that this routine stores in the state is a duplicate of the incoming block's header, but with its
state_root set to its default empty
Bytes32() value. See
process_slot() for the explanation of this.
def process_randao(state: BeaconState, body: BeaconBlockBody) -> None: epoch = get_current_epoch(state) # Verify RANDAO reveal proposer = state.validators[get_beacon_proposer_index(state)] signing_root = compute_signing_root(epoch, get_domain(state, DOMAIN_RANDAO)) assert bls.Verify(proposer.pubkey, signing_root, body.randao_reveal) # Mix in RANDAO reveal mix = xor(get_randao_mix(state, epoch), hash(body.randao_reveal)) state.randao_mixes[epoch % EPOCHS_PER_HISTORICAL_VECTOR] = mix
A good source of randomness is foundational to the operation of the beacon chain. Security of the protocol depends significantly on being able to unpredictably and uniformly select block proposers and committee members. In fact, the very name "beacon chain" was inspired by Dfinity's concept of a randomness beacon.
The current mechanism for providing randomness is a RANDAO, in which each block proposer provides some randomness and all the contributions are mixed together over the course of an epoch. This is not unbiasable (a malicious proposer may choose to skip a block if it is to its advantage to do so), but is good enough. In future, Ethereum might use a verifiable delay function (VDF) to provide unbiasable randomness.
Early designs had the validators pre-committing to "hash onions", peeling off one layer of hashing at each block proposal. This was changed to using a BLS signature over the epoch number as the entropy source. Using signatures is both a simplification, and an enabler for multi-party (distributed) validators. The (reasonable) assumption is that sufficient numbers of validators generated their secret keys with good entropy to ensure that the RANDAO's entropy is adequate.
process_randao() function simply uses the proposer's public key to verify that the RANDAO reveal in the block is indeed the epoch number signed with the proposer's private key. It then mixes the hash of the reveal into the current epoch's RANDAO accumulator. The hash is used in order to reduce the signature down from 96 to 32 bytes, and to make it uniform.
EPOCHS_PER_HISTORICAL_VECTOR past values of the RANDAO accumulator at the ends of epochs are stored in the state.
From Justin Drake's notes:
process_randaois (slightly) more secure than using
hash. To illustrate why, imagine an attacker can grind randomness in the current epoch such that two of his validators are the last proposers, in a different order, in two resulting samplings of the next epochs. The commutativity of
xormakes those two samplings equivalent, hence reducing the attacker's grinding opportunity for the next epoch versus
hash(which is not commutative). The strict security improvement may simplify the derivation of RANDAO security formal lower bounds.
Note that the
assert statement means that the whole block is invalid if the RANDAO reveal is incorrectly formed.
def process_eth1_data(state: BeaconState, body: BeaconBlockBody) -> None: state.eth1_data_votes.append(body.eth1_data) if state.eth1_data_votes.count(body.eth1_data) * 2 > EPOCHS_PER_ETH1_VOTING_PERIOD * SLOTS_PER_EPOCH: state.eth1_data = body.eth1_data
Blocks may contain
Eth1Data which is supposed to be the proposer's best view of the Eth1 chain and the deposit contract at the time. There is no incentive to get this data correct, or penalty for it being incorrect.
If there is a simple majority of the same vote being cast by proposers during each voting period of
EPOCHS_PER_ETH1_VOTING_PERIOD epochs (6.8 hours) then the Eth1 data is committed to the beacon state. This updates the chain's view of the deposit contract, and new deposits since the last update will start being processed.
This mechanism has proved to be fragile in the past, but appears to be workable if not perfect.
def process_operations(state: BeaconState, body: BeaconBlockBody) -> None: # Verify that outstanding deposits are processed up to the maximum number of deposits assert len(body.deposits) == min(MAX_DEPOSITS, state.eth1_data.deposit_count - state.eth1_deposit_index) def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None: for operation in operations: fn(state, operation) for_ops(body.proposer_slashings, process_proposer_slashing) for_ops(body.attester_slashings, process_attester_slashing) for_ops(body.attestations, process_attestation) for_ops(body.deposits, process_deposit) for_ops(body.voluntary_exits, process_voluntary_exit)
Just a dispatcher for handling the various optional contents in a block.
Deposits are optional only in the sense that some blocks have them and some don't. However, as per the
assert statement, if, according to the beacon chain's view of the Eth1 chain, there are deposits pending, then the block must include them, otherwise the block is invalid. On the face of it, this suggests that it is important for a block proposer to have access to an Eth1 node, so as to be able to obtain the deposit data. In practice, this turns out to be not so important, although. with Altair, the proposer reward was increased by a factor of four, which increases the importance of the Eth1 node.
Block proposers are explicitly rewarded for including any available attestations and slashing reports. There is a validity condition, and thus an implicit reward, related to including deposit messages. The incentive for including voluntary exits is that a smaller validator set means higher rewards for the remaining validators.
def process_proposer_slashing(state: BeaconState, proposer_slashing: ProposerSlashing) -> None: header_1 = proposer_slashing.signed_header_1.message header_2 = proposer_slashing.signed_header_2.message # Verify header slots match assert header_1.slot == header_2.slot # Verify header proposer indices match assert header_1.proposer_index == header_2.proposer_index # Verify the headers are different assert header_1 != header_2 # Verify the proposer is slashable proposer = state.validators[header_1.proposer_index] assert is_slashable_validator(proposer, get_current_epoch(state)) # Verify signatures for signed_header in (proposer_slashing.signed_header_1, proposer_slashing.signed_header_2): domain = get_domain(state, DOMAIN_BEACON_PROPOSER, compute_epoch_at_slot(signed_header.message.slot)) signing_root = compute_signing_root(signed_header.message, domain) assert bls.Verify(proposer.pubkey, signing_root, signed_header.signature) slash_validator(state, header_1.proposer_index)
ProposerSlashing is a proof that a proposer has signed two blocks at the same height. Up to
MAX_PROPOSER_SLASHINGS of them may be included in a block. It contains the evidence in the form of a pair of
The proof is simple: the two proposals come from the same slot, have the same proposer, but differ in one or more of
body_root. In addition, they were both signed by the proposer. The conflicting blocks do not need to be valid: any pair of headers that meet the criteria, irrespective of the blocks' contents, are liable to be slashed.
As ever, the
assert statements ensure that the containing block is invalid if it contains any invalid slashing claims.
Fun fact: the first slashing to occur on the beacon chain was a proposer slashing. Two clients running side-by-side with the same keys will often produce the same attestations since the protocol is designed to encourage that. Independently producing the same block is very unlikely as blocks contain much more data.
def process_attester_slashing(state: BeaconState, attester_slashing: AttesterSlashing) -> None: attestation_1 = attester_slashing.attestation_1 attestation_2 = attester_slashing.attestation_2 assert is_slashable_attestation_data(attestation_1.data, attestation_2.data) assert is_valid_indexed_attestation(state, attestation_1) assert is_valid_indexed_attestation(state, attestation_2) slashed_any = False indices = set(attestation_1.attesting_indices).intersection(attestation_2.attesting_indices) for index in sorted(indices): if is_slashable_validator(state.validators[index], get_current_epoch(state)): slash_validator(state, index) slashed_any = True assert slashed_any
AttesterSlashings are similar to proposer slashings in that they just provide the evidence of the two aggregate
IndexedAttestations that conflict with each other. Up to
MAX_ATTESTER_SLASHINGS of them may be included in a block.
The validity checking is done by
is_slashable_attestation_data(), which checks the double vote and surround vote conditions, and by
is_valid_indexed_attestation() which verifies the signatures on the attestations.
Any validators that appear in both attestations are slashed. If no validator is slashed, then the attester slashing claim was not valid after all, and therefore its containing block is invalid.
def process_attestation(state: BeaconState, attestation: Attestation) -> None: data = attestation.data assert data.target.epoch in (get_previous_epoch(state), get_current_epoch(state)) assert data.target.epoch == compute_epoch_at_slot(data.slot) assert data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot <= data.slot + SLOTS_PER_EPOCH assert data.index < get_committee_count_per_slot(state, data.target.epoch) committee = get_beacon_committee(state, data.slot, data.index) assert len(attestation.aggregation_bits) == len(committee) # Participation flag indices participation_flag_indices = get_attestation_participation_flag_indices(state, data, state.slot - data.slot) # Verify signature assert is_valid_indexed_attestation(state, get_indexed_attestation(state, attestation)) # Update epoch participation flags if data.target.epoch == get_current_epoch(state): epoch_participation = state.current_epoch_participation else: epoch_participation = state.previous_epoch_participation proposer_reward_numerator = 0 for index in get_attesting_indices(state, data, attestation.aggregation_bits): for flag_index, weight in enumerate(PARTICIPATION_FLAG_WEIGHTS): if flag_index in participation_flag_indices and not has_flag(epoch_participation[index], flag_index): epoch_participation[index] = add_flag(epoch_participation[index], flag_index) proposer_reward_numerator += get_base_reward(state, index) * weight # Reward proposer proposer_reward_denominator = (WEIGHT_DENOMINATOR - PROPOSER_WEIGHT) * WEIGHT_DENOMINATOR // PROPOSER_WEIGHT proposer_reward = Gwei(proposer_reward_numerator // proposer_reward_denominator) increase_balance(state, get_beacon_proposer_index(state), proposer_reward)
Block proposers are rewarded here for including attestations during block processing, while attesting validators receive their rewards and penalties during epoch processing.
This routine processes each attestation included in the block. First a bunch of validity checks are performed. If any of these fails, then the whole block is invalid (it is most likely from a proposer on a different fork, and so useless to us):
- The target vote of the attestation must be either the previous epoch's checkpoint or the current epoch's checkpoint.
- The target checkpoint and the attestation's slot must belong to the same epoch.
- The attestation must be no newer than
MIN_ATTESTATION_INCLUSION_DELAYslots, which is one. So this condition rules out attestations from the current or future slots.
- The attestation must be no older than
SLOTS_PER_EPOCHslots, which is 32.
- The attestation must come from a committee that existed when the attestation was created.
- The size of the committee and the size of the aggregate must match (
- The (aggregate) signature on the attestation must be valid and must correspond to the aggregated public keys of the validators that it claims to be signed by. This (and other criteria) is checked by
Once the attestation has passed the checks it is processed by converting the votes from validators that it contains into flags in the state.
It's easy to skip over amidst all the checking, but the actual attestation processing is done by
get_attestation_participation_flag_indices(). This takes the source, target, and head votes of the attestation, along with its inclusion delay (how many slots late was it included in a block) and returns a list of up to three flags corresponding to the votes that were both correct and timely, in
For each validator that signed the attestation, if each flag in
participation_flag_indices is not already set for it in its
epoch_participation record, then the flag is set, and the proposer is rewarded. Recall that the validator making the attestation is not rewarded until the end of the epoch. If the flag is already set in the corresponding epoch for a validator, no proposer reward is accumulated: the attestation for this validator was included in an earlier block.
The proposer reward is accumulated, and weighted according to the weight assigned to each of the flags (timely source, timely target, timely head).
If a proposer includes all the attestations only for one slot, and all the relevant validators vote, then its reward will be, in the notation established earlier,
Where is the total maximum reward per epoch for attesters, calculated in
get_flag_index_deltas(). The total available reward in an epoch for proposers including attestations is 32 times this.
def get_validator_from_deposit(state: BeaconState, deposit: Deposit) -> Validator: amount = deposit.data.amount effective_balance = min(amount - amount % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE) return Validator( pubkey=deposit.data.pubkey, withdrawal_credentials=deposit.data.withdrawal_credentials, activation_eligibility_epoch=FAR_FUTURE_EPOCH, activation_epoch=FAR_FUTURE_EPOCH, exit_epoch=FAR_FUTURE_EPOCH, withdrawable_epoch=FAR_FUTURE_EPOCH, effective_balance=effective_balance, )
Create a newly initialised validator object from a deposit. This was factored out of
process_deposit() for better code reuse between the Phase 0 spec and the sharding spec.
state parameter in the input argument list is an oversight: it is not used or required.
def process_deposit(state: BeaconState, deposit: Deposit) -> None: # Verify the Merkle branch assert is_valid_merkle_branch( leaf=hash_tree_root(deposit.data), branch=deposit.proof, depth=DEPOSIT_CONTRACT_TREE_DEPTH + 1, # Add 1 for the List length mix-in index=state.eth1_deposit_index, root=state.eth1_data.deposit_root, ) # Deposits must be processed in order state.eth1_deposit_index += 1 pubkey = deposit.data.pubkey amount = deposit.data.amount validator_pubkeys = [validator.pubkey for validator in state.validators] if pubkey not in validator_pubkeys: # Verify the deposit signature (proof of possession) which is not checked by the deposit contract deposit_message = DepositMessage( pubkey=deposit.data.pubkey, withdrawal_credentials=deposit.data.withdrawal_credentials, amount=deposit.data.amount, ) domain = compute_domain(DOMAIN_DEPOSIT) # Fork-agnostic domain since deposits are valid across forks signing_root = compute_signing_root(deposit_message, domain) # Initialize validator if the deposit signature is valid if bls.Verify(pubkey, signing_root, deposit.data.signature): state.validators.append(get_validator_from_deposit(state, deposit)) state.balances.append(amount) state.previous_epoch_participation.append(ParticipationFlags(0b0000_0000)) state.current_epoch_participation.append(ParticipationFlags(0b0000_0000)) state.inactivity_scores.append(uint64(0)) else: # Increase balance by deposit amount index = ValidatorIndex(validator_pubkeys.index(pubkey)) increase_balance(state, index, amount)
Here, we process a deposit from a block. If the deposit is valid, either a new validator is created or the deposit amount is added to an existing validator.
The call to
is_valid_merkle_branch() ensures that it is not possible to fake a deposit. The
eth1data.deposit_root from the deposit contract has been agreed by the beacon chain and includes all pending deposits visible to the beacon chain. The deposit itself contains a Merkle proof that it is included in that root. The
state.eth1_deposit_index counter ensures that deposits are processed in order. In short, the proposer provides
branch, but neither
Deposits are signed with the private key of the depositing validator, and the corresponding public key is included in the deposit data. This constitutes a "proof of possession" of the private key, and prevents nastiness like the rogue key attack. Note that
compute_domain() is used directly here when validating the deposit's signature, rather than the more usual
get_domain() wrapper. This is because deposit messages are valid across beacon chain forks (such as Phase 0 and Altair), so we don't want to mix the fork version into the domain. In addition, deposits can be made before
genesis_validators_root is known.
If the Merkle branch check fails, then the whole block is invalid. However, individual deposits can fail the signature check without invalidating the block. This allows incorrectly signed deposits to be de-queued (via
state.eth1_deposit_index += 1) without blocking further progress (that increment would have to be reverted if the block were invalid).
Note that it is not possible to change a validator's withdrawal credentials after the initial deposit: the withdrawal credentials of subsequent deposits for the same validator are ignored; only the credentials appearing on the initial deposit are stored on the beacon chain. This is an important security measure. If an attacker steals a validator's signing key, we don't want them to be able to change the withdrawal credentials in order to steal the stake for themselves. However, it works both ways, and a vulnerability was identified for staking pools in which a malicious operator could potentially front-run a deposit transaction with a 1 ETH deposit to set the withdrawal credentials to their own.
def process_voluntary_exit(state: BeaconState, signed_voluntary_exit: SignedVoluntaryExit) -> None: voluntary_exit = signed_voluntary_exit.message validator = state.validators[voluntary_exit.validator_index] # Verify the validator is active assert is_active_validator(validator, get_current_epoch(state)) # Verify exit has not been initiated assert validator.exit_epoch == FAR_FUTURE_EPOCH # Exits must specify an epoch when they become valid; they are not valid before then assert get_current_epoch(state) >= voluntary_exit.epoch # Verify the validator has been active long enough assert get_current_epoch(state) >= validator.activation_epoch + SHARD_COMMITTEE_PERIOD # Verify signature domain = get_domain(state, DOMAIN_VOLUNTARY_EXIT, voluntary_exit.epoch) signing_root = compute_signing_root(voluntary_exit, domain) assert bls.Verify(validator.pubkey, signing_root, signed_voluntary_exit.signature) # Initiate exit initiate_validator_exit(state, voluntary_exit.validator_index)
A voluntary exit message is submitted by a validator to indicate that it wishes to cease being an active validator. A proposer receives voluntary exit messages via gossip or via its own API and then includes the message in a block so that it can be processed by the network.
Most of the checks are straightforward, as per the comments in the code. Note the following.
- Voluntary exits are ignored if they are included in blocks before the given
epoch, so nodes might buffer any future-dated exits they see before putting them in a block.
- A validator must have been active for at least
SHARD_COMMITTEE_PERIODepochs (27 hours). See there for the rationale.
- Voluntary exits are signed with the validator's usual signing key. There is some discussion about changing this to also allow signing of a voluntary exit with the validator's withdrawal key.
If the voluntary exit message is valid then the validator is added to the exit queue by calling
def process_sync_aggregate(state: BeaconState, sync_aggregate: SyncAggregate) -> None: # Verify sync committee aggregate signature signing over the previous slot block root committee_pubkeys = state.current_sync_committee.pubkeys participant_pubkeys = [pubkey for pubkey, bit in zip(committee_pubkeys, sync_aggregate.sync_committee_bits) if bit] previous_slot = max(state.slot, Slot(1)) - Slot(1) domain = get_domain(state, DOMAIN_SYNC_COMMITTEE, compute_epoch_at_slot(previous_slot)) signing_root = compute_signing_root(get_block_root_at_slot(state, previous_slot), domain) assert eth_fast_aggregate_verify(participant_pubkeys, signing_root, sync_aggregate.sync_committee_signature) # Compute participant and proposer rewards total_active_increments = get_total_active_balance(state) // EFFECTIVE_BALANCE_INCREMENT total_base_rewards = Gwei(get_base_reward_per_increment(state) * total_active_increments) max_participant_rewards = Gwei(total_base_rewards * SYNC_REWARD_WEIGHT // WEIGHT_DENOMINATOR // SLOTS_PER_EPOCH) participant_reward = Gwei(max_participant_rewards // SYNC_COMMITTEE_SIZE) proposer_reward = Gwei(participant_reward * PROPOSER_WEIGHT // (WEIGHT_DENOMINATOR - PROPOSER_WEIGHT)) # Apply participant and proposer rewards all_pubkeys = [v.pubkey for v in state.validators] committee_indices = [ValidatorIndex(all_pubkeys.index(pubkey)) for pubkey in state.current_sync_committee.pubkeys] for participant_index, participation_bit in zip(committee_indices, sync_aggregate.sync_committee_bits): if participation_bit: increase_balance(state, participant_index, participant_reward) increase_balance(state, get_beacon_proposer_index(state), proposer_reward) else: decrease_balance(state, participant_index, participant_reward)
Similarly to how attestations are handled, the beacon block proposer includes in its block an aggregation of sync committee votes that agree with its local view of the chain. Specifically, the sync committee votes are for the head block that the proposer saw in the previous slot. (If the previous slot is empty, then the head block will be from an earlier slot.)
We validate these votes against our local view of the chain, and if they agree then we reward the participants that voted. If they do not agree with our local view, then the entire block is invalid: it is on another branch.
To perform the validation, we form the signing root of the block at the previous slot, with
DOMAIN_SYNC_COMMITTEE mixed in. Then we check if the aggregate signature received in the
SyncAggregate verifies against it, using the aggregate public key of the validators who claimed to have signed it. If either the signing root (that is, the head block) is wrong, or the list of participants is wrong, then the verification will fail and the block is invalid.
Like proposer rewards, but unlike attestation rewards, sync committee rewards are not weighted with the participants' effective balances. This is already taken care of by the committee selection process that weights the probability of selection with the effective balance of the validator.
Running through the calculations:
total_active_increments: the sum of the effective balances of the entire active validator set normalised with the
EFFECTIVE_BALANCE_INCREMENTto give the total number of increments.
total_base_rewards: the maximum rewards that will be awarded to all validators for all duties this epoch. It is at most in the notation established earlier.
max_participant_rewards: the amount of the total reward to be given to the entire sync committee in this slot.
participant_reward: the reward per participating validator, and the penalty per non-participating validator.
proposer_reward: one seventh of the participant reward.
Each committee member that voted receives a reward of
participant_reward, and the proposer receives one seventh of this in addition.
Each committee member that failed to vote receives a penalty of
participant_reward, and the proposer receives nothing.
In our notation the maximum issuance (reward) due to sync committees per slot is as follows.
The per-epoch reward is thirty-two times this. The maximum reward for the proposer in respect of sync aggregates: