Preface

Upgrading Ethereum

Welcome to my book about upgrading Ethereum: Ethereum on proof of stake and beyond.

Back in 2018, I was working with the Teku client team as we helped to create and implement Ethereum's new proof of stake protocol from the ground up. I soon became concerned that the bare specification on its own was missing many of the insights and subtleties that had shaped it, so I began a project to annotate the spec with my own commentary.

A core part of this book is the maintenance of that annotated specification as it evolves through upgrades over time. This has become an increasingly valuable exercise since the way that the official consensus specs are structured and maintained means that there is nowhere to see the entire current spec in one place.

Once the annotated spec was done, it seemed good to start adding some more narrative chapters to try to teach and explain Ethereum's proof of stake mechanisms without being constrained by the structure of the spec. I try not to assume too much in these chapters. The idea is to begin fairly gently and to gradually work up to the more complex aspects.

Two significant challenges hinder anyone wanting to learn about Ethereum's inner workings. For one, although there is quite a lot of information available, it is scattered and can be hard to find and assemble into a coherent picture. For another, articles and explainers tend to get out of date rather quickly, and it's difficult to know what is current and what is stale. I am aiming to help on both these fronts, first by gathering and ordering the information, and second by carefully versioning everything: hence the Altair, Bellatrix, Capella and so on editions.

With all this in mind, it is my hope that this book will serve both as a useful reference to the consensus specification, and as a textbook for exploring Ethereum's consensus layer.

Work in progress!

I am writing this book backwards. Bottom up. Starting with the details and working towards the big picture.

The first pretty much complete part is Part 3: The Annotated Spec. These are the guts of the machine. Like the innards of a computer, all the components are showing and the wires are hanging out: everything is on display. But with the guts in place, everything else can be built around them with the messiness all neatly tucked away.

I'm now working on Part 2: Technical Overview which wraps a first, hopefully more accessible, layer around the Annotated Spec. Again, I'm writing this backwards, starting with the protocol's Building Blocks and its Incentive Mechanisms and working forwards towards a higher level narrative of how it all fits together. The current focus is on the Consensus chapter.

Warning: until Edition 1.0 is out, anything may change. I'll try not to change URLs and anchors in the Annotated Spec part, but no promises. Anything else, including entire chapters and sections, should be considered unstable.

What to expect

This is a book for those who want to understand Ethereum 2.0 – Ethereum on proof of stake – at a technical level. What does it do? How does it work? Why is it like this?

Who am I writing for? For people like me! People who enjoy understanding how things work. But more than that, who like to know why things are the way they are. This is not intended to be an academic work; I am more interested in insight than in rigour. But I try to link out to relevant academic papers and formal analyses where relevant.

Although I am an Ethereum staker and an Ethereum user, I am not writing primarily for stakers or users here. Some of the generic material on Staking might be relevant (if I ever get round to writing it), but you will find better help in places like the excellent EthStaker community.

The scope of the book concerns (what I consider to be) the Ethereum 2.0 protocol. Ethereum 2.0 has become a less well-defined term recently. But for me, it broadly includes,

• all things proof of stake and the beacon chain,
• the process of The Merge by which Ethereum moved to proof of stake,
• in-protocol data sharding, and
• an array of potential future enhancements.

I will not be covering any of the historic Ethereum 1.0 protocol, including the Ethereum Virtual Machine (EVM), except as it touches upon The Merge. The Mastering Ethereum book is an excellent resource, though rather out of date now. Although rollups and other so-called layer 2 solutions have rapidly become part of the overall Ethereum 2.0 system, they are by definition not in-protocol, and I will not be covering them here. I will not be discussing, DeFi, DAOs, NFTs, or any of the wonderful things that can be built on top of this amazing technology.

It's a chunky list of exclusions, but there's still plenty to talk about.

Versions

This edition covers the Capella version of the deployed Ethereum 2.0 consensus layer. The beacon chain went live with Phase 0 on December 1st, 2020. It was upgraded to Altair on October 27th, 2021, to Bellatrix on September 6th, 2022, and to Capella on April 12th, 2023.

Specifically, unless otherwise stated, any reference to the consensus specifications is to the version tagged v1.3.0 (the Gamlum release).

Historical and current versions of Upgrading Ethereum are available online:

• The old Altair spec version,
• The old Bellatrix spec version,
• The current Capella spec version, and
• The latest version, which is currently Capella.

My original annotated specification for the Phase 0 version is also available, though largely of historical interest and now subsumed into this book.

In defence of “Eth2”

When I started this writing project, Ethereum on proof of stake was universally known as Ethereum 2.0, Eth2 for short, as it had been since 2014. This reflects the fact that the Ethereum 2.0 vision diverged from the original Ethereum 1.0 implementation in many respects. The direction of travel was well-captured in the Ethereum 2.0 Mauve Paper in 2016. The details of our eventual implementation differ from what's presented there, but the elements were all in place even then. Eth2 was to deliver not only proof of stake, but new cryptography, new peer-to-peer networking, new cryptoeconomics, and new directions in scalability, among other things. This is what we achieved.

In 2022, the Ethereum Foundation declared that we must emphasise continuity over discontinuity by rebranding Ethereum 2.0 back to plain old Ethereum. With the way that the Merge was coming together, that made some sense from an Ethereum user's point of view. The user and developer experience was to be almost unchanged by the move to proof of stake.

However, I am not writing primarily as a user of Ethereum or as a developer on Ethereum. From the point of view of a protocol engineer, the Merge was a moment of profound discontinuity. Ethereum on proof of stake is utterly different from Ethereum on proof of work - the size and complexity of this book is ample evidence of that. With Ethereum's new architecture we have delivered a good chunk of what the Mauve Paper laid out all those years ago.

This is why I often continue to refer to Ethereum on proof of stake as Ethereum 2.0 or Eth2¹, and I make no apology for that. I am confident that you will understand what I mean, and that it will help clarify rather than confuse.

In defence of the English language

While we're here, another thing I won't be apologising for is using British English spelling, punctuation, and quaint idioms. Consider it a feature rather than a bug.

Acknowledgements

Above all, I want to thank my former employer, Consensys. Much of the writing has been done in my own time, but Consensys was wonderfully supportive of me working on this in the course of my day job. In particular, during Q2 2023, the company granted me three months of writing leave to work full-time on the book. They also allowed me to put a liberal licence on everything. All this is extremely cool of them, and I am deeply grateful. Consensys is a wonderful employer, a terrific force for good in the ecosystem, and an incredible place to work.

So much of what I do involves writing about other people's work, and pretty much everything in this book is other people's work. I deeply value the openness and generosity of the Ethereum community. For me, this is one of its defining characteristics. Many people's contributions are cited throughout this book, and I am indebted to all of you. Being part of the Eth2 dev community has been the best experience of my life.

Thank you to the many GitCoin grant supporters who donated in support of the original annotated specification and my regular What's New in Eth2 newsletter. And to generous crypto friends, anon and otherwise, for your kind gifts over the years. Your support has encouraged me hugely as I've wrestled with the minutiae of the spec. I bloody love this community.