Crate reth_engine_tree

source ·
Expand description

This crate includes the core components for advancing a reth chain.

§Functionality

The components in this crate are involved in:

§Design and motivation

The node must keep up with the state of the chain and validate new updates to the chain state.

In order to respond to consensus messages and advance the chain quickly, validation code must avoid database write operations and perform as much work as possible in-memory. This requirement is what informs the architecture of the components this crate.

§Chain synchronization

When the node receives a block with an unknown parent, or cannot find a certain block hash, it needs to download and validate the part of the chain that is missing.

This can happen during a live sync when the node receives a forkchoice update from the consensus layer which causes the node to have to walk back from the received head, downloading the block’s parents until it reaches a known block.

This can also technically happen when a finalized block is fetched, before checking distance, but this is a very unlikely case.

There are two mutually-exclusive ways the node can bring itself in sync with the chain:

  • Backfill sync: downloading and validating large ranges of blocks in a structured manner, performing different parts of the validation process in sequence.
  • Live sync: By responding to new blocks from a connected consensus layer and downloading any missing blocks on the fly.

To determine which sync type to use, the node checks how many blocks it needs to execute to catch up to the tip of the chain. If this range is large, backfill sync will be used. Otherwise, live sync will be used.

The backfill sync is driven by components which implement the BackfillSync trait, like PipelineSync.

§Handling consensus messages

Consensus message handling is performed by three main components:

  1. The EngineHandler, which takes incoming consensus mesesages and manages any requested backfill or download work.
  2. The EngineApiRequestHandler, which processes messages from the EngineHandler and delegates them to the EngineApiTreeHandler.
  3. The EngineApiTreeHandler, which processes incoming tree events, such as new payload events, sending back requests for any needed backfill or download work.

§Chain representation

The chain is represented by the TreeState data structure, which keeps tracks of blocks by hash and number, as well as keeping track of parent-child relationships between blocks. The hash and number of the current head of the canonical chain is also tracked in the TreeState.

§Persistence model

Because the node minimizes database writes in the critical path for handling consensus messages, it must perform database writes in the background.

Performing writes in the background has two advantages:

  1. As mentioned, writes are not in the critical path of request processing.
  2. Writes can be larger and done at a lower frequency, allowing for more efficient writes.

This is achieved by spawning a separate thread which is sent different commands corresponding to different types of writes, for example a command to write a list of transactions, or remove a specific range of blocks.

The persistence service must also respond to these commands, to ensure that any in-memory state that is on-disk can be cleaned up, conserving memory and allowing us to add new blocks indefinitely.

§Feature Flags

  • test-utils: Export utilities for testing

Modules§

  • backfill
    Support for backfill sync mode. It is expected that the node has two sync modes:
  • chain
    The type that drives the chain forward.
  • download
    Support for downloading blocks on demand for live sync. Handler that can download blocks on demand (e.g. from the network).
  • engine
    Engine Api chain handler support. An engine API handler for the chain.
  • error
    Re-export of the blockchain tree API. Error handling for the blockchain tree
  • metrics
    Metrics support.
  • persistence
    The background writer service, coordinating write operations on static files and the database.
  • test_utilstest-utils
    Test utilities.
  • tree
    Support for interacting with the blockchain tree.

Enums§

  • BlockAttachment
    Re-export of the blockchain tree API. Represents what kind of block is being executed and validated.
  • BlockStatus
    Re-export of the blockchain tree API. From Engine API spec, block inclusion can be valid, accepted or invalid. Invalid case is already covered by error, but we need to make distinction between valid blocks that extend canonical chain and the ones that fork off into side chains (see BlockAttachment). If we don’t know the block parent we are returning Disconnected status as we can’t make a claim if block is valid or not.
  • BlockStatus2
    Re-export of the blockchain tree API. Block inclusion can be valid, accepted, or invalid. Invalid blocks are returned as an error variant.
  • BlockValidationKind
    Re-export of the blockchain tree API. Represents the kind of validation that should be performed when inserting a block.
  • CanonicalOutcome
    Re-export of the blockchain tree API. All possible outcomes of a canonicalization attempt of BlockchainTreeEngine::make_canonical.
  • InsertPayloadOk
    Re-export of the blockchain tree API. How a payload was inserted if it was valid.
  • InsertPayloadOk2
    Re-export of the blockchain tree API. How a payload was inserted if it was valid.

Traits§