From e7856125edf025d59cc4c136adf7089dc537d888 Mon Sep 17 00:00:00 2001 From: Jinsuk Park Date: Wed, 28 Aug 2024 08:54:04 +0900 Subject: [PATCH] move changes to builder/README.md --- README.md | 400 ++++++++++++++++++++++++++++++++++------------ builder/README.md | 153 ++++++++++++++++-- 2 files changed, 435 insertions(+), 118 deletions(-) diff --git a/README.md b/README.md index f973ef83c9..0d5b787212 100644 --- a/README.md +++ b/README.md @@ -1,166 +1,358 @@ -# Flashbots Op-Geth Builder +## Go Ethereum -This is an experimental project that implements the block builder for Optimism, based on [op-geth](https://github.com/ethereum-optimism/op-geth). +Golang execution layer implementation of the Ethereum protocol. -## How it works +[![API Reference]( +https://pkg.go.dev/badge/github.com/ethereum/go-ethereum +)](https://pkg.go.dev/github.com/ethereum/go-ethereum?tab=doc) +[![Go Report Card](https://goreportcard.com/badge/github.com/ethereum/go-ethereum)](https://goreportcard.com/report/github.com/ethereum/go-ethereum) +[![Travis](https://app.travis-ci.com/ethereum/go-ethereum.svg?branch=master)](https://app.travis-ci.com/github/ethereum/go-ethereum) +[![Discord](https://img.shields.io/badge/discord-join%20chat-blue.svg)](https://discord.gg/nthXNEv) -### GetPayload Builder API +Automated builds are available for stable releases and the unstable master branch. Binary +archives are published at https://geth.ethereum.org/downloads/. -We introduced a new API endpoint in the builder to get the payload for a given slot and parent hash. The sequencer can request the payload from the builder by calling this API and insert the payload to the engine. +## Building the source -#### GET /eth/v1/builder/payload/{slot:[0-9]+}/{parent_hash:0x[a-fA-F0-9]+} +For prerequisites and detailed build instructions please read the [Installation Instructions](https://geth.ethereum.org/docs/getting-started/installing-geth). -* {slot}: The numerical slot number for which the payload is requested. -* {parent_hash}: The hexadecimal hash of the parent block, prefixed with 0x. +Building `geth` requires both a Go (version 1.21 or later) and a C compiler. You can install +them using your favourite package manager. Once the dependencies are installed, run -* Response: A JSON object adhering to the [VersionedSubmitBlockRequest](https://github.com/attestantio/go-builder-client/blob/master/spec/versionedsubmitblockrequest.go) schema. +```shell +make geth +``` + +or, to build the full suite of utilities: + +```shell +make all +``` + +## Executables -The we are using different payload versions for following optimism fork names: +The go-ethereum project comes with several wrappers/executables found in the `cmd` +directory. -| Fork Name | Payload Version | -| --------- | --------------- | -| Bedrock | Bellatrix | -| Canyon | Capella | -| Ecotone | Deneb | +| Command | Description | +| :--------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **`geth`** | Our main Ethereum CLI client. It is the entry point into the Ethereum network (main-, test- or private net), capable of running as a full node (default), archive node (retaining all historical state) or a light node (retrieving data live). It can be used by other processes as a gateway into the Ethereum network via JSON RPC endpoints exposed on top of HTTP, WebSocket and/or IPC transports. `geth --help` and the [CLI page](https://geth.ethereum.org/docs/fundamentals/command-line-options) for command line options. | +| `clef` | Stand-alone signing tool, which can be used as a backend signer for `geth`. | +| `devp2p` | Utilities to interact with nodes on the networking layer, without running a full blockchain. | +| `abigen` | Source code generator to convert Ethereum contract definitions into easy-to-use, compile-time type-safe Go packages. It operates on plain [Ethereum contract ABIs](https://docs.soliditylang.org/en/develop/abi-spec.html) with expanded functionality if the contract bytecode is also available. However, it also accepts Solidity source files, making development much more streamlined. Please see our [Native DApps](https://geth.ethereum.org/docs/developers/dapp-developer/native-bindings) page for details. | +| `bootnode` | Stripped down version of our Ethereum client implementation that only takes part in the network node discovery protocol, but does not run any of the higher level application protocols. It can be used as a lightweight bootstrap node to aid in finding peers in private networks. | +| `evm` | Developer utility version of the EVM (Ethereum Virtual Machine) that is capable of running bytecode snippets within a configurable environment and execution mode. Its purpose is to allow isolated, fine-grained debugging of EVM opcodes (e.g. `evm --code 60ff60ff --debug run`). | +| `rlpdump` | Developer utility tool to convert binary RLP ([Recursive Length Prefix](https://ethereum.org/en/developers/docs/data-structures-and-encoding/rlp)) dumps (data encoding used by the Ethereum protocol both network as well as consensus wise) to user-friendlier hierarchical representation (e.g. `rlpdump --hex CE0183FFFFFFC4C304050583616263`). | +## Running `geth` -### Op-Node modification +Going through all the possible command line flags is out of scope here (please consult our +[CLI Wiki page](https://geth.ethereum.org/docs/fundamentals/command-line-options)), +but we've enumerated a few common parameter combos to get you up to speed quickly +on how you can run your own `geth` instance. -Modified version of op-node is required for this builder to work. It is available in our flashbots optimism repository. +### Hardware Requirements -Similar to the original beacon chain proposer-builder-separation, op-node publishes attributes to the event stream, from which the builder can pick up the latest head update and start building blocks. +Minimum: -A simplified sequence diagram of the process is as follows: +* CPU with 2+ cores +* 4GB RAM +* 1TB free storage space to sync the Mainnet +* 8 MBit/sec download Internet service + +Recommended: + +* Fast CPU with 4+ cores +* 16GB+ RAM +* High-performance SSD with at least 1TB of free space +* 25+ MBit/sec download Internet service + +### Full node on the main Ethereum network + +By far the most common scenario is people wanting to simply interact with the Ethereum +network: create accounts; transfer funds; deploy and interact with contracts. For this +particular use case, the user doesn't care about years-old historical data, so we can +sync quickly to the current state of the network. To do so: + +```shell +$ geth console +``` -```mermaid -sequenceDiagram - box rgba(103, 189, 220, 0.8) Sequencer - participant SEQ as Sequencer - participant ENGINE as Local Engine - end - box rgba(210, 209, 71, 0.8) Builder - participant BOP as Builder Op-Node - participant BUILDER as Builder Op-Geth - end +This command will: + * Start `geth` in snap sync mode (default, can be changed with the `--syncmode` flag), + causing it to download more data in exchange for avoiding processing the entire history + of the Ethereum network, which is very CPU intensive. + * Start the built-in interactive [JavaScript console](https://geth.ethereum.org/docs/interacting-with-geth/javascript-console), + (via the trailing `console` subcommand) through which you can interact using [`web3` methods](https://github.com/ChainSafe/web3.js/blob/0.20.7/DOCUMENTATION.md) + (note: the `web3` version bundled within `geth` is very old, and not up to date with official docs), + as well as `geth`'s own [management APIs](https://geth.ethereum.org/docs/interacting-with-geth/rpc). + This tool is optional and if you leave it out you can always attach it to an already running + `geth` instance with `geth attach`. + +### A Full node on the Görli test network + +Transitioning towards developers, if you'd like to play around with creating Ethereum +contracts, you almost certainly would like to do that without any real money involved until +you get the hang of the entire system. In other words, instead of attaching to the main +network, you want to join the **test** network with your node, which is fully equivalent to +the main network, but with play-Ether only. - SEQ->>BOP: Send latest head update (p2p channel) - BOP->>BUILDER: Publish attributes to the event stream - activate BUILDER - BUILDER->>BUILDER: Start building blocks - SEQ->>BUILDER: GetPayload Builder API request - BUILDER-->>SEQ: VersionedSubmitBlockRequest - deactivate BUILDER - SEQ->>ENGINE: Insert payload +```shell +$ geth --goerli console ``` -Key Components: -* Sequencer: Optimism sequencer -* Local Engine: Optimism vanilla op-geth engine. -* Builder Op-Node: Op-node connected to the network to get the latest head update. -* Builder: Flashbots block builder for Optimism. +The `console` subcommand has the same meaning as above and is equally +useful on the testnet too. + +Specifying the `--goerli` flag, however, will reconfigure your `geth` instance a bit: -Process: -1. Sequencer publishes the latest head update in p2p channel -2. Builder Op-Node receives the head update and publishes the attributes to the event stream. -3. Builder listens to the event stream and triggers the build process when the attributes are available. -4. Sequencer requests the builder for the payload by `GetPayload` API. -5. Sequencer inserts the payload to the local engine. + * Instead of connecting to the main Ethereum network, the client will connect to the Görli + test network, which uses different P2P bootnodes, different network IDs and genesis + states. + * Instead of using the default data directory (`~/.ethereum` on Linux for example), `geth` + will nest itself one level deeper into a `goerli` subfolder (`~/.ethereum/goerli` on + Linux). Note, on OSX and Linux this also means that attaching to a running testnet node + requires the use of a custom endpoint since `geth attach` will try to attach to a + production node endpoint by default, e.g., + `geth attach /goerli/geth.ipc`. Windows users are not affected by + this. -## How to run on devnet. +*Note: Although some internal protective measures prevent transactions from +crossing over between the main network and test network, you should always +use separate accounts for play and real money. Unless you manually move +accounts, `geth` will by default correctly separate the two networks and will not make any +accounts available between them.* -### Clone the repository +### Configuration + +As an alternative to passing the numerous flags to the `geth` binary, you can also pass a +configuration file via: ```shell -git clone git@github.com:flashbots/optimism.git +$ geth --config /path/to/your_config.toml ``` -### Running with default flashbots/op-geth builder image. +To get an idea of how the file should look like you can use the `dumpconfig` subcommand to +export your existing configuration: ```shell -$ DEVNET_BUILDER=true make devnet-up +$ geth --your-favourite-flags dumpconfig ``` -To additionally enable load testing through [tx-fuzz](https://github.com/MariusVanDerWijden/tx-fuzz), you can run the following command: +*Note: This works only with `geth` v1.6.0 and above.* + +#### Docker quick start + +One of the quickest ways to get Ethereum up and running on your machine is by using +Docker: ```shell -$ DEVNET_LOAD_TEST=true DEVNET_BUILDER=true make devnet-up +docker run -d --name ethereum-node -v /Users/alice/ethereum:/root \ + -p 8545:8545 -p 30303:30303 \ + ethereum/client-go +``` + +This will start `geth` in snap-sync mode with a DB memory allowance of 1GB, as the +above command does. It will also create a persistent volume in your home directory for +saving your blockchain as well as map the default ports. There is also an `alpine` tag +available for a slim version of the image. + +Do not forget `--http.addr 0.0.0.0`, if you want to access RPC from other containers +and/or hosts. By default, `geth` binds to the local interface and RPC endpoints are not +accessible from the outside. + +### Programmatically interfacing `geth` nodes + +As a developer, sooner rather than later you'll want to start interacting with `geth` and the +Ethereum network via your own programs and not manually through the console. To aid +this, `geth` has built-in support for a JSON-RPC based APIs ([standard APIs](https://ethereum.github.io/execution-apis/api-documentation/) +and [`geth` specific APIs](https://geth.ethereum.org/docs/interacting-with-geth/rpc)). +These can be exposed via HTTP, WebSockets and IPC (UNIX sockets on UNIX based +platforms, and named pipes on Windows). + +The IPC interface is enabled by default and exposes all the APIs supported by `geth`, +whereas the HTTP and WS interfaces need to manually be enabled and only expose a +subset of APIs due to security reasons. These can be turned on/off and configured as +you'd expect. + +HTTP based JSON-RPC API options: + + * `--http` Enable the HTTP-RPC server + * `--http.addr` HTTP-RPC server listening interface (default: `localhost`) + * `--http.port` HTTP-RPC server listening port (default: `8545`) + * `--http.api` API's offered over the HTTP-RPC interface (default: `eth,net,web3`) + * `--http.corsdomain` Comma separated list of domains from which to accept cross origin requests (browser enforced) + * `--ws` Enable the WS-RPC server + * `--ws.addr` WS-RPC server listening interface (default: `localhost`) + * `--ws.port` WS-RPC server listening port (default: `8546`) + * `--ws.api` API's offered over the WS-RPC interface (default: `eth,net,web3`) + * `--ws.origins` Origins from which to accept WebSocket requests + * `--ipcdisable` Disable the IPC-RPC server + * `--ipcapi` API's offered over the IPC-RPC interface (default: `admin,debug,eth,miner,net,personal,txpool,web3`) + * `--ipcpath` Filename for IPC socket/pipe within the datadir (explicit paths escape it) + +You'll need to use your own programming environments' capabilities (libraries, tools, etc) to +connect via HTTP, WS or IPC to a `geth` node configured with the above flags and you'll +need to speak [JSON-RPC](https://www.jsonrpc.org/specification) on all transports. You +can reuse the same connection for multiple requests! + +**Note: Please understand the security implications of opening up an HTTP/WS based +transport before doing so! Hackers on the internet are actively trying to subvert +Ethereum nodes with exposed APIs! Further, all browser tabs can access locally +running web servers, so malicious web pages could try to subvert locally available +APIs!** + +### Operating a private network + +Maintaining your own private network is more involved as a lot of configurations taken for +granted in the official networks need to be manually set up. + +#### Defining the private genesis state + +First, you'll need to create the genesis state of your networks, which all nodes need to be +aware of and agree upon. This consists of a small JSON file (e.g. call it `genesis.json`): + +```json +{ + "config": { + "chainId": , + "homesteadBlock": 0, + "eip150Block": 0, + "eip155Block": 0, + "eip158Block": 0, + "byzantiumBlock": 0, + "constantinopleBlock": 0, + "petersburgBlock": 0, + "istanbulBlock": 0, + "berlinBlock": 0, + "londonBlock": 0 + }, + "alloc": {}, + "coinbase": "0x0000000000000000000000000000000000000000", + "difficulty": "0x20000", + "extraData": "", + "gasLimit": "0x2fefd8", + "nonce": "0x0000000000000042", + "mixhash": "0x0000000000000000000000000000000000000000000000000000000000000000", + "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000", + "timestamp": "0x00" +} ``` -### Running with custom op-geth builder image. +The above fields should be fine for most purposes, although we'd recommend changing +the `nonce` to some random value so you prevent unknown remote nodes from being able +to connect to you. If you'd like to pre-fund some accounts for easier testing, create +the accounts and populate the `alloc` field with their addresses. + +```json +"alloc": { + "0x0000000000000000000000000000000000000001": { + "balance": "111111111" + }, + "0x0000000000000000000000000000000000000002": { + "balance": "222222222" + } +} +``` -You first need to build the op-geth docker image with builder API support. +With the genesis state defined in the above JSON file, you'll need to initialize **every** +`geth` node with it prior to starting it up to ensure all blockchain parameters are correctly +set: ```shell -$ git clone git@github.com:flashbots/op-geth.git -$ cd op-geth -$ docker build . -t +$ geth init path/to/genesis.json ``` -Then you can run the devnet with the following command: +#### Creating the rendezvous point + +With all nodes that you want to run initialized to the desired genesis state, you'll need to +start a bootstrap node that others can use to find each other in your network and/or over +the internet. The clean way is to configure and run a dedicated bootnode: ```shell -$ BUILDER_IMAGE= DEVNET_BUILDER=true make devnet-up +$ bootnode --genkey=boot.key +$ bootnode --nodekey=boot.key ``` -## Configuration +With the bootnode online, it will display an [`enode` URL](https://ethereum.org/en/developers/docs/networking-layer/network-addresses/#enode) +that other nodes can use to connect to it and exchange peer information. Make sure to +replace the displayed IP address information (most probably `[::]`) with your externally +accessible IP to get the actual `enode` URL. -These are the configuration options to enable PBS for the devnet. +*Note: You could also use a full-fledged `geth` node as a bootnode, but it's the less +recommended way.* -### Sequencer +#### Starting up your member nodes -There are three flags that configure the sequencer to request payloads from the builder API endpoint: +With the bootnode operational and externally reachable (you can try +`telnet ` to ensure it's indeed reachable), start every subsequent `geth` +node pointed to the bootnode for peer discovery via the `--bootnodes` flag. It will +probably also be desirable to keep the data directory of your private network separated, so +do also specify a custom `--datadir` flag. -| Flag | Description | Default Value | -|----------------------------|-------------------------------------------------------------------------|---------------| -| `l2.builder.enabled` | Enable the builder API request to get payloads built from the builder. | `false` | -| `l2.builder.endpoint` | The URL of the builder API endpoint. | `""` | -| `l2.builder.timeout` | The timeout for the builder API request. | `500ms` | +```shell +$ geth --datadir=path/to/custom/data/folder --bootnodes= +``` -### Builder op-node +*Note: Since your network will be completely cut off from the main and test networks, you'll +also need to configure a miner to process transactions and create new blocks for you.* -The op-geth builder requires the op-node to publish the latest attributes as server-sent events in order to start building the payloads. +#### Running a private miner -| Flag | Description | Default Value | -|--------------------------------|----------------------------------------------------------------------------------|---------------| -| `sequencer.publish-attributes` | Set to true to enable the sequencer to publish attributes to the event stream. | `false` | -| `eventstream.addr` | The address of the eventstream server. | `127.0.0.1` | -| `eventstream.port` | The port of the eventstream server. | `9546` | -### Builder +In a private network setting a single CPU miner instance is more than enough for +practical purposes as it can produce a stable stream of blocks at the correct intervals +without needing heavy resources (consider running on a single thread, no need for multiple +ones either). To start a `geth` instance for mining, run it with all your usual flags, extended +by: + +```shell +$ geth --mine --miner.threads=1 --miner.etherbase=0x0000000000000000000000000000000000000000 +``` -These are the builder flags to enable the builder service in op-geth: +Which will start mining blocks and transactions on a single CPU thread, crediting all +proceedings to the account specified by `--miner.etherbase`. You can further tune the mining +by changing the default gas limit blocks converge to (`--miner.targetgaslimit`) and the price +transactions are accepted at (`--miner.gasprice`). -| Flag | Description | Default Value | -|----------------------------------|----------------------------------------------------------------------------------------------|---------------| -| `builder` | Enable the builder service. | `false` | -| `builder.beacon_endpoints` | The op-node address to get the payload attributes from. Should be set to `builder-op-node`. | `""` | -| `builder.block_retry_interval` | The interval to retry building the payload. | `500ms` | -| `builder.block_time` | Block time of the network. | `2s` | +## Contribution -## Metrics +Thank you for considering helping out with the source code! We welcome contributions +from anyone on the internet, and are grateful for even the smallest of fixes! -### Sequencer +If you'd like to contribute to go-ethereum, please fork, fix, commit and send a pull request +for the maintainers to review and merge into the main code base. If you wish to submit +more complex changes though, please check up with the core devs first on [our Discord Server](https://discord.gg/invite/nthXNEv) +to ensure those changes are in line with the general philosophy of the project and/or get +some early feedback which can make both your efforts much lighter as well as our review +and merge procedures quick and simple. -The sequencer exposes the builder API related prometheus metrics to the same `/metrics` endpoint of op-node. +Please make sure your contributions adhere to our coding guidelines: + * Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) + guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)). + * Code must be documented adhering to the official Go [commentary](https://golang.org/doc/effective_go.html#commentary) + guidelines. + * Pull requests need to be based on and opened against the `master` branch. + * Commit messages should be prefixed with the package(s) they modify. + * E.g. "eth, rpc: make trace configs optional" -| Metric Name | Description | -|--------------------------------------------------------------|-------------------------------------------------------------------| -| `op_node_default_sequencer_builder_request_duration_seconds` | The duration of the builder API request. | -| `op_node_default_sequencer_builder_request_errors` | The total number of errors in the builder API request. | -| `op_node_default_sequencer_builder_request_timeout` | The total number of builder API requests. | -| `op_node_default_sequencer_builder_payloads_bytes` | The total size of the payloads built by the builder. | -| `op_node_default_sequencer_profit` | The total profit made by the sequencer. | -| `op_node_default_sequencer_payload_inserted` | The total number of payloads inserted into the local engine. | -| `op_node_default_sequencer_payload_gas_total` | The total gas used by the payloads. | +Please see the [Developers' Guide](https://geth.ethereum.org/docs/developers/geth-developer/dev-guide) +for more details on configuring your environment, managing project dependencies, and +testing procedures. +### Contributing to geth.ethereum.org -# Security +For contributions to the [go-ethereum website](https://geth.ethereum.org), please checkout and raise pull requests against the `website` branch. +For more detailed instructions please see the `website` branch [README](https://github.com/ethereum/go-ethereum/tree/website#readme) or the +[contributing](https://geth.ethereum.org/docs/developers/geth-developer/contributing) page of the website. -If you find a security vulnerability in this project or any other initiative -related to proposer/builder separation in Ethereum, please let us know sending -an email to security@flashbots.net. +## License -# License +The go-ethereum library (i.e. all code outside of the `cmd` directory) is licensed under the +[GNU Lesser General Public License v3.0](https://www.gnu.org/licenses/lgpl-3.0.en.html), +also included in our repository in the `COPYING.LESSER` file. -The code in this project is free software under the [LGPL License](COPYING.LESSER). +The go-ethereum binaries (i.e. all code inside of the `cmd` directory) are licensed under the +[GNU General Public License v3.0](https://www.gnu.org/licenses/gpl-3.0.en.html), also +included in our repository in the `COPYING` file. diff --git a/builder/README.md b/builder/README.md index 2d4940c1d5..0b031f8b17 100644 --- a/builder/README.md +++ b/builder/README.md @@ -1,25 +1,55 @@ -# Op-Geth Builder +# Flashbots Op-Geth Builder +This is an experimental project that implements the block builder for Optimism, based on [op-geth](https://github.com/ethereum-optimism/op-geth). [![Docker hub](https://badgen.net/docker/size/flashbots/op-geth?icon=docker&label=image)](https://hub.docker.com/r/flashbots/op-geth/tags) -This is an implementation of a block builder for proposer / builder separation (PBS) in the optimism stack. Please see [this guide](https://github.com/flashbots/optimism/blob/daa43f158ffca0bfaba18391f688fed1d8a8f3d9/pbs/README.md) on how to run and test with a local devnet. +## How it works -## Block Builder API Interation +### GetPayload Builder API -The builder requires an [op-node](https://github.com/flashbots/optimism/tree/pbs) that publishes an event stream with the block payload attributes to trigger block building. +We introduced a new API endpoint in the builder to get the payload for a given slot and parent hash. The sequencer can request the payload from the builder by calling this API and insert the payload to the engine. + +#### GET /eth/v1/builder/payload/{slot:[0-9]+}/{parent_hash:0x[a-fA-F0-9]+} + +* {slot}: The numerical slot number for which the payload is requested. +* {parent_hash}: The hexadecimal hash of the parent block, prefixed with 0x. + +* Response: A JSON object adhering to the [VersionedSubmitBlockRequest](https://github.com/attestantio/go-builder-client/blob/master/spec/versionedsubmitblockrequest.go) schema. + +The we are using different payload versions for following optimism fork names: + +| Fork Name | Payload Version | +| --------- | --------------- | +| Bedrock | Bellatrix | +| Canyon | Capella | +| Ecotone | Deneb | + + +### Op-Node modification + +The builder requires an [op-node](https://github.com/flashbots/optimism/tree/pbs) that publishes an event stream with the block payload attributes to trigger block building. + +Similar to the original beacon chain proposer-builder-separation, op-node publishes attributes to the event stream, from which the builder can pick up the latest head update and start building blocks. To request a block, the proposer does a http request to the builder in parallel with its local node for a block as a failsafe. The following sequence diagram describes the interactions between the builder and the proposer. +A simplified sequence diagram of the process is as follows: + ```mermaid sequenceDiagram + box rgba(103, 189, 220, 0.8) Sequencer participant EES as Exec Engine (Proposer) participant OPS as Op-Node (Proposer) + end + box rgba(210, 209, 71, 0.8) Builder participant OPB as Op-Node (Block Builder) participant BB as Block Builder + end + BB-->>OPB: payload_attributes events OPS-->> OPB: Fork Choice Update (p2p) OPB-->>BB: PayloadAttributes - + Note right of BB: timespan for building blocks OPS->> BB: /eth/v1/builder/payload/{slot}/{parent_hash} BB-->>OPS: BuilderPayload @@ -29,19 +59,114 @@ sequenceDiagram OPS ->> EES: engine_forkchoiceUpdated ``` -## Components +Key Components: +* Sequencer: Optimism sequencer +* Local Engine: Optimism vanilla op-geth engine. +* Builder Op-Node: Op-node connected to the network to get the latest head update. +* Builder: Flashbots block builder for Optimism. + +Process: +1. Sequencer publishes the latest head update in p2p channel +2. Builder Op-Node receives the head update and publishes the attributes to the event stream. +3. Builder listens to the event stream and triggers the build process when the attributes are available. +4. Sequencer requests the builder for the payload by `GetPayload` API. +5. Sequencer inserts the payload to the local engine. + +## How to run on devnet. + +### Clone the repository + +```shell +git clone git@github.com:flashbots/optimism.git +``` + +### Running with default flashbots/op-geth builder image. + +```shell +$ DEVNET_BUILDER=true make devnet-up +``` + +To additionally enable load testing through [tx-fuzz](https://github.com/MariusVanDerWijden/tx-fuzz), you can run the following command: + +```shell +$ DEVNET_LOAD_TEST=true DEVNET_BUILDER=true make devnet-up +``` + +### Running with custom op-geth builder image. + +You first need to build the op-geth docker image with builder API support. + +```shell +$ git clone git@github.com:flashbots/op-geth.git +$ cd op-geth +$ docker build . -t +``` + +Then you can run the devnet with the following command: + +```shell +$ BUILDER_IMAGE= DEVNET_BUILDER=true make devnet-up +``` + +## Configuration + +These are the configuration options to enable PBS for the devnet. + +### Sequencer + +There are three flags that configure the sequencer to request payloads from the builder API endpoint: + +| Flag | Description | Default Value | +|----------------------------|-------------------------------------------------------------------------|---------------| +| `l2.builder.enabled` | Enable the builder API request to get payloads built from the builder. | `false` | +| `l2.builder.endpoint` | The URL of the builder API endpoint. | `""` | +| `l2.builder.timeout` | The timeout for the builder API request. | `500ms` | + +### Builder op-node + +The op-geth builder requires the op-node to publish the latest attributes as server-sent events in order to start building the payloads. + +| Flag | Description | Default Value | +|--------------------------------|----------------------------------------------------------------------------------|---------------| +| `sequencer.publish-attributes` | Set to true to enable the sequencer to publish attributes to the event stream. | `false` | +| `eventstream.addr` | The address of the eventstream server. | `127.0.0.1` | +| `eventstream.port` | The port of the eventstream server. | `9546` | + +### Builder + +These are the builder flags to enable the builder service in op-geth: + +| Flag | Description | Default Value | +|----------------------------------|----------------------------------------------------------------------------------------------|---------------| +| `builder` | Enable the builder service. | `false` | +| `builder.beacon_endpoints` | The op-node address to get the payload attributes from. Should be set to `builder-op-node`. | `""` | +| `builder.block_retry_interval` | The interval to retry building the payload. | `500ms` | +| `builder.block_time` | Block time of the network. | `2s` | + +## Metrics + +### Sequencer + +The sequencer exposes the builder API related prometheus metrics to the same `/metrics` endpoint of op-node. -There are two main components of the builder. -* `miner` module responsible for block building logic. Receives payload attributes as block building parameters and build blocks from its transaction pool. +| Metric Name | Description | +|--------------------------------------------------------------|-------------------------------------------------------------------| +| `op_node_default_sequencer_builder_request_duration_seconds` | The duration of the builder API request. | +| `op_node_default_sequencer_builder_request_errors` | The total number of errors in the builder API request. | +| `op_node_default_sequencer_builder_request_timeout` | The total number of builder API requests. | +| `op_node_default_sequencer_builder_payloads_bytes` | The total size of the payloads built by the builder. | +| `op_node_default_sequencer_profit` | The total profit made by the sequencer. | +| `op_node_default_sequencer_payload_inserted` | The total number of payloads inserted into the local engine. | +| `op_node_default_sequencer_payload_gas_total` | The total gas used by the payloads. | -* `builder` module for handling communication with the proposer and its op-node. Triggers a continuous block building job in the `miner` module on receiving the payload attributes from its op-node. -## Usage +# Security -To enable the builder: +If you find a security vulnerability in this project or any other initiative +related to proposer/builder separation in Ethereum, please let us know sending +an email to security@flashbots.net. - * `--builder` Enable the Builder module - * `--builder.beacon_endpoints` list of op-node SSE event stream endpoints to subscribe from +# License -Run `geth --help` for the full list of builder configurations. \ No newline at end of file +The code in this project is free software under the [LGPL License](COPYING.LESSER).