Create MPT scroll-reth

[frisitano]

Describe the feature

We currently have a single node implementation for scroll that uses the BMPT state commitment. As we undergo our transition to using the native Ethereum MPT it would be of value to provide an MPT implementation so that we can leverage it for both comparison with l2geth (MPT) and for witness generation in for the stateless block verifier.

Implementation

Rename ScrollNode as ScrollNodeBMPT. Introduce a new node ScrollNodeMPT with the same configuration. On this node when implementing NodeTypes we should specify the MerklePatriciaTrie for the state commitment. Introduce an additional scroll binary for the MPT node. Ensure that this node can not be compiled with the scroll feature flag.

References below:

reth/crates/scroll/node/src/lib.rs

Lines 38 to 79 in 133d9c0

	/// The Scroll node implementation.
	#[derive(Clone, Debug)]
	pub struct ScrollNode;
	impl<N> Node<N> for ScrollNode
	where
	N: FullNodeTypes,
	N::Types: NodeTypesWithDB
	+ NodeTypesWithEngine<
	ChainSpec = ScrollChainSpec,
	Primitives = EthPrimitives,
	Engine = EthEngineTypes,
	Storage = ScrollStorage,
	>,
	{
	type ComponentsBuilder = ComponentsBuilder<
	N,
	ScrollPoolBuilder,
	ScrollPayloadBuilder,
	ScrollNetworkBuilder,
	ScrollExecutorBuilder,
	ScrollConsensusBuilder,
	>;
	type AddOns = ScrollAddOns<
	NodeAdapter<N, <Self::ComponentsBuilder as NodeComponentsBuilder<N>>::Components>,
	>;
	fn components_builder(&self) -> Self::ComponentsBuilder {
	ComponentsBuilder::default()
	.node_types::<N>()
	.pool(ScrollPoolBuilder)
	.payload(ScrollPayloadBuilder)
	.network(ScrollNetworkBuilder)
	.executor(ScrollExecutorBuilder)
	.consensus(ScrollConsensusBuilder)
	}
	fn add_ons(&self) -> Self::AddOns {
	ScrollAddOns::default()
	}
	}

reth/crates/scroll/node/src/types.rs

Lines 8 to 18 in 133d9c0

	impl NodeTypesWithEngine for ScrollNode {
	type Engine = EthEngineTypes;
	}
	impl NodeTypes for ScrollNode {
	// TODO(scroll): update to scroll primitives when we introduce the revm SDK pattern.
	type Primitives = EthPrimitives;
	type ChainSpec = ScrollChainSpec;
	type StateCommitment = BinaryMerklePatriciaTrie;
	type Storage = ScrollStorage;
	}

reth/crates/scroll/bin/src/main.rs

Lines 1 to 46 in 133d9c0

	//! Scroll binary
	#![cfg(all(feature = "scroll", not(feature = "optimism")))]
	use clap::Parser;
	use reth_node_builder::{engine_tree_config::TreeConfig, EngineNodeLauncher, Node};
	use reth_provider::providers::BlockchainProvider2;
	use reth_scroll_cli::{Cli, ScrollChainSpecParser, ScrollRollupArgs};
	use reth_scroll_node::{ScrollAddOns, ScrollNode};
	#[global_allocator]
	static ALLOC: reth_cli_util::allocator::Allocator = reth_cli_util::allocator::new_allocator();
	fn main() {
	reth_cli_util::sigsegv_handler::install();
	// Enable backtraces unless a RUST_BACKTRACE value has already been explicitly provided.
	if std::env::var_os("RUST_BACKTRACE").is_none() {
	std::env::set_var("RUST_BACKTRACE", "1");
	}
	if let Err(err) = Cli::<ScrollChainSpecParser, ScrollRollupArgs>::parse().run(
	|builder, rollup_args| async move {
	let engine_tree_config = TreeConfig::default()
	.with_persistence_threshold(rollup_args.persistence_threshold)
	.with_memory_block_buffer_target(rollup_args.memory_block_buffer_target);
	let handle = builder
	.with_types_and_provider::<ScrollNode, BlockchainProvider2<_>>()
	.with_components(ScrollNode.components_builder())
	.with_add_ons(ScrollAddOns::default())
	.launch_with_fn(|builder| {
	let launcher = EngineNodeLauncher::new(
	builder.task_executor().clone(),
	builder.config().datadir(),
	engine_tree_config,
	);
	builder.launch_with(launcher)
	})
	.await?;
	handle.node_exit_future.await
	},
	) {
	eprintln!("Error: {err:?}");
	std::process::exit(1);
	}
	}

Distribution

We have two possibilities for the distribution of scroll reth.

Node client binary + state dump / import

We can distribute the node client and also provide a state dump. A user can then spin up the node and import the state dump using the init-state command. This should provide them with a node with a database that allows them to invoke desired RPC commands on it. Alternatively, the user can use the import command which allows them to import a set of rlp encoded blocks from a file. This should be confirmed and tested that this pattern works. This is a more technical option but allows consumers of reth to become more familiar with its operation and is more flexible as different users can import different blocks/states as they require.

Hosted Node Client

We can deploy servers (EC2 instances or kube pods) to host the reth clients and expose the RPC for consumers. This is a simple approach for consumers and the clients are "managed" however the consumers will have less flexibility about what blocks are included in the chain.

Conclusion

It may be best to provide both solutions. We host nodes for both the mpt and bmpt implementations of reth synced to mainnet and we also distribute the node client and allow consumers to import state / create chains that satisfy their use case. We should discuss with SRE how we want to deploy these nodes, as dedicated EC2 instances or as containers on kubernetes.

Additional context

No response

[frisitano]

Outstanding work: We need to make the CLI run method generic over the node type:

reth/crates/scroll/cli/src/lib.rs

Lines 103 to 154 in fdf74bb

	pub fn run<L, Fut>(mut self, launcher: L) -> eyre::Result<()>
	where
	L: FnOnce(WithLaunchContext<NodeBuilder<Arc<DatabaseEnv>, C::ChainSpec>>, Ext) -> Fut,
	Fut: Future<Output = eyre::Result<()>>,
	{
	// add network name to logs dir
	self.logs.log_file_directory =
	self.logs.log_file_directory.join(self.chain.chain().to_string());
	let _guard = self.init_tracing()?;
	info!(target: "reth::cli", "Initialized tracing, debug log directory: {}", self.logs.log_file_directory);
	// Install the prometheus recorder to be sure to record all metrics
	let _ = install_prometheus_recorder();
	let runner = CliRunner::default();
	match self.command {
	//TODO(scroll): make all these commands generic over the node
	Commands::Node(command) => {
	runner.run_command_until_exit(|ctx| command.execute(ctx, launcher))
	}
	Commands::Init(command) => {
	runner.run_blocking_until_ctrl_c(command.execute::<ScrollNodeBmpt>())
	}
	Commands::InitState(command) => {
	runner.run_blocking_until_ctrl_c(command.execute::<ScrollNodeBmpt>())
	}
	Commands::Import(command) => runner.run_blocking_until_ctrl_c(
	command.execute::<ScrollNodeBmpt, _, _>(ScrollExecutorProvider::scroll),
	),
	Commands::DumpGenesis(command) => runner.run_blocking_until_ctrl_c(command.execute()),
	Commands::Db(command) => {
	runner.run_blocking_until_ctrl_c(command.execute::<ScrollNodeBmpt>())
	}
	Commands::Stage(command) => runner.run_command_until_exit(|ctx| {
	command.execute::<ScrollNodeBmpt, _, _, EthNetworkPrimitives>(
	ctx,
	ScrollExecutorProvider::scroll,
	)
	}),
	Commands::P2P(command) => {
	runner.run_until_ctrl_c(command.execute::<EthNetworkPrimitives>())
	}
	Commands::Config(command) => runner.run_until_ctrl_c(command.execute()),
	Commands::Recover(command) => {
	runner.run_command_until_exit(|ctx| command.execute::<ScrollNodeBmpt>(ctx))
	}
	Commands::Prune(command) => {
	runner.run_until_ctrl_c(command.execute::<ScrollNodeBmpt>())
	}
	}
	}

[frisitano]

closed by #107