Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase upstream with Reth & refactor #29

Conversation

CeciliaZ030
Copy link
Contributor

@CeciliaZ030 CeciliaZ030 commented Feb 14, 2024

  • Cherry pick a bunch of commits for Risc0 upstream, the main breaking change is the hook register mechanism from revm
  • Refactor our fork of revm Sync taiko v3.5 CeciliaZ030/revm#1
  • Refactor our block-building mechanism with a compatible design

Host

Screen Shot 2024-02-15 at 11 36 16 PM

Guest: SGX

  • TODO: the build_from(.) is run twice with exactly the same args. we can definitely remove one instance.*
Screen Shot 2024-02-15 at 11 37 10 PM

Guest: Powdr [RFC]

Screen Shot 2024-02-15 at 11 37 43 PM

intoverflow and others added 27 commits February 5, 2024 01:38
* Introduce op-info tool

* Whitespace

* Update host/Cargo.toml

Co-authored-by: Wolfgang Welz <[email protected]>

* Update Cargo.lock

---------

Co-authored-by: Wolfgang Welz <[email protected]>
…e for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
#
# Author: Wolfgang Welz <[email protected]>
#
# interactive rebase in progress; onto 99f2229
# Last commands done (5 commands done):
#    pick 69402de Introduce op-info tool (taikoxyz#48)
#    pick eed232b upgrade risc0 to v0.19 (taikoxyz#50)
# Next commands to do (14 remaining commands):
#    pick 2dc0bc6 chore(deps): bump risc0 to v0.19.1 (taikoxyz#54)
#    pick 2290e8b cleanup transaction code (taikoxyz#53)
# You are currently rebasing branch 'risc0-main' on '99f2229'.
#
# Changes to be committed:
#	modified:   Cargo.lock
#	modified:   Cargo.toml
#	new file:   guests/eth-block/Cargo.lock
#	new file:   guests/eth-block/Cargo.toml
#	new file:   guests/op-block/Cargo.lock
#	new file:   guests/op-block/Cargo.toml
#	new file:   host/src/main.rs
#	new file:   host/tests/blocks.rs
#	modified:   testing/ef-tests/testguest/Cargo.lock
#	modified:   testing/ef-tests/testguest/Cargo.toml
#	modified:   testing/ef-tests/tests/executor.rs
#
* upgrade dependencies

* upgrade dependencies
* MPT cleanups

* avoid clone when calling hash

* fix clippy
* cleanup host crate

* minor cleanups

* fix trie creation

* separate node maps

* fix tests

* fix proof generation

* fix integration test

* add documentation

* inline public mpt methods

* move mpt functions

* fix verification of zero RPC accounts
…and non-Unicode sequences in filepath (taikoxyz#65)

* fix: use PathBuf and Path for compatibility

* fix: use PathBuf as cache path type
* Update and fix CI

* fix cargo-install version

* try without cache

* Revert "try without cache"

This reverts commit d91547ab1697989050cfdfce35229e8899b0b27f.

* add GITHUB_TOKEN

* review suggestions

* make version env
* transactions module

* TxEssence trait

* Generic Transaction struct

* tx essence as a generic parameter

* OptimismTxEssence

* op chain spec

* OpTxExecStrategy

* redundant block builder type params

* strategy bundles

* host binary parameters

* profiling flag

* optimism

* new derivation binary skeleton, copy over libs

* providers and conversion utils

* epoch transitioning

* basic derive flow

* host-side derivation

* read metadata from op head

* op-derive guest

* disable guest memory leaks

* heapless batch derivation

* Add missing import

* Remove heapless BinaryHeap

* Remove heapless

* Introduce op-derive tool

* Remove ethers types from BatcherDb trait

* Verify new op block has correct transaction list

* Move derive logic into library

* Fix bug in transaction trie reconstruction

* Introduce get_op_header() to BatcherDb

* Clippy warning

* Default Serde value for FileProvider::receipts

* Disable bloom filter checks

* Clippy

* Clippy

* Fix parsing of from and to fields for deposits

* Reintroduce filtering by log bloom

* fmt

* Add support for local exec to op-derive

* Enforce block_number is correct in MemDb

* Split derive() into multiple functions

* Remove redundant check for batch parent hash

* Remove redundant copy of system config

* Remove redundant block number check

* Cleanup

* Remove redundant vector of eth blocks

* Add base_fee_per_gas to Epoch

* Store deposits in Epoch

* Add Eth tail to DeriveOutput

* Cleanup

* Move deque_next_epoch_if_none to State

* Move eth block processing to Batches

* Reorg and cleanup

* More cleanup

* run cargo fmt --all

* fix clippy warnings

* remove unused imports

* format guest code

* add op-derive cmd test

* add cmd tests

* cleanup optimism/mod

* Rename command line args

* Fix test arguments

* Rename config field to max_channel_bank_size

* Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL

* Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity

* Import from std instead of alloc/core

* Re-enable core::mem::forget() optimization

* Replace asserts with ensures; enforce absence of receipts for Op blocks

* Simplfy iteration through derived transactions

* Add Bonsai support to op-derive. Also add Bonsai session status to output when polling

* More println

* Rework Batcher initialization

* More logging

* More log output if Bonsai workflow fails

---------

Co-authored-by: Rami Khalil <[email protected]>
Co-authored-by: Wolfgang Welz <[email protected]>
* cleanup frame parsing

* fix frame loading

* move byte wrapping into RLP encoding

* improve documentation

* fmt

* use an actual Batch for the tests

* log batcher transaction

* add unit tests
* fix channel bank

* channel_index is not mut
* do not default source_hash and mint

* use revm optimism execution

* update testdata

* add optimism tests

* upgrade revm to latest commit

* upgrade revm with latest op changes

* Upgrade `revm` in Optimism execution (taikoxyz#76)

* Update dependencies versions

* fix revm commit

* use correct tags

* use EVM in comments

* update copyright

* register optimism revm handle

* update revm

* update k256

---------

Co-authored-by: john xu <[email protected]>

---------

Co-authored-by: john xu <[email protected]>
* sort batches in a multimap

* update copyright
* verify op block loaded from DB

* improve errors

* Simplify BatcherDb (taikoxyz#81)

* fix warnings

---------

Co-authored-by: Tim Carstens <[email protected]>
commit 0aa4be1
Author: ceciliaz030 <[email protected]>
Date:   Mon Feb 12 18:42:01 2024 +0800

    feature taiko compile

commit f3b6a0f
Author: ceciliaz030 <[email protected]>
Date:   Mon Feb 12 01:31:59 2024 +0800

    TaikoProvider, TaikoSystemInfo, preflight

commit 1e9ffe2
Author: ceciliaz030 <[email protected]>
Date:   Mon Feb 12 01:31:15 2024 +0800

    host refactor

commit 851b817
Author: ceciliaz030 <[email protected]>
Date:   Sat Feb 10 02:51:09 2024 +0800

    fix

commit 39d8654
Author: ceciliaz030 <[email protected]>
Date:   Sat Feb 10 02:35:08 2024 +0800

    fix lib/hosts

commit f87ffd7
Author: Cecilia Zhang <[email protected]>
Date:   Sat Feb 10 01:39:55 2024 +0800

    refactor host Taiko API

commit ef4172f
Author: ceciliaz030 <[email protected]>
Date:   Wed Feb 7 21:54:13 2024 +0800

    wip

commit 6867af4
Author: Wolfgang Welz <[email protected]>
Date:   Tue Jan 23 11:43:23 2024 +0100

    Fix OP block header validation (taikoxyz#74)

    * verify op block loaded from DB

    * improve errors

    * Simplify BatcherDb (taikoxyz#81)

    * fix warnings

    ---------

    Co-authored-by: Tim Carstens <[email protected]>

commit 3a99983
Author: Wolfgang Welz <[email protected]>
Date:   Mon Jan 22 22:10:46 2024 +0100

    Fix batch queue handling in Optimism derivation  (taikoxyz#73)

    * sort batches in a multimap

    * update copyright

commit 7280614
Author: Wolfgang Welz <[email protected]>
Date:   Mon Jan 22 21:21:48 2024 +0100

    feat: Use revm Optimism execution (taikoxyz#44)

    * do not default source_hash and mint

    * use revm optimism execution

    * update testdata

    * add optimism tests

    * upgrade revm to latest commit

    * upgrade revm with latest op changes

    * Upgrade `revm` in Optimism execution (taikoxyz#76)

    * Update dependencies versions

    * fix revm commit

    * use correct tags

    * use EVM in comments

    * update copyright

    * register optimism revm handle

    * update revm

    * update k256

    ---------

    Co-authored-by: john xu <[email protected]>

    ---------

    Co-authored-by: john xu <[email protected]>

commit fbe4255
Author: Wolfgang Welz <[email protected]>
Date:   Fri Dec 22 15:52:52 2023 +0100

    Fix channel bank handling in Optimism derivation (taikoxyz#71)

    * fix channel bank

    * channel_index is not mut

commit 0b2efed
Author: Wolfgang Welz <[email protected]>
Date:   Tue Dec 19 13:06:40 2023 +0100

    Fix frame parsing and loading in Optimism derivation (taikoxyz#68)

    * cleanup frame parsing

    * fix frame loading

    * move byte wrapping into RLP encoding

    * improve documentation

    * fmt

    * use an actual Batch for the tests

    * log batcher transaction

    * add unit tests

commit 4c0496c
Author: Tim Carstens <[email protected]>
Date:   Mon Dec 18 04:25:01 2023 -0800

    Optimism L1 -> L2 derivation (taikoxyz#51)

    * transactions module

    * TxEssence trait

    * Generic Transaction struct

    * tx essence as a generic parameter

    * OptimismTxEssence

    * op chain spec

    * OpTxExecStrategy

    * redundant block builder type params

    * strategy bundles

    * host binary parameters

    * profiling flag

    * optimism

    * new derivation binary skeleton, copy over libs

    * providers and conversion utils

    * epoch transitioning

    * basic derive flow

    * host-side derivation

    * read metadata from op head

    * op-derive guest

    * disable guest memory leaks

    * heapless batch derivation

    * Add missing import

    * Remove heapless BinaryHeap

    * Remove heapless

    * Introduce op-derive tool

    * Remove ethers types from BatcherDb trait

    * Verify new op block has correct transaction list

    * Move derive logic into library

    * Fix bug in transaction trie reconstruction

    * Introduce get_op_header() to BatcherDb

    * Clippy warning

    * Default Serde value for FileProvider::receipts

    * Disable bloom filter checks

    * Clippy

    * Clippy

    * Fix parsing of from and to fields for deposits

    * Reintroduce filtering by log bloom

    * fmt

    * Add support for local exec to op-derive

    * Enforce block_number is correct in MemDb

    * Split derive() into multiple functions

    * Remove redundant check for batch parent hash

    * Remove redundant copy of system config

    * Remove redundant block number check

    * Cleanup

    * Remove redundant vector of eth blocks

    * Add base_fee_per_gas to Epoch

    * Store deposits in Epoch

    * Add Eth tail to DeriveOutput

    * Cleanup

    * Move deque_next_epoch_if_none to State

    * Move eth block processing to Batches

    * Reorg and cleanup

    * More cleanup

    * run cargo fmt --all

    * fix clippy warnings

    * remove unused imports

    * format guest code

    * add op-derive cmd test

    * add cmd tests

    * cleanup optimism/mod

    * Rename command line args

    * Fix test arguments

    * Rename config field to max_channel_bank_size

    * Enforce decompression limit of MAX_RLP_BYTES_PER_CHANNEL

    * Use constant OPTIMISM_DEPOSITED_TX_TYPE when checking batch validity

    * Import from std instead of alloc/core

    * Re-enable core::mem::forget() optimization

    * Replace asserts with ensures; enforce absence of receipts for Op blocks

    * Simplfy iteration through derived transactions

    * Add Bonsai support to op-derive. Also add Bonsai session status to output when polling

    * More println

    * Rework Batcher initialization

    * More logging

    * More log output if Bonsai workflow fails

    ---------

    Co-authored-by: Rami Khalil <[email protected]>
    Co-authored-by: Wolfgang Welz <[email protected]>

commit 916f8c5
Author: Wolfgang Welz <[email protected]>
Date:   Tue Dec 12 19:39:39 2023 +0100

    Update CI (taikoxyz#60)

    * Update and fix CI

    * fix cargo-install version

    * try without cache

    * Revert "try without cache"

    This reverts commit d91547ab1697989050cfdfce35229e8899b0b27f.

    * add GITHUB_TOKEN

    * review suggestions

    * make version env

commit c24c7b8
Author: John Smith <[email protected]>
Date:   Tue Dec 12 22:02:16 2023 +0800

    fix: use PathBuf instead of String to compatible different platforms and non-Unicode sequences in filepath  (taikoxyz#65)

    * fix: use PathBuf and Path for compatibility

    * fix: use PathBuf as cache path type

commit ee88f6d
Author: John Smith <[email protected]>
Date:   Tue Dec 12 00:11:27 2023 +0800

    fix: clippy complain (taikoxyz#64)

commit 3e70593
Author: Wolfgang Welz <[email protected]>
Date:   Mon Nov 27 09:43:01 2023 +0100

    Pre-flight and block building improvements (taikoxyz#55)

    * cleanup host crate

    * minor cleanups

    * fix trie creation

    * separate node maps

    * fix tests

    * fix proof generation

    * fix integration test

    * add documentation

    * inline public mpt methods

    * move mpt functions

    * fix verification of zero RPC accounts

commit 59c23fd
Author: Wolfgang Welz <[email protected]>
Date:   Tue Nov 21 17:23:03 2023 +0100

    chore: MPT cleanups (taikoxyz#56)

    * MPT cleanups

    * avoid clone when calling hash

    * fix clippy

commit 71135b7
Author: Tim Carstens <[email protected]>
Date:   Tue Nov 21 07:09:20 2023 -0800

    Reintroduce data() on TxEssence (taikoxyz#57)

commit 2290e8b
Author: Wolfgang Welz <[email protected]>
Date:   Sat Nov 18 10:24:53 2023 +0100

    cleanup transaction code (taikoxyz#53)

commit 2dc0bc6
Author: Wolfgang Welz <[email protected]>
Date:   Sat Nov 18 10:23:30 2023 +0100

    chore(deps): bump risc0 to v0.19.1 (taikoxyz#54)

    * upgrade dependencies

    * upgrade dependencies

commit eed232b
Author: Wolfgang Welz <[email protected]>
Date:   Wed Nov 1 19:05:41 2023 +0100

    upgrade risc0 to v0.19 (taikoxyz#50)

commit 69402de
Author: Tim Carstens <[email protected]>
Date:   Thu Oct 26 13:07:33 2023 -0700

    Introduce op-info tool (taikoxyz#48)

    * Introduce op-info tool

    * Whitespace

    * Update host/Cargo.toml

    Co-authored-by: Wolfgang Welz <[email protected]>

    * Update Cargo.lock

    ---------

    Co-authored-by: Wolfgang Welz <[email protected]>

commit 2f12782
Author: Wolfgang Welz <[email protected]>
Date:   Thu Oct 19 23:10:56 2023 +0200

    fix clippy (taikoxyz#45)

commit 4db793c
Author: John Smith <[email protected]>
Date:   Thu Oct 19 21:14:19 2023 +0800

    fix: extra data can equal to MAX_EXTRA_DATA_BYTES (taikoxyz#43)

commit ddb349b
Author: Tim Carstens <[email protected]>
Date:   Thu Oct 19 04:35:16 2023 -0700

    Enable RPC retries (taikoxyz#42)
@CeciliaZ030 CeciliaZ030 changed the title Rebase upstream with Reth Rebase upstream with Reth & refactor Feb 14, 2024
Fix unused variables, results and imports lints
Refactor some format arg captures
Refactor nested if statements into guard clauses
Refactor if-else chains into match pattern
@CeciliaZ030
Copy link
Contributor Author

@petarvujovic98 Thanks for the lints :) The purpose of this PR is to minimize the diff of Risc0 upstream, so ur fix kinda increase the amount of diffs. Maybe you can make a PR against the upstream, and then I'll cherry-pick it back.

@petarvujovic98
Copy link
Contributor

petarvujovic98 commented Feb 19, 2024

@CeciliaZ030 Sorry, I didn't catch that... I'll check the upstream and make a PR there.

Update: Here is the upstream PR

@CeciliaZ030 CeciliaZ030 closed this Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants