Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ICRC-Rosetta): updated rosetta to support icrc3 standard #2607

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

NikolasHai
Copy link
Contributor

@NikolasHai NikolasHai commented Nov 14, 2024

This MR proposes the following changes:

  1. Update ICRC-1 ledger to use latest_block_hash instead of tip_hash in the ICRC3 certificate
  2. Update ICRC-1 ledger to use leb128 encoding for the latest_block_index in the ICRC3 certificate
  3. Use feature flags for the ICRC3 update
  4. Use the public crate for ic-certificates to construct the hash tree
  5. Update icrc agent to support the changes made to the icrc1 ledger

@github-actions github-actions bot added the feat label Nov 14, 2024
@NikolasHai NikolasHai marked this pull request as ready for review November 14, 2024 08:32
@NikolasHai NikolasHai requested a review from a team as a code owner November 14, 2024 08:32
@NikolasHai NikolasHai marked this pull request as draft November 14, 2024 11:05
@NikolasHai NikolasHai marked this pull request as ready for review November 14, 2024 13:46
Ok(last_block_index_bytes) => last_block_index_bytes,
Err(_) => {
return Err(Icrc1AgentError::VerificationFailed(format!(
let last_block_index_leb128_encoded = lookup_leaf(&hash_tree, "last_block_index")?;
Copy link
Contributor

@bogwar bogwar Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the new code still handel big endian byte encodings?
Can't the two encodings coincide? In which case the decision on what the encoding is should be based on the presence of tip_hash branch in the certificate.

rs/ledger_suite/icrc1/ledger/src/lib.rs Outdated Show resolved Hide resolved
}

pub fn construct_hash_tree(&self) -> MixedHashTree {
pub fn construct_hash_tree(&self) -> HashTree {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What triggered this change? I couldn't find any test that verify that they produce the same value for this function, do we have one?

In general, the only test I found that checks the certificate is the //rs/tests/financial_integrations:icrc1_agent_test system test. Would it be possible to add a deterministic StateMachine test that verifies the certificate both with and without the feature flag, and ideally also with the MixedHashTree just to make sure that nothing changes with the move to HashTree?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the system tests for rosetta test the certificate as it is part of the block synchronization.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used HashTree since that is the official crate for ic-certification released on crates.io.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the system tests for rosetta test the certificate as it is part of the block synchronization.

I see, thanks for the pointer! I do find that a rather heavyweight approach, and a unit/integration test closer to the function itself would be useful. IIUC, the Rosetta system tests now also only test Rosetta with a ledger where the feature flag is enabled, but not with it disabled, and also only test with a client that understands both certificates, but not with one that only supports the legacy certificate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The icrc1 agent test tests the version of the icrc1 ledger where the feature flag is not set.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The client understands both certificates. It is tested with both the icrc1 ledger with the feature flag (Rosetta tests) and without the feature flag (icrc1 agent test).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The icrc1 agent test tests the version of the icrc1 ledger where the feature flag is not set.

Do you mean test_simple_start_of_synchronizing_blocks in rs/rosetta-api/icrc1/tests/integration_test_components/blocks_synchronizer/fetching_blocks_interval_test.rs? I couldn't find any tests in packages/icrc-ledger-agent. The aforementioned test also only calls the legacy get_data_certificate endpoint, and I couldn't find any code that actually calls icrc3_get_tip_certificate, which I feel should also be tested.

I find the current situation with only having Rosetta tests for this ledger feature to be a bit cumbersome and unintuitive, and I don't see a good reason for not having some simple tests under rs/ledger_suite/icrc1/ledger that call icrc3_get_tip_certificate with and without the feature flag.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant: //rs/tests/financial_integrations:icrc1_agent_test.
I can add explicit tests for both endpoints too.

rs/ledger_suite/icrc1/ledger/src/lib.rs Outdated Show resolved Hide resolved
rs/ledger_suite/icrc1/ledger/src/lib.rs Outdated Show resolved Hide resolved
rs/ledger_suite/icrc1/ledger/src/lib.rs Outdated Show resolved Hide resolved
}

pub fn construct_hash_tree(&self) -> MixedHashTree {
pub fn construct_hash_tree(&self) -> HashTree {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the system tests for rosetta test the certificate as it is part of the block synchronization.

I see, thanks for the pointer! I do find that a rather heavyweight approach, and a unit/integration test closer to the function itself would be useful. IIUC, the Rosetta system tests now also only test Rosetta with a ledger where the feature flag is enabled, but not with it disabled, and also only test with a client that understands both certificates, but not with one that only supports the legacy certificate.

Comment on lines +352 to +358
match block_hash.clone().try_into() {
Ok(last_block_hash) => Ok(last_block_hash),
Err(_) => {
return Err(Icrc1AgentError::VerificationFailed(format!(
"DataCertificate last_block_hash bytes: {}, cannot be decoded as last_block_hash",
hex::encode(last_block_hash_vec)
)))
Err(Icrc1AgentError::VerificationFailed(format!(
"DataCertificate last_block_hash bytes: {}, cannot be decoded as last_block_hash",
hex::encode(block_hash)
)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this could be simplified as
block_hash.clone().try_into().or(Err(Icrc1AgentError::...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants