Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpc-alt: CoinMetadata data loader #21382

Merged
merged 2 commits into from
Mar 7, 2025
Merged

rpc-alt: CoinMetadata data loader #21382

merged 2 commits into from
Mar 7, 2025

Conversation

amnn
Copy link
Contributor

@amnn amnn commented Feb 28, 2025

Description

Use a DataLoader for fetching CoinMetadata, instead of issuing queries separately. This is fairly important as CoinMetadata queries will be quite common from Wallet clients.

The loader is built on top of the following query:

EXPLAIN ANALYZE SELECT DISTINCT ON (
    candidates.package,
    candidates.module,
    candidates.name,
    candidates.instantiation
)
    candidates.*
FROM
    obj_info candidates
LEFT JOIN
    obj_info newer
ON (
    candidates.object_id = newer.object_id
AND candidates.cp_sequence_number < newer.cp_sequence_number
)
WHERE
    newer.object_id IS NULL
AND candidates.package = '\x0000000000000000000000000000000000000000000000000000000000000002'::bytea
AND candidates.module = 'coin'
AND candidates.name = 'CoinMetadata'
AND candidates.instantiation IN (
    '\x01070000000000000000000000000000000000000000000000000000000000000002037375690353554900'::bytea,
    '\x010700008006563579284526db4e89d73f7bfe3180f0c00bb0e494845e9edbef29bb0a636174616c6f7269616e0a434154414c4f5249414e00'::bytea,
    '\x01070006f9c0b5fd4aed846466683df8c9ce67551715526951130219f2ddcb0806840d66697877616c6c657473706c730d46495857414c4c455453504c5300'::bytea,
    '\x0107000b0aa96d6a3ca2388cff30124761c2dae0759583c72efae600ed5c2df005de037775660357554600'::bytea,
    '\x0107000768051314fa31471f61e7dd0db5282724a1020c41b16f98f242f1875f3dec046d616761044d41474100'::bytea
)
ORDER BY
    candidates.package,
    candidates.module,
    candidates.name,
    candidates.instantiation,
    candidates.cp_sequence_number DESC,
    candidates.object_id DESC;

Test plan

Test that the aggregate query is efficient (check the query plan):

 Unique  (cost=3300.10..3302.50 rows=479 width=159) (actual time=23.142..23.149 rows=5 loops=1)
   ->  Sort  (cost=3300.10..3301.30 rows=479 width=159) (actual time=23.141..23.143 rows=5 loops=1)
         Sort Key: candidates.instantiation, candidates.cp_sequence_number DESC, candidates.object_id DESC
         Sort Method: quicksort  Memory: 25kB
         ->  Nested Loop Anti Join  (cost=1.52..3278.78 rows=479 width=159) (actual time=0.212..23.125 rows=5 loops=1)
               ->  Index Scan using obj_info_inst_object_id_desc on obj_info candidates  (cost=0.82..2585.72 rows=719 width=159) (actual time=0.198..23.046 rows=5 loops=1)
                     Index Cond: ((package = '\x0000000000000000000000000000000000000000000000000000000000000002'::bytea) AND (module = 'coin'::text) AND (name = 'CoinMetadata'::text) AND (instantiation = ANY ('{"\\x01070000000000000000000000000000000000000000000000000000000000000002037375690353554900","\\x010700008006563579284526db4e89d73f7bfe3180f0c00bb0e494845e9edbef29bb0a636174616c6f7269616e0a434154414c4f5249414e00","\\x01070006f9c0b5fd4aed846466683df8c9ce67551715526951130219f2ddcb0806840d66697877616c6c657473706c730d46495857414c4c455453504c5300","\\x0107000b0aa96d6a3ca2388cff30124761c2dae0759583c72efae600ed5c2df005de037775660357554600","\\x0107000768051314fa31471f61e7dd0db5282724a1020c41b16f98f242f1875f3dec046d616761044d41474100"}'::bytea[])))
               ->  Index Only Scan using obj_info_pkey on obj_info newer  (cost=0.70..5.01 rows=11 width=41) (actual time=0.011..0.011 rows=0 loops=5)
                     Index Cond: ((object_id = candidates.object_id) AND (cp_sequence_number > candidates.cp_sequence_number))
                     Heap Fetches: 0
 Planning Time: 6.832 ms
 Execution Time: 23.209 ms
(12 rows)

Existing tests to make sure behaviour is preserved


Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

  • Protocol:
  • Nodes (Validators and Full nodes):
  • gRPC:
  • JSON-RPC:
  • GraphQL:
  • CLI:
  • Rust SDK:

@amnn amnn requested review from emmazzz, gegaowp and wlmyng February 28, 2025 13:21
@amnn amnn self-assigned this Feb 28, 2025
Copy link

vercel bot commented Feb 28, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
sui-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 7, 2025 1:36am
2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
multisig-toolkit ⬜️ Ignored (Inspect) Visit Preview Mar 7, 2025 1:36am
sui-kiosk ⬜️ Ignored (Inspect) Visit Preview Mar 7, 2025 1:36am

Copy link
Contributor

@wlmyng wlmyng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ty

amnn added 2 commits March 6, 2025 18:33
## Description

Use a DataLoader for fetching CoinMetadata, instead of issuing queries
separately. The loader is built on top of the following query:

```sql
EXPLAIN ANALYZE SELECT DISTINCT ON (
    candidates.package,
    candidates.module,
    candidates.name,
    candidates.instantiation
)
    candidates.*
FROM
    obj_info candidates
LEFT JOIN
    obj_info newer
ON (
    candidates.object_id = newer.object_id
AND candidates.cp_sequence_number < newer.cp_sequence_number
)
WHERE
    newer.object_id IS NULL
AND candidates.package = '\x0000000000000000000000000000000000000000000000000000000000000002'::bytea
AND candidates.module = 'coin'
AND candidates.name = 'CoinMetadata'
AND candidates.instantiation IN (
    '\x01070000000000000000000000000000000000000000000000000000000000000002037375690353554900'::bytea,
    '\x010700008006563579284526db4e89d73f7bfe3180f0c00bb0e494845e9edbef29bb0a636174616c6f7269616e0a434154414c4f5249414e00'::bytea,
    '\x01070006f9c0b5fd4aed846466683df8c9ce67551715526951130219f2ddcb0806840d66697877616c6c657473706c730d46495857414c4c455453504c5300'::bytea,
    '\x0107000b0aa96d6a3ca2388cff30124761c2dae0759583c72efae600ed5c2df005de037775660357554600'::bytea,
    '\x0107000768051314fa31471f61e7dd0db5282724a1020c41b16f98f242f1875f3dec046d616761044d41474100'::bytea
)
ORDER BY
    candidates.package,
    candidates.module,
    candidates.name,
    candidates.instantiation,
    candidates.cp_sequence_number DESC,
    candidates.object_id DESC;
```

## Test plan

Test that the aggregate query is efficient (check the query plan):

```
 Unique  (cost=3300.10..3302.50 rows=479 width=159) (actual time=23.142..23.149 rows=5 loops=1)
   ->  Sort  (cost=3300.10..3301.30 rows=479 width=159) (actual time=23.141..23.143 rows=5 loops=1)
         Sort Key: candidates.instantiation, candidates.cp_sequence_number DESC, candidates.object_id DESC
         Sort Method: quicksort  Memory: 25kB
         ->  Nested Loop Anti Join  (cost=1.52..3278.78 rows=479 width=159) (actual time=0.212..23.125 rows=5 loops=1)
               ->  Index Scan using obj_info_inst_object_id_desc on obj_info candidates  (cost=0.82..2585.72 rows=719 width=159) (actual time=0.198..23.046 rows=5 loops=1)
                     Index Cond: ((package = '\x0000000000000000000000000000000000000000000000000000000000000002'::bytea) AND (module = 'coin'::text) AND (name = 'CoinMetadata'::text) AND (instantiation = ANY ('{"\\x01070000000000000000000000000000000000000000000000000000000000000002037375690353554900","\\x010700008006563579284526db4e89d73f7bfe3180f0c00bb0e494845e9edbef29bb0a636174616c6f7269616e0a434154414c4f5249414e00","\\x01070006f9c0b5fd4aed846466683df8c9ce67551715526951130219f2ddcb0806840d66697877616c6c657473706c730d46495857414c4c455453504c5300","\\x0107000b0aa96d6a3ca2388cff30124761c2dae0759583c72efae600ed5c2df005de037775660357554600","\\x0107000768051314fa31471f61e7dd0db5282724a1020c41b16f98f242f1875f3dec046d616761044d41474100"}'::bytea[])))
               ->  Index Only Scan using obj_info_pkey on obj_info newer  (cost=0.70..5.01 rows=11 width=41) (actual time=0.011..0.011 rows=0 loops=5)
                     Index Cond: ((object_id = candidates.object_id) AND (cp_sequence_number > candidates.cp_sequence_number))
                     Heap Fetches: 0
 Planning Time: 6.832 ms
 Execution Time: 23.209 ms
(12 rows)
```

Existing tests to make sure behaviour is preserved
@amnn amnn force-pushed the amnn/rpc-coin-meta-dl branch from 1381eda to 4bbb0b4 Compare March 7, 2025 01:34
@amnn amnn temporarily deployed to sui-typescript-aws-kms-test-env March 7, 2025 01:34 — with GitHub Actions Inactive
@amnn amnn merged commit 611a28b into main Mar 7, 2025
46 of 47 checks passed
@amnn amnn deleted the amnn/rpc-coin-meta-dl branch March 7, 2025 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants