Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce Solana support #252

Merged
merged 60 commits into from
Mar 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
b5a66e3
save work
eldargab Dec 10, 2023
3e13f1e
save work
eldargab Dec 13, 2023
ee0c972
save work
eldargab Dec 17, 2023
e5fc938
save work
eldargab Dec 17, 2023
1e1c351
save work
eldargab Dec 20, 2023
8c83123
Merge branch 'master' into solana
eldargab Dec 29, 2023
881046d
Merge branch 'master' into solana
eldargab Dec 29, 2023
c276444
computeUnitsConsumed is optional
eldargab Dec 31, 2023
6d32c0d
fixes
eldargab Dec 31, 2023
4bed738
Merge branch 'master' into solana
eldargab Jan 2, 2024
dc6e015
save work
eldargab Jan 3, 2024
b037cec
Merge branch 'master' into solana
eldargab Jan 5, 2024
184571f
save work
eldargab Jan 6, 2024
332487d
Merge branch 'master' into solana
eldargab Jan 7, 2024
c7ebdf8
refactor dump CLI
eldargab Jan 8, 2024
4a77eda
solana data ingestion works
eldargab Jan 8, 2024
d8ef196
Merge branch 'master' into solana
eldargab Jan 10, 2024
18de161
expand raw data validation
eldargab Jan 10, 2024
0747b43
solana-dump image
eldargab Jan 10, 2024
cc6059c
Merge branch 'master' into solana
eldargab Jan 11, 2024
7e30502
expose fetch settings
eldargab Jan 13, 2024
2cd87e7
reserve endpoint capacity
eldargab Jan 13, 2024
79be1c4
fix unsafe integers
eldargab Jan 14, 2024
85f4e1e
fix not ok HTTP responses handling
eldargab Jan 15, 2024
c792621
fix getFinalizedTop() errors
eldargab Jan 16, 2024
52ee7fe
make .stack the first property of JSON serialized Error
eldargab Jan 16, 2024
3766c94
log rpc call during connection failures
eldargab Jan 16, 2024
7b80429
fix prev commit
eldargab Jan 16, 2024
f9525e7
log parser
eldargab Jan 17, 2024
34d4ff9
expand log parsing
eldargab Jan 17, 2024
d2e7623
save work
eldargab Jan 20, 2024
3edfc9b
Merge branch 'master' into solana
eldargab Feb 9, 2024
1b2f130
extend retry warning with httpResponseBody when it is readable
eldargab Feb 9, 2024
12ee244
don't try to append completed ranges
eldargab Feb 9, 2024
7c3e477
save work
eldargab Feb 10, 2024
ff4ceff
reduce getBlock batch calls on RPC error and retry
eldargab Feb 10, 2024
08b0440
handle null getBlock responses
eldargab Feb 11, 2024
7ed5fdb
save work
eldargab Feb 14, 2024
48e5392
Merge branch 'master' into solana
eldargab Feb 17, 2024
aa1bda8
save work
eldargab Feb 21, 2024
9005142
save work
eldargab Feb 21, 2024
c944c09
save work
eldargab Feb 21, 2024
c11e48c
save work
eldargab Feb 21, 2024
734f47c
save work
eldargab Feb 21, 2024
a36e815
add Ed25519SigVerify111111111111111111111111111 to invoke less programs
eldargab Feb 24, 2024
8ff9b58
map all Solana data
eldargab Feb 25, 2024
59665bc
Merge branch 'master' into solana
eldargab Feb 25, 2024
dc4a629
change Reward.commission type to js number
eldargab Feb 25, 2024
3513dea
save work
eldargab Feb 25, 2024
b228a1a
save work
eldargab Feb 25, 2024
4f4b774
save work
eldargab Feb 26, 2024
5fc3ab8
index only changed token balances
eldargab Feb 26, 2024
f788b26
save work
eldargab Feb 28, 2024
fc1b943
Merge branch 'master' into solana
eldargab Feb 29, 2024
caf5f03
save work
eldargab Mar 3, 2024
03d0d8c
save work
eldargab Mar 4, 2024
20eb893
save work
eldargab Mar 5, 2024
428d216
first data where processed
eldargab Mar 5, 2024
5c9c017
don't publish solana packages just yet
eldargab Mar 5, 2024
0dfcdc4
changes
eldargab Mar 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,15 @@
/common/autoinstallers/*/.npmrc
**/.rush/temp/

# Our temprorary files (e.g. ad-hoc scripts)
*.temp.*
*.temp

# IDE and tools
.git
.idea
*.log


# READMEs and docs
*.md

Expand Down
20 changes: 9 additions & 11 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,15 @@ RUN rm common/config/rush/build-cache.json
RUN node common/scripts/install-run-rush.js build


FROM builder AS solana-dump-builder
RUN node common/scripts/install-run-rush.js deploy --project @subsquid/solana-dump


FROM node AS solana-dump
COPY --from=solana-dump-builder /squid/common/deploy /squid
ENTRYPOINT ["node", "/squid/solana/solana-dump/bin/run.js"]


FROM builder AS substrate-dump-builder
RUN node common/scripts/install-run-rush.js deploy --project @subsquid/substrate-dump

Expand All @@ -36,14 +45,3 @@ RUN node common/scripts/install-run-rush.js deploy --project @subsquid/substrate
FROM node AS substrate-metadata-service
COPY --from=substrate-metadata-service-builder /squid/common/deploy /squid
ENTRYPOINT ["node", "/squid/substrate/substrate-metadata-service/bin/run.js"]


FROM builder AS chain-status-service-builder
RUN node common/scripts/install-run-rush.js deploy --project chain-status-service


FROM node AS chain-status-service
COPY --from=chain-status-service-builder /squid/common/deploy /squid
ENTRYPOINT ["node", "/squid/util/chain-status-service/lib/main.js"]
CMD ["/squid/util/chain-status-service/config.json"]
EXPOSE 3000
10 changes: 10 additions & 0 deletions common/changes/@subsquid/http-client/solana_2024-03-05-20-15.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/http-client",
"comment": "expose `FetchResponse` type",
"type": "minor"
}
],
"packageName": "@subsquid/http-client"
}
10 changes: 10 additions & 0 deletions common/changes/@subsquid/openreader/solana_2024-03-05-20-15.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/openreader",
"comment": "",
"type": "none"
}
],
"packageName": "@subsquid/openreader"
}
15 changes: 15 additions & 0 deletions common/changes/@subsquid/rpc-client/solana_2024-03-05-20-15.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"changes": [
{
"packageName": "@subsquid/rpc-client",
"comment": "add `.fixUnsafeIntegers` request option",
"type": "minor"
},
{
"packageName": "@subsquid/rpc-client",
"comment": "refine retry warning message",
"type": "patch"
}
],
"packageName": "@subsquid/rpc-client"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/substrate-dump",
"comment": "use new dumper CLI framework",
"type": "minor"
}
],
"packageName": "@subsquid/substrate-dump"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/substrate-ingest",
"comment": "use new ingest CLI framework",
"type": "minor"
}
],
"packageName": "@subsquid/substrate-ingest"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/substrate-metadata-service",
"comment": "migrate to a new version of `util-internal-http-server`",
"type": "minor"
}
],
"packageName": "@subsquid/substrate-metadata-service"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/substrate-processor",
"comment": "",
"type": "none"
}
],
"packageName": "@subsquid/substrate-processor"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal-archive-layout",
"comment": "add `ArchiveLayout.getRawBlocks()`",
"type": "minor"
},
{
"packageName": "@subsquid/util-internal-archive-layout",
"comment": "add `.topDirSize` option",
"type": "minor"
},
{
"packageName": "@subsquid/util-internal-archive-layout",
"comment": "relax short block hash regex",
"type": "minor"
},
{
"packageName": "@subsquid/util-internal-archive-layout",
"comment": "fix: `ArchiveLayout.append()` should not try to extend passed block range when it is already completed",
"type": "patch"
}
],
"packageName": "@subsquid/util-internal-archive-layout"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal-dump-cli",
"comment": "",
"type": "none"
}
],
"packageName": "@subsquid/util-internal-dump-cli"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal-http-server",
"comment": "new app framework",
"type": "major"
}
],
"packageName": "@subsquid/util-internal-http-server"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal-ingest-cli",
"comment": "",
"type": "none"
}
],
"packageName": "@subsquid/util-internal-ingest-cli"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal-json-fix-unsafe-integers",
"comment": "",
"type": "none"
}
],
"packageName": "@subsquid/util-internal-json-fix-unsafe-integers"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal-json",
"comment": "make `.stack` the first property of JSON serialized Error",
"type": "patch"
}
],
"packageName": "@subsquid/util-internal-json"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal-prometheus-server",
"comment": "adapt to a new version of `util-internal-http-server`",
"type": "minor"
}
],
"packageName": "@subsquid/util-internal-prometheus-server"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal-range",
"comment": "add `isRange()`",
"type": "minor"
}
],
"packageName": "@subsquid/util-internal-range"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal-validation",
"comment": "add `tuple()` validator",
"type": "minor"
},
{
"packageName": "@subsquid/util-internal-validation",
"comment": "add `B58`, `BIG_NAT` primitives",
"type": "minor"
}
],
"packageName": "@subsquid/util-internal-validation"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"changes": [
{
"packageName": "@subsquid/util-internal",
"comment": "add `concurrentWriter()` function",
"type": "minor"
},
{
"packageName": "@subsquid/util-internal",
"comment": "fix unhandled promise rejection crash in `concurrentMap()`",
"type": "patch"
}
],
"packageName": "@subsquid/util-internal"
}
4 changes: 2 additions & 2 deletions common/config/rush/deploy.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@
* and then specify those projects in the "additionalProjectsToInclude" list.
*/
"deploymentProjectNames": [
"@subsquid/solana-dump",
"@subsquid/substrate-dump",
"@subsquid/substrate-ingest",
"@subsquid/substrate-metadata-service",
"chain-status-service"
"@subsquid/substrate-metadata-service"
],

/**
Expand Down
Loading
Loading