Index getting bloated when the commit interval is set to 10 or some lower value

[krrish-sehgal]

Issue: High Storage Usage While Syncing Ordinals Index

System Specifications:

• RAM: 16GB
• Storage: 200GB

Command Used:

sudo /usr/local/bin/ord \
  --bitcoin-rpc-username=btc_rpc_user \
  --bitcoin-rpc-password=pass \
  --bitcoin-rpc-url=http://ip:8332 \
  --index-runes \
  --index /blockchain/ord-index/mainnet/index-0.22-without-878500.redb \
  --commit-interval 10 \
  --index-cache-size=15032385536 \
  server --http-port 9003

Context:

I am using a prebuilt index https://ordstuff.info/indexes/0.22/index-0.22-without-878500.redb.gz.torrent, which unpacks to around 61GB. After starting with this, I sync the remaining blocks. This approach is based on discussions regarding the looping issue (e.g., Issue #3804)

However, at around block 879,200, I encountered the same looping issue. To mitigate this, I set --commit-interval to 10. Upon reaching block 880,100, I noticed a significant increase in storage usage.

Question:

• The index file has grown significantly. Could the low commit interval be causing excessive disk usage?
• Given my 200GB storage limit, is there an optimal way to manage storage while ensuring the index syncs successfully?