sync2: ATX integration

[ivan4th]

---

Motivation

We need more efficient ATX sync.

Description

This adds set reconciliation for ATXs.
There are per-epoch syncers, with lower FPTree depth (16 by default)
used for older epochs and greater FPTree depth (21 by default) used
for current epoch.
Both active syncv2 and passive (server-only) syncv2 are disabled by
default. It is possible to enable syncv2 in server-only or
full (active) mode.

Test Plan

Test on testnet and mainnet nodes

TODO

• Explain motivation or link existing issue(s)
• Test changes and document test plan
• Update documentation as needed
• Update changelog as needed

[fasmat]

I don't like how this config is structured, what is the meaning of OldAtxSyncCfg and NewAtxSyncCfg?
Is there maybe a better name for them?

Why are they part of "main" -> "Syncer" -> "V2"? Shouldn't they be part of the "main" -> "Syncer" config object?

Also if these are the defaults anyway, no need to overwrite them with the same values here again 🙂

[fasmat]

EDIT: fixed spelling

[ivan4th]

This might cause confusion as there's currently V1 sync for ATXs, too, which is also configured in the syncer config.
When V2 is used in server-only mode, both V1 and V2 syncs are used at the same time.

[ivan4th]

The idea is to use reasonable defaults so that most users will not have to adjust sync settings.
Then at some time in the future we might want to restructure the config to remove all the v1 stuff and keep v2 sync only, moving it out of this v2 field.

[ivan4th]

old vs new in this context means old epochs vs the current epoch

[fasmat]

Ah so it is actually PrevEpochSyncConfig and CurrentEpochSyncConfig? 🙂

[fasmat]

The idea is to use reasonable defaults so that most users will not have to adjust sync settings. Then at some time in the future we might want to restructure the config to remove all the v1 stuff and keep v2 sync only, moving it out of this v2 field.

The reasonable defaults are already specified as part of the syncer.DefaultConfig, my point was that we don't need to overwrite this values again with the same values in config.MainnetConfig 🙂

[fasmat]

This might cause confusion as there's currently V1 sync for ATXs, too, which is also configured in the syncer config. When V2 is used in server-only mode, both V1 and V2 syncs are used at the same time.

But this is a technical detail that I think should not be exposed via the config, or at least in a different way. Maybe instead of V2 we can call it something like "reconciliationSync" or similar. Especially since "V2" has negative connotations: https://en.wikipedia.org/wiki/V-2_rocket

[ivan4th]

Actually there's a bit of confusion re current vs prev, and I was imprecise in my above comment.
When a new epoch starts, activations don't start to be published in that epoch immediately, and consequently, the more efficient (but more memory and CPU hungry per missing/extra set element) sync is to be applied to the new epoch some time later (EpochEndFraction in the existing sync config). That's why the "new" epoch is not necessarily the current one, and the "old" epoch is not necessarily the previous. We could probably come up with better naming but I'm unsure current / prev is the right choice here.

Otherwise agree, will fix

[codecov]

Codecov Report

Attention: Patch coverage is 76.08069% with 83 lines in your changes missing coverage. Please review.

Project coverage is 79.8%. Comparing base (042d0d7) to head (294bc6c).

Files with missing lines	Patch %	Lines
sync2/atxs.go	75.1%	40 Missing and 8 partials ⚠️
syncer/syncer.go	74.5%	21 Missing and 7 partials ⚠️
fetch/fetch.go	0.0%	4 Missing ⚠️
fetch/mesh_data.go	72.7%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@                  Coverage Diff                  @@
##           sync2/fix-multipeer   #6448     +/-   ##
=====================================================
- Coverage                 79.9%   79.8%   -0.1%     
=====================================================
  Files                      352     353      +1     
  Lines                    46209   46534    +325     
=====================================================
+ Hits                     36928   37172    +244     
- Misses                    7193    7254     +61     
- Partials                  2088    2108     +20     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests