Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wal Recovery #39

Merged
merged 15 commits into from
Jan 13, 2025
Merged

Wal Recovery #39

merged 15 commits into from
Jan 13, 2025

Conversation

samliok
Copy link
Collaborator

@samliok samliok commented Jan 2, 2025

No description provided.

@samliok samliok marked this pull request as draft January 2, 2025 20:05
@samliok samliok changed the base branch from main to epoch January 2, 2025 20:05
consts/consts.go Outdated Show resolved Hide resolved
consts/consts.go Outdated Show resolved Hide resolved
encoding.go Outdated Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
epoch_test.go Outdated Show resolved Hide resolved
epoch.go Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
@@ -340,6 +433,28 @@ func (e *Epoch) maybeCollectFinalizationCertificate(round *Round) error {
return e.assembleFinalizationCertificate(round)
}

func (e *Epoch) NewFinalizationCertificate(finalizations []*Finalization) (FinalizationCertificate, error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand you refactored this method outside of assembleFinalizationCertificate to use it in a test, but we shouldn't use Epoch methods to create messages that are used to test the Epoch.

If there is a bug in Epoch logic we don't want to repeat it in the test.

Copy link
Collaborator Author

@samliok samliok Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

separated to a new file called record_test.go. Now that creating finalization/notarization certificates don't rely on epoch, would it make sense to use newNotarization & newFinalizationCertificate in epoch? Since they are decoupled, don't see why epoch should re-implement these functions. However if our test notarizations/finalizations are going to deviate from production then we shouldn't combine.

epoch.go Outdated Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
epoch_test.go Outdated Show resolved Hide resolved
epoch_test.go Outdated Show resolved Hide resolved
epoch_test.go Outdated Show resolved Hide resolved
epoch_test.go Outdated Show resolved Hide resolved
epoch_test.go Outdated Show resolved Hide resolved
Base automatically changed from epoch to main January 3, 2025 17:56
epoch.go Show resolved Hide resolved
epoch.go Outdated Show resolved Hide resolved
encoding.go Outdated Show resolved Hide resolved
encoding.go Outdated Show resolved Hide resolved
epoch_test.go Outdated
require.NoError(t, err)
require.Len(t, records, 2)
// expectedNotarizationRecord, err := newNotarizationRecord(signatureAggregator, block, nodes[0:quorum], conf.Signer)
// require.NoError(t, err)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this commented out?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because the bytes of the notarization record are not always guaranteed to be the same every time you create it. I set up a PR #44 addressing this

epoch_test.go Outdated Show resolved Hide resolved
epoch_test.go Outdated Show resolved Hide resolved
epoch_test.go Outdated Show resolved Hide resolved
epoch_test.go Outdated
require.NoError(t, err)

rounds := uint64(100)
for i := uint64(0); i < rounds; i++ {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible not to start the test from a 0 sequence? Maybe have the instance finalize into storage a couple of blocks before recovering it?

Let's be sure that the logic that loops through the records in the WAL and ingests the messages works when the records start from an arbitrary round and not zero.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TestRecoverFromWalWithStorage includes a test for this

epoch.go Outdated Show resolved Hide resolved
@samliok samliok marked this pull request as ready for review January 11, 2025 00:32
@@ -370,8 +370,7 @@ func (e *Epoch) isFinalizationCertificateValid(fCert *FinalizationCertificate) (
return false, nil
}

e.Logger.Debug("Received finalization without any signatures in it")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure what this log was doing here, if its valid then there should be signatures on it

epoch_multinode_test.go Outdated Show resolved Hide resolved
@yacovm yacovm merged commit e0c1431 into main Jan 13, 2025
1 check passed
@yacovm yacovm deleted the wal-rebase branch January 13, 2025 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants