Introduce the Env::nested_read_txn from an RwTxn
#307
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Kerollmops
force-pushed
the
allow-nested-rtxn-from-wtxn
branch
from
9df5ad8 to
09d1167
Compare
Kerollmops
force-pushed
the
allow-nested-rtxn-from-wtxn
branch
from
95d39b5 to
f7651f0
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is a first attempt at patching LMDB to generate multiple
RoTxnfrom anRwTxnand, therefore, be able to read the content of an uncommittedRwTxnin parallel. We can run many algorithms in Meilisearch to process the written content and generate new data structures before committing the transaction.This first attempt uses a broken fork of LMDB that ignores the number of nested write transactions associated with the parent one. Therefore, it lets heed create multiple nested write transactions from the parent one (dangerous). We store those write transactions in
RoTxns shells to disallow writing anything with them at the compilation level.However, while thinking about this, it is probably still possible to open databases (already existing) and commit them from multiple threads concurrently. It should be either disallowed or will, anyway, be fixed by the correct LMDB patch later.
Note the CI is broken but it works locally so it's maybe a submodule issue (always a submodule issue) 🤔It was...