Update sequence number assignment method in WBWIMemTable
#13400
Conversation
|
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| @@ -161,6 +163,9 @@ struct WriteBatchIndexEntry { | |||
| // SeekForPrev() will see all the keys with the same key. | |||
| size_t offset; | |||
| uint32_t column_family; // column family of the entry. | |||
| // The following three fields are only maintained when the WBWI is created | |||
| // with overwrite_key = true. | |||
| uint32_t update_count; // number of updates for this key up to this entry. | |||
There was a problem hiding this comment.
Let's mention explicitly somewhere that this is a 1-based count, not 0-based
| // | ||
| // WBWI with overwrite mode keeps track of the most recent update for each key, | ||
| // so this memtable contains one update per key usually. However, there is a | ||
| // special case where this memtable needs to emit an extra SingleDelete even |
There was a problem hiding this comment.
I assume discussion of special case for merge comes later?
There was a problem hiding this comment.
Yes, it will come later. It should be a small change where for merge we don't track only the most recent update.
cbi42
force-pushed
the
wbwi-seqno-update-count
branch
from
c54ef30 to
45e9204
Compare
|
@cbi42 has updated the pull request. You must reimport the pull request before landing. |
| // | ||
| // WBWI with overwrite mode keeps track of the most recent update for each key, | ||
| // so this memtable contains one update per key usually. However, there is a | ||
| // special case where this memtable needs to emit an extra SingleDelete even |
There was a problem hiding this comment.
Yes, it will come later. It should be a small change where for merge we don't track only the most recent update.
| @@ -161,6 +163,9 @@ struct WriteBatchIndexEntry { | |||
| // SeekForPrev() will see all the keys with the same key. | |||
| size_t offset; | |||
| uint32_t column_family; // column family of the entry. | |||
| // The following three fields are only maintained when the WBWI is created | |||
| // with overwrite_key = true. | |||
| uint32_t update_count; // number of updates for this key up to this entry. | |||
|
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary: This is a preparation for supporting merge in
WBWIMemTable. This PR updates the sequence number assignment method so that it allows efficient and simple assignment when there are multiple entries with the same user key. This can happen when the WBWI contains Merge operations. This assignment relies on tracking the number of updates issued for each key in each WBWI entry (WriteBatchIndexEntry::update_count). Some refactoring is done in WBWI to removelast_entry_offsetas part of the WBWI state which I find it harder to use correctly.Test plan: updated unit tests to check that update count is tracked correctly and WBWIMemTable is assigning sequence number as expected.