Blog post about new hash table (WIP) #195
Conversation
Signed-off-by: Viktor Söderqvist <[email protected]>
| | Valkey 8.0 | ? bytes | | ||
| | Valkey 8.1 | ? bytes | | ||
|
|
||
| The benchmarks below were run using a key size of N and a value size of M bytes, without pipelining. |
There was a problem hiding this comment.
Let's add something for set/zset/hash and see if we get even more performance and memory savings since those datatypes are hashtables inside of a hashtable. :)
There was a problem hiding this comment.
Yes... Feel free to replace these tables with some completely different tests.
There's a fixed overhead for the key and then per field-value. Still I'd like to see a table of memory savings per element/field/etc. for these types.
I want to do hash value embedding (to save the value pointer and an extra allocation) and Ran noticed that our embedded sds (key and field) are sds8 even when they should be sds5, so we could save a two more bytes for those. That's because they're copied from an EMBSTR robj value and those are always sds8. I have some idea to fix that too though.
| Why not use an open-source state-of-the-art hash table implementation such as | ||
| Swiss tables? The answer is that we require some specific features, apart from | ||
| the basic operations like add, lookup, replace, delete: |
There was a problem hiding this comment.
Another reason: Swiss table is very fast, but it stores the elements directly in a contiguous array, which requires that the elements all be the same size. Because our elements vary in size, we had to choose a different design - we chose cache-line sized buckets with element pointers. (This idea was mentioned at the end of the swiss table talk - up to you if you want to make that reference though.)
There was a problem hiding this comment.
No, you can store pointers in a Swiss table, just like how we store pointers in our bucket layout. The pointers are the fixed-size elements, no?
I don't think it allows a custom key-value entry design like we do though. It can be either a set or a map (key and value) IIUC.
There was a problem hiding this comment.
I think we could have picked an off-the-shelf implementation even if we couldn't embed key and value the way we do, as long as would be better than dict. It's good to use a battle-tested ready-to-use one too. It's easier to get it right, and less work... I think scan and incremental rehashing were clearly blockers though.
There was a problem hiding this comment.
Overall I think the content is really good. Is @SoftlyRaining planning on adding more information about benchmark results?
| title= "A new hash table" | ||
| date= 2025-03-20 00:00:00 | ||
| description= "Designing a state-of-the art hash table implementation" | ||
| authors= [ "zuiderkwast", "SoftlyRaining"] |
There was a problem hiding this comment.
Yall need author pages, or it won't render correctly.
There was a problem hiding this comment.
I'll just blank out authors then and write my name in the bottom of the article text instead. 😆
| authors= [ "zuiderkwast", "SoftlyRaining"] | |
| authors= [] |
| Memory usage for keys of length N and value of length M bytes. TBD. | ||
|
|
||
| | Version | Memory usage per key | | ||
| |------------|------------------------| | ||
| | Valkey 7.2 | ? bytes | | ||
| | Valkey 8.0 | ? bytes | | ||
| | Valkey 8.1 | ? bytes | | ||
|
|
There was a problem hiding this comment.
@SoftlyRaining Are you working on generating these numbers?
| memory usage by roughly 20 bytes per key-value pair and improve the latency and | ||
| CPU usage by rougly 10% for instances without I/O threading. | ||
|
|
||
| Results |
There was a problem hiding this comment.
A typical blog would leave the results till the end, and talk about the constraints up front.
There was a problem hiding this comment.
Just move the Results section to the end?
I'll keep brief text in the abstract above "we have managed to reduce the memory usage by roughly 20 bytes per key-value pair and improve the latency and CPU usage by rougly 10% for instances without I/O threading"?
| The slowest operation when looking up a key-value pair is by far reading from | ||
| the main RAM memory. A key point when optimizing a hash table is therefore to | ||
| make sure we have as few memory accesses as possible. Ideally, the memory | ||
| reading is already in the CPU cache, which is much faster memory that belong to | ||
| the CPU. |
There was a problem hiding this comment.
I think this background is OK, but I think it would be beneficial to show the previous implementation, and then explain how we iterated on it to the new one. Then it becomes easier to show the memory accesses that are getting removed. You already show the previous design later on, so we can probably move it earlier.
A good general framing is, show the current state (the dict), highlight the gaps (lots of memory accesses and allocations), introduce a new concept that solves that (cache friendly), then discuss the solution. Most of that content is here, just needs a little bit of moving around.
|
|
||
| When a computer loads some data from the main memory into the CPU cache, it does | ||
| so in blocks of one cache line. The cache-line size is 64 bytes on almost all | ||
| modern hardware. Recent work on hash tables, such as "Swiss tables", are highly |
There was a problem hiding this comment.
Should probably link the swiss table design notes or youtube video here.
| To lookup a key "FOO" and access the value "BAR", Valkey still had to read from | ||
| memory four times. If there is a hash collission, it has to follow two more | ||
| pointers for each hash collission and thus read twice more from memory (the key | ||
| and the next pointer). | ||
|
|
||
| In Valkey 8.0, an optimization was made to embed the key ("FOO" in the drawing | ||
| above) in the dictEntry, eliminating one pointer and one memory access. |
There was a problem hiding this comment.
We have a full blog post dedicated to embedding the key, so let's just start with the state of the world in 7.2.
| authors= [ "zuiderkwast", "SoftlyRaining"] | ||
| +++ | ||
|
|
||
| Valkey is essentially a giant hash table attached to the network. A hash table |
There was a problem hiding this comment.
Probably want a better hook. We can talk about how many caching workloads are bound on storing data, so being able to to store more data allows you to reduce the size of your clusters.
| In the new hash table designed for Valkey 8.1, the table consists of buckets of | ||
| 64 bytes, one cache line. Each bucket can store up to seven elements. Keys that | ||
| map to the same bucket are all stored in the same bucket. The bucket also has a | ||
| metadata section which contains a one byte secondary hash for each key. This is |
There was a problem hiding this comment.
Need to explain a bit about what the secondary hash is doing here, it's not really explained what it is or how it solve hash collisions.
I'm not sure we did any testing, but this is also really good for caching workloads, since we save a memory access most of the time when there is a cache miss. A common cache hit rate is like 80%, so that 20% gets a nice speed boost as well.
| Iterator prefetching | ||
| -------------------- |
There was a problem hiding this comment.
This is technically a separate point. It addresses some of the content of the blog, but ultimately feels a bit random at the end. Maybe we drop it and write another followup blog on memory prefetching?
There was a problem hiding this comment.
We could include it in the 8.1 overview blog post. I'm not sure anyone wants to write a prefetching blog post specifically.
Signed-off-by: Viktor Söderqvist <[email protected]>
The new new hash table is one of the highlights of the upcoming 8.1 release.