Lifetimes and ownership on keys and values could use more elaboration

[dpc]

If you look at the current https://docs.rs/redb/latest/redb/index.html

The example uses &str:

const TABLE: TableDefinition<&str, u64> = TableDefinition::new("my_data");

and that's quite confusing. TableDefinition mantions:

Note that the lifetime of the K and V type parameters does not impact the lifetimes of the data that is stored or retreived from the table

And that's about all the explanation given.

It's unclear if the example could use TableDefinition<String, u64>. Is this OK? Worse? Better? etc. Does the whole example work because only string literals are used?

Ideally some examples of avoiding copying etc. could be given as well. E.g. someone might have:

struct SomeKey {
  kind: u64,
  big_value: Vec<u8>,
}

and might want to avoid cloning big_value just to create a key to do a lookup, so I think using:

struct SomeKey {
  kind: u64,
  big_value: Cow<'_, [u8]>,
}

with some #[repr(transparent)] and transmute etc. might allow it, but it's actually very confusing and hard to come up with, especially in the presence of yet another component with it's own generics etc.

[dpc]

I am having a hard time coming up with all of these in redb-bincode. I'm trying to make a zero-copy keys (avoid allocating big_value: Vec<u8> just for lookup). It's even more tricky, because with redb-bincode everything always is getting de/serialized so somewhat different, but I first need to make sure I actually understand underlying redb properties/expecattions, so I can figure how to handle it at higher level, and I am not confident I get it.

[adamreichold]

I'm trying to make a zero-copy keys (avoid allocating big_value: Vec just for lookup).

I don't think this is possible generically, the problem being that

impl Value for Bincode<T> where T: for<'a> Deserialize<'a> {
    type SelfType<'a> = T;

    fn from_bytes<'a>(data: &'a [u8]) -> Self::SelfType<'a> {
        let val = bincode::deserialize(data)?;

        Ok(val)
    }

    ...
}

cannot unify an unnameable lifetime inside T with the SelfType lifetime parameter 'a which is also why you need the for<'a> Deserialize<'a> bound a.k.a. DeserializeOwned.

There is no problem though when T is some concrete type which borrows from the binary data, e.g.

#[derive(Deserialize)]
struct Foo<'a> {
    bar: &'a str,
}

impl Value for Foo<'_> {
    type SelfType<'a> = Foo<'a>;

    fn from_bytes<'a>(data: &'a [u8]) -> Self::SelfType<'a> {
        let val = bincode::deserialize(data)?;

        Ok(val)
    }

    ...
}

[adamreichold]

(I think this can be made to work by having users of your library implement a custom GAT variant of Deserialize which does tie the lifetime to the trait itself. But that does complicate things somewhat for them, especially for third-party types.)

[dpc]

@adamreichold Huh. I wasn't expecting anyone to actually be looking into redb-bincode.

Today I wrote a little cheatsheet ensuring I actually understand the problem.

In redb-bincode all insert/lookups need to go through serialization anyway, but this re-uses a thread-local buffers so it actually doesn't allocate.

I actually "solved" the issue by using Arc<[u8]> for keys, so I can clone it cheaply, but I'm still unsure if I'm doing something wrong, and that's why I'm wondering what's the approach in redb is, so I can compare.