Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Codecs composable #18

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 56 additions & 48 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ serde_bytes = { version = "0.11.5", default-features = false, optional = true }

[dev-dependencies]
serde_derive = "1.0.197"
serde_ipld_dagcbor = "0.6.0"
serde_ipld_dagjson = "0.2.0"
serde_ipld_dagcbor = { version = "*", git = "https://github.com/expede/serde_ipld_dagcbor", branch = "from_u64" }
serde_ipld_dagjson = { version = "*", git = "https://github.com/expede/serde_ipld_dagjson", branch = "update-codec" }
serde_json = "1.0.79"
serde_test = "1.0.132"

Expand Down
18 changes: 8 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Here's a full example of a function that can encode data with both [serde_ipld_d
```rust
use std::str;

use ipld_core::codec::Codec;
use ipld_core::codec::{Codec, Links};
use serde::{Deserialize, Serialize};
use serde_ipld_dagcbor::codec::DagCborCodec;
use serde_ipld_dagjson::codec::DagJsonCodec;
Expand All @@ -32,11 +32,9 @@ struct Tree {
age: u8,
}

fn encode_generic<C, T>(value: &T) -> Result<Vec<u8>, C::Error>
where
C: Codec<T>,
fn encode_generic<C: Codec<T>, T>(codec: C, value: &T) -> Result<Vec<u8>, C::Error>
{
C::encode_to_vec(value)
codec.encode_to_vec(value)
}

fn main() {
Expand All @@ -45,7 +43,7 @@ fn main() {
age: 91,
};

let cbor_encoded = encode_generic::<DagCborCodec, _>(&tree);
let cbor_encoded = encode_generic::<DagCborCodec, Tree>(DagCborCodec, &tree);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised this was necessary.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably not; it's having a hard time picking up the codec in the doctests and I'm troubleshooting. Because this spans multiple repos I need to push to git, so I've flipped the PR to Draft

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this spans multiple repos

Multiple repos with a dependency cycle, even (on the dev-dependencies)

Copy link
Author

@expede expede Jun 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I know that it's not required because in the library that I'm actually working on I now actually need fewer type annotations. For example:

//Before
let bytes = <DagCborCodec as Codec<Ipld>>::encode_to_vec(dag).unwrap();

// After
let bytes = DagCborCodec.encode_to_vec(dag).unwrap();

#[allow(clippy::format_collect)]
let cbor_hex = cbor_encoded
.unwrap()
Expand All @@ -54,7 +52,7 @@ fn main() {
.collect::<String>();
// CBOR encoded: https://cbor.nemo157.com/#value=a2666865696768740c63616765185b
println!("CBOR encoded: https://cbor.nemo157.com/#value={}", cbor_hex);
let json_encoded = encode_generic::<DagJsonCodec, _>(&tree).unwrap();
let json_encoded = encode_generic(DagJsonCodec, &tree).unwrap();
// JSON encoded: {"height":12,"age":91}
println!("JSON encoded: {}", str::from_utf8(&json_encoded).unwrap());
}
Expand All @@ -65,17 +63,17 @@ fn main() {
If you are only interested in the links (CIDs) of an encoded IPLD object, then you can extract them them directly with [`Codec::links()`]:

```rust
use ipld_core::{codec::{Codec, Links}, ipld, cid::Cid};
use ipld_core::{codec::{Codec, Links}, ipld, ipld::Ipld, cid::Cid};
use serde_ipld_dagjson::codec::DagJsonCodec;

fn main() {
let cid = Cid::try_from("bafkreibme22gw2h7y2h7tg2fhqotaqjucnbc24deqo72b6mkl2egezxhvy").unwrap();
let data = ipld!({"some": {"nested": cid}, "or": [cid, cid], "more": true});

let mut encoded = Vec::new();
DagJsonCodec::encode(&mut encoded, &data).unwrap();
DagJsonCodec.encode(&mut encoded, &data).unwrap();

let links = DagJsonCodec::links(&encoded).unwrap().collect::<Vec<_>>();
let links = DagJsonCodec.links(&encoded).unwrap().collect::<Vec<_>>();
// Extracted links: [Cid(bafkreibme22gw2h7y2h7tg2fhqotaqjucnbc24deqo72b6mkl2egezxhvy), Cid(bafkreibme22gw2h7y2h7tg2fhqotaqjucnbc24deqo72b6mkl2egezxhvy), Cid(bafkreibme22gw2h7y2h7tg2fhqotaqjucnbc24deqo72b6mkl2egezxhvy)]
println!("Extracted links: {:?}", links);
}
Expand Down
21 changes: 12 additions & 9 deletions src/codec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,25 +10,28 @@ use std::io::{BufRead, Write};
/// Each IPLD codec implementation should implement this Codec trait. This way codecs can be more
/// easily exchanged or combined.
pub trait Codec<T>: Links {
/// The multicodec code of the IPLD codec.
const CODE: u64;
/// The error that is returned if encoding or decoding fails.
type Error;

/// The multicodec code of the IPLD codec.
fn to_code(&self) -> u64;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd just call this code().

/// Attempt to convert from a `u64` code to this `Codec`.
fn try_from_code(code: u64) -> Option<Self> where Self: Sized;

/// Decode a reader into the desired type.
fn decode<R: BufRead>(reader: R) -> Result<T, Self::Error>;
fn decode<R: BufRead>(&self, reader: R) -> Result<T, Self::Error>;
/// Encode a type into a writer.
fn encode<W: Write>(writer: W, data: &T) -> Result<(), Self::Error>;
fn encode<W: Write>(&self, writer: W, data: &T) -> Result<(), Self::Error>;

/// Decode a slice into the desired type.
fn decode_from_slice(bytes: &[u8]) -> Result<T, Self::Error> {
Self::decode(bytes)
fn decode_from_slice(&self, bytes: &[u8]) -> Result<T, Self::Error> {
self.decode(bytes)
}

/// Encode a type into bytes.
fn encode_to_vec(data: &T) -> Result<Vec<u8>, Self::Error> {
fn encode_to_vec(&self, data: &T) -> Result<Vec<u8>, Self::Error> {
let mut output = Vec::new();
Self::encode(&mut output, data)?;
self.encode(&mut output, data)?;
Ok(output)
}
}
Expand All @@ -39,5 +42,5 @@ pub trait Links {
type LinksError;

/// Return all links (CIDs) that the given encoded data contains.
fn links(bytes: &[u8]) -> Result<impl Iterator<Item = Cid>, Self::LinksError>;
fn links(&self, bytes: &[u8]) -> Result<impl Iterator<Item = Cid>, Self::LinksError>;
}