Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port boostd-data #24

Closed
serg-temchenko opened this issue May 15, 2024 · 4 comments
Closed

Port boostd-data #24

serg-temchenko opened this issue May 15, 2024 · 4 comments
Assignees

Comments

@serg-temchenko
Copy link
Contributor

serg-temchenko commented May 15, 2024

As described in the piecestore and polka-index sections of our guidelines, we need to port the boostd-data functionality since it contains LID, which is a replacement for the DAGstore component.

Related issues:

@jmg-duarte
Copy link
Contributor

jmg-duarte commented May 15, 2024

As far as I understood from my research, this task is somewhat blocked by #25 because the main entry point mechanism — importing files — requires files to be in the CAR format, as made evident in the ClientImport implementation.

This goes against the UML diagram in the Data Storage Workflow but the code seems to imply that it is not the Lotus daemon doing the conversion.

While we don't need the creation of CAR files to be implemented to unblock this task, we do need the reader.

Given that the upload is always done with CAR files, we can also consider how much we need to be able to write the CAR files ourselves. In other words, if we can get away with just using another tool to create the CAR files from arbitrary data.

@jmg-duarte
Copy link
Contributor

jmg-duarte commented May 21, 2024

What I suggested during the meeting was to start scaffolding the interface serving the boostd-data client.

A component that will be similar for use is the LevelDB service, though that for our case we'll be using RocksDB.

@serg-temchenko
Copy link
Contributor Author

What I suggested during the meeting was to start scaffolding the interface serving the boostd-data client.

A component that will be similar for use is the LevelDB service, though that for our case we'll be using RocksDB.

Created a separate issue #41 for that.

@serg-temchenko
Copy link
Contributor Author

serg-temchenko commented Jun 18, 2024

@serg-temchenko todo update reasons about closing..

EDIT(@jmg-duarte): we've decided to close this for now and use the current LID indexing as a library rather than a service for the time being.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants