Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New parameter value binary format #174

Open
manuelma opened this issue Apr 12, 2022 · 2 comments
Open

New parameter value binary format #174

manuelma opened this issue Apr 12, 2022 · 2 comments
Labels
Milestone

Comments

@manuelma
Copy link
Collaborator

manuelma commented Apr 12, 2022

At the moment we store values in the database as BLOB, but those BLOBs are just 'binarised' JSON strings, so we don't really exploit the capability. This is for historical reasons: originally, we stored values as TEXT so it made a lot of sense to use JSON. Then, we changed to BLOB but didn't ditch the JSON - to take things slowly. But at some point, we should ditch the JSON and either adopt a known binary standard or develop our own.

@manuelma
Copy link
Collaborator Author

manuelma commented Apr 12, 2022

The problem with JSON is efficiency. Currently, we need to convert the bytes from the db to string, and then JSON-parse the string into an object. Ideally we should never do any of that and just keep the bytes. This would require developing a high level interface to access the bytes at specific points to retrieve specific indices in the parameter value. Anyone knows a standard?

@jkiviluo
Copy link
Member

In principle I'm really in favor of something like this (and I've written as much into Mopo proposal). We should just do careful research to land on the right approach. Maybe even ask around - e.g. Inti might know something about this.

@jkiviluo jkiviluo added this to the v0.8.2 milestone Aug 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants