[BUG]: sqlite database is not persisted (in dev environment at least)

[aittalam]

Description

Not sure if the problem is limited to the dev environment only, but the sqlite db is not persisted

Reproduction

1. make local-up
2. save a dataset
3. check with sqlite3 client that the new data is in the local.db file in the container: ✅
4. check with sqlite3 client that the new data is in the local.db file on the host machine: ❌
5. make local-down
6. check that the lumigator_database_volume contains something: ❌ (size=0 bytes)
7. make local-up
8. check with sqlite3 client that the data is in the local.db file in the container: ❌

Relevant log output

Expected behavior

The volume should persist the database data, or there should be a sync with the local.db file on the host and the one in the container.
If the volume works, perhaps we do not need to also have the file?

System Info

• OS: MacOs X
• Lumigator: e6c3768

Have you searched for similar issues before submitting this one?

• Yes, I have searched for similar issues

[ividal]

@aittalam Can this be closed?

[aittalam]

The main reason for pausing this was that information about jobs was taken live from ray and not having ray persisted broke things. Now we have ray persistence and experiment information in mlflow, so in theory we need the DB just for datasets. As soon as the old experiments is removed in favor of the new workflows we can prune whatever is not needed anymore and merge this to make datasets persistent!