Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add persistence to datasqrl run command #958

Open
henneberger opened this issue Oct 30, 2024 · 1 comment
Open

Add persistence to datasqrl run command #958

henneberger opened this issue Oct 30, 2024 · 1 comment
Assignees
@wutianchen
Labels
enhancement New feature or request
Milestone
v0.6.0

Comments

@henneberger
Copy link
Contributor

Sets a shared folder for the various embedded services and flink in the run command.
@henneberger henneberger added the enhancement New feature or request label Oct 30, 2024
@github-project-automation github-project-automation bot moved this to Backlog in DataSQRL Oct 30, 2024
@henneberger henneberger moved this from Backlog to Ready in DataSQRL Oct 30, 2024
@henneberger henneberger self-assigned this Nov 11, 2024
@mbroecheler mbroecheler added this to the v0.6.0 milestone Feb 25, 2025
@mbroecheler
Copy link
Contributor

For now, we are only persisting the data in the kafka topics so that it can be reprocessed when running the script.
Also persisting the flink and database state can cause issues when the user makes changes to the script that aren't compatible - which is very often when using the run command.
So, rather than making the user deal with those issues, we reprocess all data which shouldn't take long.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wutianchen wutianchen

Labels
enhancement New feature or request
Projects
DataSQRL
Status: Ready
Milestone
v0.6.0
Development

No branches or pull requests
3 participants
@mbroecheler @wutianchen @henneberger