Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce metric data parity between Snowflake and DynamoDb #1198

Open
1 of 2 tasks
manasaV3 opened this issue Aug 2, 2023 · 0 comments
Open
1 of 2 tasks

Enforce metric data parity between Snowflake and DynamoDb #1198

manasaV3 opened this issue Aug 2, 2023 · 0 comments
Assignees
Labels

Comments

@manasaV3
Copy link
Collaborator

manasaV3 commented Aug 2, 2023

As the hub is not fetching data directly from GitHub/ PyPi and is relying on the snowflake data, we treat snowflake data as the single source of truth, and the data in dynamo should maintain parity with snowflake.

In the cases where data gets added/removed from snowflake tables/views that the hub consumes from, for any reason, we should be able to update the hub data to reflect that. We need an automated workflow to validate that our data is in sync with Snowflake and heal the discrepancies.

This workflow could even be set to run periodically to ensure data quality.

@manasaV3 manasaV3 converted this from a draft issue Aug 2, 2023
@manasaV3 manasaV3 added the epic label Aug 2, 2023
@manasaV3 manasaV3 added backend and removed epic labels Aug 14, 2023
@richaagarwal richaagarwal moved this from Backlog to In Progress in napari hub backlog Aug 17, 2023
@manasaV3 manasaV3 moved this from In Progress to Ready in napari hub backlog Sep 5, 2023
@manasaV3 manasaV3 moved this from Ready to In Progress in napari hub backlog Sep 5, 2023
@richaagarwal richaagarwal moved this from In Progress to Backlog in napari hub backlog Sep 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Backlog
Development

No branches or pull requests

1 participant