You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As the hub is not fetching data directly from GitHub/ PyPi and is relying on the snowflake data, we treat snowflake data as the single source of truth, and the data in dynamo should maintain parity with snowflake.
In the cases where data gets added/removed from snowflake tables/views that the hub consumes from, for any reason, we should be able to update the hub data to reflect that. We need an automated workflow to validate that our data is in sync with Snowflake and heal the discrepancies.
This workflow could even be set to run periodically to ensure data quality.
As the hub is not fetching data directly from GitHub/ PyPi and is relying on the snowflake data, we treat snowflake data as the single source of truth, and the data in dynamo should maintain parity with snowflake.
In the cases where data gets added/removed from snowflake tables/views that the hub consumes from, for any reason, we should be able to update the hub data to reflect that. We need an automated workflow to validate that our data is in sync with Snowflake and heal the discrepancies.
This workflow could even be set to run periodically to ensure data quality.
The text was updated successfully, but these errors were encountered: