Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix streaming support for multi property configurations #257

Merged
merged 5 commits into from
Aug 2, 2023

Conversation

adamribaudo-velir
Copy link
Collaborator

Description & motivation

Resolves #254

As noted by @FlorianASchroeder , PR #250 allowed the union of intraday data only for single-property configurations.

This update removes that limitation by leveraging the fact that both daily and intraday event tables start with 'events_'. This means we can use a wildcard operation to query events_* to pull both daily and intraday records. Duplicate records will be deduped as part of the base_ga4__events qualify statement.

As a result, we can remove the intraday source from src_ga4.yml

Checklist

  • I have verified that these changes work locally
  • [na] I have updated the README.md (if applicable)
  • [na] I have added tests & descriptions to my models (and macros if applicable)
  • I have run dbt test and python -m pytest . to validate existing tests

Copy link
Collaborator

@dgitis dgitis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern with this PR is that the qualify statement at the end won't deduplicate in all situations. I am not aware of any event_params that get added when Google processes the GA4 data, but just one field would cause the deduplication to stop working.

I'm also concerned that deduplicating this way will take a lot of unnecessary processing, but I can't think of a better way of doing this.

@adamribaudo-velir adamribaudo-velir merged commit a921996 into main Aug 2, 2023
1 check passed
@adamribaudo-velir adamribaudo-velir deleted the fix-streaming-support-multi-property branch August 2, 2023 23:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Multi-Property Support with streaming intraday tables
2 participants