Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support for unique where clauses for each relation passed to union_relations #897

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

thisismath
Copy link

@thisismath thisismath commented May 23, 2024

resolves: #895

This is a:

  • documentation update
  • bug fix with no breaking changes
  • new functionality
  • a breaking change

All pull requests from community contributors should target the main branch (default).

Description & motivation

currently when a where clause is passed to union_relations, it is used for every relation that was passed. In some situations, this approach is not ideal. The proposed code change here maintains current functionality of passing strings to the where param but also allows for lists to be passed just like how the relations are passed to the relations parameter. Support for specific where clauses for each relation allows for more flexibility in the way the macro can be deployed

Checklist

  • This code is associated with an Issue which has been triaged and accepted for development.
  • I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
    • BigQuery
    • Postgres
    • Redshift
    • Snowflake
  • I followed guidelines to ensure that my changes will work on "non-core" adapters by:
    • dispatching any new macro(s) so non-core adapters can also use them (e.g. the star() source)
    • using the limit_zero() macro in place of the literal string: limit 0
    • using dbt.type_* macros instead of explicit datatypes (e.g. dbt.type_timestamp() instead of TIMESTAMP
  • I have updated the README.md (if applicable)
  • I have added tests & descriptions to my models (and macros if applicable)
  • I have added an entry to CHANGELOG.md

…lations

currently when a where clause is passed to union_relations, it is used for every relation that was passed. In some situations, this approach is not ideal. The proposed code change here maintains current functionality of passing strings to the where param but also allows for lists to be passed just like how the relations are passed to the relations parameter. Support for specific where clauses for each relation allows for more flexibility in the way the macro can be deployed
@thisismath
Copy link
Author

thisismath commented May 23, 2024

For example:

prior functionality is maintained when passing a string as designed:

{{ dbt_utils.union_relations(
    relations=[ref('my_model'), source('my_source', 'my_table')]
    , exclude=["_loaded_at"]
    , where='1=1' 
) }}

and allows for specific where clauses when passing a list:

{% set max_dateframe = "'2023-06-01'" %}

{{ dbt_utils.union_relations(
    relations=[ref('my_model'), ref('my_model_view')]
    , where=[
      'ts >= '~max_dateframe
      , 'ts < '~max_dateframe
    ]
) }}

@dbeatty10 dbeatty10 added the enhancement New feature or request label Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

support for specific where clauses for each relation in the union_relations macro
2 participants