Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Target name doesn't invalidate partial parse #11279

Open
2 tasks done
dichenqiandbt opened this issue Feb 6, 2025 · 0 comments
Open
2 tasks done

[Bug] Target name doesn't invalidate partial parse #11279

dichenqiandbt opened this issue Feb 6, 2025 · 0 comments
Labels
bug Something isn't working triage

Comments

@dichenqiandbt
Copy link

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Hi Team, we've observed an unexpected behavior when switching schemas in development credentials while using a custom generate_schema_name macro with Alpha feature enabled. There seems to be an issue with project re-parsing, which does not properly recognize the schema switch even after deleting the target folder. This issue does not occur when the Alpha feature is disabled.

Custom generate_schema_name macro:

{% macro generate_schema_name(custom_schema_name, node) -%}
{%- set default_schema = target.schema -%}
{%- if target.name == 'default' and custom_schema_name is not none -%}
{{ custom_schema_name }}
{%- else -%}
{{ default_schema }}
{%- endif -%}
{%- endmacro %}

dbt_project.yml:

models:
my_new_project:
example:
+materialized: table
+schema: dbt_raw_gseda_19

Model: example/model1.sql

Step 1: Enable Alpha (Internal) feature under Settings/Profile

Step 2: Ensure the following Dev credential settings:

Dataset: dbt_gseda19

Target Name: default

Step 3: Run dbt build and validate result:
Expectation: model1 should build in dbt_raw_gseda_19
Result: Model built in dbt_raw_gseda_19 as expected.

OK created sql table model dbt_raw_gseda_19.model1

Step 4: Change the Dev credential settings (Target Name) to dev. IDE will prompt to restart.

Dataset: dbt_gseda19

Target Name: dev

Step 5: Run dbt build and validate result:
Expectation: model1 should now build in dbt_gseda19
Result: Model still resolves to dbt_raw_gseda_19 instead of dbt_gseda19

OK created sql table model dbt_raw_gseda_19.model1

Step 6: Delete the target folder then run dbt build again.
Result: Model still resolves to dbt_raw_gseda_19 instead of dbt_gseda19

OK created sql table model dbt_raw_gseda_19.model1

Step 7: Make a small (without affecting the logic) tweak to generate_schema_name (e.g., adding an extra line), save it, and build model1 again.
Result: It resolved to dbt_gseda19 as expected:

OK created sql table model dbt_gseda19.model1

Expected Behavior

In step 5, model should resolve to be dbt_gseda19.

Steps To Reproduce

You could use any dbt project with the macro generate_schema_name mentioned above.

  • Use default as target name.
  • Build any model, output debug level messages.
  • Change target name as dev.
  • Build the same model, output debug level messages. You will see the table name is incorrect from generated SQL.

Relevant log output

Environment

- OS: MACOS
- Python: 3.11
- dbt: 1.9

Which database adapter are you using with dbt?

No response

Additional Context

I believe the root cause is partial parse because when we disable partial parse, model name could be resolved correctly.

I read the source code and notice target name isn't been calculated when invalidate partial parse, this seems to be unexpected.

@dichenqiandbt dichenqiandbt added bug Something isn't working triage labels Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

1 participant