-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix incremental delete+insert SQL #9459
Conversation
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @ataft |
Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide. |
changelog for Fix incremental delete+insert SQL
Thanks for raising this PR @ataft ! Could you resolve the merge conflict? |
Here's the merge conflict message I'm getting: Oddly, I don't see the |
Hi @ataft sorry for the confusion but we're in the middle of a large scale refactor of dbt.adapters. Once that has been wrapped up (in the next couple weeks) we will move your PR to the new dbt-adapters repo and review it there. More info: #9171 |
hey @ataft thanks for taking the time to open this PR. Since opening, this code now lives in a separate repo: dbt-adapters. A consequence of the decoupling is that PR can't be merged anymore as is, so we're closing it. For more context see #9171. The linked issue has already been transferred. Please don't hesitate to re-open on the |
All good, just moved the PR over to dbt-labs/dbt-adapters#151 |
resolves dbt-labs/dbt-adapters#150
Problem
The delete query for the 'delete+insert' incremental_strategy with 2+ unique_key columns is VERY inefficient. In many cases, it will hang and never return for deleting small amounts of data (<100K rows).
Solution
Improve the query by switching to a much more efficient delete strategy:
Checklist