Skip to content

Commit

Permalink
Pass USE_SDK_V2_{RESOURCES,DATA_SOURCES} to terraform (#2207)
Browse files Browse the repository at this point in the history
## Changes
- Propagate env vars USE_SDK_V2_RESOURCES and $USE_SDK_V2_DATA_SOURCES
to terraform
- This are troubleshooting helpers for resources migrated to new plugin
framework, recommended here:
https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/troubleshooting#plugin-framework-migration-problems
- This current unblocks deploying quality monitors, see
databricks/terraform-provider-databricks#4229 (comment)

## Tests
Manually testing that I can deploy quality monitor after this change
with `USE_SDK_V2_RESOURCES="databricks_quality_monitor"` set

### Main branch:
```
~/work/databricks_quality_monitor_repro % USE_SDK_V2_RESOURCES="databricks_quality_monitor" ../cli/cli-main bundle deploy
Uploading bundle files to /Workspace/Users/[email protected]/.bundle/quality_monitor_bundle/default/files...
Deploying resources...
Updating deployment state...
Deployment complete!
Error: terraform apply: exit status 1

Error: Provider produced inconsistent result after apply

When applying changes to databricks_quality_monitor.monitor_trips, provider
"provider[\"registry.terraform.io/databricks/databricks\"]" produced an
unexpected new value: .data_classification_config: block count changed from 0
to 1.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.
```

### This branch:
```
~/work/databricks_quality_monitor_repro % USE_SDK_V2_RESOURCES="databricks_quality_monitor" ../cli/cli bundle deploy
Uploading bundle files to /Workspace/Users/[email protected]/.bundle/quality_monitor_bundle/default/files...
Deploying resources...
Updating deployment state...
Deployment complete!
```

### Config:
```
~/work/databricks_quality_monitor_repro % cat databricks.yml
bundle:
  name: quality_monitor_bundle

resources:
  quality_monitors:
    monitor_trips:
      table_name: main.denis-bilenko-cuj-pe34.trips_sanitized_1
      output_schema_name: main.denis-bilenko-cuj-pe34
      assets_dir: /Workspace/Users/${workspace.current_user.userName}/quality_monitor_issue
      snapshot: {}
```
  • Loading branch information
denik authored Jan 23, 2025
1 parent 1f63aa0 commit ddd45e2
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions bundle/deploy/terraform/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,14 @@ var envCopy = []string{
// Include $TF_CLI_CONFIG_FILE to override terraform provider in development.
// See: https://developer.hashicorp.com/terraform/cli/config/config-file#explicit-installation-method-configuration
"TF_CLI_CONFIG_FILE",

// Include $USE_SDK_V2_RESOURCES and $USE_SDK_V2_DATA_SOURCES, these are used to switch back from plugin framework to SDKv2.
// This is used for mitigation issues with resource migrated to plugin framework, as recommended here:
// https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/troubleshooting#plugin-framework-migration-problems
// It is currently a workaround for deploying quality_monitors
// https://github.com/databricks/terraform-provider-databricks/issues/4229#issuecomment-2520344690
"USE_SDK_V2_RESOURCES",
"USE_SDK_V2_DATA_SOURCES",
}

// This function inherits some environment variables for Terraform CLI.
Expand Down

0 comments on commit ddd45e2

Please sign in to comment.