Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enums are now treated as type INTEGER in BigQuery #54

Merged
merged 8 commits into from
Jan 24, 2025

Conversation

stephenoken
Copy link
Contributor

@stephenoken stephenoken commented Jan 15, 2025

Referring to the BigQuery schema compatibility page. Enums are mapped as integers and not strings. The following changes have been made to address this issue:

  • Enums are now represented as INTEGERS in BigQuery schemas.
  • Updated plugin_test to reflect the change.

* Enums are now represented as INTEGERS in BigQuery schemas.
* Updated pluging_test to reflect the change.
Copy link

google-cla bot commented Jan 15, 2025

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@stephenoken stephenoken changed the title Enums are now treated as type STRING in BigQuery Enums are now treated as type INTEGER in BigQuery Jan 16, 2025
* Enums are now represented as INTEGERS in BigQuery schemas.
* Updated pluging_test to reflect the change.
@stephenoken
Copy link
Contributor Author

Hello, can anyone take a look at this PR?

@stephenoken
Copy link
Contributor Author

Hi again, I've updated the PR to address the failing build.

@mescanne
Copy link
Collaborator

Stephen -

I'm conscious that this is a change of behaviour for people who are using it today -- enums to strings can be a reasonable approach as well as integer.

Could we either bump the version to v3 -- similar to this PR #52 -- or put this behind a feature flag? I'm happy with either approach. Then let's merge away.

Apologies for all of the work involved with this!

Mark

@stephenoken
Copy link
Contributor Author

Stephen -

I'm conscious that this is a change of behaviour for people who are using it today -- enums to strings can be a reasonable approach as well as integer.

Could we either bump the version to v3 -- similar to this PR #52 -- or put this behind a feature flag? I'm happy with either approach. Then let's merge away.

Apologies for all of the work involved with this!

Mark

Hi Mark,

No worries, I'm happy to do that. I suggest we might actually want to do both suggestions. Except treat enums as integers by default and have a feature flag that treats them as string.

What do you think?

Stephen

* By default enum field types will be treated as integer columns in
  BigQuery schemas.
* A new enum-as-string flag has been added to have enums be treated as
  strings.
@stephenoken
Copy link
Contributor Author

Hi Mark, when you have time, please let me know your thoughts on the PR.

@mescanne
Copy link
Collaborator

Looks good to me -- can you run go fmt on the convert.go and push? Seems to have a formatting issue.

@stephenoken
Copy link
Contributor Author

Thanks! I've fixed the formatting issue.

@stephenoken
Copy link
Contributor Author

Hi Mark, could you please rerun the checks 🙂

@mescanne mescanne merged commit 8f85408 into GoogleCloudPlatform:master Jan 24, 2025
3 checks passed
@mescanne
Copy link
Collaborator

Merged! Thanks a lot Stephen.

@stephenoken
Copy link
Contributor Author

Hi Mark, would it be possible to publish this new version? Or is there a release schedule that needs to be followed?

@mescanne
Copy link
Collaborator

mescanne commented Jan 28, 2025

Nothing as formal a release schedule. I just haven't automated it so have to remember all of the steps. It's done now.

@mfridman
Copy link
Contributor

mfridman commented Jan 28, 2025

@mescanne Can you git tag and github release a v3.0.0. I'm afraid Go isn't picking up the v3.0 tag and is treating it as a pseduo-version.

https://pkg.go.dev/github.com/GoogleCloudPlatform/protoc-gen-bq-schema/v3

go install github.com/GoogleCloudPlatform/protoc-gen-bq-schema/v3@latest
go: downloading github.com/GoogleCloudPlatform/protoc-gen-bq-schema/v3 v3.0.0-20250124122649-8f85408915e3

Notice the v3.0.0-20250124122649-8f85408915e3 version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants