Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-51334][CONNECT] Add java/scala version in analyze spark_version response #50102

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

garlandz-db
Copy link
Contributor

@garlandz-db garlandz-db commented Feb 27, 2025

What changes were proposed in this pull request?

  • piggyback off the spark version response and include other env properties like java/scala version

Why are the changes needed?

  • client is not able to know what type of java/scala version the server uses which can be problematic with udfs and client has different build env than server

Does this PR introduce any user-facing change?

  • no. the proto includes an additional response which doesn't change for existing users

How was this patch tested?

existing tests

Was this patch authored or co-authored using generative AI tooling?

No

@garlandz-db garlandz-db changed the title Add java/scala version in response [SPARK-51334] Add java/scala version in response Feb 27, 2025
@garlandz-db garlandz-db changed the title [SPARK-51334] Add java/scala version in response [SPARK-51334] Add java/scala version in analyze spark_version response Feb 27, 2025
@the-sakthi
Copy link
Member

I think we could add some tests here to verify the change in the response?

@HyukjinKwon HyukjinKwon changed the title [SPARK-51334] Add java/scala version in analyze spark_version response [SPARK-51334][CONNECT] Add java/scala version in analyze spark_version response Feb 28, 2025
@garlandz-db
Copy link
Contributor Author

@the-sakthi do you know where i can write them? im not seeing a relevant test suite that already includes tests for analyze plan handler

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants