Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix/Address server_url breaking change #226

Merged
merged 12 commits into from
Feb 20, 2025
Merged

Conversation

awalker4
Copy link
Collaborator

@awalker4 awalker4 commented Feb 18, 2025

In order to add the platform-api functions, we now need to handle two openapi specs. When these are merged, we get a breaking change with the server_url arg:

# The passed url is ignored and we go to our default Serverless URL. 
# You suddenly get an invalid api key error
s = UnstructuredClient(server_url="my_own_url")
s.general.partition()

#  This does the right thing
s = UnstructuredClient()
s.general.partition(server_url="my_own_url")

We published the 0.30.0 client with this change, but let's do the right thing before it breaks a whole lot of users. For now, let's directly edit the endpoint functions and make sure the client level url is still accepted. We can work with Speakeasy to roll this back into the auto generated SDK.

Changes

  • Add a helper function to our clean_server_url hook file to select the right url from three options: the endpoint, the client, or the default from the config
  • Call this helper from every generated method (plus async versions)
  • Add a test_server_urls.py to test every URL case against every endpoint. The user can pass the URL in either location, and the SDK will strip the path if it's given, for backwards compat.
  • Add the endpoint files to .genignore. This will unblock the generate action for now, and our test suite will tell us if these files have gotten stale.

Other:

  • Update workflow.yaml to reorder the openapi specs. Both specs include HttpValidationError, and the later spec will overwrite the duplicate. This schema changed a bit in Serverless, so some tests are failing on main. Note that the make client-generate-sdk does the right thing, but this got reverted when the github generate action ran again.

Testing

  • Check out this branch and verify that your old SDK code with a custom URL is still working

@awalker4 awalker4 changed the title Fix/base url breaking change fix/Address server_url breaking change Feb 18, 2025
Copy link
Contributor

@pawel-kmiecik pawel-kmiecik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks fine - just one important question added.

base_url = operations.CREATE_DESTINATION_SERVERS[
client_url, *_ = self.sdk_configuration.get_server_details()

base_url = choose_server_url(
Copy link
Contributor

@pawel-kmiecik pawel-kmiecik Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I get why do we need to overwrite the platform-specific operations? As I understand that it didn't break anything yet and users can only use our prod environment, hardcoded in each operation.
The only thing we want to overwrite is partition operation which got broken.

I'd limit the changes only to that operation - it's rather "frozen" while we'll be working on platform-specific operations and might need regenerating these files. WDYT?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, that's an excellent point!

if not base_url:
return ""
# -- add a url scheme if not present (urllib.parse does not work reliably without it)
if "http" not in base_url:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would parse the URL first and then check if a url has a scheme by using parsed_url.scheme. Then we can prepend http:// or https://

Copy link
Collaborator Author

@awalker4 awalker4 Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That comment has been around for a while - I tried parsing without the scheme and it does look like we need to make sure http:// is there first or it gets confused.

In [2]: urlparse("localhost:8000/general/v0/general")
Out[2]: ParseResult(scheme='localhost', netloc='', path='8000/general/v0/general', params='', query='', fragment='')

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, good to know!

parsed_url: ParseResult = urlparse(base_url)
# First, see if the endpoint has a url:
# s.general.partition(server_url=...)
if endpoint_url is not None:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means the endpoint_url is treated as an optional kwarg and it's currently not.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can come in from the generated code as None over here

# -- add a url scheme if not present (urllib.parse does not work reliably without it)
if "http" not in base_url:
base_url = "http://" + base_url
def choose_server_url(endpoint_url, client_url, default_endpoint_url) -> str:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this entire helper function can be written as a oneliner:
url = endpoint_url if endpoint_url is not None else (client_url or default_endpoint_url)
the logic and reasoning behind this can be explained in the docstring.

@awalker4 awalker4 requested a review from PastelStorm February 19, 2025 22:01
@awalker4 awalker4 enabled auto-merge (squash) February 20, 2025 16:52
@awalker4 awalker4 merged commit e0b8220 into main Feb 20, 2025
11 checks passed
@awalker4 awalker4 deleted the fix/base-url-breaking-change branch February 20, 2025 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants