Separate Polaris-native API from Iceberg Rest Catalog API spec

[HonahX]

Not ready for review

This PR fixes: #553 by separating Polaris-native catalog APIs from Iceberg REST Catalog spec. Dev list discussion: https://lists.apache.org/thread/1fqocs00pno0xfr4ss2p69d6dv5h8qzf

• Move notifications API to polaris-apis/notifications-api.yaml. The new yaml contains necessary paths and components def for the notifications API
• Create polaris-catalog-service.yaml which will be the root spec file that groups IRC Apis from rest-catalog-open-api.yaml and polaris-apis/notifications-api.yaml. This one will be used in the builder to generate open api service.
• Add generated bundle generated/bundled-polaris-catalog-service.yaml for website to render the preview of APIs

There will be 2 follow-ups of this PR

1. Update hugo site's rest-catalog-open-api.md to point to the generated bundle generated/bundled-polaris-catalog-service.yaml
2. Make rest-catalog-open-api.yaml match the released version (currently 1.7.1)

[snazy]

I thought we do not want to have two PRs #906 and #856 for the same thing (see #884 and https://lists.apache.org/thread/bcnh1dwgoxd2dvtqk6z935gfzmh4q0jq)

[HonahX]

@snazy Thanks for bringing this up. Sorry for the confusion, the 2 PR should for the different changes but I forgot to update the title. I will pay attention to this in the future to better follow the guideline

[jackye1995]

do we need to also commit the generated file? If possible I think we should have that in .gitignore

[HonahX]

Yeah, it is a weird but it is necessary for Polaris's hugo site to render preview.
https://github.com/apache/polaris/blob/main/site/content/in-dev/unreleased/rest-catalog-open-api.md?plain=1#L27

polaris/site/layouts/shortcodes/redoc-polaris.html

Line 39 in 569bf02

{{- $url = printf "https://raw.githubusercontent.com/apache/polaris/refs/heads/main/spec/%s" $spec }}

The site need a url of the yaml to render the preview: https://polaris.apache.org/in-dev/unreleased/rest-catalog-open-api/, so we have to push it to the github

We can remove the generated one from github if our site can render a local yaml, but that to my current understanding will require non-trivial change to our site so we may explore later.

[jackye1995]

we might want to rename this as iceberg-rest-catalog-open-api.yaml just to be clear.

[HonahX]

Good point! I plan to do the rename in a follow-up PR because

1. The current rest-catalog-open-api.yaml is referenced in our hugo site setting
2. We need to extract additional stuff from rest-catalog-open-api.yaml to make it match a released version of API.

I think we can do the above 2 together in one separate PR so the current one won't contain too many changes

[jackye1995]

this does not include the management APIs?

[HonahX]

The Management APIs and Catalog APIs have different scopes and are typically referenced separately for service/client generation, as seen here:
https://github.com/HonahX/polaris/blob/4c7ae00b542806928dc05d1e4f56d51014dedac5/api/iceberg-service/build.gradle.kts#L51
https://github.com/HonahX/polaris/blob/4c7ae00b542806928dc05d1e4f56d51014dedac5/api/management-service/build.gradle.kts#L48

They also have different URL prefixes, security configurations, etc. Given these differences, I think it makes sense to keep them in separate YAML files for better flexibility.

The goal of this PR is to clearly separate Polaris-specific catalog APIs from Iceberg catalog APIs.

[jackye1995]

looks good to me!

[flyrain]

Is re-generation manual or part of build process? I think we could make it part of build process if it's manual. Not a blocker for this PR though.

[HonahX]

Currently it's manual. I think once we find a way to render local rest spec YAML instead of referring to GitHub main branch one, we can make this automatic.

[flyrain]

Thanks @HonahX for working on it. Thanks @snazy @jackye1995 for the review.