Skip to content

Commit

Permalink
[MINOR] Fix README and CI fir min Spark 3.3 patch version support (ap…
Browse files Browse the repository at this point in the history
…ache#12431)

* [MINOR] Fix README and CI fir min Spark 3.3 patch version support

* updte release guide for hudi-metaserver module

* fix typo and spark version
  • Loading branch information
codope authored Dec 6, 2024
1 parent 07452fb commit 24c2526
Show file tree
Hide file tree
Showing 3 changed files with 5 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/bot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -509,7 +509,7 @@ jobs:
- scalaProfile: 'scala-2.12'
flinkProfile: 'flink1.14'
sparkProfile: 'spark3.3'
sparkRuntime: 'spark3.3.1'
sparkRuntime: 'spark3.3.4'
steps:
- uses: actions/checkout@v3
- name: Set up JDK 8
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Refer to the table below for building with different Spark and Scala versions.
| Maven build options | Expected Spark bundle jar name | Notes |
|:--------------------------|:---------------------------------------------|:-------------------------------------------------|
| (empty) | hudi-spark3.5-bundle_2.12 | For Spark 3.5.x and Scala 2.12 (default options) |
| `-Dspark3.3` | hudi-spark3.3-bundle_2.12 | For Spark 3.3.x and Scala 2.12 |
| `-Dspark3.3` | hudi-spark3.3-bundle_2.12 | For Spark 3.3.2+ and Scala 2.12 |
| `-Dspark3.4` | hudi-spark3.4-bundle_2.12 | For Spark 3.4.x and Scala 2.12 |
| `-Dspark3.5 -Dscala-2.12` | hudi-spark3.5-bundle_2.12 | For Spark 3.5.x and Scala 2.12 (same as default) |
| `-Dspark3.5 -Dscala-2.13` | hudi-spark3.5-bundle_2.13 | For Spark 3.5.x and Scala 2.13 |
Expand Down
3 changes: 3 additions & 0 deletions release/release_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,8 @@ mvn -Prelease clean install -Dspark3
- There are no release blocking JIRA issues.
- Release branch has been created.
- Release Notes have been audited and added to RELEASE_NOTES.md.
- Thrift is installed locally, if not then please follow the instructions [here](https://thrift.apache.org/docs/install/).
- A docker daemon is running (or Docker Desktop is running). This is required to build `hudi-metaserver-server` module.

# Build a release candidate

Expand Down Expand Up @@ -412,6 +414,7 @@ Set up a few environment variables to simplify Maven commands that follow. This
2. make sure your IP is not changing while uploading, otherwise it creates a different staging repo
3. Use a VPN if you can't prevent your IP from switching
4. after uploading, inspect the log to make sure all maven tasks said "BUILD SUCCESS"
5. In case you faced any issue while building `hudi-platform-service` or `hudi-metaserver-server` module, please ensure that you have docker daemon running. This is required to build `hudi-metaserver-server` module. See [checklist](#checklist-to-proceed-to-the-next-step).
5. Review all staged artifacts by logging into Apache Nexus and clicking on "Staging Repositories" link on left pane. Then find a "open" entry for apachehudi
6. Ensure it contains all 2 (2.12 and 2.13) artifacts, mainly hudi-spark-bundle-2.12/2.13, hudi-spark3-bundle-2.12/2.13, hudi-spark-2.12/2.13, hudi-spark3-2.12/2.13, hudi-utilities-bundle_2.12/2.13 and hudi-utilities_2.12/2.13.
> With 0.10.1, we had 4 bundles. spark2 with scala11, spark2 with scala12, spark3.0.x bundles and spark3.1.x bundles. Ensure each spark bundle reflects the version correctly. hudi-spark3.1.2-bundle_2.12-0.10.1.jar and hudi-spark3.0.3-bundle_2.12-0.10.1.jar are the respective bundle names for spark3 bundles.
Expand Down

0 comments on commit 24c2526

Please sign in to comment.