Add system.bucket function to Iceberg #24200
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@martint Could you please review the syntax? |
posulliv
force-pushed
the
add-iceberg-bucket-udf
branch
from
3f098db to
f5d7a1a
Compare
ebyhr
added
the
needs-docs
martint
reviewed
Nov 22, 2024
| @@ -33,6 +34,6 @@ public Iterable<ConnectorFactory> getConnectorFactories() | |||
| @Override | |||
| public Set<Class<?>> getFunctions() | |||
| { | |||
| return ImmutableSet.of(IcebergThetaSketchForStats.class); | |||
| return ImmutableSet.of(IcebergThetaSketchForStats.class, IcebergBucketFunction.class); | |||
There was a problem hiding this comment.
This should be exposed as a catalog function, not a global function.
There was a problem hiding this comment.
@martint thank you for reviewing! I'm not sure how to do this however? Is there an example of a function exposed as a catalog function in another connector I could look at to understand how to do this?
There was a problem hiding this comment.
I think you can check random_string function in Faker. Also, we could rename to bucket once it is implemented as a catalog function.
There was a problem hiding this comment.
Sent #24703 so we can avoid depending on trino-main in Iceberg connector.
ebyhr
reviewed
Nov 27, 2024
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/functions/IcebergBucketFunction.java
Outdated
Show resolved
Hide resolved
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/functions/IcebergBucketFunction.java
Outdated
Show resolved
Hide resolved
...trino-iceberg/src/test/java/io/trino/plugin/iceberg/functions/TestIcebergBucketFunction.java
Outdated
Show resolved
Hide resolved
...trino-iceberg/src/test/java/io/trino/plugin/iceberg/functions/TestIcebergBucketFunction.java
Outdated
Show resolved
Hide resolved
...trino-iceberg/src/test/java/io/trino/plugin/iceberg/functions/TestIcebergBucketFunction.java
Outdated
Show resolved
Hide resolved
...trino-iceberg/src/test/java/io/trino/plugin/iceberg/functions/TestIcebergBucketFunction.java
Outdated
Show resolved
Hide resolved
...trino-iceberg/src/test/java/io/trino/plugin/iceberg/functions/TestIcebergBucketFunction.java
Outdated
Show resolved
Hide resolved
...trino-iceberg/src/test/java/io/trino/plugin/iceberg/functions/TestIcebergBucketFunction.java
Show resolved
Hide resolved
...trino-iceberg/src/test/java/io/trino/plugin/iceberg/functions/TestIcebergBucketFunction.java
Outdated
Show resolved
Hide resolved
...trino-iceberg/src/test/java/io/trino/plugin/iceberg/functions/TestIcebergBucketFunction.java
Show resolved
Hide resolved
posulliv
force-pushed
the
add-iceberg-bucket-udf
branch
from
f5d7a1a to
f96e88a
Compare
|
@ebyhr thank you for reviewing! I think I addressed all your comments if you want to have another look when you get a chance. |
|
This pull request has gone a while without any activity. Tagging for triage help: @mosabua |
ebyhr
added
stale-ignore
ebyhr
force-pushed
the
add-iceberg-bucket-udf
branch
from
f96e88a to
8265b27
Compare
ebyhr
changed the title
ebyhr
removed
the
stale-ignore
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add a UDF to expose the iceberg bucket transform via SQL. This can be used to help determine what bucket a value would be placed in. This could be used with the
optimizeprocedure to only optimize specific buckets.Spark has a similar UDF - https://iceberg.apache.org/javadoc/1.5.1/org/apache/iceberg/spark/functions/BucketFunction.html
Release notes