Iceberg custom partitioning: use configured spec when creating the table #24952
Conversation
CI test resultstest results on build#61260
test results on build#61380
|
| @@ -249,19 +249,31 @@ checked<ss::gate::holder, coordinator::errc> coordinator::maybe_gate() { | |||
| return gate_.hold(); | |||
| } | |||
|
|
|||
| struct coordinator::table_schema_provider { | |||
There was a problem hiding this comment.
nit: just a thought, this file is getting a bit crowded to read through with the different implementations of this schema provider sprinkled throughout. Might be easier on the eyes to separate it out into its own header and cc
There was a problem hiding this comment.
I kind of like it being close to the respective functions as the only reason for existence of these classes is to factor out differences in those functions
| @@ -251,13 +251,9 @@ redpanda_cc_library( | |||
| hdrs = [ | |||
| "table_definition.h", | |||
| ], | |||
| implementation_deps = [ | |||
| # key to ensure that all partitions get some records | ||
| key=str(uuid4()), | ||
| value=record) | ||
| producer.flush() |
There was a problem hiding this comment.
Is it possible for us to get very unlucky and for the records in this batch to also be in different hours?
There was a problem hiding this comment.
good catch, I was intending to have the same timestamp_us value for all records and partition based on that, but forgot to change the spec expression.
No functional changes.
As we now set the partition spec from config property values, we don't need a hardcoded spec in the main binary.
ztlpn
force-pushed
the
iceberg-cp-use-config
branch
from
7cbce89 to
b434da9
Compare
Backports Required
Release Notes
Features
redpanda.iceberg.partition.spectopic property.