Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SUPPORT] use flink1.16 write hudi0.15.0 MOR table, have same primary key data #12903

Open
screamedddd opened this issue Mar 3, 2025 · 2 comments

Comments

@screamedddd
Copy link

Data like this :
_hoodie_record_key _hoodie_partition_path _hoodie_file_name
835735 20250101 0001.parquet
835735 20250201 0002.parquet

Image

hudi table had compaction, same primary key can query(use snap-shot or read-optimize) many diff partition data.
use MOR table how can I achieve the same primary key, even if the data is in different partitions, only one piece of data can be queried?

Expected behavior

A clear and concise description of what you expected to happen.

Environment Description

  • Hudi version : 0.15.0

  • Spark version : 3.3.2

  • Hive version : 2.1.1

  • Hadoop version : 3.0.0

  • Storage (HDFS/S3/GCS..) : HDFS

  • Running on Docker? (yes/no) : no

Additional context

Add any other context about the problem here.

Stacktrace

Add the stacktrace of the error.

@danny0405
Copy link
Contributor

what kind of index did you use then?

@screamedddd
Copy link
Author

what kind of index did you use then?

bucket index

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants