-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add StarRocks connector #24720
base: master
Are you sure you want to change the base?
Add StarRocks connector #24720
Conversation
@hackeryang |
The old PR #17330 looks better than this PR at a glance. I would recommend asking @chenjian2664 if the PR can be reopened and how to collaborate. |
@ebyhr In fact, these are two different implementations. This PR using a JDBC client can first fetch the query plan from the responsible FE, then distribute the fetched query plan as a parameter to all relevant BEs, and finally fetch the data returned by the BEs. PR #17330 using JDBC connector can only read data from a single FE at a time. Data reading is very slow. You can take a look at https://docs.starrocks.io/docs/unloading/Flink_connector/ |
This is benchmark on Starrocks 3.2.9 Using JDBC connector and Starrocks connector. Test data
Result
|
Please fix CI failures and add following tests:
Also, I would recommend reading developer guide. |
@Max-Cheng I love your implementation of the load data from BEs, great job! And I think the main difference is just between the @ebyhr Does the community still want the jdbc implementation of the Starrocks? if yes I would like to reopen that. |
Description
Support Trino access Starrocks by connector
Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
(x) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: