Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor CassandraSinkCluster #1844

Open
rukai opened this issue Nov 26, 2024 · 0 comments
Open

refactor CassandraSinkCluster #1844

rukai opened this issue Nov 26, 2024 · 0 comments
Labels
cleanup-internal Any cleanup that is not cleanup-api

Comments

@rukai
Copy link
Member

rukai commented Nov 26, 2024

We learnt a lot about what a SinkCluster should look like while writing CassandraSinkCluster, as a result there is a lot of confusing invariants to uphold and it is very difficult to understand.
In comparison, KafkaSinkCluster, which was written later, is a lot nicer.

Here are some things we can take from KafkaSinkCluster to improve the design of CassandraSinkCluster.

  • Keep the control connection as purely a control connection, do not direct its responses to the client by default
  • Combine rewrite_requests with route_requests
    • We need to be handling the effects of requests in the order they were received in, but with these two stages, sometimes the effect is handled in rewrite_requests and sometimes in route_requests, this causes particular issues with use statements.
  • Maybe split up sending requests into route_requests -> send_requests stages, need to check if this actually brings value for CassandraSinkCluster.
@rukai rukai added the cleanup-internal Any cleanup that is not cleanup-api label Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cleanup-internal Any cleanup that is not cleanup-api
Projects
None yet
Development

No branches or pull requests

1 participant