Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do we have a plan to implement Kafka RackAffinityGroupBalancer ? #26

Open
Derfirm opened this issue Feb 25, 2025 · 1 comment
Open

Do we have a plan to implement Kafka RackAffinityGroupBalancer ? #26

Derfirm opened this issue Feb 25, 2025 · 1 comment

Comments

@Derfirm
Copy link

Derfirm commented Feb 25, 2025

I'd like to configure my application (currently EKS, but is should be configurable) and use rack-affinity
link to the KIP
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1123:+Rack-aware+partitioning+for+Kafka+Producer

also example of implementation is here
https://github.com/segmentio/kafka-go/blob/main/groupbalancer.go

@stewartboyd119
Copy link
Member

This library wouldn't take on that complexity. zkafka offloads the consumer assignment to confluent-kafka-go (which uses librdkafka. They have a couple of configurable strategies that can be used (see config.md in librdkafka). As of today that includes range, roundrobin, cooperativesticky.

zkafka enables these configs to be specified, but if memory serves, isn't well suited for cooperativesticky. If confluent-kafka-go library adds rackaffinity as a configurable strategy, I'd be happy to look into how to make it available (and work well) with in zkafka.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants