We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Describe the bug
Currently, goflow produces to single partition. I tried enabling hashing in order to achieve random partitioning but it seems to have no effect.
To Reproduce
-transport.kafka.hashing=true
(
-transport.kafka.brokers=... \ -transport.kafka.version=3.5.0 \ -transport.kafka.hashing=1 \ -format=bin \ -listen 'sflow://:1234?count=16'
)
goflow2/transport/kafka/kafka.go
Line 156 in a1750f5
But maybe the key is not nil by default? 🤔
https://github.com/IBM/sarama/blob/d09a287f30bdf73b1c9f53c3f7a4b2ecc2bd41d3/partitioner.go#L171-L174
Expected behavior
Messages produces to all partitions in the topic randomly
GoFlow2:
The text was updated successfully, but these errors were encountered:
It also seems setting transport.kafka.hashing=1 is redundant as the NewHashPartitioner is default
transport.kafka.hashing=1
Sorry, something went wrong.
Thanks for the report! will have a look
May be the type ProtoProducerMessage struct function baseKey(h hash.Hash) had m.formatter == nil || len(m.formatter.key) == 0.
type ProtoProducerMessage struct
baseKey(h hash.Hash)
m.formatter == nil || len(m.formatter.key) == 0
Hoping to fix with #246
Should be fixed. Let me know if still having the issue.
No branches or pull requests
Describe the bug
Currently, goflow produces to single partition. I tried enabling hashing in order to achieve random partitioning but it seems to have no effect.
To Reproduce
-transport.kafka.hashing=true
(
)
goflow2/transport/kafka/kafka.go
Line 156 in a1750f5
But maybe the key is not nil by default? 🤔
https://github.com/IBM/sarama/blob/d09a287f30bdf73b1c9f53c3f7a4b2ecc2bd41d3/partitioner.go#L171-L174
Expected behavior
Messages produces to all partitions in the topic randomly
GoFlow2:
The text was updated successfully, but these errors were encountered: