DBZ-7921 Kafka offset/history store support

[jcechace]

https://issues.redhat.com/browse/DBZ-7921

[jcechace]

@jpechane This wont make it into 2.7 unfortunately

[ryanvanhuuksloot]

👋 Hello,

I just wanted to hop into this issue (I haven't setup Redhat JIRA yet) and say we'd love to help / test this new feature at Shopify. We've been testing the operator and like the direction - this is a big feature so we're around to support.

[jcechace]

@ryanvanhuuksloot Hey Ryan, it's great hearing that you are testing the operator. I'd be happy hearing any feedback you can provide, or any ideas/request in terms of functionality you might have. Of course PR are definitively welcomed. In any case this will be available soon -- it's nothing problematic, I just haven't gotten around to implementing it for 2.7.

Note that this feature is really more about the cosmetics than anything else -- technically you should be able to just put the configuration under spec.source.config and it would work.

[ryanvanhuuksloot]

I'd be happy hearing any feedback you can provide, or any ideas/request in terms of functionality you might have.

I'd love to setup some time to discuss how we are thinking about using the operator. We have some fundamental requirements that we must have to fully migrate but we also see the potential of the operator. I imagine our team can really leverage and contribute to this system if roadmaps align. Would you be open to putting something in the calendar with a few folks from our team (we are primarily EST)?

Of course PR are definitively welcomed.

Amazing - as we continue to test we'll likely be coming back with bugs/fixes.

In any case this will be available soon -- it's nothing problematic, I just haven't gotten around to implementing it for 2.7.
Note that this feature is really more about the cosmetics than anything else -- technically you should be able to just put the configuration under spec.source.config and it would work.

No problem. I didn't realize it was possible. I've had very mixed success with other offset types but I'll try with Kafka and the "less than ideal" config. Or I'll do a custom build with this PR :)

[jcechace]

Or I'll do a custom build with this PR :)

Once it is resolved. Right now this is just a preparation change which was required. I will try to find the time this week and have this resolved in full.

Would you be open to putting something in the calendar with a few folks from our team (we are primarily EST)?

Anything in the morning (EST) should work.

[jcechace]

@ryanvanhuuksloot

I've had very mixed success with other offset types but I'll try with Kafka

Perhaps this is something you could elaborate on during our call.

[obabec]

Looks good from my POV, although we have to prepare some tests to verify this.