Custom HTTP correlation support

[stijnmoreels]

Currently, we don't allow custom HTTP correlation. This, because we wanted to have a stable functionality when Web API and Messaging is used at the same time. We created dedicated interfaces for this so the application always gets the right kind of correlation info accessor.

But, we received a request to use custom HTTP correlation by @carolinasantos0. This raises the question: should we come up with a way to allow custom HTTP correlation, while still being strict when using with Messaging?
We could make our dedicated interfaces an open-generic one, and make sure that the custom correlation model is either inheriting from HttpCorrelationInfo or MessageCorrelationInfo, so we're still always sure from where the correlation comes.

[stijnmoreels]

@fgheysels @pim-simons @gverstraete @meersschautarnaud, would this be a valuable addition?
Currently, our correlation in web API can't be extended so you only have the operation ID and the transaction ID. The request from Carolina, though, is the first time we encountered something was maybe missing.

[gverstraete]

Although, we should try to stick to standard (which we currently do I thought?) - This could be an interesting addition. But be careful to then also allow this in the arcus.observability as this will then be the next struggle. And over there it might not be that easy?

[stijnmoreels]

We already have support for custom correlation support in observability. That repo can actually be seen as an abstraction, while web API and messaging are concrete implementations. The question is if we should allow the custom-support bubble up from observability to HTTP and (possibly?) messaging.
We do would make it more complex in HTTP and messaging and combinations when accessing the correlation information within your applications.

[fgheysels]

I think extensibility is always a good thing. Your suggestion regarding the interfaces makes sense.
But I'm wondering: what is currently missing in the standard correlation implementation that a custom correlation is required ? What are the exact requirements of @carolinasantos22 ?

I might be wrong, but I think building a custom correlation implementation is quite hard ?

[stijnmoreels]

What are the exact requirements of @carolinasantos22 ?

She wanted to add a custom Correlation ID to the response headers, besides the Operation ID and Transaction ID. Since our HTTP correlation model is closed, that can't be done (yet). But our observability correlation model isn't closed and can be extended.

I might be wrong, but I think building a custom correlation implementation is quite hard ?

I think the hardest parts are already done, which is done in the observability. We should come up with a way to integrate the custom correlation model into Serilog. We could just document that they should write their own enricher (or extend the current one). We could make the HTTP correlation interface open generic so that accessing correlation information in your application still stays somewhat straightforward.