OTel: Map user fields to semver fields instead of labels

[marclop]

Map the semconv user.* attributes to ECS user.* fields in APM Server. This may be considered a breaking change since it changes the destination fields in the indexed documents.

[ericywl]

Few questions to consider here:

• It seems the OTel user.* attributes are still experimental. Should we wait until its stable?
• We also previously added process.owner into APMEvent.User.Name. If we fix this issue, it will overwrite the previous value. Not sure how we should handle this?

[simitt]

It seems the OTel user.* attributes are still experimental. Should we wait until its stable?

We support several experimental attributes, so I don't see a problem with adding it already.

We also previously added process.owner into APMEvent.User.Name. If we fix this issue, it will overwrite the previous value. Not sure how we should handle this?

Setting process.owner to user seems a bit odd. IMO we should find a different solution for the process.owner. Other process.* fields are stored as system.process.* , I can't find any hints why owner was treated differently and would suggest to add as system.process.user.* fields.
Let's wait whether @lahsivjar or @carsonip have any more input here on anything I might overlook.

[carsonip]

Setting process.owner to user seems a bit odd. IMO we should find a different solution for the process.owner. Other process.* fields are stored as system.process.* , I can't find any hints why owner was treated differently and would suggest to add as system.process.user.* fields.

Nice catch. Instead of mapping process.owner to user.name, it should actually be process.user.name. I'd consider this a bug, if can confirm. See https://www.elastic.co/guide/en/ecs/current/ecs-process.html

I've also left some comments on the PR.

[lahsivjar]

IIRC, user.name is used in the processes list in the Kibana UI (a quick search returned this but I didn't dig deeper into this). We might need to talk with the UI team if we decide to look for an alternative for process.owner.

[ericywl]

It seems like elastic/opentelemetry-lib#42 also adds process.owner as user.name, so just changing it on apm-data might not be enough.

[carsonip]

IIRC, user.name is used in the processes list in the Kibana UI (a quick search returned this but I didn't dig deeper into this). We might need to talk with the UI team if we decide to look for an alternative for process.owner.

Good call. We'll need to align with UI team if we change where process.owner is stored, and there might be backcompat implications.

[simitt]

ugh, good catch
I wonder if it is worth the effort, given the overall direction with otel native. If customers are eager to have the user.* fields mapped, we can offer them docs for how to achieve adding the mapping themselves with ingest pipelines.

[lahsivjar]

If customers are eager to have the user.* fields mapped, we can offer them docs for how to achieve adding the mapping themselves with ingest pipelines.

I agree with this approach. At this point, we would need to investigate a lot more on the overall impact of making a change with already mapped fields, and given our approach to OTel native, this doesn't feel like a high-priority requirement at the moment.

[simitt]

Let's close this then and not move forward with this change.