feat: use otel in core metrics

[jjaakola-aiven]

About this change - What it does

References: #xxxxx

Why this way

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
src/karapace
__main__.py					9, 22, 29, 41-71
rapu.py
statsd.py
src/karapace/api
container.py
controller.py
factory.py					18-23, 40, 47
user.py
src/karapace/api/routers
compatibility.py					15
config.py
mode.py					15
schemas.py					15
subjects.py					16
src/karapace/api/telemetry
container.py
metrics.py
setup.py					38
src/karapace/core
auth.py
auth_container.py
metrics_container.py
schema_reader.py
schema_registry.py
stats.py					62-65, 70, 73, 76-77, 80-88, 91
Project Total

This report was generated by python-coverage-comment-action

[nosahama]

I like the implementation, really nice, also shows how the DI containers simplify things. It's nice that you added dedicated containers.
[nit]

• since we now have 2 containers in the core package, maybe we want to further group the container modules (auth_container and metrics_container) under a single package.
• maybe we need to document somewhere that the config tags are added to metric labels and issue tracking.
• would be nice to see the actual calls in the tests for the stats client.
• there were also some commented code.
• we no longer needstatsd-exporter.
• grafana dashboard is broken, as we now have _total added to the metric names and thankfully we've also now changed the name of the exception metric 😅 , I can fix the dashboard in another PR.