New Blog Post: Collecting file-based Java logs with OpenTelemetry

[cyrille-leclerc]

Title of the blog post

Collecting file-based Java logs with OpenTelemetry

Authors

@cyrille-leclerc, @zeitlinger

Short description of the blog post

This blog post shows how the experimental-otlp/stdout export format recently introduced in OTel Java Instrumentation (see otel-java#6675) enables a turnkey ingestion pipeline when logs can't be transported through OTLP.

Prior to experimental-otlp/stdout, practitioners had to combine painful enrichment & formatting of logs in their Java app with cumbersome parsing rules in the OTel Collector.

Outline of blog post

• Problem description: the challenges to use file based logs for Java instrumentation with OTel
• Solution description: OTLP/JSON logs is now a turnkey feature both in the OTel Java Instrumentation and in the OTel Collector
• Reference architecture: architecture diagram including a Java app instrumented with OTel, the routing of OTel traces, metrics, and logs, and the OTel Collector pipelines
• Step by step guide to setup OTLP/JSON logs through stdout coverign both the Java app and the OTel Collector
• Step by step guide to do the same in Kubernetes, adding to the steps above the parsing of the container log prefixes
• Conclusion

Technology used

• OTel Java Instrumentation with SLF4J/Logback instrumentation and the experimental-otlp/stdout export format.
• OTel Collector
  • File Log receiver, including the container parser
  • OTLP/JSON connector and

SIG

• Java for
  • add stdout log record exporter opentelemetry-java#6675
• Specification for
  • Add OTLP stdout exporter opentelemetry-specification#4183

Sponsor

[svrnm]

Thanks @cyrille-leclerc, looks good to me overall!

@open-telemetry/java-maintainers please take a look

[jack-berg]

Makes sense to me. 👍

[svrnm]

awesome. then let's go for it.

to make things easy, I assume that @zeitlinger is the sponsor for this PR and provides a review accordingly.