Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

httpjson: add event publishing metrics #42340

Open
andrewkroh opened this issue Jan 17, 2025 · 2 comments · May be fixed by #42442
Open

httpjson: add event publishing metrics #42340

andrewkroh opened this issue Jan 17, 2025 · 2 comments · May be fixed by #42442
Assignees
Labels
enhancement Filebeat Filebeat Team:Security-Service Integrations Security Service Integrations Team

Comments

@andrewkroh
Copy link
Member

Describe the enhancement:

The CEL input provides metrics (docs) about the number of events published and some details about the amount of time it took to publish the data (which gives a signal about amount of time to ACK). For example, CEL has metrics like

Metric Description
batches_published_total Number of event arrays published.
events_published_total Number of events published.
batch_processing_time Histogram of the elapsed successful batch processing times in nanoseconds (time of receipt to time of ACK for non-empty batches).

I would like to have similar metrics available for httpjson.

Describe a specific use case for the enhancement or feature:

  • In diagnostic dumps from Agent in input_metrics.json, I would like to have a metric that shows the total number of events published. Batch stats would be useful too.
@andrewkroh andrewkroh added enhancement Filebeat Filebeat Team:Security-Service Integrations Security Service Integrations Team labels Jan 17, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@efd6 efd6 self-assigned this Jan 28, 2025
@efd6
Copy link
Contributor

efd6 commented Jan 28, 2025

There is a good correspondence between events published in the two inputs, but batches published in CEL does not have a good equivalent in HTTPJSON; the closest is probably pages published. This would mean that the existing httpjson_interval_pages_execution_time would satisfy the batch_processing_time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Filebeat Filebeat Team:Security-Service Integrations Security Service Integrations Team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants