[O11y][MySQL] Rally benchmark `mysql.error` #8957

ali786XI · 2024-01-24T09:23:24Z

Proposed commit message

This PR adds benchmarking templates to the error data stream of MySQL

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.

How to test this PR locally

Run this command from package root

elastic-package benchmark rally --benchmark error-benchmark -v
elastic-package benchmark stream --benchmark error-benchmark -v

Related issues

Relates [Rally] Add benchmarking templates to data streams for set of packages #8719

Screenshots

--- Benchmark results for package: mysql - START ---
╭────────────────────────────────────────────────────────────────────────────────────╮
│ info                                                                               │
├────────────────────────┬───────────────────────────────────────────────────────────┤
│ benchmark              │                                           error-benchmark │
│ description            │               Benchmark 20000 mysql.error events ingested │
│ run ID                 │                      1431c4ca-552c-473e-9041-13ff6af5e7bf │
│ package                │                                                     mysql │
│ start ts (s)           │                                                1706088097 │
│ end ts (s)             │                                                1706088130 │
│ duration               │                                                       33s │
│ generated corpora file │ /root/.elastic-package/tmp/rally_corpus/corpus-2476068247 │
╰────────────────────────┴───────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────╮
│ parameters                                                          │
├─────────────────────────────────┬───────────────────────────────────┤
│ package version                 │                            1.17.0 │
│ data_stream.name                │                             error │
│ corpora.generator.total_events  │                             20000 │
│ corpora.generator.template.path │ ./error-benchmark/template.ndjson │
│ corpora.generator.template.raw  │                                   │
│ corpora.generator.template.type │                            gotext │
│ corpora.generator.config.path   │      ./error-benchmark/config.yml │
│ corpora.generator.config.raw    │                             map[] │
│ corpora.generator.fields.path   │      ./error-benchmark/fields.yml │
│ corpora.generator.fields.raw    │                             map[] │
╰─────────────────────────────────┴───────────────────────────────────╯
╭───────────────────────╮
│ cluster info          │
├───────┬───────────────┤
│ name  │ elasticsearch │
│ nodes │             1 │
╰───────┴───────────────╯
╭──────────────────────────────────────────────────╮
│ data stream stats                                │
├────────────────────────────┬─────────────────────┤
│ data stream                │ logs-mysql.error-ep │
│ approx total docs ingested │               20000 │
│ backing indices            │                   1 │
│ store size bytes           │             2797082 │
│ maximum ts (ms)            │       1706088093000 │
╰────────────────────────────┴─────────────────────╯
╭───────────────────────────────────────╮
│ disk usage for index .ds-logs-mysql.e │
│ rror-ep-2024.01.24-000001 (for all fi │
│ elds)                                 │
├──────────────────────────────┬────────┤
│ total                        │ 2.5 MB │
│ inverted_index.total         │ 562 kB │
│ inverted_index.stored_fields │ 1.6 MB │
│ inverted_index.doc_values    │ 178 kB │
│ inverted_index.points        │ 167 kB │
│ inverted_index.norms         │    0 B │
│ inverted_index.term_vectors  │    0 B │
│ inverted_index.knn_vectors   │    0 B │
╰──────────────────────────────┴────────╯
╭──────────────────────────────────────────────────────────────────────────────╮
│ pipeline logs-mysql.error-1.17.0 stats in node 8obINwBVTpalSDJ2GV3UVQ        │
├────────────────────────────────────┬─────────────────────────────────────────┤
│ Totals                             │ Count: 20000 | Failed: 0 | Time: 1.116s │
│ set ()                             │   Count: 20000 | Failed: 0 | Time: 88ms │
│ set ()                             │    Count: 20000 | Failed: 0 | Time: 8ms │
│ set ()                             │   Count: 20000 | Failed: 0 | Time: 30ms │
│ grok ()                            │  Count: 20000 | Failed: 0 | Time: 237ms │
│ grok ()                            │   Count: 20000 | Failed: 0 | Time: 49ms │
│ rename ()                          │   Count: 20000 | Failed: 0 | Time: 20ms │
│ date ()                            │         Count: 0 | Failed: 0 | Time: 0s │
│ date ()                            │  Count: 20000 | Failed: 0 | Time: 509ms │
│ date ()                            │         Count: 0 | Failed: 0 | Time: 0s │
│ remove ()                          │    Count: 20000 | Failed: 0 | Time: 8ms │
│ set ()                             │   Count: 20000 | Failed: 0 | Time: 11ms │
│ append ()                          │    Count: 20000 | Failed: 0 | Time: 9ms │
│ append ()                          │    Count: 20000 | Failed: 0 | Time: 7ms │
│ append ()                          │     Count: 4938 | Failed: 0 | Time: 3ms │
│ remove ()                          │   Count: 20000 | Failed: 0 | Time: 22ms │
│ pipeline (logs-mysql.error@custom) │    Count: 20000 | Failed: 0 | Time: 2ms │
╰────────────────────────────────────┴─────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ rally stats                                                                                 │
├────────────────────────────────────────────────────────────────┬────────────────────────────┤
│ Cumulative indexing time of primary shards                     │                0.42195 min │
│ Min cumulative indexing time across primary shards             │                      0 min │
│ Median cumulative indexing time across primary shards          │                 0.0019 min │
│ Max cumulative indexing time across primary shards             │    0.26306666666666667 min │
│ Cumulative indexing throttle time of primary shards            │                      0 min │
│ Min cumulative indexing throttle time across primary shards    │                      0 min │
│ Median cumulative indexing throttle time across primary shards │                      0 min │
│ Max cumulative indexing throttle time across primary shards    │                      0 min │
│ Cumulative merge time of primary shards                        │    0.04953333333333333 min │
│ Cumulative merge count of primary shards                       │                         39 │
│ Min cumulative merge time across primary shards                │                      0 min │
│ Median cumulative merge time across primary shards             │  0.0005833333333333334 min │
│ Max cumulative merge time across primary shards                │   0.010483333333333334 min │
│ Cumulative merge throttle time of primary shards               │                      0 min │
│ Min cumulative merge throttle time across primary shards       │                      0 min │
│ Median cumulative merge throttle time across primary shards    │                      0 min │
│ Max cumulative merge throttle time across primary shards       │                      0 min │
│ Cumulative refresh time of primary shards                      │    0.12648333333333334 min │
│ Cumulative refresh count of primary shards                     │                       1625 │
│ Min cumulative refresh time across primary shards              │                      0 min │
│ Median cumulative refresh time across primary shards           │  0.0013333333333333333 min │
│ Max cumulative refresh time across primary shards              │    0.05026666666666667 min │
│ Cumulative flush time of primary shards                        │     0.6471833333333333 min │
│ Cumulative flush count of primary shards                       │                       1389 │
│ Min cumulative flush time across primary shards                │ 1.6666666666666667e-05 min │
│ Median cumulative flush time across primary shards             │                 0.0228 min │
│ Max cumulative flush time across primary shards                │   0.056716666666666665 min │
│ Total Young Gen GC time                                        │                    0.019 s │
│ Total Young Gen GC count                                       │                          2 │
│ Total Old Gen GC time                                          │                        0 s │
│ Total Old Gen GC count                                         │                          0 │
│ Store size                                                     │     0.01303890347480774 GB │
│ Translog size                                                  │    0.024319728836417198 GB │
│ Heap used for segments                                         │                       0 MB │
│ Heap used for doc values                                       │                       0 MB │
│ Heap used for terms                                            │                       0 MB │
│ Heap used for norms                                            │                       0 MB │
│ Heap used for points                                           │                       0 MB │
│ Heap used for stored fields                                    │                       0 MB │
│ Segment count                                                  │                        311 │
│ Total Ingest Pipeline count                                    │                      20022 │
│ Total Ingest Pipeline time                                     │                    1.841 s │
│ Total Ingest Pipeline failed                                   │                          0 │
│ Min Throughput                                                 │             3258.36 docs/s │
│ Mean Throughput                                                │             4067.98 docs/s │
│ Median Throughput                                              │             4214.20 docs/s │
│ Max Throughput                                                 │             4585.17 docs/s │
│ 50th percentile latency                                        │       94.85651151044294 ms │
│ 90th percentile latency                                        │      112.31609176611528 ms │
│ 100th percentile latency                                       │      187.14893993455917 ms │
│ 50th percentile service time                                   │       94.85651151044294 ms │
│ 90th percentile service time                                   │      112.31609176611528 ms │
│ 100th percentile service time                                  │      187.14893993455917 ms │
│ error rate                                                     │                     0.00 % │
╰────────────────────────────────────────────────────────────────┴────────────────────────────╯

--- Benchmark results for package: mysql - END   ---
Done

elasticmachine · 2024-01-24T09:44:32Z

🚀 Benchmarks report

Package `mysql` 👍(0) 💚(0) 💔(2)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
`error`	12987.01	4444.44	-8542.57 (-65.78%)	💔
`slowlog`	18518.52	9615.38	-8903.14 (-48.08%)	💔

To see the full report comment with /test benchmark fullreport

ali786XI · 2024-02-02T07:51:26Z

packages/mysql/_dev/benchmark/rally/error-benchmark/template.ndjson

+    "version": "8.8.0",
+    "snapshot": false
+  },
+  "message": "{{$timestamp.Format "2006-01-02 15:04:05"}} {{ $thread_id }} [{{ $log_level }}] {{ generate "message" }}",


Given that we need to improve the message generated in logs as per this, I couldn't find any fixed format over the error logs. Please refer here in case.

mysqld_safe Starting mysqld daemon with databases from /usr/local/var/mysql

InnoDB: The InnoDB memory heap is disabled

InnoDB: Mutexes and rw_locks use GCC atomic builtins

Should we go like generating an enum of these messages which would have around 10 to 12 error logs?

i think having somethign more realistic would be good if it's not too much effort.

Thanks. Provided the messages as per the log_level. Please have a look

tommyers-elastic · 2024-02-12T12:22:07Z

packages/mysql/_dev/benchmark/rally/error-benchmark/config.yml

+    enum: ["Note", "Warning", "System", "ERROR"]
+  - name: message
+    type: text
+  - name: thread_id


would we expect this to have such high cardinality in real life?

@tommyers-elastic Are you referring the range of values this field is generating? I agree that the values are not that high in the logs here. Let me update this to maybe max value as 20-30.

… into mysql_benchmark_error

elasticmachine · 2024-02-13T06:22:38Z

💚 Build Succeeded

Buildkite Build
Commit: 6be93d2

History

💚 Build #8867 succeeded efb1c11
💚 Build #8359 succeeded 105a4b2
💚 Build #8308 succeeded 99bbe8b

cc @aliabbas-elastic

elastic-sonarqube · 2024-02-13T06:22:39Z

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

tommyers-elastic

nice work - thanks!

rally benchmark mysql.error

99bbe8b

ali786XI added enhancement New feature or request Integration:mysql MySQL labels Jan 24, 2024

ali786XI self-assigned this Jan 24, 2024

ali786XI requested review from tommyers-elastic and milan-elastic January 24, 2024 09:24

ali786XI marked this pull request as ready for review January 24, 2024 09:25

ali786XI requested a review from a team as a code owner January 24, 2024 09:25

This was referenced Jan 24, 2024

[Rally] Add benchmarking templates to data streams for set of packages #8719

Closed

[Meta] Add more benchmark schema templates to integrations #8636

Closed

update timestamp

105a4b2

milan-elastic approved these changes Jan 31, 2024

View reviewed changes

ali786XI commented Feb 2, 2024

View reviewed changes

tommyers-elastic reviewed Feb 12, 2024

View reviewed changes

ali786XI added 3 commits February 12, 2024 19:43

update range

efb1c11

improve error messages

7fb4bbd

Merge branch 'main' of https://github.com/aliabbas-elastic/integrations…

6be93d2

… into mysql_benchmark_error

ali786XI requested a review from tommyers-elastic February 13, 2024 05:59

tommyers-elastic approved these changes Feb 13, 2024

View reviewed changes

ali786XI merged commit 94d1cea into elastic:main Feb 15, 2024
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[O11y][MySQL] Rally benchmark `mysql.error` #8957

[O11y][MySQL] Rally benchmark `mysql.error` #8957

ali786XI commented Jan 24, 2024

elasticmachine commented Jan 24, 2024 •

edited

Loading

ali786XI Feb 2, 2024

tommyers-elastic Feb 12, 2024

ali786XI Feb 13, 2024

tommyers-elastic Feb 12, 2024

ali786XI Feb 12, 2024

elasticmachine commented Feb 13, 2024

elastic-sonarqube bot commented Feb 13, 2024

tommyers-elastic left a comment

[O11y][MySQL] Rally benchmark mysql.error #8957

[O11y][MySQL] Rally benchmark mysql.error #8957

Conversation

ali786XI commented Jan 24, 2024

Proposed commit message

Checklist

How to test this PR locally

Related issues

Screenshots

elasticmachine commented Jan 24, 2024 • edited Loading

🚀 Benchmarks report

Package mysql 👍(0) 💚(0) 💔(2)

ali786XI Feb 2, 2024

Choose a reason for hiding this comment

tommyers-elastic Feb 12, 2024

Choose a reason for hiding this comment

ali786XI Feb 13, 2024

Choose a reason for hiding this comment

tommyers-elastic Feb 12, 2024

Choose a reason for hiding this comment

ali786XI Feb 12, 2024

Choose a reason for hiding this comment

elasticmachine commented Feb 13, 2024

💚 Build Succeeded

History

elastic-sonarqube bot commented Feb 13, 2024

Quality Gate passed

tommyers-elastic left a comment

Choose a reason for hiding this comment

[O11y][MySQL] Rally benchmark `mysql.error` #8957

[O11y][MySQL] Rally benchmark `mysql.error` #8957

elasticmachine commented Jan 24, 2024 •

edited

Loading

Package `mysql` 👍(0) 💚(0) 💔(2)