Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[AO] Add integration test to the Threshold Rule #160510

Closed
Tracked by #161237
fkanout opened this issue Jun 26, 2023 · 7 comments · Fixed by #160633
Closed
Tracked by #161237

[AO] Add integration test to the Threshold Rule #160510

fkanout opened this issue Jun 26, 2023 · 7 comments · Fixed by #160633
Labels
Team: Actionable Observability - DEPRECATED For Observability Alerting and SLOs use "Team:obs-ux-management", for AIops "Team:obs-knowledge" v8.10.0

Comments

@fkanout
Copy link
Contributor

fkanout commented Jun 26, 2023

📝 Summary

The Threshold rule code was cloned and refactored completely. A suite of integration tests is required at this stage as we will update the rule inputs and do more refactoring.

✅ AC

  • Write integration tests that create a rule and wait for the alert to fire
  • Verify the fired alerts
@fkanout fkanout added Team: Actionable Observability - DEPRECATED For Observability Alerting and SLOs use "Team:obs-ux-management", for AIops "Team:obs-knowledge" v8.10.0 labels Jun 26, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/actionable-observability (Team: Actionable Observability)

@maryam-saeidi
Copy link
Member

Example of an integration test for metric threshold: #157489

@maryam-saeidi
Copy link
Member

Things to consider while testing:

  1. Having multiple conditions
  2. No data
  3. KQL filtering
  4. Grouping

@katrin-freihofner You already had a list of scenarios for this rule that we could use for testing, I forgot, where can I find them?

@fkanout Just a heads up, I am working on saving action variables in the alert document, so we can postpone checking the action context variables until that work is done to avoid conflict or extra unnecessary work :)

@maryam-saeidi
Copy link
Member

maryam-saeidi commented Jun 26, 2023 •
edited
Loading

@fkanout There is this proposal from @mikecote regarding how to write an integration test for alerting rules.

I wonder if we can try it while working on this ticket, what do you think?

@fkanout fkanout self-assigned this Jun 26, 2023
@fkanout
Copy link
Contributor Author

fkanout commented Jun 26, 2023 •
edited
Loading

@fkanout There is this proposal from @mikecote regarding how to write an integration test for alerting rules.

I wonder if we can try it while working on this ticket, what do you think?

@maryam-saeidi, thanks for sharing. But The proposal is about the E2E test, not the integration(which this issue is about).
For me, E2E is when we involve UI in the tests.
However, conducting the E2E tests using a live Elasticsearch cluster would be great. Do we have an example for that?

@maryam-saeidi
Copy link
Member

I think we use the term E2E for both functional and API integration tests. The example that I shared for metric threshold in the comments above uses Elasticsearch, so it should be fine to follow that and adjust it according to the proposal (unless I am missing something)

@fkanout fkanout mentioned this issue Jun 27, 2023
Merged
@katrin-freihofner
Copy link
Contributor

@katrin-freihofner You already had a list of scenarios for this rule that we could use for testing, I forgot, where can I find them?

I don't think that is applicable here.

fkanout added a commit that referenced this issue Jun 30, 2023
@fkanout
[AO] Add integration test for the Threshold Rule (#160633)
2b45cf6 
## Summary
It fixes #160510 by:
- Adding integration tests for the Threshold Rule.
- Update the consumer value. 
- Update the action group.
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jun 30, 2023
@fkanout
[AO] Add integration test for the Threshold Rule (elastic#160633)
e86122a 
## Summary
It fixes elastic#160510 by:
- Adding integration tests for the Threshold Rule.
- Update the consumer value.
- Update the action group.

(cherry picked from commit 2b45cf6)
kibanamachine added a commit that referenced this issue Jun 30, 2023
@kibanamachine @fkanout
[8.9] [AO] Add integration test for the Threshold Rule (#160633) (#16…
f0ee8c0 
…1006)

# Backport

This will backport the following commits from `main` to `8.9`:
- [[AO] Add integration test for the Threshold Rule
(#160633)](#160633)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Faisal
Kanout","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-06-30T14:11:58Z","message":"[AO]
Add integration test for the Threshold Rule (#160633)\n\n##
Summary\r\nIt fixes #160510 by:\r\n- Adding integration tests for the
Threshold Rule.\r\n- Update the consumer value. \r\n- Update the action
group.","sha":"2b45cf628f1391e7918b23e63df44e23f1c07bcc","branchLabelMapping":{"^v8.10.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:
Actionable
Observability","backport:prev-minor","v8.9.0","v8.10.0"],"number":160633,"url":"https://github.com/elastic/kibana/pull/160633","mergeCommit":{"message":"[AO]
Add integration test for the Threshold Rule (#160633)\n\n##
Summary\r\nIt fixes #160510 by:\r\n- Adding integration tests for the
Threshold Rule.\r\n- Update the consumer value. \r\n- Update the action
group.","sha":"2b45cf628f1391e7918b23e63df44e23f1c07bcc"}},"sourceBranch":"main","suggestedTargetBranches":["8.9"],"targetPullRequestStates":[{"branch":"8.9","label":"v8.9.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.10.0","labelRegex":"^v8.10.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/160633","number":160633,"mergeCommit":{"message":"[AO]
Add integration test for the Threshold Rule (#160633)\n\n##
Summary\r\nIt fixes #160510 by:\r\n- Adding integration tests for the
Threshold Rule.\r\n- Update the consumer value. \r\n- Update the action
group.","sha":"2b45cf628f1391e7918b23e63df44e23f1c07bcc"}}]}]
BACKPORT-->

Co-authored-by: Faisal Kanout <[email protected]>
@maryam-saeidi maryam-saeidi mentioned this issue Jul 31, 2023
Closed
4 tasks
@maryam-saeidi maryam-saeidi changed the title [AO] Add integration test to the Threshold Rule. [AO] Add integration test to the Threshold Rule Aug 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Team: Actionable Observability - DEPRECATED For Observability Alerting and SLOs use "Team:obs-ux-management", for AIops "Team:obs-knowledge" v8.10.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[AO] Add integration test for the Threshold Rule fkanout/kibana
4 participants
@fkanout @maryam-saeidi @elasticmachine @katrin-freihofner