Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] StandardVersusLogsStoredSourceChallengeRestIT.testMatchAllQuery failing with OOME #120432

Closed
elasticsearchmachine opened this issue Jan 18, 2025 · 3 comments
Closed

[CI] StandardVersusLogsStoredSourceChallengeRestIT.testMatchAllQuery failing with OOME #120432

elasticsearchmachine opened this issue Jan 18, 2025 · 3 comments
Assignees
@kkrik-es
Labels
needs:risk Requires assignment of a risk label (low, medium, blocker) :StorageEngine/Logs You know, for Logs Team:StorageEngine >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

Build Scans:

Reproduction Line:

./gradlew ":x-pack:plugin:logsdb:javaRestTest" --tests "org.elasticsearch.xpack.logsdb.qa.StandardVersusLogsStoredSourceChallengeRestIT.testHistogramAggregation" -Dtests.seed=4CAA7452D418406E -Dtests.locale=ff-Latn-SN -Dtests.timezone=Africa/Malabo -Druntime.java=17 -Dtests.fips.enabled=true

Applicable branches:
8.17

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.RuntimeException: Request execution cancelled

Issue Reasons:

  • [8.17] 2 failures in class org.elasticsearch.xpack.logsdb.qa.StandardVersusLogsStoredSourceChallengeRestIT (0.8% fail rate in 244 executions)
  • [8.17] 2 failures in step openjdk17_checkpart2_java-fips-matrix (22.2% fail rate in 9 executions)
  • [8.17] 2 failures in pipeline elasticsearch-periodic (22.2% fail rate in 9 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.
@elasticsearchmachine elasticsearchmachine added :StorageEngine/Logs You know, for Logs >test-failure Triaged test failures from CI Team:StorageEngine needs:risk Requires assignment of a risk label (low, medium, blocker) labels Jan 18, 2025
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@kkrik-es kkrik-es self-assigned this Jan 20, 2025
@kkrik-es
Copy link
Contributor

Logs show OOMEs

REPRODUCE WITH: ./gradlew ":x-pack:plugin:logsdb:javaRestTest" --tests "org.elasticsearch.xpack.logsdb.qa.LogsDbVersusLogsDbReindexedIntoStandardModeChallengeRestIT.testMatchAllQuery" -Dtests.seed=4CAA7452D418406E -Dtests.locale=lv -Dtests.timezone=Asia/Singapore -Druntime.java=17 -Dtests.fips.enabled=true

...
       Caused by:
        java.lang.OutOfMemoryError: Java heap space

REPRODUCE WITH: ./gradlew ":x-pack:plugin:logsdb:javaRestTest" --tests "org.elasticsearch.xpack.logsdb.qa.StandardVersusLogsStoredSourceChallengeRestIT.testMatchAllQuery" -Dtests.seed=4CAA7452D418406E -Dtests.locale=ff-Latn-SN -Dtests.timezone=Africa/Malabo -Druntime.java=17 -Dtests.fips.enabled=true

Not sure if anything changed here. We can look into comparing pairs of results directly instead of keeping them all in memory, @lkts wdyt?

@kkrik-es kkrik-es changed the title [CI] StandardVersusLogsStoredSourceChallengeRestIT class failing [CI] StandardVersusLogsStoredSourceChallengeRestIT.testMatchAllQuery class failing with OOME Jan 20, 2025
@kkrik-es kkrik-es changed the title [CI] StandardVersusLogsStoredSourceChallengeRestIT.testMatchAllQuery class failing with OOME [CI] StandardVersusLogsStoredSourceChallengeRestIT.testMatchAllQuery failing with OOME Jan 20, 2025
@kkrik-es kkrik-es mentioned this issue Jan 20, 2025
Merged
elasticsearchmachine pushed a commit that referenced this issue Jan 20, 2025
@kkrik-es
Reduce number of docs in randomized testing for logsdb (#120451)
82295cf 
Related to #120432
kkrik-es added a commit to kkrik-es/elasticsearch that referenced this issue Jan 20, 2025
@kkrik-es
Reduce number of docs in randomized testing for logsdb (elastic#120451)
56060fc 
Related to elastic#120432
elasticsearchmachine pushed a commit that referenced this issue Jan 20, 2025
@kkrik-es
Reduce number of docs in randomized testing for logsdb (#120451) (#12…
9c37e46 
…0452)

Related to #120432
kkrik-es added a commit to kkrik-es/elasticsearch that referenced this issue Jan 20, 2025
@kkrik-es
Reduce number of docs in randomized testing for logsdb (elastic#120451)
aed86b6 
Related to elastic#120432

(cherry picked from commit 82295cf)

# Conflicts:
#	x-pack/plugin/logsdb/src/javaRestTest/java/org/elasticsearch/xpack/logsdb/qa/StandardVersusLogsIndexModeChallengeRestIT.java
@kkrik-es
Copy link
Contributor

Oh 8.17 was using way more docs, we've already reduced the limit in 8.x. This should be good.

kkrik-es added a commit that referenced this issue Jan 20, 2025
@kkrik-es
Reduce number of docs in randomized testing for logsdb (#120451) (#12…
27fa993 
…0460)

Related to #120432

(cherry picked from commit 82295cf)

# Conflicts:
#	x-pack/plugin/logsdb/src/javaRestTest/java/org/elasticsearch/xpack/logsdb/qa/StandardVersusLogsIndexModeChallengeRestIT.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kkrik-es kkrik-es

Labels
needs:risk Requires assignment of a risk label (low, medium, blocker) :StorageEngine/Logs You know, for Logs Team:StorageEngine >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@elasticsearchmachine @kkrik-es