Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve retry logic for Bulk Requests #1905

Open
gareth-ellis opened this issue Jan 21, 2025 · 0 comments
Open

Improve retry logic for Bulk Requests #1905

gareth-ellis opened this issue Jan 21, 2025 · 0 comments
Labels
enhancement Improves the status quo :Usability Makes Rally easier to use

Comments

@gareth-ellis
Copy link
Member

When trying to push Elasticsearch to extremes, some requests may not succeed due to e.g circuit breakers triggering 429s amongst other things.

To ensure that we ingest all documents that we planned to, we need to look at improving the robustness of the BulkIndex action.

Currently rally relies on the elasitcsearch client for retries - as such most runners don't retry - only certain runners that are not normally really part of the benchmark, but rather just setting up configuration / waiting for cluster to be healthy etc.

One approach is #1868 this is aimed at document level 429s - where a single request may return as 200, but certain documents maybe failed to ingest due to CBE or similar.

Where we have an entire request that fails, we could look at wrapping Bulk in Retry - https://github.com/elastic/rally/blob/master/esrally/driver/runner.py#L2953

@gareth-ellis gareth-ellis added enhancement Improves the status quo :Usability Makes Rally easier to use labels Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improves the status quo :Usability Makes Rally easier to use
Projects
None yet
Development

No branches or pull requests

1 participant