Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(server): optimize writing performance #1530

Merged
merged 3 commits into from
Feb 14, 2025
Merged

perf(server): optimize writing performance #1530

merged 3 commits into from
Feb 14, 2025

Conversation

hubcio
Copy link
Contributor

@hubcio hubcio commented Feb 14, 2025

This commit optimizes the log writing process by removing the
stream_position() call, which uses lseek and negatively
impacts performance. Afterwards, the new implementation
directly updates the log file size using fetch_add, resulting
in a 40% performance improvement on writing (Linux).
On MacOS, degradation was not visible.

spetz
spetz previously approved these changes Feb 14, 2025
@coveralls
Copy link

coveralls commented Feb 14, 2025

Pull Request Test Coverage Report for Build 13337959792

Details

  • 26 of 30 (86.67%) changed or added relevant lines in 10 files are covered.
  • 2 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.01%) to 74.75%

Changes Missing Coverage Covered Lines Changed/Added Lines %
server/src/streaming/segments/logs/persister_task.rs 11 15 73.33%
Files with Coverage Reduction New Missed Lines %
server/src/streaming/segments/logs/persister_task.rs 1 50.32%
server/src/streaming/systems/personal_access_tokens.rs 1 77.52%
Totals Coverage Status
Change from base Build 13337166152: 0.01%
Covered Lines: 25243
Relevant Lines: 33770

💛 - Coveralls

This commit optimizes the log writing process by removing the
stream_position() call, which uses lseek and negatively
impacts performance. Afterwards, the new implementation
directly updates the log file size using fetch_add, resulting
in a 40% performance improvement on writing (Linux).
On MacOS, degradation was not visible.
spetz
spetz previously approved these changes Feb 14, 2025
spetz
spetz previously approved these changes Feb 14, 2025
@hubcio hubcio merged commit 4adba54 into master Feb 14, 2025
17 checks passed
@hubcio hubcio deleted the remove-lseek branch February 14, 2025 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants