Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(types): 3x speedup MakePartSet (#3117) (backport #83) #84

Merged
merged 3 commits into from
May 31, 2024

Conversation

mergify[bot]
Copy link

@mergify mergify bot commented May 29, 2024

This PR adds some benchmarks, and significantly speeds up types.MakePartSet, and Partset.AddPart. (Used by the block proposer, and every consensus instance) It does so by doing two things:

  • Saving mutexes on the newly created bit array, by defaulting every value to True (rather than setting it in a loop that goes through a mutex)
  • Uses the same hash object throughout, and avoids an extra copy of every leaf. (main speedup)

I do the same hash optimization for proof.Verify, which is used in the add block part codepath for both the proposer and every full node.

New:

BenchmarkMakePartSet/nParts=1-12         	   38616	     29817 ns/op	     568 B/op	      12 allocs/op
BenchmarkMakePartSet/nParts=2-12         	   19888	     59866 ns/op	    1000 B/op	      22 allocs/op
BenchmarkMakePartSet/nParts=3-12         	   12979	     95691 ns/op	    1528 B/op	      33 allocs/op
BenchmarkMakePartSet/nParts=4-12         	    8688	    128192 ns/op	    2024 B/op	      44 allocs/op
BenchmarkMakePartSet/nParts=5-12         	    7308	    155224 ns/op	    2888 B/op	      57 allocs/op

Old:

BenchmarkMakePartSet/nParts=1-12         	   16647	    106545 ns/op	   74169 B/op	      12 allocs/op
BenchmarkMakePartSet/nParts=2-12         	   10000	    106361 ns/op	  148329 B/op	      23 allocs/op
BenchmarkMakePartSet/nParts=3-12         	    6992	    337644 ns/op	  222587 B/op	      35 allocs/op
BenchmarkMakePartSet/nParts=4-12         	    3488	    480109 ns/op	  296811 B/op	      47 allocs/op
BenchmarkMakePartSet/nParts=5-12         	    2228	    557768 ns/op	  371404 B/op	      61 allocs/op

System wide, this is definitely not our issue (looks like roughly .1ms per blockpart), but still definitely useful time to remove


  • Tests written/updated
  • Changelog entry added in .changelog (we use unclog to manage our changelog)
  • Updated relevant documentation (docs/ or spec/) and code comments
  • Title follows the Conventional Commits spec

PR checklist

  • Tests written/updated
  • Changelog entry added in .changelog (we use unclog to manage our changelog)
  • Updated relevant documentation (docs/ or spec/) and code comments

This is an automatic backport of pull request #83 done by [Mergify](https://mergify.com).

ValarDragon and others added 2 commits May 29, 2024 14:13
<!--

Please add a reference to the issue that this PR addresses and indicate
which
files are most critical to review. If it fully addresses a particular
issue,
please include "Closes #XXX" (where "XXX" is the issue number).

If this PR is non-trivial/large/complex, please ensure that you have
either
created an issue that the team's had a chance to respond to, or had some
discussion with the team prior to submitting substantial pull requests.
The team
can be reached via GitHub Discussions or the Cosmos Network Discord
server in
the #cometbft channel. GitHub Discussions is preferred over Discord as
it
allows us to keep track of conversations topically.
https://github.com/cometbft/cometbft/discussions

If the work in this PR is not aligned with the team's current
priorities, please
be advised that it may take some time before it is merged - especially
if it has
not yet been discussed with the team.

See the project board for the team's current priorities:
https://github.com/orgs/cometbft/projects/1

-->

This PR adds some benchmarks, and significantly speeds up
types.MakePartSet, and Partset.AddPart. (Used by the block proposer, and
every consensus instance) It does so by doing two things:
- Saving mutexes on the newly created bit array, by defaulting every
value to True (rather than setting it in a loop that goes through a
mutex)
- Uses the same hash object throughout, and avoids an extra copy of
every leaf. (main speedup)

I do the same hash optimization for proof.Verify, which is used in the
add block part codepath for both the proposer and every full node.

New:
```
BenchmarkMakePartSet/nParts=1-12         	   38616	     29817 ns/op	     568 B/op	      12 allocs/op
BenchmarkMakePartSet/nParts=2-12         	   19888	     59866 ns/op	    1000 B/op	      22 allocs/op
BenchmarkMakePartSet/nParts=3-12         	   12979	     95691 ns/op	    1528 B/op	      33 allocs/op
BenchmarkMakePartSet/nParts=4-12         	    8688	    128192 ns/op	    2024 B/op	      44 allocs/op
BenchmarkMakePartSet/nParts=5-12         	    7308	    155224 ns/op	    2888 B/op	      57 allocs/op
```

Old:
```
BenchmarkMakePartSet/nParts=1-12         	   16647	    106545 ns/op	   74169 B/op	      12 allocs/op
BenchmarkMakePartSet/nParts=2-12         	   10000	    106361 ns/op	  148329 B/op	      23 allocs/op
BenchmarkMakePartSet/nParts=3-12         	    6992	    337644 ns/op	  222587 B/op	      35 allocs/op
BenchmarkMakePartSet/nParts=4-12         	    3488	    480109 ns/op	  296811 B/op	      47 allocs/op
BenchmarkMakePartSet/nParts=5-12         	    2228	    557768 ns/op	  371404 B/op	      61 allocs/op
```

System wide, this is definitely not our issue (looks like roughly .1ms
per blockpart), but still definitely useful time to remove

---

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

(cherry picked from commit 7b1c1f8)
(cherry picked from commit 104624c)

# Conflicts:
#	CHANGELOG.md
@mergify mergify bot added the conflicts label May 29, 2024
Copy link
Author

mergify bot commented May 29, 2024

Cherry-pick of 104624c has failed:

On branch mergify/bp/osmo-v25/v0.37.4/pr-83
Your branch is ahead of 'origin/osmo-v25/v0.37.4' by 1 commit.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 104624c94.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   CHANGELOG.md

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@ValarDragon ValarDragon merged commit 89a346f into osmo-v25/v0.37.4 May 31, 2024
12 of 16 checks passed
@mergify mergify bot deleted the mergify/bp/osmo-v25/v0.37.4/pr-83 branch May 31, 2024 03:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant