Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(types): 3x speedup MakePartSet (#3117) #83

Merged
merged 2 commits into from
May 29, 2024
Merged

Conversation

ValarDragon
Copy link
Member

This PR adds some benchmarks, and significantly speeds up types.MakePartSet, and Partset.AddPart. (Used by the block proposer, and every consensus instance) It does so by doing two things:

  • Saving mutexes on the newly created bit array, by defaulting every value to True (rather than setting it in a loop that goes through a mutex)
  • Uses the same hash object throughout, and avoids an extra copy of every leaf. (main speedup)

I do the same hash optimization for proof.Verify, which is used in the add block part codepath for both the proposer and every full node.

New:

BenchmarkMakePartSet/nParts=1-12         	   38616	     29817 ns/op	     568 B/op	      12 allocs/op
BenchmarkMakePartSet/nParts=2-12         	   19888	     59866 ns/op	    1000 B/op	      22 allocs/op
BenchmarkMakePartSet/nParts=3-12         	   12979	     95691 ns/op	    1528 B/op	      33 allocs/op
BenchmarkMakePartSet/nParts=4-12         	    8688	    128192 ns/op	    2024 B/op	      44 allocs/op
BenchmarkMakePartSet/nParts=5-12         	    7308	    155224 ns/op	    2888 B/op	      57 allocs/op

Old:

BenchmarkMakePartSet/nParts=1-12         	   16647	    106545 ns/op	   74169 B/op	      12 allocs/op
BenchmarkMakePartSet/nParts=2-12         	   10000	    106361 ns/op	  148329 B/op	      23 allocs/op
BenchmarkMakePartSet/nParts=3-12         	    6992	    337644 ns/op	  222587 B/op	      35 allocs/op
BenchmarkMakePartSet/nParts=4-12         	    3488	    480109 ns/op	  296811 B/op	      47 allocs/op
BenchmarkMakePartSet/nParts=5-12         	    2228	    557768 ns/op	  371404 B/op	      61 allocs/op

System wide, this is definitely not our issue (looks like roughly .1ms per blockpart), but still definitely useful time to remove


  • Tests written/updated
  • Changelog entry added in .changelog (we use unclog to manage our changelog)
  • Updated relevant documentation (docs/ or spec/) and code comments
  • Title follows the Conventional Commits spec

PR checklist

  • Tests written/updated
  • Changelog entry added in .changelog (we use unclog to manage our changelog)
  • Updated relevant documentation (docs/ or spec/) and code comments

ValarDragon and others added 2 commits May 29, 2024 12:40
<!--

Please add a reference to the issue that this PR addresses and indicate
which
files are most critical to review. If it fully addresses a particular
issue,
please include "Closes #XXX" (where "XXX" is the issue number).

If this PR is non-trivial/large/complex, please ensure that you have
either
created an issue that the team's had a chance to respond to, or had some
discussion with the team prior to submitting substantial pull requests.
The team
can be reached via GitHub Discussions or the Cosmos Network Discord
server in
the #cometbft channel. GitHub Discussions is preferred over Discord as
it
allows us to keep track of conversations topically.
https://github.com/cometbft/cometbft/discussions

If the work in this PR is not aligned with the team's current
priorities, please
be advised that it may take some time before it is merged - especially
if it has
not yet been discussed with the team.

See the project board for the team's current priorities:
https://github.com/orgs/cometbft/projects/1

-->

This PR adds some benchmarks, and significantly speeds up
types.MakePartSet, and Partset.AddPart. (Used by the block proposer, and
every consensus instance) It does so by doing two things:
- Saving mutexes on the newly created bit array, by defaulting every
value to True (rather than setting it in a loop that goes through a
mutex)
- Uses the same hash object throughout, and avoids an extra copy of
every leaf. (main speedup)

I do the same hash optimization for proof.Verify, which is used in the
add block part codepath for both the proposer and every full node.

New:
```
BenchmarkMakePartSet/nParts=1-12         	   38616	     29817 ns/op	     568 B/op	      12 allocs/op
BenchmarkMakePartSet/nParts=2-12         	   19888	     59866 ns/op	    1000 B/op	      22 allocs/op
BenchmarkMakePartSet/nParts=3-12         	   12979	     95691 ns/op	    1528 B/op	      33 allocs/op
BenchmarkMakePartSet/nParts=4-12         	    8688	    128192 ns/op	    2024 B/op	      44 allocs/op
BenchmarkMakePartSet/nParts=5-12         	    7308	    155224 ns/op	    2888 B/op	      57 allocs/op
```

Old:
```
BenchmarkMakePartSet/nParts=1-12         	   16647	    106545 ns/op	   74169 B/op	      12 allocs/op
BenchmarkMakePartSet/nParts=2-12         	   10000	    106361 ns/op	  148329 B/op	      23 allocs/op
BenchmarkMakePartSet/nParts=3-12         	    6992	    337644 ns/op	  222587 B/op	      35 allocs/op
BenchmarkMakePartSet/nParts=4-12         	    3488	    480109 ns/op	  296811 B/op	      47 allocs/op
BenchmarkMakePartSet/nParts=5-12         	    2228	    557768 ns/op	  371404 B/op	      61 allocs/op
```

System wide, this is definitely not our issue (looks like roughly .1ms
per blockpart), but still definitely useful time to remove

---

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [x] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
@ValarDragon ValarDragon added the S:backport/v25 backport to the osmo-v25/v0.37.4 branch label May 29, 2024
@ValarDragon ValarDragon merged commit a6a064d into osmo/v0.37.4 May 29, 2024
17 of 18 checks passed
ValarDragon added a commit that referenced this pull request May 31, 2024
…pr-83

perf(types): 3x speedup MakePartSet (cometbft#3117) (backport #83)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S:backport/v25 backport to the osmo-v25/v0.37.4 branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant