Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor benchmarks #161

Merged
merged 3 commits into from
Dec 23, 2024
Merged

Refactor benchmarks #161

merged 3 commits into from
Dec 23, 2024

Conversation

cupicmarko
Copy link
Contributor

Revamped benchmarks:

  • Benchmarks no longer run any code outside of bench_function block
  • Using latticefold/benches/config.toml to run benchmarks
  • No macros used in benchmarks
  • Using build.rs to generate benchmarks code from latticefold/benches/config.toml

Copy link
Contributor Author

@cupicmarko cupicmarko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments for clarity, diff is hard to read

latticefold/benches/config.toml Show resolved Hide resolved
latticefold/benches/config.toml Outdated Show resolved Hide resolved
latticefold/benches/macros.rs Outdated Show resolved Hide resolved
latticefold/benches/utils.rs Show resolved Hide resolved
@v0-e
Copy link
Member

v0-e commented Dec 18, 2024

I think it looks pretty cool! 💯
Just a small improvement: with my experience in benchmarking this repo 😄, it would be nice to have the possibility to choose which subprotocol to bench, for example, just run the folding benches. Also the same thing for scalar, non-scalar, 3 non-scalar.
You can make fields an Option in the benches config, and generate the benches based on their existence 👍

@cupicmarko
Copy link
Contributor Author

I think it looks pretty cool! 💯 Just a small improvement: with my experience in benchmarking this repo 😄, it would be nice to have the possibility to choose which subprotocol to bench, for example, just run the folding benches. Also the same thing for scalar, non-scalar, 3 non-scalar. You can make fields an Option in the benches config, and generate the benches based on their existence 👍

I have something that can filter benchmarks based on environment variables, I just have to verify it works and I'll push one commit

@maksimryndin
Copy link
Collaborator

awesome job, @cupicmarko! 👍

@cupicmarko
Copy link
Contributor Author

It was updated to allow filtering, also bounds.sage was updated based on @ivolkg changes. I've noticed that it takes significantly more time to compile and link benches because there is more benchmark tests, so for local running, I would suggest either building benches incrementally or linking dynamically, to reduce build time.

@ElijahVlasov ElijahVlasov merged commit 71debaf into main Dec 23, 2024
5 checks passed
@ElijahVlasov ElijahVlasov deleted the cupicmarko/benches branch December 23, 2024 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants