Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roofline estimator: simplify #1783

Open
wants to merge 3 commits into
base: gh/vkuzo/44/head
Choose a base branch
from
Open

Conversation

vkuzo
Copy link
Contributor

@vkuzo vkuzo commented Feb 26, 2025

Summary:

  1. remove estimating torch.compile limitations (interesting but hasn't
    been useful)
  2. make clearer distinction between roofline and benchmarked values
  3. fix formatting of sympy-generated numbers in pandas by casting to float
  4. general var names cleanup

Test Plan:

(pytorch) [[email protected] ~/local/ao (20250225_mx_roofline)]$ python benchmarks/float8/float8_roofline.py ~/local/tmp/20250225_test.csv
do_benchmarks: True
shape_gen_name: square
bf16_gemm_time_sympy 4.44444444444444e-15*K*M*N
fp8_gemm_time_sympy 2.22222222222222e-15*K*M*N

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:17<00:00,  2.14s/it]
   fwd_M  fwd_K  fwd_N  r_bf16_gemm_s  r_fp8_gemm_s  r_fp8_ovhd_s  r_fp8_gemm_and_ovhd_s  r_fp8_gemm_and_ovhd_spdp  b_bf16_gemm_s  b_fp8_gemm_s  b_bf16_e2e_s  b_fp8_e2e_s  b_fp8_e2e_spdp
0    256    256    256       7.46e-08      3.73e-08      6.12e-06               6.16e-06                      0.01       9.41e-06      9.72e-06      2.35e-05     4.73e-05            0.50
1    512    512    512       5.97e-07      2.98e-07      6.50e-06               6.80e-06                      0.09       1.10e-05      1.03e-05      2.63e-05     5.00e-05            0.53
2   1024   1024   1024       4.77e-06      2.39e-06      7.99e-06               1.04e-05                      0.46       1.71e-05      1.29e-05      3.41e-05     6.02e-05            0.57
3   2048   2048   2048       3.82e-05      1.91e-05      1.40e-05               3.31e-05                      1.15       4.61e-05      2.83e-05      8.72e-05     1.29e-04            0.67
4   4096   4096   4096       3.05e-04      1.53e-04      3.79e-05               1.91e-04                      1.60       3.31e-04      1.67e-04      4.08e-04     4.22e-04            0.97
5   8192   8192   8192       2.44e-03      1.22e-03      1.34e-04               1.36e-03                      1.80       2.69e-03      8.37e-04      2.71e-03     2.01e-03            1.35
6  16384  16384  16384       1.95e-02      9.77e-03      5.17e-04               1.03e-02                      1.90       2.19e-02      1.01e-02      2.75e-02     1.51e-02            1.82
7  32768  32768  32768       1.56e-01      7.82e-02      2.05e-03               8.02e-02                      1.95       2.67e-01      6.22e-02      2.51e-01     1.27e-01            1.99

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
Copy link

pytorch-bot bot commented Feb 26, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1783

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit e9ab762 with merge base 1ab1b77 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 26, 2025
vkuzo added a commit that referenced this pull request Feb 26, 2025
Summary:

1. remove estimating torch.compile limitations (interesting but hasn't
   been useful)
2. make clearer distinction between roofline and benchmarked values

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 3348fff4abce61a52265e94badbe58426bd342ab
ghstack-comment-id: 2683886949
Pull Request resolved: #1783
[ghstack-poisoned]
vkuzo added a commit that referenced this pull request Feb 26, 2025
Summary:

1. remove estimating torch.compile limitations (interesting but hasn't
   been useful)
2. make clearer distinction between roofline and benchmarked values
3. sympy float -> float cast to fix pandas df formatting

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: dacab7f909eb7214a25a068d7d6ebc9a12a7614d
ghstack-comment-id: 2683886949
Pull Request resolved: #1783
@vkuzo vkuzo added the topic: not user facing Use this tag if you don't want this PR to show up in release notes label Feb 26, 2025
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. topic: not user facing Use this tag if you don't want this PR to show up in release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants