Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate golden tensors for TTIR llama prefill #1057

Open
tapspatel opened this issue Jan 16, 2025 · 3 comments
Open

Generate golden tensors for TTIR llama prefill #1057

tapspatel opened this issue Jan 16, 2025 · 3 comments
Assignees

Comments

@tapspatel
Copy link

tapspatel commented Jan 16, 2025

llama_prefill.mlir.txt

I've attached the llama prefill model given from @dgolubovicTT. We have mechanisms in MLIR itself to run golden comparisons if the golden data exists. Part of the bringup is being able to generate the golden intermediates/output for each of the ttir ops in the llama model.

I'm requesting for there to be some support to dump the goldens as .pt tensors for each op in the ttir graph. You can also save the Goldens in a golden_map in the flatbuffer itself, as the support for that already exists (second way is preferred). There should be a way to correlate the dumped tensor to the ttir op it came from. I will then take the Goldens and the ttir graph and run it in our existing infra in MLIR. Ultimately, this will help in the future once we integrate golden into tt-forge itself.

Relevant issue: tenstorrent/tt-mlir#1768 (stage 4)

@tapspatel
Copy link
Author

fyi @nvukobratTT

@nvukobratTT
Copy link
Contributor

Thanks @tapspatel! We're planing to fix up and enable our intermediate verifications in next period. We'll include this as additional set of requirements

@tapspatel
Copy link
Author

thanks @nvukobratTT !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants