Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script for updating the model ops test failures as xfail #1199

Closed
chandrasekaranpradeep opened this issue Feb 7, 2025 · 0 comments · Fixed by #1198
Closed

Script for updating the model ops test failures as xfail #1199

chandrasekaranpradeep opened this issue Feb 7, 2025 · 0 comments · Fixed by #1198
Assignees

Comments

@chandrasekaranpradeep
Copy link
Contributor

The model ops test generation workflow(i.e .github/workflows/model-analysis.yml with generate_models_ops_test = True), generates model ops test in forge/test/models_ops/ path by extracting the unique ops configuration across all the models and then create PR for the generated models ops test and since all the generated model ops are marked with push pytest marker, ON PR workflow (.github/workflows/on-pr.yml) will run all the pytest including the generated models ops test. There will failure for the generated model ops test.

Instead of manually searching the failed test and update the failed test with xfail marker with failure reason. Create a script which will take a pytest logs fails which contains failures of the generated models ops test and search for failed test with respect to generated models ops test and update the failed test with xfail marker and failure reason extracted from the same pytest logs file for the respective test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment