You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When one inference + eval workflow is run, two datasets are generated by the same inference job instead of just one.
Reproduction
Run a new evaluation workflow given a dataset which already has GT
Check the list of datasets at the end of the workflow: two new datasets with predictions will be generated by the same model
Relevant log output
Expected behavior
Only one dataset should be generated.
I already pinpointed the issue which is that dataset generation has been enabled automatically both at job level and at workflow level.
I have a possible solution in mind, will self-assign this issue and push a PR soon for it.
Description
When one inference + eval workflow is run, two datasets are generated by the same inference job instead of just one.
Reproduction
Relevant log output
Expected behavior
Only one dataset should be generated.
I already pinpointed the issue which is that dataset generation has been enabled automatically both at job level and at workflow level.
I have a possible solution in mind, will self-assign this issue and push a PR soon for it.
System Info
Have you searched for similar issues before submitting this one?
The text was updated successfully, but these errors were encountered: