Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix(BMZ): Relax model output validation kwargs; extract weights and c…
…onfig file following new `spec` and `core` release (#279) ### Description - **What**: Relaxing the model output validation kwargs, both absolute and relative tolerance, from the default, `1e-4`, to `1e-2`. - **Why**: The defaults are pretty strict and some of our uploaded models are stuck in pending because of slightly mismatching input and outputs. - e.g. (Actually maybe absolute tolerance should be put to 0, otherwise it still might not pass after this PR) ```console Output and expected output disagree: Not equal to tolerance rtol=0.0001, atol=0.00015 Mismatched elements: 40202 / 1048576 (3.83%) Max absolute difference: 0.1965332 Max relative difference: 0.0003221 ``` - **How**: In the model description config param, added the new test kwargs. Additionally, updated `bmz_export` so that the test kwargs in the model description are used during model testing at export time. ### Changes Made - **Modified**: Describe existing features or files modified. - `create_model_description`: added test_kwargs to config param - `export_to_bmz`: use test_kwargs in model description for model testing at export time. ### Related Issues - Resolves - last checkbox in #278 EDIT: This PR also fixes loading from BMZ following an incompatible release of `bioimageio/core` (`0.7.0`) and `bioimageio/spec` (`0.5.3.5`). The problem was `load_model_description` no longer unzips the archive file but only streams the `rdf.yaml` file data. This means we have to now extract the weights and careamics config from the zip to load them, which can be done using `bioimageio.spec._internal.io.resolve_and_extract` --- **Please ensure your PR meets the following requirements:** - [x] Code builds and passes tests locally, including doctests - [ ] New tests have been added (for bug fixes/features) - [x] Pre-commit passes - [ ] PR to the documentation exists (for bug fixes / features) --------- Co-authored-by: Joran Deschamps <[email protected]>
- Loading branch information