Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(io): Load files with AFMReader #1031

Merged
merged 7 commits into from
Dec 4, 2024
Merged

Conversation

ns-rse
Copy link
Collaborator

@ns-rse ns-rse commented Dec 2, 2024

Refactors topostats/io.py to use AFMReader to load all files.

Almost everything works as expected with the exception of .ibw where AFMReader raises an UnboundLocalError if a file is not as image is created as None rather than raising FileNotFound error. This is captured in Consistently raise FileNotFound errors across each load_*() functions. #80 and for now the on test that is affected has been marked as pytest.mark.xfail() as it is expected to fail.

I was perhaps a little hasty in making a release of AFMReader v0.0.2 which I realised early on as being required so that tests pass in Continuous Integration.


Items that are Not Applicable have been struck through.

  • Existing tests pass.
  • Documentation has been updated and builds. Remember to update as required...
    • docs/configuration.md
    • docs/usage.md
    • docs/data_dictionary.md
    • docs/advanced.md and new pages it should link to.
  • Pre-commit checks pass.
  • New functions/methods have typehints and docstrings.
  • New functions/methods have tests which check the intended behaviour is correct.

Sorry, something went wrong.

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard
@ns-rse ns-rse force-pushed the ns-rse/1025-use-afmreader branch from 8f1c59e to 0282945 Compare December 2, 2024 16:36
@ns-rse ns-rse marked this pull request as ready for review December 2, 2024 16:48
@ns-rse ns-rse changed the title refactor(io): Load .spm with AFMReader refactor(io): Load fileswith AFMReader Dec 2, 2024
@ns-rse ns-rse changed the title refactor(io): Load fileswith AFMReader refactor(io): Load files with AFMReader Dec 2, 2024
@ns-rse
Copy link
Collaborator Author

ns-rse commented Dec 2, 2024

I might have to think about/revise the use of .pop() in run_modules.process() as it might be more prudent to remove these conveniences that AFMReader returns when using LoadScans.get_data() so that we only have to do it once and don't have to do it for every single module in run_modules as the "swiss army knife" work will require loading .topostats files for subsequent processing/re-processing.

Copy link
Collaborator

@SylviaWhittle SylviaWhittle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love a PR that reduces the codebase!

This is neat and tidy, agree that using .pop would be nice to replace with a more elegant solution in the future, perhaps by automatically removing them from the returned dictionary in AFMReader and passing the image, pixel scaling and file version as separate returns (image, scaling, version, data) so we don't have to pop them in each stage like you mentioned we will have to.

@ns-rse
Copy link
Collaborator Author

ns-rse commented Dec 3, 2024

Thanks for the review and approval.

agree that using .pop would be nice to replace with a more elegant solution in the future, perhaps by automatically removing them from the returned dictionary in AFMReader and passing the image, pixel scaling and file version as separate returns (image, scaling, version, data) so we don't have to pop them in each stage like you mentioned we will have to.

I'm going to change this now, otherwise it will linger and get forgotten about.

@ns-rse
Copy link
Collaborator Author

ns-rse commented Dec 3, 2024

Also just noticed I van remove _ibw_pixel_to_nm_scaling() definition too as that has migrated to AFMReader and isn't used here.

ns-rse and others added 5 commits December 3, 2024 17:22

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard
In light of switching to AFMReader to load files and consistently loading and failing with a `FileNotFoundError` when
files do not exist we can parameterize the tests that check the appropriate error is raised.

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard
@ns-rse ns-rse force-pushed the ns-rse/1025-use-afmreader branch from 0282945 to 0ef9be3 Compare December 3, 2024 17:27
@ns-rse
Copy link
Collaborator Author

ns-rse commented Dec 3, 2024

Change of heart, going to address this as part of the Swiss army knife work as I don't wish to hold up the pending version release, but have removed the redundant _ibw_pixel_to_nm_scaling() function (and kept history clean by using git commit --fixup to add it to the relevant commit then git push --force 😇 )

@ns-rse ns-rse merged commit 5ba3ffe into main Dec 4, 2024
11 checks passed
@ns-rse ns-rse deleted the ns-rse/1025-use-afmreader branch December 4, 2024 07:39
ns-rse added a commit that referenced this pull request Dec 11, 2024

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard
Closes #1036

With the delegation of much I/O (Input/Output) functionality to @SylviaWhittle `AFMReader` (see #1031) we have removed
most of the methods to the `io.LoadScan()` class. One that slipped through was `_spm_pixel_to_nm_scaling()` and this
commit removes it along with its associated test.

Further the delegation of reading AFM files using `AFMReader` means `TopoStats` no longer has explicit dependencies on
the `igor2`, `pySPM` or `tifffile` packages and so these have been removed from the `dependencies` section of
`pyproject.toml`. These are dependencies of `AFMReader` and will still be installed in virtual environments.
ns-rse added a commit that referenced this pull request Dec 11, 2024

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard
Closes #1036

With the delegation of much I/O (Input/Output) functionality to @SylviaWhittle `AFMReader` (see #1031) we have removed
most of the methods to the `io.LoadScan()` class. One that slipped through was `_spm_pixel_to_nm_scaling()` and this
commit removes it along with its associated test.

Further the delegation of reading AFM files using `AFMReader` means `TopoStats` no longer has explicit dependencies on
the `igor2`, `pySPM` or `tifffile` packages and so these have been removed from the `dependencies` section of
`pyproject.toml`. These are dependencies of `AFMReader` and will still be installed in virtual environments.
ns-rse added a commit that referenced this pull request Dec 11, 2024

Verified

This commit was signed with the committer’s verified signature.
ns-rse Neil Shephard
Closes #1036

With the delegation of much I/O (Input/Output) functionality to @SylviaWhittle `AFMReader` (see #1031) we have removed
most of the methods to the `io.LoadScan()` class. One that slipped through was `_spm_pixel_to_nm_scaling()` and this
commit removes it along with its associated test.

Further the delegation of reading AFM files using `AFMReader` means `TopoStats` no longer has explicit dependencies on
the `igor2`, `pySPM` or `tifffile` packages and so these have been removed from the `dependencies` section of
`pyproject.toml`. These are dependencies of `AFMReader` and will still be installed in virtual environments.

Moved `pytest-testmon` from `tests` to `dev` under optional dependencies as its not used when running the test suite,
only when making commits locally.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants