v0.7.0

v0.7.0

This minor release updates the mimumum supported python to python = "^3.8", addresses dependency vulnerabilities, and updates typing to conform with current mypy and pytest standards.

Updates / New Features

Build

• New minimum supported python changed to python = "^3.8".

CI

• Updated codecov action version to 3.

• Added explicit use of codecov token to facilitate successful coverage
  submission.

• Updated publish/release CI workflow.

Dependencies

• Updated notebook dependency due to a vulnerability alert.

• Periodic update of locked dep versions within abstract version constraints.

• Updated sphinx versions to fix local documentation building issue.

• Updated python minimum requirement to 3.8 (up from 3.6). This involved a
  number of updates and bifurcations of abstract requirements, an update to
  pinned versions for development/CI, and expansion of CI to cover python
  versions 3.10 and 3.11 (latest current release).

Fixes

Docs

• Added missing step to the release process about creating the release on
  GitHub's Releases section.

Examples

• Added a note to each example about restarting the runtime for compatibility
  with Colab, as well as a step to create a data directory if necessary.

v0.6.1

v0.6.1

This patch specializes the release CI workflow for this organization and
repository as opposed to previously relying on the remote workflow.

Fixes

CI

• Fix the publish workflow to use appropriate values and version for the
  containing org and this repository.

• Update CI workflows to also run for update-to-v* branches.

v0.6.0

v0.6.0

This minor release notably adds new high-level interfaces for black-box object
detector and image similarity saliency generation.
We provide some reference implementations for these interfaces, notably the
D-RISE and SBSM algorithms, respectively.

Other improvements include the addition of more examples notebooks,
improvements/fixes to existing implementations and a revision to the CLI object
detection saliency generation tool.

See below for more details.

Updates / New Features

CI

• Added the ATARI example notebook to the list of notebooks to run during CI.

Documentation

• Update saliency algorithms table with perturbation-based saliency for reinforcement learning
  and add corresponding section to README.

• Added a lighter color version of the logo that will appear better is both
  light- and dark-theme contexts. The main README file has been updated to refer
  to this image.

• Added introductory sentence to the style sheet document.

• Updated the release process to be incrementally more comprehensive and now
  includes the specification of a release branch with better patch release
  instructions. This also now includes a step to update the version referenced
  in the xaitk.org source.

Examples

• Updated demo resource download links from Google Drive to data.kitware.com

• Added example using saliency to qualitatively compare two object detection
  models.

• Updated SimilarityScoring example to use new high-level image similarity
  saliency interface and follow new similarity interface inputs.

Interfaces

• Added new high-level interface for black-box object detector saliency,
  GenerateObjectDetectorBlackboxSaliency.

• Updated image similarity interface GenerateDescriptorSimilaritySaliency to
  accept multiple query images and compute a saliency map for each one, relative
  to the reference image.

• Added new high-level interface for image similarity saliency,
  GenerateImageSimilarityBlackboxSaliency.

Implementations

• Added three GenerateObjectDetectorBlackboxSaliency implementations: the
  generic PerturbationOcclusion, and two usable classes DRISEStack
  and RandomGridStack.

• Updated behavior of the SlidingWindow PerturbImage implementation. For
  a given stride, the number of masks generated is now agnostic to the window
  size.

• Updated SimilarityScoring to return [N x H x W] instead of [1 x H x W]
  saliency heatmaps matrix. This is inline with the similarity interface update.

• Added two implementations of GenerateImageSimilarityBlackboxSaliency:
  PerturbationOcclusion and SBSMStack.

Misc.

• Updated poetry-core build backend to version 1.0.8, which now supports
  pip editable installs (pip install -e .).

Utils

• Updated COCO utility functions to use new high-level detector interface.
  gen_coco_sal() is now deprecated in exchange for parse_coco_dset() which
  parses a kwcoco.CocoDataset object into the inputs used with an
  implementation of GenerateObjectDetectorBlackboxSaliency.

Fixes

Dependency Versions

• Update pinned jupyter notebook transitive dependency version due to
  vulnerability warning.

Examples

• Fixed inconsistency of dependency package installs at the head of the
  examples/SerializedDetectionSaliency.ipynb notebook.

v0.5.0

v0.5.0

Updates / New Features

CI

• Updated notebooks CI workflow to include notebook data caching.

Documentation

• Added text discussing black box methods to introduction.rst.

• Added a section to introduction.rst that describes the links between saliency algorithms and implementations.

• Edited all text.

• Update top-level README.md file to have more useful content.

• Update misc. doc on local SonarQube scanning.

Examples

• Add example notebook for saliency on Atari deep RL agent, including updates
  on top of the original work to normalize saliency maps and conform to our
  API standards.

• Add example demonstrating saliency map generation for COCO formatted
  serialized detections.

• Updated examples to all use a common data sub-directory when downloading or
  saving generated data.

Implementations

• Add SquaredDifferenceScoring implementation of the GenerateClassifierConfidenceSaliency
  interface that uses squared difference.

• Add RandomGrid implementation of PerturbImage. This generates masks
  of randomly occluded cells with a given size in pixels.

Utilities

• Add gen_coco_sal function to compute saliency maps for detections in a
  kwcoco dataset, with accompanying cli script sal-on-coco-dets which
  does this on a COCO formatted json file and writes saliency maps to disk.

• Add multi-threaded functionality to occlude_image_batch utility.

Containerization

• Added Dockerfile and compose file that create base xaitk_saliency image.

Fixes

Build

• Fix incorrect specification of actually-optional papermill in relation to
  its intended inclusion in the example_deps extra.

• Update patch version of Pillow transitive dependency locked in the
  poetry.lock file to address CVE-2021-23437.

• Update the developer dependency and locked version of ipython to address a
  security vulnerability.

Implementations

• Fix incorrect cosine similarity computation and mask inversion in implementation of
  DRISEScoring detector saliency.

Examples

• Updated example Jupyter notebooks with more consistent dependency checks and
  also fixed minor header formatting issues.

Tests

• Fix deprecation warnings around the use of numpy.random.random_integers.

Utilities

• Fix xaitk_saliency.utils.detection.format_detection to not upcast the
  data type output when objectness is None.

• Fix xaitk_saliency.utils.masking.weight_regions_by_scalar to not upcast
  the data type output when inv_masks is True.

• Update xaitk_saliency.utils.masking.weight_regions_by_scalar to not use
  fully vectorized operation which significantly improves efficiency.

v0.4.0

This minor release expands our documentation and examples pool.
We additionally provide an the D-RISE implementation for the
GenerateDetectorProposalSaliency interface.

Updates / New Features

CI

• Added workflow for test running some example notebooks.

• Update CodeCov action used to version 2.

Documentation

• Added text discussing white box methods to introduction.rst.

• Added some review process documentation.

• Add initial FAQ documentation file.

• Add background material for saliency maps to introduction.rst.

• Added API docs section, which includes descriptions of all interfaces.

• Added content to the CONTRIBUTING.md file on:

  • including notes here for added updates, features and fixes

  • Jupyter notebook CI workflow inclusion

• Add implementations section.

• Update example Jupyter notebooks to work with Google Colab.

Examples

• Add example notebook using classifier-based interfaces and implementations
  with scikit-learn on the MNIST dataset.

• Edited notebook examples.

Implementations

• Add DRISEScoring implementation of the GenerateDetectorProposalSaliency
  interface using detection output and associated occlusion masks.

• Add SlidingRadial implementation of the PerturbImage interface that
  slides radial occlusion areas across an image.

Tests

• Removed use of unittest.TestCase as it is not utilized directly in any way
  that PyTest does not provide.

Utilities

• Add type annotation, documentation and unit-tests for using image matrices as
  the fill option instead of just a solid color.

• Add format_detection helper function to form the input for
  GenerateDetectorProposalSaliency from separated components.

• Add example notebook showing the use of SlidingRadial perturbation and
  the use of occlude_image_batch with blurred-image alpha blending.

Fixes

Implementations

• Fixed ValueError messages raised in the SimilarityScoring
  implementation. Added unittests to check the raising and message content.

v0.3.1

v0.3.1

This patch release focuses on removing some listed dependencies that are in
reality unused.

Fixes

CI

• Enable the running of CI for branches and PRs targeting release branches.

Package

• Remove required dependencies that are in fact not used.

v0.3.0

v0.3.0

This minor release provides updates to example notebooks, interface naming
updating for consistency, as well as added a "high level" interface for visual
saliency map generation from black-box classifiers.

Updates / New Features

Documentation

• Add "SuperPixelSaliency" notebook to demonstrate use of arbitrary perturbation
  masks based on superpixels for saliency map generation.

• Add "VIAME_OcclusionSaliency" notebook demonstrating integration with VIAME
  toolkit based on saliency map generation for a fish classification task.

• Add "covid_classification" notebook demonstrating integration with MONAI
  based on saliency map generation for a COVID-19 X-ray classification task.

• Updated notebook demonstration for SimilarityScoring usage to better track
  the notebook structures across the repo.

• Introduce class naming philosophy in the CONTRIBUTING.md file.

• Updated notebooks to make use of GenerateImageClassifierBlackboxSaliency
  interface/implementations where appropriate.

• Updated wording of the SuperPixelSaliency.ipynb notebook as to cover a
  use-case where the GenerateImageClassifierBlackboxSaliency interface API
  is not appropriate, i.e. already have masks computed.

• Added support for doc building on Windows by adding a platform check so that "make.bat" is called for and not "make html".

Interfaces

• Update PerturbImage to only output perturbation masks, dropping physical
  image perturbation output. Since this output generation was the same across
  all known implementations, that part has been split out into a utility
  function.

• Added support for positive and negative saliency values as output by the
  saliency map generation interfaces.

• Updated PerturbImage interface to take in numpy.ndarray as the image
  data structure.

• Added new, higher-level GenerateImageClassifierBlackboxSaliency interface
  for transforming an image and black-box classifier into visual saliency maps.

• Renamed ImageClassifierSaliencyMapGenerator interface to be
  GenerateClassifierConfidenceSaliency.

• Renamed ImageSimilaritySaliencyMapGenerator interface to be
  GenerateDescriptorSimilaritySaliency.

• Renamed ImageDetectionSaliencyMapGenerator interface to be
  GenerateDetectorProposalSaliency.

Implementations

• Add RISEScoring implementation, with the ability to also compute a
  de-biased form of RISE with an optional input parameter.

• Add RISEStack implementation of the GenerateImageClassifierBlackboxSaliency
  interface as a simple way to invoke the combination of RISE component
  algorithms.

• Add SlidingWindowStack implementation of the GenerateImageClassifierBlackboxSaliency
  interface as a simple way to invoke the combination of the Sliding Window
  perturbation method and the occlusion-based based scoring method.

Misc.

• Update locked dependency versions to latest defined by abstract requirements.

Utils

• Masking

  • Added utility functions for occluded image generation that was previously
    duplicated across PerturbImage implementations. Added both batch and
    streaming versions of this utility.

Fixes

Documentation

• Fixed misspelled "miscellaneous" file.

Implementations

• Fix saliency map normalization in both OcclusionScoring as well as
  SimilarityScoring to disallow cross-class pollution in the norm.

Misc.

• Fixed up module naming inconsistencies.

v0.2.0

v0.2.0

This initial alpha release introduces the xaitk-saliency toolkit for computing
visual saliency heat-maps for input imagery over based on black-box operations.

Updates / New Features

CI

• Added properties file for SonarQube scans.

• Add CodeCov integration.

Documentation

• Updated the "Occlusion Saliency" notebook to flow smoother and include
  un-commentable RISE perturbation algorithm option. The narrative has
  been more explicitly tuned to follow an "application" narrative.

• Add miscellaneous documentation on how to run a local SonarQube scan and
  experimental documentation on setting up scanning as a CI workflow job.

Interfaces

• Add new interfaces in accordance to the v0.2 API draft.

  • Added to doc-strings to expand on detail around saliency heatmap return
    value range and meaning.

  • Updated image perturbation interface to function in a streaming iterator
    fashion instead of in-bulk as a means of performance optimization as well
    as to allow it to function on larger image sizes and larger perturbation
    quantities at the same time.

• Removed old interface classes "ImageSaliencyMapGenerator", "SaliencyBlackbox"
  and "ImageSaliencyAugmenter".

Implementations

• Add new occlusion based classifier scoring in accordance to the v0.2 API draft for ImageClassifierSaliencyMapGenerator.

• Add new RISE based perturbation algorithm in accordance to the v0.2 API draft for PerturbImage

• Add new similarity based scoring algorithm in accordance to the v0.2 API draft for ImageSimilaritySaliencyMapGenerator

• Remove old "stub" implementations in transitioning to the new API draft

  • Removed "LogitImageSaliencyAugmenter" implementation class.

  • Removed "LogitImageSaliencyMapGenerator" implementation class.

  • Removed old RISE implementation classes.

  • Removed old SBSM implementation classes.

Fixes

• Update Read the Docs documentation link in README

• Address various "code smells" as reported by SonarQube/SonarCloud.