Skip time slices for which the transmission calculation fails

[backmari]

Description of work:

Bio-SANS has an option "useTimeSliceTransmission" where the transmission correction will be calculated separately for each time slice using the time sliced sample run rather than a separate sample transmission run. Currently, the whole reduction can fail if one time slice (usually the last smaller slice) has insufficient statistics and the transmission calculation results in NaNs. The Bio-SANS IS wants the reduction to proceed without the slice for which the transmission calculation failed.

This PR changes the RuntimeError in the transmission calculation to a custom error TransmissionNanError. A new sanity check is added for the relative size of the transmission error compared to the transmission value and a custom error TransmissionErrorToleranceError is thrown if the value is larger than the sample transmission parameter "errorTolerance". When "useTimeSliceTransmission" is used, a time slice will be skipped if TransmissionNanError or TransmissionErrorToleranceError is thrown.

• new sample transmission parameter "errorTolerance" with default value 0.01
• when using "useTimeSliceTransmission", skip slices with relative transmission error higher than the tolerance or when all transmission values are NaN

Check all that apply:

• added release notes (if not, provide an explanation in the work description)
• updated documentation
• Source added/refactored
• Added unit tests
• Added integration tests
• Verified that tests requiring the /SNS and /HFIR filesystems pass without fail

References:

• Links to IBM EWM items: Defect 8722: [DRTSANS] The time-slicing reduction raises when dividing by the empty transmission value
• Links to related issues or pull requests:

Manual test for the reviewer

Check list for the reviewer

• release notes updated, or an explanation is provided as to why release notes are unnecessary
• best software practices
  • clearly named variables (better to be verbose in variable names)
  • code comments explaining the intent of code blocks
• All the tests are passing
• The documentation is up to date
• code comments added when explaining intent

Execution of tests requiring the /SNS and /HFIR filesystems

It is strongly encouraged that the reviewer runs the following tests in their local machine
because these tests are not run by the GitLab CI. It is assumed that the reviewer has the /SNS and /HFIR filesystems
remotely mounted in their machine.

cd /path/to/my/local/drtsans/repo/
git fetch origin merge-requests/<MERGE_REQUEST_NUMBER>/head:mr<MERGE_REQUEST_NUMBER>
git switch mr<MERGE_REQUEST_NUMBER>
conda activate <my_drtsans_dev_environment>
pytest -m mount_eqsans ./tests/unit/ ./tests/integration/

In the above code snippet, substitute <MERGE_REQUEST_NUMBER> for the actual merge request number. Also substitute
<my_drtsans_dev_environment> with the name of the conda environment you use for development. It is critical that
you have installed the repo in this conda environment in editable mode with pip install -e . or conda develop .

[codecov-commenter]

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

[jmborr]

the errorTolerance should go inside the sample transmission schema. In file common.json:

"sample": {
  "transmission": {
            "type": "object",
        "properties": {
          "runNumber": {
            "$ref": "common.json#/definitions/runNumberOptionalTypes",
            "description": "The run number(s) for the transmission sample. More than one run number can be specified by passing a list or by passing a string containing the run numbers separated by commas.",
            "examples": ["12345", 12345]
          },
          "value": {
            "$ref": "common.json#/definitions/transmissionValueTypes"
          }
          "errorTolerance": {
             "$ref": "common.json#/definitions/transmissionValueTypes",
             "description": "maximum allowed absolute value of the error in the calculated transmission"
             "default": 0.01,
          }
  }
}
This problem arises when slicing and since we're slicing just the sample, the problems arises when slicing the sample-transmission run (which also happens to be the sample run)

[backmari]

@jmborr Should we only check the transmission error when time slicing the sample transmission run? That is, we don't care about this sanity check otherwise?

[jmborr]

@jmborr Should we only check the transmission error when time slicing the sample transmission run? That is, we don't care about this sanity check otherwise?

I can't think of any scenario when the transmission will have a big error. If we're not slicing, then the transmission run will have zillion events. I'd go "cheap" and check only the slicing case. If you want to be generous and it's not too much extra work, I'd check the sample-transmission (not the other cases like the background-transmission or empty transmission) for the non-slicing case. It's up to you.

[backmari]

@jmborr Should we only check the transmission error when time slicing the sample transmission run? That is, we don't care about this sanity check otherwise?

I can't think of any scenario when the transmission will have a big error. If we're not slicing, then the transmission run will have zillion events. I'd go "cheap" and check only the slicing case. If you want to be generous and it's not too much extra work, I'd check the sample-transmission (not the other cases like the background-transmission or empty transmission) for the non-slicing case. It's up to you.

Sounds good. I will probably err on the side of not upsetting users with new error messages and check only the slicing case.

[jmborr]

Sounds good. I will probably err on the side of not upsetting users with new error messages and check only the slicing case.

Fine by me. Let me know when the PR is ready to be reviewed

[backmari]

@jmborr This is ready for a second pass.