Add verbose to Step and Pipeline

[izkgao]

This PR adds a new verbose argument to Step and Pipeline which may fix issue #49.

However, changing the logging level from DEBUG to CRITICAL also makes it impossible to record log messages (as you can see in the tests I use record_logs to check if there is any output). I am not sure if there is a way to prevent log messages from being output to the console and still record the logs.

Another concern is that the get_config_from_reference class method uses the delegator logger instead of the logger created by the pipeline. This delegator logger cannot be controlled by verbose before the pipeline is initiated. So some logs may still be output to the console.
https://github.com/spacetelescope/stpipe/blob/9e7c4416fa8776c00843c18390a1dc2d7b4179d0/src/stpipe/pipeline.py#L143C11-L143C11

Although not good enough, this PR can still work as a workaround for those who do not want the logs printed to the console.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9e7c441) 59.15% compared to head (e4ef6a2) 55.25%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #131      +/-   ##
==========================================
- Coverage   59.15%   55.25%   -3.90%     
==========================================
  Files          24       24              
  Lines        1611     1616       +5     
==========================================
- Hits          953      893      -60     
- Misses        658      723      +65     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jdavies-st]

Thanks for having a crack at this @izkgao. stpipe logging has been broken for a very long time, and there's been a few attempts to fix it, none successful.

The basic problem is that the library code fiddles with handlers and log levels, and that's a big no-no. Only the CLI should be using handlers and setting levels, and the pipeline and step library code should be just emitting messages.

For background, see spacetelescope/jwst#4658

But it does need to be fixed, but it will involve changes to both stpipe and to jwst, where null handlers and DEBUG levels are set routinely within modules. And there's a whole other logger and handler in jwst.associations. 🤯