Backport PR #3227 on branch v4.1.x (more spectral extraction improvements for downstream use) #11483

Job	Run time
Code style checks	18s
PEP 517	22s
Security audit	19s
Python 3.11 with coverage checking, all deps, and remote data	36m 32s
OS X - Python 3.12	8m 3s
Windows - Python 3.10	21m 4s
Python 3.12 with latest dev versions of key dependencies, and remote data	27m 59s
Python 3.11 with stable versions of dependencies and Roman	12m 50s
Upload Coverage	10s
	1h 47m 37s

Provide feedback