Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

now keeping tile, petal, and loc #1060

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Conversation

iprafols
Copy link
Collaborator

This should propagate the read values of tile, petal and night values to the delta files, if they are available in the catalogue. They are stored as a string. Lines of sight with multiple observations contain all the values separated by "-".

@iprafols
Copy link
Collaborator Author

@julienguy , this should fix your complaint the other day about not having the tile information in the delta files. Can you review that this is indeed what you needed?

@iprafols iprafols self-assigned this Feb 19, 2024
@iprafols iprafols requested a review from julienguy February 19, 2024 11:57
@iprafols iprafols removed their assignment Feb 19, 2024
Copy link
Contributor

@Waelthus Waelthus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this change assuming it's actually keeping the info with it. Could add more fields potentially

@@ -338,6 +338,12 @@ def format_data(self,
"z": row['Z'],
}
args["log_lambda"] = np.log10(spec['WAVELENGTH'])
if "TILEID" in row:
args["tile"] = row['TILE']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it called TILEID?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I have fixed the bug (fixed it in a commit). Should we rename the tile member in DesiForest to tileid as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe that would be good for consistency... Esp given that e.g. eBOSS did have a PLATE instead of TILEID anyway, so it's a DESI specific thing.

if "PETAL_LOC" in row:
args["petal"] = row['PETAL_LOC']
if "NIGHT" in row:
args["night"] = row['NIGHT']

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add e.g. FIBER or EXPID? Are these fields all read earlier from HDU2 (EXP_FIBERMAP) of the coadd-files, or just when reading spectra-files?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just checked and here we'd just copy over info from the catalogue, is all this info in the actual quasar cats? or would we need to assemble the data from the coadd-files?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possibly, the FIBER would be nice to have. I'm not sure about the EXPID. @julienguy , what do you think?

Copy link
Contributor

@Waelthus Waelthus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think at least the typo needs fixing

Copy link
Contributor

@julienguy julienguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just ran a test with this branch and this is just not working. Nothing is saved for NIGHT PETAL TILEID in the METADATA of the delta files.

Example run, ini file:
/global/cfs/cdirs/desicollab/users/jguy/lya/instrumental-systematics/deltas/delta-civ-tile-2-sub-1/picca_delta.ini
And delta files in :

/global/cfs/cdirs/desicollab/users/jguy/lya/instrumental-systematics/deltas/delta-civ-tile-2-sub-1/Delta

@Waelthus
Copy link
Contributor

In case we still want to add this feature:

  • I think the issue is that in this PR we tried to read the relevant information from the catalogs, but those only contain the summarized info like LASTNIGHT etc iirc
  • I think the fix could be as easy as reading the info from the coadd-files (which iirc store both summarized and per-exp data). Those are open anyway at the point those attributes are set in this version of the code, we probably just need to read the additional headers...
  • Not sure if this PR would still be wanted/needed though or if some part of the functionality has been added elsewhere

@corentinravoux
Copy link
Contributor

@iprafols or @calumgordon if I remember from the latest Lya soft telecon, you wanted to rework this PR, any news ?

@Waelthus
Copy link
Contributor

I guess given this PR is based on an out-of-date master with merge conflicts, it might be easier to start from scratch.
I tried to generate a fix like I suggested above, it's part of #1087 and so far works for the desi fast healpix mode only.
I guess a fix for other modes could look similar...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants