Public archive of high-level products from MAGIC papers

[stefanomarchesi88]

Hi @bkhelifi @GernotMaier everyone,

I am Stefano Marchesi, a UniBO/CTA/INAF researcher and a recent MAGIC associate. I've recently started working with @micheledoro and Elisa Prandini on generating high level data (e.g., light curves, spectral energy distributions) in ecsv format for MAGIC published extragalactic papers, covering both the MAGIC information and the multi-wavelength data, when possible.

As a reference, I include here some files we originated from a recent MAGIC collaboration paper on the blazar 1RS 0647+250 (Long-term multi-wavelength study of 1ES 0647+250; https://ui.adsabs.harvard.edu/abs/2023A%26A...670A..49M/abstract).

1. A yaml file that summarizes the paper and reports the list of files made available in ecsv format.
   magic_2023a_yaml.txt

2. Two light curves (the MAGIC one and the OVRO 15 GHz band one) in ecsv format.
   magic_2023a_fig1a_lc_ecsv.txt
   magic_2023a_fig1h_lc_ecsv.txt

3. The multi-wavelength SED for the first of the four MAGIC epochs studied in this paper, in ecsv format.
   magic_2023a_fig3a_SED_ecsv.txt

We would like to take the opportunity to ask your opinion about the kind of information stored on files, if those are enough for you. We also note that we can easily convert these files in a gamma-cat format, if you deem this useful.

[GernotMaier]

Hello Stefano,

good to hear on this topic. Lot's of information in these files and this is good!

Some random comments / questions:

• noticed that you use sometimes Target and sometimes Source. Is this intended?
• on the same topic: in VTSCat we have struggled a while with the source naming, given that we had papers on the same source using different names used. We decided to keep for the files the numerical source identifiers from gamma-cat, and then use consistently VTS-names (see source table) for any human-related output (with the 'common' name as possible additional output).
• Fver is a File version which is changed by hand for any update? As this is text-based information and you probably keep it in a git repository, is this really necessary?
• I assume that for all paper-related data (e..g the author list), you are using the nice python API to ADS? I am asking, as I don't see the ADS identifiers anywhere (which are of difficult format for Unix directories).
• what does 0.00 mean in the SEDs for same of the values in the SED mean? Is that value really 0 or is it undetermined?
• en == Energy: I guess that is the energy where the point is drawn in the plots. Possibly at the bin center (but not always)?

[stefanomarchesi88]

Hi Gernot,

thanks a lot for the comments! Here are the answers:

• Target vs Source: there was no specific reason to use one term or the other, we now uniformed everything to "Source".

• Source name: thanks for the feedback on this, it's very useful. At the moment we used the source name reported in the paper we analyzed, but I agree that this could generate some inhomogeneities. We'll discuss this with @micheledoro and Elisa Prandini, and take into account your approach.

• Fvers: the idea was to update this number when major changes take place (i.e., datafiles are replaced, added or deleted), but I agree with you that this information basically comes for free when uploading the files on Git, so we can consider delete it.

• Paper related data: so far we manually downloaded them from the ADS archive, since it wasn't too time-consuming, but we can consider automatising the procedure in the next phase of the project. As for the ADS identifier, we have a Pads line in the yaml file, were you referring to something different?

• 0.0 points in SED: when a datapoint is actually a 95% confidence upper limit, we report the value of the upper limit in the flux best-fit value column, and we then flag with 0.0 the lower and upper uncertainties on the flux, so to identify this datapoint as an upper limit. We can consider adopting a different approach, if this doesn't seem clear enough.

• Ene point: these are taken directly from the files we got from the corresponding authors of the papers. In this specific paper it seems to me that this is indeed the bin center, as you mentioned.

Thanks a lot again for your comments, please let us know if you have any follow-up question or suggestion.

[GernotMaier]

Yes, I was referring to what is in the "PADS" field - didn't see this, all good!

[micheledoro]

Hi @GernotMaier thanks a lot for the comments! And thanks @stefanomarchesi88 for the replies.

The tag for the version of the file was thought because a) considering this file format is easily readable, it is possible it’s not downloaded from git but in other means (mail) b) considering that after publication some revisions could be made. In my mind these are still valid motives.

I am not sure I got the suggestion for the naming of the sources. What procedure did you use?

MD

[GernotMaier]

We use the git versioning for any changes, considering the different tools provided by git for sufficient to trace changes.

For the naming of the sources we use fields called common_name (how the field generally knows the source, but even this is sometimes not unique), veritas_name (which pushed us naming really all VERITAS detections, we had not been consistent with that), and other_names, which allows for readability to add a list of other names of the sources. For non-detection, the veritas_name is obviously not filled. As gamma_cat, in the data files (seds, lightcurves, ...) we only use the numerical source_id.

[micheledoro]

Hi @Gernot, thanks. Ok then I propose both things

• remove versioning
• use tag common_name, other_names and also why not adding magic_name which makes all easier to reference in the future internally.

Thanks!

[stefanomarchesi88]

Hi @micheledoro and @GernotMaier,

perfect, I'll update the file following your suggestions.