Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add utility tool to validate histogram contents against a reference #149

Merged
merged 7 commits into from
Jun 7, 2023

Conversation

eguiraud
Copy link
Contributor

@eguiraud eguiraud commented May 29, 2023

A reference file for the configuration with N_FILES_MAX_PER_SAMPLE=1 is provided as well.

The tool:

  • compares histogram edges and contents against a JSON reference file
  • can also dump the contents of the histograms as json in order to produce new reference files
  • ignores mismatches in the contents (but not the edges) of histograms with name containing "pt_res_up", because the content of these histograms depends on a random number generator
  • only considers the highest namecycle in case there are multiple cycles for the same histogram (e.g. name;1 and name;2)

Fixes #136 .

eguiraud added 2 commits May 29, 2023 15:11
Example usage:
python validate_histograms.py --reference reference/histos_1_file_per_process.json
python validate_histograms.py --dump-json | jq
Copy link
Member

@alexander-held alexander-held left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This is going to be very helpful. Given that we'll want to add more references for different settings and also will pick up some new systematics that will cause histograms to change, could you perhaps also add a script example for the JSON reference generation?

@eguiraud
Copy link
Contributor Author

That would be just python validate_histograms.py --dump-json (assuming the input has the default file name histograms.root, otherwise you also pass that to the --histos option).

Is that what you mean? I'm not sure where to put that other than the --help as it's now?

@alexander-held
Copy link
Member

Perfect, I hadn't looked closely enough at the code yet. No need to change anything then!

@ekauffma
Copy link
Collaborator

ekauffma commented Jun 1, 2023

Can we add a description of this tool to the README for cms-open-data-ttbar?

@eguiraud
Copy link
Contributor Author

eguiraud commented Jun 1, 2023

@ekauffma sure, done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a way to validate output histograms against a trusted reference
3 participants