Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First version of an MCMC sampler #829

Merged
merged 17 commits into from
Nov 8, 2024
Merged

First version of an MCMC sampler #829

merged 17 commits into from
Nov 8, 2024

Conversation

JanWeldert
Copy link
Collaborator

@JanWeldert JanWeldert commented Sep 27, 2024

First version of an simple MCMC sampler as part of the analysis class. You need to manually install emcee at the moment.

This PR is part of #804.

I also sneaked in a fix for the search for shared parameters in the detectors class.

(Again ignore the first 7 commits)

Copy link
Contributor

@thehrh thehrh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see inline

pisa/analysis/analysis.py Show resolved Hide resolved
pisa/analysis/analysis.py Outdated Show resolved Hide resolved
pisa/analysis/analysis.py Show resolved Hide resolved
@philippeller
Copy link
Collaborator

When the used metric is chi2, it should be halved so that it can be interpreted as a makeshift log-likelihood. I don't see that happening in the code. Can you verify you get compatible results using chi2 and llh? And does the result also give compatible results with our standard analysis?

@JanWeldert
Copy link
Collaborator Author

Ah right, good point. I will add the check for chi2 metrics.
I haven't compared it to any frequentist analysis so far, mainly because it is pretty slow.

@JanWeldert JanWeldert marked this pull request as ready for review November 6, 2024 16:06
@JanWeldert
Copy link
Collaborator Author

BTW, I have a little toy example comparing the Frequentist and Bayesian results. I won't put it into pisa, but maybe can go into the fridge?

Same as scaled_chain, but for the burn in phase.

"""
import emcee

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May want to add a comment specifying which sampling algorithm the emcee package uses, for future reference.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the option to define the sampling algorithm and pointed to the emcee docs where one can find the different options and the default

"The sampler will assume that llh=0.5*chi2.")

params_truth = hypo_maker.params.free
x0 = np.array([p._rescaled_value for p in params_truth])
Copy link

@JoshHenryPeterson JoshHenryPeterson Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, x0 is only used in the next line, and we only seem to use the length of the array. May be able to save some time and memory by extracting the length directly from params_truth.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I simplified the code a bit

@JoshHenryPeterson
Copy link

If you are going to put a toy example in the fridge please put it somewhere where it can easily be found, like maybe the bootcamp directory? I had to dig pretty deep to find the previous bayesian sampling work and I don't want that to happen to future analyzers.

@JanWeldert
Copy link
Collaborator Author

Could also have its own MCMC sampling or Bayesian analysis example folder

@thehrh thehrh merged commit 95eadf3 into icecube:master Nov 8, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants