Prior sensitivity

[maxbiostat]

Our whole argument hinges on the fact that the prior might remotely matter. After a very informal search of the literature, I've found many instances of people claiming that "the inferences were robust to the choice of prior". A couple possibilities are:

• people lie;
• people are idiots: inferences actually do change, but they haven't looked closely enough to notice, since it'd be an inconvenience;
• Many/most likelihoods employed in the literature are very dominant w.r.t. the posterior.

If that last one is also the case [the first two are a certainty :) ] it would be very nice to know why and when the likelihood is robust.

[lsbastos]

The idea here should be see how sensitive is the posterior distribution by changes on the prior hyperparameters.

If there is a measure or a quantity, lets say Q(), from the posterior, we could evaluate how changes on the prior hyperparameters affect this quantity. Q would be a function of the hyperparameters (a1,a2,...,ap), so we could then try several combinations of (a1,a2,...,ap) and see the impact of each hyperparameter on Q (which could also be a vector of posterior quantities).

Well, I know that each combination would lead to a different MCMC which is computationally expensive, but this is how computer model people work, see for instance [https://en.wikipedia.org/wiki/Sensitivity_analysis](Sensitity analysis).

My suggestion would be:

• for a given data sample N observations from the prior hyperparameters space (using a Latin hypercube)
• for each set of hyperparameters run the MCMC,
• after all runs we analyse how each parameter a_i affects Q (in a simple scatter plot).

Probably this kind of prior sensitivity analysis has already been done out there, perhaps even done in a more clever way. I am just not aware of (which doesn't mean anything really)

[maxbiostat]

See also #25.

We'll need an up-to-date discussion of prior sensitivity. In our case, we have the advantage that our problem is domain-specific, so we have the possibility of building custom loss functions. For example, we can look at the impact of priors on the time of peak, total number of cases (integral of I(t) in t), etc.

[maxbiostat]

@lsbastos I used some of the stuff we collected for COVID-19 but ended up not using and put in these notes.