Extensions for approximate inference vignette

[athowes]

In PR #69 we added a vignette fitting a simple model with four inference techniques. This includes Pathfinder, which I also presented about here.

There are a variety of improvements I think could be made to the vignette. Some of these improvements are closer to research projects:

1. A demonstration of some more sophisticated or complex model which can be fit quickly with approximate methods.
2. A more complete simulation study which allows recommendations to be made about when to use approximate inference methods.
3. Discussion about the use of approximate methods for initialising HMC. Perhaps could include code testing performance of randomly initialised HMC versus e.g. Pathfinder initialised HMC. Could also include PR implementing e.g. Pathfinder initialised HMC as default within epidist.
4. Could post case-study to Stan forum demonstrating a case where Pathfinder fails. This may require some more exposition of the failure to help with debugging. We may also have suggestions about how to make the algorithm more robust.

• If, at some point, the algorithm is more robust then we should use multi-Pathfinder rather than single Pathfinder in the vignette.

5. Similar to above, could include some exploration of how different the Gaussians in multi-Pathfinder are, including exlcluding ones which had optimisation paths that failed. Found use of the option to save individual paths hard to use as discussed in the vignette.

[athowes]

We should consider making the vignette more accessible by moving some material to an appendix or providing clearer explanations:

• As a Bayesian newcomer, I also found the explanations quite happy technically. Could you consider moving that to an "appendix" in the vignette so that it doesn't distract the reader who only cares about the immediate results?

Originally posted by @jamesmbaazam in #69 (comment)

[seabbs]

I agree that these are all intereting research topics that could be explored.

I think

We should consider making the vignette more accessible by moving some material to an appendix or providing clearer explanations:

Is really its own issue and we should consider doing this sooner rather than later (potentially @jamesmbaazam would be interested in tackling).

[athowes]

See also this thread on the Stan forum: https://discourse.mc-stan.org/t/using-pathfinder-and-other-algorithms-to-infer-epidemiological-delay-distributions-and-to-initialise-rt-estimates/35929/3

[seabbs]

I'd be tempted to move this to a discussion topic or a epinowcast community post as I think its really a research project that could result in a vignette vs a vignette plan as it stands?

[athowes]

Yes agree it's more of a research project.