zeus has a hard time with a simple transit fit problem (as compared to emcee)

[nespinoza]

Hi @minaskar,

First of all: thanks a ton for this package! We were excited to read about it in your paper, and it looks so promising that our team has started an implementation to add it into our juliet package (nespinoza/juliet#59).

Typically, the first tests we do when implementing a new sampler into juliet is to try it on a simple exoplanet transit fit (i.e., fitting the relative flux of an exoplanet --- we typically use TESS lightcurves for this). In particular, this is an 8-dimensional problem (so, "easy"). Our initial tests with juliet typically "failed" with zeus, meaning, the posteriors looked wonky.

At first I thought maybe we made some mistake implementing zeus, so we decided to re-code a minimal working example for you to look at; you can find that here: https://github.com/nespinoza/zeus-transit-example/blob/master/zeus_tests.ipynb. There, I compare zeus against two other samplers:

1. emcee, using the exact same log-probability function and starting point for the walkers.
2. MultiNest (via our juliet package), a nested sampling scheme (i.e., sampling directly from the prior, so no need to define starting points) for which this problem is a very easy one.

Interestingly, zeus typically fails 1/3 of the way with an error that reads:

RuntimeError: Number of contractions exceeded maximum limit!
Make sure that the pdf is well-defined.
Otherwise increase the maximum limit (maxiter=10^4 by default).

If I try enlarging the maxiter, it just gets stuck again in general. If you run it a handful of times, though, one might get lucky and you would be able to sample from the posterior (as it is done in the notebook --- but I encourage you to re-run the notebook so you can see the error I mentioned above). Here's the comparison of zeus versus emcee samples:

As in our juliet tests (not shown here, but we can share them with you if you want), the zeus posteriors look a bit weird. Some walkers apparently go crazy, and try to do their own sampling away from the rest. As a comparison, here's the posterior with MultiNest:

As you see, emcee and MultiNest agree very well with the posterior. But, again, don't know why zeus has such a hard time with this particular problem. So, two questions:

1. Why is zeus getting stuck in this particular problem?
2. Why when it doesn't get stuck, it produces such weird-looking posteriors?

Thanks in advance for any pointers!
Néstor

[minaskar]

Hi @nespinoza,

Thanks for the detailed example. I'll investigate and I'll you know.

Cheers,
Minas