Automated Testing

[mperrin]

The Keck OSIRIS development branch has already worked out how to connect pytest and IDL. Since the OSIRIS pipeline is one of the parents of the GPI data pipeline, it ought to be reasonably straightforward to port this stuff over.

https://github.com/Keck-DataReductionPipelines/OsirisDRP/tree/develop

[mperrin]

Looks like some amount of what's that subtree is equivalent to functionality we already have in gpipy - i.e. tools for parsing XML files etc. So we can use gpipy instead of that stuff.

[mperrin]

Notes - @mperrin will handle setup of infrastructure. @kfollette will make some tests. Let's assume for now the tests will run on a machine that has a local copy of the full Dropbox (e.g. at Berkeley, Cornell, or STScI) and a local working IDL.

Let's not worry about running this in the cloud on travis or anything like that. Once we have the basic test framework functional, we can use Github web hooks to trigger a local test run on one of our computers for every github commit. See https://developer.github.com/webhooks/

[mperrin]

End-to-end functional test suite

• basic make-a-cube
• spec mode - beta pic or ASU data - make cube and check sat spots
• pol mode - 4796A check center and check polarized flux exists in radial pol image
• calibrations - make some quick wave cal

Implementation

• Each test will be a subdirectory of tests, which contains at minimum one XML recipe and one python function that implements the test criteria.
• The input files for the tests will exist in Dropbox\Pipeline Releases\Data_for_DRP_Tests, with a subdirectory for each test. In addition to the input raw files this can optionally contain output files we want to compare to. We can refer to this directory as $GPIESTESTDATA in recipes.
• We assume the user will already have a working calibration database and don't have to do anything manually about that.

[semaphoreP]

Is there anything left to do on this issue? I recall testing the testing code and being happy with the current state. I guess the one thing mentioned that we haven't done is adding a hook so that the tests are run locally on our machines?

[mperrin]

Yeah, the remaining task is setting up some continuous integration setup so these tests actually get executed semi regularly somewhere.