Docker build update

New Features:

• Update docker build to run on releases and include ephem_template repo using a secret token

Bug Fixes:

• Only plot dynamic spectra if more than 1 nsub and assume there is already a dynspec file
• Use a while loop to keep reducing until dynamic spectra image under 1 MB
• Updated SBAND_1 freq chop range and test to cover it
• Change calc_max_nsub output slightly

General updates

New Features:

• Rewrote the channel chop function and created tests to check it works with different bands and nchans
• Made dockerhub actions
• Added calc_max_nsub script
• Updated argument names and improved handling of raw-only obs
• Updated fluxcal argument names and only flux calibrate on a single file
• Flux density is now an input argument instead of a calculation
• Use json file as DM results file
• Renamed the RM cat file and added TPA and RelBin values

Bug Fixes:

• Rename the fluxcal script to fluxcal_meerkat to prevent psrchive being called
• No longer output chi2r into tres in the result.json file
• Reduced dpi to 90 to reduce dynamic spectra image size to below 1 MB

Chop edge channels

Added a script to chop the edge channels off an archive and can now create images for just raw archvies

Docker image

Made a docker image that will be useful for testing and updated psrdb version to 3.0.2

Python package for main nf-core Nextflow

The main pipeline has now been moved to nf-core-meerpipe so all scripts that are no longer used have been removed and some scripts like dylfix have been integrated. This is now a support repo of python scripts for the nf-core-meerpipe repo.

PSRDB intergration

Operational version of MeerPipe on the OzStar HPC cluster as of May, 2023. Implements a number of significant upgrades and incremental fixes over the previous version.

Major New features

• Fully integrated with PSRDB, a backend database which records details of all MeerTIME observations and their subsequent processing through MeerPIPE. PSRDB integration also allows results to be displayed on the MeerTIME Data Portal (pulsars.org.au), along with the download of selected products.
• Full automation, such that new observations are processed as soon as they are transferred onto OzStar from the PTUSE machines.
• Expansion of manual launch options for efficient selection and reprocessing of individual observations.
• Production of reduced data products per the specification of individual projects (PTA / TPA / RelBin, etc.), allowing for the scientific needs of each individual project.
• Complete UHF and L-Band flux and polarisation calibration, thanks to significant input from Simon Johnston, Bettina Posselt & Lucy Oswald.
• Production of diagnostic images, including TOA residuals showing the state of the pulsar's timing.
• Automatic logging of per-observation flux density, S/N and DM values, as well as the fraction of the observation zapped during cleaning.
• Dynamic resource requests for SLURM jobs (RAM & walltime) based on the size of the observation to be processed, allowing for more efficient processing.

Original version

The original version of MeerPipe before PSRDB database interactions were introduced.