I found a few issues with pintk. They are demonstrated with the following files (remove .txt from the filenames after downloading).
J0437phoff.par.txt
test.tim.txt
J0437test.par.txt
- Post-Fit Chi2 is bonkers. To see this, run
pintk J0437test.par test.tim, then click "Fit". Here is what is printed:
------------------------------------
Pre-Fit Chi2: 127.87657
Pre-Fit DOF: 97
Pre-Fit reduced-Chi2: 1.3183152
Pre-Fit Weighted RMS: 5.6247845 us
------------------------------------
Post-Fit Chi2: 1465222
Has Correlated Errors False
Post-Fit DOF: 97
Post-Fit Reduced-Chi2: 15105.382
Post-Fit Weighted RMS: 5.5043546 us
------------------------------------
Parameter Pre-Fit Post-Fit Uncertainty Difference Diff/Unc
------------------------------------------------------------------------------------------------------------------------------------
F0 Hz 173.6879456649309113 173.6879456650772781 1.1830554e-10 1.4636679e-10 134.85374
F1 Hz / s -1.7283484555202134978e-15 -1.7309811914243010753e-15 1.9126623e-18 -2.6327359e-18 -821.45156
Akaike information criterion = -2314.022850977091
Notice that the weighted rms drops from 5.62 to 5.50 µs, but Chi2 is crazy.
I found that this is reversed if there is a PHOFF in the initial .par file, pintk J0437phoff.par test.tim. Here the pre-fit is huge, but the post-fit is fine.
------------------------------------
Pre-Fit Chi2: 1328707.7
Pre-Fit DOF: 96
Pre-Fit reduced-Chi2: 13840.705
Pre-Fit Weighted RMS: 5.6247845 us
------------------------------------
Post-Fit Chi2: 121.19168
Has Correlated Errors False
Post-Fit DOF: 96
Post-Fit Reduced-Chi2: 1.2624134
Post-Fit Weighted RMS: 5.5043547 us
------------------------------------
Parameter Pre-Fit Post-Fit Uncertainty Difference Diff/Unc
------------------------------------------------------------------------------------------------------------------------------------
F0 Hz 173.6879456649309113 173.68794566507727815 1.1830554e-10 1.4636685e-10 134.85379
F1 Hz / s -1.7283484555202134978e-15 -1.7309811924439748781e-15 1.9126623e-18 -2.6327369e-18 -821.45188
PHOFF 0.1 0.10511710065360719 0.0035830273 0.0051171007
Akaike information criterion = -2314.02284612503
-
In the previous output, notice that Diff/Unc is computed using the uncertainty that was in the initial par file, rather than the uncertainty on that parameter resulting from the current fit. This is confusing and wrong.
-
If you edit J0437phoff.par to have PHOFF 0.0 then the DownhillWLSFitter does not work, with a StepProblem. Changing to the WLSFitter does work, however. Here is what happens:
------------------------------------
Pre-Fit Chi2: 127.87657
Pre-Fit DOF: 96
Pre-Fit reduced-Chi2: 1.3320476
Pre-Fit Weighted RMS: 5.6247845 us
------------------------------------
Post-Fit Chi2: 127.87657
WARNING (pint.logging ): /Users/paulr/src/PINT/src/pint/fitter.py:1031 StepProblem: Unable to improve chi2 even with very small steps
Has Correlated Errors False
Post-Fit DOF: 96
Post-Fit Reduced-Chi2: 1.3320476
Post-Fit Weighted RMS: 5.6247845 us
------------------------------------
Parameter Pre-Fit Post-Fit Uncertainty Difference Diff/Unc
------------------------------------------------------------------------------------------------------------------------------------
F0 Hz 173.6879456649309113 173.6879456649309113 1.1830554e-10 0 0
F1 Hz / s -1.7283484555202134978e-15 -1.7283484555202134978e-15 1.9126623e-18 0 0
PHOFF 0.0 0.0 0.0035830273 0
Akaike information criterion = -2307.3379556298382
So, something is wrong in how Chi2 is computed and maybe it got introduced when PHOFF was added?
I found a few issues with pintk. They are demonstrated with the following files (remove .txt from the filenames after downloading).
J0437phoff.par.txt
test.tim.txt
J0437test.par.txt
pintk J0437test.par test.tim, then click "Fit". Here is what is printed:Notice that the weighted rms drops from 5.62 to 5.50 µs, but Chi2 is crazy.
I found that this is reversed if there is a PHOFF in the initial .par file,
pintk J0437phoff.par test.tim. Here the pre-fit is huge, but the post-fit is fine.In the previous output, notice that Diff/Unc is computed using the uncertainty that was in the initial par file, rather than the uncertainty on that parameter resulting from the current fit. This is confusing and wrong.
If you edit J0437phoff.par to have
PHOFF 0.0then the DownhillWLSFitter does not work, with a StepProblem. Changing to the WLSFitter does work, however. Here is what happens:So, something is wrong in how Chi2 is computed and maybe it got introduced when PHOFF was added?