You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The PR #55 fixed a bug with the header check (the connector was checking all FITS header before starting the job. Good idea until you have +10,000 files....). The fix is however temporary, as you can see below:
// Check that all the files have the same Schema// in order to perform the union. Return the HDU type.// NOTE: This operation is very long for hundreds of files!// NOTE: Limit that to the first 10 files.// NOTE: Need to be fixed!valimplemented=if (listOfFitsFiles.size <10) {
checkSchemaAndReturnType(listOfFitsFiles)
} else{
checkSchemaAndReturnType(listOfFitsFiles.slice(0, 10))
}
The idea would be then to add an option to the DataFrameReader (e.g. checkHeader), false by default, that would trigger the header checker.
The text was updated successfully, but these errors were encountered:
The PR #55 fixed a bug with the header check (the connector was checking all FITS header before starting the job. Good idea until you have +10,000 files....). The fix is however temporary, as you can see below:
The idea would be then to add an option to the DataFrameReader (e.g.
checkHeader
),false
by default, that would trigger the header checker.The text was updated successfully, but these errors were encountered: