Inconsistent Data Extraction in pet-surface Pipeline for Specific Subjects

[yunyunjin]

Description
What is happening?

Pipeline
Which Clinica's Pipeline is concerned?

• pet-surface
  ...

Hello Clinica Team,

I've encountered a problem with the pet-surface pipeline while working on my dataset within a Docker environment on Ubuntu 20, using the latest version of Clinica. Despite most subjects being processed successfully, there seems to be a specific issue with 50 out of 200 subjects.

Issue Description:
For these 50 subjects, the output Desikan and Destrieux TSV files are missing mean_scalar values. Here’s an example of the path to one of the files for reference:

CAPS_Dataset/subjects/sub-H095/ses-M000/pet/surface/atlas_statistics/sub-H095_ses-M000_trc-18FFBB_pet_space-desikan_pvc-iy_suvr-cerebellumPons_statistics.tsv

I have re-run the pipeline for these subjects but the issue persists with the same mean_scalar values missing.

What might be causing this issue for a subset of the dataset? If there are specific logs or diagnostic steps that could help isolate the issue, I would greatly appreciate your guidance.

Thank you very much for your support and for developing Clinica!

Best regards,
Yunjin

[AliceJoubert]

Hello Yunjin,

Thanks for reporting ! I could not reproduce with the data I have. You could indeed try to look at the difference between the logs between a successful run and a 'failed' one, but it is difficult to say which logs to look at without reproducing. If that is alright with you, could you send us an image that had a mean_scalar value and one that did not, or a least a snapshot ?

Best,

[yunyunjin]

Hello Alice

I'm attaching the log file that came out when I reran the PET-surface pipeline by collecting only the cases where the mean_scalar value has not been derived. I'm also sending you the well-processed person's PET DICOM file and the PET DICOM file of the unprocessed person. I hope these materials can help solve the problem. Please let me know if there is anything else I need to do.

I’m hoping these files help figure out the next steps. Please let me know if there’s anything else you need from my side.
Thanks for your help and looking forward to your insights.

Best regards,

pypeline.log
pet_dicom.zip

[AliceJoubert]

Hello !

Thanks for sending this, I will be looking into the problem. Do you actually mind sending the whole volume (if you have the niftis ?) and the others parameters required for the command line ? I might need to run it on my side to explore more !

Best,

[yunyunjin]

Hello,

I am sending the nifiti file of pet. My command is as below.

clinica run pet-surface /data/pet/5th_BIDS_Dataset_centered/ /data/pet/5th_CAPS_Dataset_test2/ 18FFBB cerebellumPons
 /data/pet/psf_values_test2.tsv

I cannot send the freesurfer outputs stored in CAPS due to security concerns. Could the issue possibly be related to the freesurfer outputs?

5th_BIDS_Dataset_centered.zip
psf_values_test2.zip

Thanks for your help.
Best regards

[NicolasGensollen]

Hi @yunyunjin

Looking at the logs you provided, it seems like there are some NaNs in the freesurfer projection images. For example:

stdout 2024-09-13T11:17:41.230034:Reading curvature file /data/pet/4th_CAPS_Dataset/subjects/sub-H095/ses-M000/t1/freesurfer_cross_sectional/sub-H095_ses-M000/surf/lh.averaged_projection_on_cortical_surface.mgh
240913-11:17:41,240 nipype.interface INFO:
	 stdout 2024-09-13T11:17:41.230034:NaN found at voxel (0, 0, 0, 0)
240913-11:17:41,240 nipype.interface INFO:
	 stdout 2024-09-13T11:17:41.230034:error: No such file or directory
240913-11:17:41,241 nipype.interface INFO:
	 stdout 2024-09-13T11:17:41.230034:error: WARNING: 121504 NaNs found in volume /data/pet/4th_CAPS_Dataset/subjects/sub-H095/ses-M000/t1/freesurfer_cross_sectional/sub-H095_ses-M000/surf/lh.averaged_projection_on_cortical_surface.mgh...

It seems to be the only kind of errors/warnings in the logs so I suspect this might potentially be the reason you're getting unexpected outputs.

One thing you can do to test this hypothesis a bit is to check the logs of the pet-surface pipeline for subjects for which you didn't have the issue and see whether you have the same NaN related warnings or not.

You can also inspect the images in your CAPS (for example /data/pet/4th_CAPS_Dataset/subjects/sub-H174/ses-M000/t1/freesurfer_cross_sectional/sub-H174_ses-M000/surf/lh.averaged_projection_on_cortical_surface.mgh) to see how they look, where are the NaNs and so on.

These images come from the t1-freesurfer pipeline, so it could be interesting to re-run the pipeline on these specific subjects. If the outputs still contain NaN values, checking the logs of this pipeline would probably tell us more.

Hope this helps, let us know how it goes !

[yunyunjin]

Thank you for your help!! As you indicated, there were no NaN errors for those who were processed correctly. For those who did not complete the pet-surface processing, it seemed there might be issues with the T1 images. Addressing these problems was challenging, so as an alternative, I switched from 3D T1 images to spgr images for the reconstructions. This change allowed successful processing for several additional subjects.

However, a new error has surfaced during the process.

# clinica run pet-surface /data/pet/4th_BIDS_Dataset_centered_t1_pet/ /data/pet/4th_CAPS_Dataset 18FFBB cerebellumPons /data/pet/psf_values_4th.tsv -np 10

Traceback (most recent call last):
  File "/data/pet/clinicaEnv/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/data/pet/clinicaEnv/lib/python3.10/site-packages/nipype/interfaces/utility/wrappers.py", line 142, in _run_interface
    out = function_handle(**args)
  File "<string>", line 64, in make_label_conversion
Exception: Could not find label 1000 on conversion table. Add it manually in CSV file to correct error

It looks like this error is about a missing label in a conversion table used during the pipeline's run. Could you shed some light on what might be causing this issue, or show me how to fix it?

Thank you again for your help!

pypeline.log