Vertebra missegmentation in Gold Standard CT dataset

[Smir-Nik]

Hello Dr. Wasserthal,

My name is Nikolai Smirnov, I'm a radiologist.
I recently conducted a review of the data from the TotalSegmentator CT set. To be precise, I randomly selected about 100 cases from your set of Gold Standards downloaded from here.
During the review I considered only the segmentation of the vertebrae and noticed that some of the cases (about 10-15% in my set) have a well-marked external contour, but the vertebrae are mixed together.
Perhaps these inaccuracies are the reason that the TotalSegmentator itself periodically mixes the vertebrae as well.
I am attaching several screenshots and patient numbers that you can check to see the errors.

S0941
S0350
S0513
S0603
S0676
S0071
S0128

I will be glad to help if you need any support with reviewing and correct these segmentations.

Best regards,
Nikolai Smirnov

[lassoan]

I had a look at s0941. The "ground truth" segmentation indeed has errors in separation of 3 vertebrae. Segmentation result mostly good, but there is still an error in 1 vertebra.

[wasserth]

There are indeed several cases with errors in the vertebrae ground truth in the training data (the same is true for the ribs). Fixing these kind of errors is very time consuming. On our agenda it has low priority. Therefore it did not happen so far. If anybody wants to correct these errors and submit fixed ground truth files I would be very happy to retrain the model.

[Smir-Nik]

I see.
Thanks for your reply.

If I provide you with corrections in the form of .nrrd segmentations files containing all the objects of the spine, is this suitable for you?

Unfortunately, I'm familiar and fast in 3DSlicer, but not very good at writing scripts, and exporting it all separately to NIFTI manually takes a very long time.

Best,
Nikolai

[lassoan]

Unfortunately, I'm familiar and fast in 3DSlicer, but not very good at writing scripts, and exporting it all separately to NIFTI manually takes a very long time.

This should be no problem at all. ChatGPT should be able to write all the scripts that are needed for this, but I can help with that, too, if you get to that point.

[gokceay]

@Smir-Nik If you need any help, I can assist with correcting the segmentations. I have been using Slicer and TotalSegmentator for my projects for a while, and it would be ideal to use high-quality segmentations to train TotalSegmentator. I was also lookong ways to improve the totalsegmentator model.

@lassoan If we provide TotalSegmentator with more data, its segmentation predictions will improve, right? Also, is it possible to increase the resolution of TotalSegmentator by reducing the slice distance from 1.5 to 0.5?

I found this https://annotate.totalsegmentator.com/

[wasserth]

If providing TotalSegmentator with more and better data, then the segmentations will improve. But it is not guaranteed by how much they will improve.
TotalSegmentator is based on 1.5mm resolution. This is a good tradeoff between accuracy and speed and memory requirements. At the moment I am not planning to switch to 0.5mm resolution. This result in a lot higher runtime and memory requirements.

https://annotate.totalsegmentator.com/ is a very new website I created for people to help with the annotations. So this is now the best way if you want to contribute to the totalsegmentator dataset.

[fedorov]

@wasserth I am not experienced with your new website (and I am not a radiologist to contribute any of the corrections for the annotations), but can you confirm - are you saying that the so called "Gold Standards" images with incorrect segmentations of vertebrae and ribs is already included in that website, and are prioritized for review by its users?

It seems that correcting the known problems with the "Gold Standards" data should be absolutely top priority.

For your reference, we applied TotalSegmentator to segment >120,000 lung screening CT scans, and we see invalid vertebrae/ribs segmentations even in the first handful of random images picked from that large cohort.

[Smir-Nik]

Dear dr @wasserth.

I have to offer my deepest apologies.

Since the opening of this discussion, in my free time I have conducted a review of the entire CT dataset and made corrections for almost 200 cases in which similar errors were observed.

On average, 5-7 vertebrae were corrected in such cases.

I wanted already share it to you for free.

Unfortunately, my company forbade me to give you this data, citing a conflict of interest (we are developing Spine AI).

I recently joined development from a medical environment where sharing knowledge is more accepted, so I didn't expect a trick when I promised to make corrections.

I've been trying to persuade my superiors for the last month, but I haven't been able to do it.

I apologize again.

Best regards,
Nikolai Smirnov