seqkit may silently output incorrect results when I/O errors occur

[yoshiF7d]

this is inherently not reproducible and I'm not 100% certain about this. but I think that seqkit may produce incorrect output without displaying any error messages when encountering I/O errors.

My file system experienced frequent I/O errors (detected and reported by other programs like gzip,cp), I observed seqkit sample command produced the corrupted output. which was fine at the start but at some lines, the structure was broken. example:

+
FFFFFFFFFFFFTFFFF1:N:0:TFFFFFFFACAGGGGACAAGAAGCCAAAACGGATC80GGAC8:HTLKJDGAAAACGGT891TTTGFFFFF:FFFFFFFF0:TGGCCGGT+GCGCTCTA
GGGGACAAGAAATACTGATCGAAGTCATGGTCTATTTAGCCAAAACGGATGGGATGGGTGGTFFFFFFFFFFFFFFFFFFFFFFFFFFFFFCAGGGGACAAGAAGCCAAAACGGATGGGATGCTATTTAC0:TFFFFFFFFFFFFFFFFFFFF:FG2GTTTG
+FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFF
@A01090:973:HTLKJDMXFFFFFFF+GCGCTCTFFFFHTLK973:HTLGGCCCTCTAGGTTTG
+GATGGGTGGLKJDMTAGG0GGAC8
+

I checked the stderr output but there's no io error message in it.

[shenwei356]

What did cp/gzip say about the error?

The gzip read and write package seqkit uses should report errors of incomplete gzip files.

[yoshiF7d]

@shenwei356 thank you very much for your responses. The error message was something like "IO error has occurred," though I’m not sure of the exact wording. Aditionally, commands like "ls" were hanging, and Trimmomatic was reporting "IO errors" as well.
I think the gzip file is not corrupt. I received more information about our filesystem today, and it seems that write failures occur occasionally, but not read failures.

[shenwei356]

I see. It's better to wait for the server administrator to fix the IO error, which might be a file system or disk failure.