squeue & CephFS cheat sheet

Table of contents

• squeue
• CephFS

squeue

For all possible options visit SLURM documentation.

Check how many jobs are currently in the queue

squeue -h | wc -l

Check how many jobs have you submitted to the queue

squeue -u $USER -h | wc -l

Check how many jobs are currently in the running state

squeue -h -t r | wc -l

Check how many jobs are currently in the pending state

squeue -h -t pd | wc -l

Check how many jobs each user has submitted to the queue

squeue -h -o "%u" | sort | uniq -c | sort -nr -k2

Display the actual command, runtime, node and user who submitted jobs to the queue

squeue -h -o "%o %A %M %u"

CephFS

List statistics of your $HOME directory:

getfattr -n "<option>" $HOME

Where <option> in double quotes can be:

• ceph.dir.files -- number of files, including hidden
• ceph.dir.subdirs -- number of subdirectories, including hidden
• ceph.dir.entries -- sum of ceph.dir.files and ceph.dir.subdirs
• ceph.dir.rfiles -- same as ceph.dir.files, but recursively summed
• ceph.dir.rsubdirs -- same as ceph.dir.subdirs, but recursively summed
• ceph.dir.rentries -- sum of ceph.dir.rfiles and ceph.dir.rsubdirs
• ceph.dir.rbytes -- total number of bytes allocated by the directory

The getfattr command works with directories on /scratch*, too.

If you want to delete files faster than rm, you can use rsync. First, you have to set up an empty directory at, say $HOME/empty_dir/. Assuming that the directory you want to delete is at $HOME/to_delete/, you can use the following command to delete the contents of $HOME/to_delete/:

rsync -a --delete $HOME/empty_dir/ $HOME/to_delete/

If you want to delete multiple directories in this manner, you can always mv your stuff to $HOME/to_delete/ and then run the above command. Though having to remember that long command and to create the empty can be hard, so instead you may just create a function to you .bashrc:

function rmsync(){ find $@ | xargs -n1 -I{} bash -c 'mkdir -p empty/ && rsync -r empty/ {} --delete && rm -r {} empty/'; }

This function now allows you to use also wild card paths.