Slurm examples

Job Submission

PIConGPU job submission on the Taurus cluster at TU Dresden:

  • tbg -s sbatch -c etc/picongpu/0008gpus.cfg -t etc/picongpu/taurus-tud/k80.tpl $SCRATCH/runs/test-001

Job Control

  • interactive job:

    • salloc --time=1:00:00 --nodes=1 --ntasks-per-node=2 --cpus-per-task=8 --partition gpu-interactive

    • e.g. srun "hostname"

    • GPU allocation on taurus requires an additional flag, e.g. for two GPUs --gres=gpu:2

  • details for my jobs:

    • scontrol -d show job 12345 all details for job with <job id> 12345

    • squeue -u $(whoami) -l all jobs under my user name

  • details for queues:

    • squeue -p queueName -l list full queue

    • squeue -p queueName --start (show start times for pending jobs)

    • squeue -p queueName -l -t R (only show running jobs in queue)

    • sinfo -p queueName (show online/offline nodes in queue)

    • sview (alternative on taurus: module load llview and llview)

    • scontrol show partition queueName

  • communicate with job:

    • scancel <job id> abort job

    • scancel -s <signal number> <job id> send signal or signal name to job

    • scontrol update timelimit=4:00:00 jobid=12345 change the walltime of a job

    • scontrol update jobid=12345 dependency=afterany:54321 only start job 12345 after job with id 54321 has finished

    • scontrol hold <job id> prevent the job from starting

    • scontrol release <job id> release the job to be eligible for run (after it was set on hold)