UNIX_HPC_cheat_sheet.md

Bash Cheat Sheet

A cheat sheet for bash commands.

Manage and navigate directories

pwd                       # Print current directory path
ls                        # List directories
ls -a|--all               # List directories including hidden
ls -l                     # List directories in long form
ls -l -h|--human-readable # List directories in long form with human readable sizes
cd foo                    # Go to foo sub-directory
cd                        # Go to home directory
cd ~                      # Go to home directory
cd -                      # Go to last directory

Manage files

cp foo.txt bar.txt                                # Copy file
mv foo.txt bar.txt                                # Move/Rename file
rm foo.txt                                        # Delete foo.txt
rm *                                              # Delete all files in current directory 
ln -s foo bar                                     # Create a link 'bar' to the file 'foo'
mkdir foo                 # Create foo directory
rmdir foo                 # Delete empty foo directory
rmd -r foo                # Delete foo directory including content  

Read and manipulate files

cat foo.txt            # Print all contents
less foo.txt           # Print some contents at a time (g - go to top of file, SHIFT+g, go to bottom of file, /foo to search for 'foo')
more foo.txt           # Print some content at a time, only moving forward in document
head foo.txt           # Print top 10 lines of file
tail foo.txt           # Print bottom 10 lines of file
nano foo.txt           # Simple file editor
vi foo.txt             # Advanced file editor 
wc foo.txt             # List number of lines, words and characters in the file
wc -l foo.txt          # Counts the lines
cut -d [delimiter] -f [field] foo.txt     # Cuts out the specified fields that are separated by the specified delimiter out of each line of a file
touch foo.txt           # Create empty file if it does not exist or change "modified by" date of existing file    

Input and Output

echo "Hello world!"         # Print statement to standard output
echo "foo" > bar.txt        # Direct output into file. Overwrites file if it already exists.
echo "foo" >> bar.txt       # Direct output into file and append if it already exists. 
command1 | command2         # Directs output from command1 as standard input into command2    

Variables and arrays

myvar="foobar"              # Assign string to a variable
$myvar                      # Calling the variable
output=$(command)           # Assign output of a command to a variable
array=(x1 x2 x3 x4)         # Create an array 
${array[@]}                 # Access the whole array
${array[0]}                 # Access the 1st element
${!array[@]}                # Get indeces of all array elements

Control structures

For loop:

for i in {1..n}             
do
    command $i
done 

While loop:

while [ condition ]             
do
    command $i
done 

If else:

if [ condition ]
then
    command1
else
    command2
fi

Running Singularity containers

Running a container called my_software_container that contains a command called my_software

my_software_container.sif my_software [parameters]             

SLURM Cheat Sheet

Connect to Science Cluster

ssh shortname@cluster.s3it.uzh.ch

Copy files to Science Cluster

scp foo.txt <shortname>@cluster.s3it.uzh.ch:<target directory>

Submitting jobs

In a script myjob.sh:

#!/usr/bin/env bash
#SBATCH --cpus-per-task=[number]
#SBATCH --mem=[memory]
#SBATCH --time=[hr:min:sec]
#SBATCH --job-name=[name]
#SBATCH --output=[name]_%j.out
#SBATCH --error=[name]_%j.err

# load any required modules 
module load [module name]

command1
command2
command3

Submit script as job to the computing nodes:

sbatch myjob.sh