diff --git a/README.md b/README.md index fcc5fe1607614c0e913ef39a204c0d21be8a6112..bb60ef4e228f7b57b358a80977bbfd674ebc9c8d 100644 --- a/README.md +++ b/README.md @@ -4,12 +4,15 @@ Welcome to the home page of the Microbial Bioinformatics course. ## Course material -## Introduction to UNIX and HPC + +### Introduction to UNIX and HPC - [ ] [Introduction to UNIX slides](slides/introduction_UNIX_HPC.pdf) - [ ] [Exercise instructions](exercises/UNIX_HPC_exercise_instructions.md) - [ ] [Cheat sheet](exercises/UNIX_HPC_cheat_sheet.md) +### Assembly +- [ ] [Exercise instructions](exercises/assembly_exercise_instructions.md) diff --git a/exercises/UNIX_HPC_cheat_sheet.md b/exercises/UNIX_HPC_cheat_sheet.md index 89116462a39f3183c99842921bfa8e8eebeeddb3..6a25e03e958bbc8b8a83328d4fd6d4309fb62765 100644 --- a/exercises/UNIX_HPC_cheat_sheet.md +++ b/exercises/UNIX_HPC_cheat_sheet.md @@ -27,6 +27,7 @@ cp foo.txt bar.txt # Copy file mv foo.txt bar.txt # Move/Rename file rm foo.txt # Delete foo.txt rm * # Delete all files in current directory +ln -s foo bar # Create a link 'bar' to the file 'foo' ``` ## Read and manipulate files @@ -39,6 +40,7 @@ tail foo.txt # Print bottom 10 lines of file nano foo.txt # Simple file editor vi foo.txt # Advanced file editor wc foo.txt # List number of lines words and characters in the file +cut -d [delimiter] -f [field] foo.txt # Cuts out the specified fields that are separated by the specified delimiter ``` ## Input and Output @@ -47,7 +49,43 @@ wc foo.txt # List number of lines words and characters in the file echo "Hello world!" # Print statement to standard output echo "foo" > bar.txt # Direct output into file. Overwrites file if it already exists. echo "foo" >> bar.txt # Direct output into file and append if it already exists. -command 1 | command2 # Directs output from command1 as standard input into command2 +command1 | command2 # Directs output from command1 as standard input into command2 +``` +<br><br> + + + +# SLURM Cheat Sheet + +## Connect to Science Cluster + +```bash +ssh shirtname@cluster.s3it.uzh.ch +``` + + +## Submitting jobs + +In a script `myjob.sh`: +```bash +#!/usr/bin/env bash +#SBATCH --cpus-per-task=[number] +#SBATCH --mem=[memory] +#SBATCH --time=[hr:min:sec] +#SBATCH --job-name=[name] +#SBATCH --output=[name]_%j.out +#SBATCH --error=[name]_%j.err + +# load any required modules +module load [module name] + +command1 +command2 +command3 +``` +Submit script as job to the computing nodes: +```bash +sbatch myjob.sh ``` diff --git a/exercises/UNIX_HPC_exercise_instructions.md b/exercises/UNIX_HPC_exercise_instructions.md index 436d070d71f7b26e30c8c5d019c0344e7fdf520c..c6d0c7bb0c4962cf765e60dc567a1bba55bf562a 100644 --- a/exercises/UNIX_HPC_exercise_instructions.md +++ b/exercises/UNIX_HPC_exercise_instructions.md @@ -90,4 +90,161 @@ the content of directories. </p> </details> -<br> \ No newline at end of file +<br> + +## Exercise 2 - Creating and moving directories and files + +**Objective:** learn to use the `mkdir`, `cp` and `mv` commands. + +1. **Create directories** with the `mkdir` command: + * At the root of the `exercises/` directory, create 2 new + sub-directories: `species_by_genus` and `species_by_common_name`. + * In `exercises/species_by_genus/`, create 2 new sub-directories: + `Dendrolagus` ([tree-kangaroos](https://en.wikipedia.org/wiki/Tree-kangaroo)) + and + `Crocidura` ([a genus of shrews](https://en.wikipedia.org/wiki/Crocidura)). + * In `exercises/species_by_common_name/`, create 2 new sub-directories + named `B`, and `R`. + +2. **Copy files** using the `cp` command: + * From the directory `RedList_mammals/`, make a copy of all files of the + genuses `Dendrolagus` and `Crocidura` into their respective + sub-directories in `species_by_genus/`. + * From the directory `RedList_mammals/`, copy the file for the + [Red Wolf](https://en.wikipedia.org/wiki/Red_wolf) *Canis rufus* to the + directory `species_by_common_name/`. + +3. **Move and rename files** with the `mv` command: + * Enter the `species_by_common_name/` directory. + * In the directory, move the file `Canis_rufus` into subdirectory `R`. + * Rename the `Canis_rufus` file you just moved into the subdirectory `R` to + the common name of the species: `Canis_rufus` -> `Red_wolf`. + +4. **Copy and rename files** with the `cp` command: + * Similarly to what we did for the Red Wolf file, we will now copy and + rename the file for the + [Black Rhinoceros](https://en.wikipedia.org/wiki/Black_rhinoceros) + *Diceros bicornis*, but this time in a single step with the `cp` command. + * Copy the file `Diceros_bicornis` from its original location (in + `RedList_mammals`) into `species_by_common_name/B`, while directly + renaming it to the common name of the species: `Black_rhino`. + +5. **Copy, rename and delete directories**: + * Go back to the root of the `exercises/` directory. + * Copy the entire directory `species_by_genus/Dendrolagus/` with all its + content to the root of `exercises/`. + * Rename the directory to `Tree-kangaroos`. + * Delete the directory `Tree-kangaroos` and its content **in a safe way**. + + +<details><summary><b>Exercise solution</b></summary> +<p> + +1. Create the `IUCN_species_by_genus` and `IUCN_species_by_common_name` + directories. + + ```sh + # Option 1: create one directory after the other. + mkdir species_by_genus + mkdir species_by_common_name + + # Option 2: create both directories with a single command. + mkdir species_by_genus species_by_common_name + + # Option 3: use brace expansion to avoid repeating the common part of + # the directory names. + mkdir species_by_{genus,common_name} + ``` + + Create sub-directories `Dendrolagus` and `Crocidura`: + + ```sh + # Option 1: create sub-directories from the exercises/ directory. + mkdir species_by_genus/Dendrolagus species_by_genus/Crocidura + + # Option 2: enter the species_by_genus/ directory, then create the + # sub-directories "Dendrolagus" and "Crocidura". + cd species_by_genus/ + mkdir Dendrolagus Crocidura + cd .. + + # Option 3: same as option 1, but using brace expansion to avoid repetition. + mkdir species_by_genus/{Dendrolagus,Crocidura} + ``` + + Create sub-directories `R` and `B`: + + ```sh + mkdir species_by_common_name/{R,B} + ``` + + *Note:* using the `-p` option of `mkdir`, it is possible to create + multiple levels of directories in a single command. For example, we could + create all the directories from this exercise in a single command: + + ```sh + mkdir -p species_by_{genus/{Dendrolagus,Crocidura},common_name/{R,B}} + + # Pro-tip: if you want to preview the output of a brace expansion by + # the shell, you can run the command prefixed with "echo": it will print + # the command that would be executed to the terminal without running the + # command. + echo mkdir -p species_by_{genus/{Dendrolagus,Crocidura},common_name/{R,B}} + ``` + +2. Copy files for `Dendrolagus` and `Crocidura`: + + ```sh + cp RedList_mammals/Dendrolagus_* species_by_genus/Dendrolagus/ + cp RedList_mammals/Crocidura_* species_by_genus/Crocidura/ + ``` + + Copy the file for the Red Wolf: + + ```sh + cp RedList_mammals/Canis_rufus species_by_common_name/ + ``` + +3. Move and rename the Red Wolf file: + + ```sh + cd species_by_common_name/ + mv Canis_rufus R/ # Move the file into its subdirectory. + mv R/Canis_rufus R/Red_wolf # Rename the files to the common name of the species. + ``` + +4. Copy and rename the file for teh Black Rhino in a single `cp` command: + + ```sh + # Note: this assumes you are currently in directory "species_by_common_name". + cp ../RedList_mammals/Diceros_bicornis B/Black_rhino + ``` + +5. Copy, rename and delete a directory: + + ```sh + cd .. # Go back to the root of the `exercises/` directory. + cp -r species_by_genus/Dendrolagus/ . # Copy the directory and its content. + mv Dendrolagus/ Tree-kangaroos # Rename the directory. + ls -l + + # To delete the directory in a safe way, we first delete all the files + # inside it, and then delete the empty directory with "rmdir". + # Note that "rmdir" will not delete a directory if it's not empty - this + # is a safety behavior to avoid deleting large number of files by mistake. + rm Tree-kangaroos/* + rmdir Tree-kangaroos + ``` + + *Note:* the faster way to delete the directory and all of its content would + be to use the command: `rm -rf Tree-kangaroos`. + + This recursively delete the directory, and therefore one has to be careful + to be deleting the correct directory, as you can otherwise very quickly + delete large amounts of data by mistake. + +</p> +</details> + + +