Introduction to Computational Quantum Chemistry
Introduction to Unix
Martin Novák (NCBR)
Introduction to Unix
23. září 2014
1 /35
Unix
• Developped in 1970s in C language
• Open source code
• Multiuser system
• Case-sensitive system
• Many distributions developped since:
• Ubuntu
• Debian
• BSD
• Fedora a ...
Introduction to Unix 23. zafi 2014 2/35
Cluster Wolf
Head administrátor: Mgr. Jakub Štěpán
Scientific software administrátor: RNDr. Petr Kulhánek, PhD.
tfolf.ncbr.muni.cz ;
wolfOl	wolf02	wolf03	wolf04	WDlf05
wolf03 ■ ■ ■ ■ wolf23
Computational workstations (nodes)
Martin Novák (NCBR)
Graphical interfaces
• Unity
• Native Ubuntu environment
• Gnome a Xterm
• Mac OS-like looking environment
• KDE
• Windows-like looking environment
• Several text terminals
• Accessed with CTRL+ALT+F1-F6
Martin Novák (NCBR)
Introduction to Unix
23. září 2014 4/35
User system
• Superuser
• Administrative privileges
• Can edit system files
• User
• Cannot edit system files
• Only selected items are editable/accessible
• Belongs to certain groups with respective rights (hardware/software access...)
Martin Novák (NCBR)
Introduction to Unix
23. září 2014
5/35
Filesystem
• No "Windows-like" discs
• Everything mounted under"/" (root) directory
• Slash sign is used as separator between directories
• Important paths:
• /home/username/ orQuota 1.5 GB, backed-up
• /scratch/username/: No quota, NOT backed-up
• /media/filesystem/: USB sticks, DVD discs...
• Everything is either file or process
• Arbitrary suffixes for files
Martin Novák (NCBR)
Introduction to Unix 23. září 2014
Directories and filenames
General advices aka "Good-To-Follow" rules:
• Case-sensitive system
a Do NOT use spaces in filenames (use underscore or dash)
• Good characters:
• Alphanumerics
• _ ■ - +
a Forbidden characters:
• Any kind of diacritics
• Quotation marks
• Brackets
• #%?!,*"&<§> /
Martin Novák (NCBR)
Introduction to Unix
23. září 2014
7/35
Text terminal
• Found in Applications    Accessories Terminal
• Shell interpreter translating written commands into actions
• Cygwin, PuTTY: Terminal emulators for Windows machines
• Pros:
o Fast and effective way of work
• Directly visible output from operation
• Error tracking
• No GUI needed
a Cons:
• Need of memorizing commands
Introduction to Unix 23. zafi 2014 8/35
Terminal welcome output
• Useful information: Highly advisable to read
• Contains:
• System statistics
• Last login of user and IP address
• Active site
• User and Host info
• Site documentation and support
Martin Novák (NCBR)
Introduction to Unix
23. září 2014
9/35
Useful commands I
Command	Action
cc/foo	Change current working directory to "too"
Is	List files in directory
cp source target	Copy source file to target file
cp -r source target	Copy source directory recursively into target
mvsource target	Move source file to target file
mkdir too	Create "too" directory
rmdirloo	Remove" "too" directory (only if empty)
rm too	Remove" "foo" file
rm -r too	Remove" "foo" directory recursively
cat too	Print content of a "foo" file into terminal
grep too file	Print only line containing "foo" keyword in "file"
top	See currently running processes
Removing means deleting from the disc. NOT moving into trash.
Martin Novák (NCBR)
Introduction to Unix 23. září 2014 10/35
Useful commands II
Command Action
head-n number too Print first "number" rows of "foo" file
tail -n number foo Print last "number" rows of "foo" file
echo foo Prints "foo" into terminal
printf Similar to echo but handles formatted text
chmod switch foo Changes rights of "foo" file according to switch
quota Prints current quota of user and disc usage
ssh user@host Remote access to host machine
exit Logout from the terminal
who Prints all users logged into machine
passwd Change current pasword
kill PID Kill the process with number "PID"
ps Print all current processes running in terminal
module Accessing the scientific software
Martin Novák (NCBR)
Introduction to Unix
□      S       š ► < š ►    š -OQ.O 23. září 2014 11/35
Work in terminal
• General way of use is:
• mnovak@wolf:~$ command [argumentl, [argument2, ...]]
• Switches
• Alter output of commands
• Short notation: command-a -b -C
• Long notation: command--alpha --beta --Gamma
• Use switch "-h" or "--help" for general help
• Manual pages
• Help for basic commands
• Accessed via command man
• Close help with "q" key
Martin Novák (NCBR)
Introduction to Unix 23. září 2014 12/35
Work in terminal II
• Use ArrowUp and ArrowDown for searching the command history
• Use Tabulator for word completion S
• Copy/Paste from terminal using mouse (CTRL+c/CTRL+v does NOT work here)
Will terminate current command!
Martin Novák (NCBR)
Introduction to Unix 23. září 2014 13/35
Example
• Run the following commands and compare results
• $ Is
• $ Is -a
• $ Is -I
• $ Is -h
• $ /s -lah
Martin Novák (NCBR)
Introduction to Unix 23. září 2014 14/35
Wild characters
Notation	Matches
*	Any string of characters including empty string
?	Any single character
[jklm.]	Single character j, k, 1, m or a dot
[a-m]	Single character from range a to m
[2-9]	Single number from range of 2 to 9
• Example:
• $ Is a*[0-2].??[df]   This command will print all files which:
o Start with "a"
• Then they have any string of characters
• Then there is either 0, 1, or 2
• Followed by a dot
• Then any two characters
• Last character is either "d" or "f"
• All conditions must be satisfied
Martin Novák (NCBR)
Introduction to Unix
23. září 2014 15/35
Running jobs in background
• Terminal is still usable for other tasks
• Two ways to achieve:
• $ command &
• Once job is running in terminal:
• CTRL+z # Stops current task
• $ bg      # Puts all jobs into background
Martin Novák (NCBR)
Introduction to Unix 23. září 2014 16/35
Listing and killing processes
Once command\s run, it obtains a unique process ID (PID) $ top        # Displays currently running jobs in real time $ kill PID   # Kills process with a given PID $ kill   PID # Kills process (Signal cannot be blocked)
martin@debian: "/Documents
FiLe   Edit   View   Search   Terminal. HeLp
mnovak@wolf15:~/test$ top
top - 16:40:23 up   9:09,    1 user,   load average: 0.00, 0.01, 0.05 Tasks: 119 total,     1 running, IIS sleeping,     0 stopped,     0 zombie Cpu(s] :    0.0%us,    0.3%sy,    0.0%ni, 99.7%id,    0.0%wa,    0.0%hi,    0.0%si, 0 Mem:     2032876k total,    1035056k used,     997820k free,       89876k buffers Swap:    4194300k total, 0k used,    4194300k free,     716420k cached
I   PID USER
5877 mnovak
5878 mnnvsk 767S mnovak
PR   NI    WIRT    RES    SHR S %CPU %MEM
TIME+
COMMAND
20 0 133m 222B 840 S 20 0 15976 536B 1656 S 20     0    912B 1136    852 R
.1 0:00.08 sshd .3 0:00.55 bash .1     0:00.07 top
Martin Novák (NCBR)
Password change
• $ passwd
• Insert current password
• Enter new password twice (check for typos)
• No characters are printed in terminal during typing
A valid password should be a mix of upper and lower case letters, digits, and other characters. You can use an 8 character long password with characters from at least 3 of these 4 classes, or a 7 character long password containing characters from all the classes. An upper case letter that begins the password and a digit that ends it do not count towards the number of character classes used.
Martin Novák (NCBR)
Introduction to Unix 23. září 2014 18/35
Scientific software
• System of modules located on server
• Modules must be imported prior to running
• Some modules available only at selected nodes
• $ module avail # Lists all available modules
a $ module add moduleName # Adds selected module to path
• $ module remove moduleName # Removes selected module
• Full description of imported module:
• moduleName:version:arch:build
• It is always a good practice to include version
• Full documentation at website:
https://lcc.ncbr.muni.cz//whitezone/development/infinity/
Martin Novák (NCBR)
Introduction to Unix 23. září 2014 19/35
Example
• Run following commands and see the results: $ gabedit
$ module add gabedit:2.3.0 $ gabedit
Martin Novák (NCBR)
Introduction to Unix
□      S       š ► < š ►    š -OQ.O 23. září 2014 20/35
Text editors
• With graphical interface:
• gedit
• kate
• kwrite
• gvim
• Without graphical interface (editing in terminal):
• vi / vim
a Programmed to highlight keywords of many languages/source codes
Martin Novák (NCBR)
Introduction to Unix
□     g     š ► < š ►   š -oq.o
23. září 2014      21 /35
Editor vi
• Fast and effective way to edit files in remote machine
• 3 modes:
• Command mode
• Edit mode
• Visual mode
• Enter command mode via ESC key
• Enter edit mode via Insert or "i" key
• Visual mode for editing blocks of text:
http://vimdoc.soureeforge.net/htmldoc/visual.html#Visua1
Martin Novák (NCBR)
Introduction to Unix
□     g     š ► < š ►   š -oq.o
23. září 2014 22/35
Commands of editor vi
Command	Action
:w	Save document
:w filename	Save document as "filename"
:q	Quit document
:q!	Quit without saving
:wq	Save and quit
:u	Undo
i / insert	Enter edit mode
R	Enter replace mode
99	Go to the beginning of the document
G	Go to the end of the document
dd	Delete current line
25D	Delete next 25 lines
dG	Delete all lines starting from cursor
/keyword	Search for keyword
Martin Novák (NCBR)
Introduction to Unix
□     g     š ► < š ►   š -oq.o
23. září 2014 23/35
Vi tutorial
• Writing a plain text file:
$ vi test.dat Open 'test.dat' file for editing
i / insert Enter editing mode Write some text
ESC exit editing mode and enter command mode
:w Write text to file
gg Go to first line
2D Delete two lines
:u Undo last change
:wq Write and quit
$ rm test.dat   Remove file
Martin Novák (NCBR)
Introduction to Unix
□     g     š ► < š ►   š -oq.o
23. září 2014 24/35
Remote access
• Accessing remote machine via ethernet or internet
• ssh command:
• $ ssh [username@]hostmachine
• username does not have to be specified if same as current login a If X applications should be exportable, use "-X" switch
Martin Novák (NCBR)
Introduction to Unix
□      S       š ► < š ►    š -OQ.O 23. září 2014 25/35
Example
• Access the wolf node next to yours with X server export enabled
• Find out who is logged in there a Exit from this computer
• Help: OB
Martin Novák (NCBR)
Introduction to Unix
□     g     š ► < š ►   š -oq.o
23. září 2014 26/35
Passwordless authentication within cluster
• No password required for access the host machine
• Should be used with great care only on local networks
• Procedure:
$ cd .ssh $ ssh-keygen <enter> <enter>
$ cat id_rsa.pub » authorized_keys a Try to remotely access the same machine
Martin Novák (NCBR)
Introduction to Unix
□     g     š ► < š ►   š -oq.o
23. září 2014 27/35
Copying files between machines
• $ scp source target
• Source and/or target can be on remote machine:
• mnovak@wolf12:~$ scp text.dat wolf 13:/scratch/mnovak/
• mnovak@wolf12:~$ scp -r wolfl3:/scratch/mnovak/ directory/
• $ mc
a Midnight commander - same as in Windows/Mac machines
• "Graphical interface"
• $ gftp
• "Real" graphical interface
Martin Novák (NCBR)
Introduction to Unix
□      S       š ► < š ►    š -OQ.O 23. září 2014 28/35
Absolute versus Relative paths
Absolute path:
• Total path from the root directory
• /scratch/mnovak/test
• -/Documents/
Relative path:
• ./   # Current directory
• ../   # Parent directory
• ../../../data/test/
Martin Novák (NCBR)
Introduction to Unix
23. září 2014
29/35
mnovak@wolf15:~/test$ Is -alh total 56K
drwxr-xr-x   2 mnovak nmr 4.ÖK Mar xr-xr-x 73 mnovak nmr   12K Mar 1 mnovak nmr   201 Mar
15:19 . 15:Q9 ..
15:09 aimextractor.awk
1 mnovak nmr 1.1K Mar 4 15:09 beta_master.sh
1 mnovak nmr     74 Mar 4 15:09 copy
k   1 mnovak nmr   17K Mar 4 15:09 temp_factor_into_
1 mnovak nmr 5.2K Mar 4 15:09 temp factor into fl5:-/test$ |
Access permissions
Each file has permissions for
d - Directory r - Read w- Write x - Execute
■ - Permission not granted
martin@debian: "/Documents
Martin Novák (NCBR)
Change permissions
• $ chmod switch file a examples of switches:
u+x User can execute file
go+w Group members and others can write to file
a-r Remove right to read for all users
o-rwx Remove right to read, write and execute to others
Martin Novák (NCBR)
Introduction to Unix
□      S       š ► < š ►    š -OQ.O 23. září 2014      31 /35
Variables
• Variables are used for storing values
• Various languages/interpreters use different name schemes
• bash/tcsh: CAPITALLETTERS
• awk: lowercasejetters
• C++: almostEveryFirstLetterCappitalized
• ...
• Variable names follow in general the same rules as filenames
• Variables in bash/tcsh accessible via $ sign, e.g. $RANDOM, for example
Martin Novák (NCBR)
Introduction to Unix
□     g     š ► < š ►   š -oq.o
23. září 2014 32/35
Scripts in bash
• Scripts are executable files which serve as storage of commands
• They are read line by line by interpreter and commands are run a User does not have to type everything by hand
• Script must have executable permission
• Running scripts: mnovak@wolf:~$ ./myscript.sh a Comments after hash "#" sign
mnovak@wolf:~
#!/bin/bash
# This is a script which makes a directory, enters it and, lists all files
mkdir test;   # This commands makes the directory
cd test;      # This command enters it
Is -alh;       # This command lists all files inside
Martin Novák (NCBR)
Introduction to Unix
23. září 2014
33/35
Useful script in bash
mnovak@wolf:~
#!/bin/bash
# This script loads gaussian module and performs calculations
module add gaussian:09.A2; # Loads the module
g09 input.com; # Performs the calculations
echo "All work done!";   # Prints info into terminal
return 0; # Return code for the script can be accessed via '$?' variable
Martin Novák (NCBR)
Introduction to Unix
□     g     š ► < š ►   š -oq.o
23. září 2014 34/35