Metacentrum

MetaCentrum VO is catch-all virtual organization of the Czech National Grid Organization MetaCentrum NGI. MetaCentrum operates and manages distributed computing infrastructure consisting of computing and storage resources owned by CESNET as well as resources of co-operative academic centres within the Czech Republic. Users registered in MetaCentrum get access to a wide range of application software and utilities free of charge, for example Matlab, Maple, or Gaussian.

For more information please visit documentation.

Logging in

To acces metacentrum resources you need to log in to any of the frontend machine trough ssh protocol. To view complete list of frontends please see documentation.

To log in, open terminal and type:

ssh [username]@skirit.ics.muni.cz

From WOLF cluster please add -X option:

ssh -X [username]@skirit.ics.muni.cz

You will be asked for password. When typing, you will not see any characters appeared for the security reason.

If you log in for the first time, you may be prompted by a query:

The authenticity of host 'skirit.ics.muni.cz (2001:718:ff01:1:216:3eff:fe20:382)' can't be
established. ECDSA key fingerprint is SHA256:Splg9bGTNCeVSLE0E4tB30pcLS80sWuv0ezHrH1p0xE.
Are you sure you want to continue connecting (yes/no)?

type “yes” and confirm.

Computations

There are several frontends (login nodes) to access the grid. Each frontend has a native home directory on one of the storages. There are several storages (large-capacity harddisc arrays). They are named according to their physical location (a city).

Frontend

Frontends should be used only for:

  • preparing inputs, data pre- and postprocessing
  • managing batch jobs
  • light compiling and testing

Usually no computations is done on the frontend. You need to specify your computing resource requirements first, wait for their allocation and then start computation.

Job management

There are three basic commands you can use:

  • qsub - submit computational job
  • qstat - query status of a job
  • qdel - delete job

To run a job you have two options: interactive job and batch job. Let’s start with interactive one:

qsub -I -l select=1:ncpus=2:mem=4gb:scratch_local=1gb -l walltime=2:00:00

parameter -I means that we are asking for an Interactive job. Parameters -l are for specifying resources.

Other way to submit job is through the batch job. That mean you will submit a script file and when the resources is ready the computation will start.

#!/bin/bash
#PBS -N myFirstJob
#PBS -l select=1:ncpus=4:mem=4gb:scratch_local=10gb
#PBS -l walltime=1:00:00 
#PBS -m ae
# The 4 lines above are options for scheduling system: job will run 1 hour at maximum, 1 machine with 4 processors + 4gb RAM memory + 10gb scratch memory are requested, email notification will be sent when the job aborts (a) or ends (e)

At the beginning of the file there should be #PBS lines with specification of resources.

Basic commands

Lets submit your first interactive job and try basic commands in bash.

qsub -I -l select=1:ncpus=2:mem=4gb:scratch_local=1gb -l walltime=2:00:00

There are some basic commands you need to be familiar with.

first one is pwd

pwd

which will show you the path to your current working folder.

To navigate through the directories you can use change directory command

cd .. # go to parent directory
cd /home/user/Documents # go to specific folder using absolute path
cd ../../user/Documents # go to specific folder using relative path

To list files and folders use listing command>

ls

To create a folder you can use make directory command

mkdir my_new_folder

there is a few other useful commands>

cp /old_path/my_file /new_path/my_file # copy file
mv /old_path/my_file /new_path/my_file # move file
rm /path_to_delete/file # remove file
touch new_file.txt # create file
echo "messege to be displayed" # display some messege
man rm #print manual for rm command
head myfile.txt # display first 10 lines of file
tail myfile.txt # display lasat 10 lines of file
cat file1 file2 > combined_file # combine files together
exit #exit current session