PA234 Infrastuctural and Cloud Systems

Faculty of Informatics
Spring 2025
Extent and Intensity
2/2/0. 3 credit(s) (plus extra credits for completion). Type of Completion: zk (examination).
In-person direct teaching
Teacher(s)
RNDr. Tomáš Rebok, Ph.D. (lecturer)
Guaranteed by
RNDr. Tomáš Rebok, Ph.D.
Department of Computer Systems and Communications – Faculty of Informatics
Supplier department: Department of Computer Systems and Communications – Faculty of Informatics
Prerequisites
none
Course Enrolment Limitations
The course is offered to students of any study field.
Course objectives
This course aims to bridge the gap between the theoretical concepts of infrastructural computing and storage systems (so-called HPC systems, supercomputers) and the possibilities of their application for solving various practical problems related to research and business practice. In addition to theoretical knowledge about the concepts and technologies applied in these systems, students will also gain experience from their practical use, applied to the solution of common, computationally or data-intensive tasks -- related to, for example, the application of machine learning algorithms and artificial intelligence, analysis of BigData, etc. Among others, the course will also introduce possibilities of using (national) e-infrastructures (CERIT-SC, MetaCentrum, IT4Innovations) and the hardware and software resources provided by them, which will be used for the practical implementation of assigned tasks. The course also supports the development of critical thinking and students' ability to analyze, design, and use appropriate infrastructural environments for solving advanced practical problems, emphasizing the effective use of the technologies and resources they provide.
Learning outcomes
Upon successful completion of the course, students will be able to:
- understand the concepts and technologies of infrastructure systems - students will have a deep theoretical and practical knowledge of the concepts and technologies used in infrastructure computing and storage systems - container and cloud technologies, job scheduling and batch processing in supercomputing systems, working with data and common storage systems.
- practically use the existing and future infrastructure systems - students will gain hands-on experience using existing infrastructure systems to solve typed, computationally or data-intensive problems. Thanks to the theoretical knowledge, they can use future technologies based on the concepts introduced.
- effectively use the available national e-infrastructure e-INFRA CZ to support science and research - students will gain experience with the effective use of (national) e-infrastructures in CERIT-SC, MetaCentrum, and IT4Innovations, and the resources and services provided by them.
- practically apply the acquired skills in their (future) academic and professional life - the course supports the development of critical analytical thinking and the ability of students to analyse, design and effectively use appropriate infrastructure environments to solve advanced computational and storage problems.
Syllabus
  • 1. Introduction to HPC infrastructures, national (e-INFRA CZ) and European (EGI.eu) e-infrastructure (approx. 1 lecture)
  • 2. Container technologies (approx. 3 lectures)
  • - Concept and applied technologies, advantages and disadvantages of container technologies
  • - Overview of container technologies (Docker, Kubernetes, OpenShift, etc.)
  • - Basic and advanced Docker techniques - working with images and containers, container builds, networking and storage techniques, image optimization
  • - Basic and advanced Kubernetes techniques - principles, architecture and components, networking techniques, add-ons (certificates), application configuration and management, security and authentication, HELM charts
  • - Other container technologies and tools - OpenShift, container orchestration (Docker Swarm vs. Kubernetes), container registries (Docker Hub, Harbor.io, Quay.io, etc.)
  • - Standardized practices for efficient development and deployment of containerized applications
  • 3. Cloud technologies (approx. 3 lectures)
  • - Concepts and applied technologies, virtualization, advantages and disadvantages of cloud technologies
  • - Overview of cloud services (IaaS, PaaS, SaaS) - virtual machine images, configuration of virtual machines in the cloud
  • - Basic and advanced OpenStack techniques - creating virtual infrastructures in the cloud, deploying and configuring services (EC2-compatible services, orchestration tools - API/CLI, Terraform, Ansible, etc.)
  • - Principles and architecture of clouds built on the OpenStack platform - overview of the essential operating system (micro)services, their interconnection, security and tuning of deployed virtual machines, load balancing and robustness (high availability of services)
  • - Strategies for building private cloud services (e.g. beskar.cloud), DevOps and GitOps techniques
  • - Standardized practices for effective development and deployment of cloud applications and infrastructures
  • 4. Task scheduling and management in computing infrastructure (approx. 2 lectures)
  • - Concepts, principles and importance of job scheduling and management in distributed infrastructure
  • - Overview of job scheduling and management technologies (OpenPBS, Slurm, HTCondor, etc.), advantages and disadvantages
  • - OpenPBS basics - principles of batch computing and jobs (interactive, batch), infrastructure resources and their allocation to jobs, job submission and monitoring, data processing (local and shared scratch, stagein & stageout)
  • - Advanced OpenPBS techniques - parallel and distributed computing (OpenMP and MPI), specific use cases (job dependencies, checkpointing, job arrays, etc.), PBS and containers (Singularity), job scheduling and policies, workload management and performance optimization, debugging and troubleshooting
  • - OpenPBS integration with other technologies - such as cloud services (AWS, Google Cloud, Azure, etc.) and container technologies (Docker, Kubernetes, etc.)
  • - Standardized procedures for scheduling and managing tasks in computing infrastructure and their use in practical applications
  • 5. Storage systems (approx. 1 lecture)
  • - Concepts and principles, importance of cloud storage systems
  • - Overview of types of cloud storage systems (block, file, object, etc.) and cloud storage service providers (AWS, Google Cloud, Azure, etc.), advantages and disadvantages of using cloud storage systems
  • - Cloud storage and security - basics of cloud storage security, secure data sharing and encryption in the cloud, access and identity management in cloud storage systems, working with sensitive data (SensitiveCloud)
  • - Efficient use of cloud storage systems - data transfers within and to/from cloud systems, S3 API, MinIO and CEPH
  • - Standardized practices for working with cloud storage systems
  • 6. Efficient calculations (approx. 2 lectures)
  • - GPU accelerated computing - introduction, efficient use of GPUs in e-infrastructures, GPU allocation options, practical examples
  • - Pivot Jobs and bulk processing - introduction, bulk processing and dynamic scheduling, efficient processing of large data sets in e-infrastructures, practical examples
  • - Data processing and workflow - introduction to data pipelines, data processing using workflow principles, workflow automation (orchestrators - AirFlow, Argo Workflows, Galaxy) and their case studies (AFoLab, EnviLab)
  • - OneData - introduction, unified management and access to data in a distributed environment, case studies (CEITEC), practical examples
  • - Standardized practices for effective use of e-infrastructures, applied to selected research and professional needs
Teaching methods
Lectures and practical exercises with consultations for the implementation of the assigned topics.
Assessment methods
Bachelor students - evaluation of practical projects (exercises)
Master students - final exam and evaluation of practical projects (exercises)
Náhradní absolvování
Due to the structure and content of the course (a set of practical exercises), the alternative form is not suitable. If required, an individual arrangement with the course teacher is necessary.
Language of instruction
English
Further Comments
The course is taught annually.
The course is taught: every week.
  • Permalink: https://is.muni.cz/course/fi/spring2025/PA234