Scientific Clouds
Introduction
Boris Parak, Z d e n e k Sustr
C E S N E T
M a y 9, 2 0 1 6
^CESNET
^ Meta
• Basic t e r m i n o l o g y
• laaS, PaaS, SaaS, X a a S , . . .
• opaque internal structure
• resources on demand, pay-as-you-go
• multi-tenant, multi-purpose
• V i s u a l i z a t i o n
• compute ( K V M , X E N , V M W a r e , HyperV, . . . )
storage (block, object, fs, . . .)
• network ( V L A N , V X L A N , . . . S D N )
• A p p l i c a t i o n deployment
designed for application-level horizontal scaling
replication, distributed consensus, self-healing
automated life-cycle processes, continuous integration and
deployment
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 2 / 20
^ Meta
• C o m p u t i n g intensive tasks — HPC
• D a t a intensive tasks — Big Data
• O f t e n requiring assistance as well as resources
• N a t i o n a l providers targeting a c a d e m i c / r e s e a r c h c o m m u n i t i e s
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 3 / 20
- H P C C l o u d s Boris
Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 4 / 20
^ Meta
• Require flexibility ( O S , H W , network, .. .)
• M a k e use of scalability, even on d e m a n d if supported
• User in control of the execution environment
— additional responsibilities
— higher skill requirements
• W i d e variety of types ( G P G P U , legacy, bleeding edge, . . . )
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 5 / 20
^ Meta
• H P C vs. A W S - l i k e end-user services
— scientific grants from A m a z o n / M i c r o s o f t
• N o " p a y i n g " customers (fairness in resource allocation)
• E x p e c t a t i o n of assistance ( l o o k i n g for a research partner)
• O f t e n w o r k i n g w i t h open data, w a n t i n g to share the results
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 6 / 20
^ Meta
• N o o v e r c o m m i t m e n t of resources
I/O performance often critical (disks, network)
• T o o l s for sharing d a t a / r e s u l t s or restricting access
• Heterogeneous infrastructure and user requirements
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 7 / 20
- MetaCloud Infrastructure
Example
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 8 / 20
^ Meta
• O p e n N e b u l a cloud m a n a g e m e n t framework (open source)
»- KVM with libvirt on a mixed fleet of D e b i a n / C e n t O S nodes
• Kerberos and X.509 a u t h e n t i c a t i o n for users ( G U I and A P I )
Sandbox and Reference architecture:
• http://opennebula.org/tryout/sandboxvirtualbox/
• https://goo.gl/470wce
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 9 / 20
^ Meta
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds M a y 9, 2016 10 / 20
^ Meta
• O p e n to everyone in the C z e c h a c a d e m i c c o m m u n i t y
• Resources from M e t a C e n t r u m ( C E S N E T ) and C E R I T - S C ( M U )
• User payment — publications
• Registration - https://goo.gl/3gGGjW
• GUI - https://cloud.metacentrum.cz/
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds M a y 9, 2016 11 / 20
^ Meta
1. Get a personal certificate at h t t p : / / g o o . g l / C D s d I j
2. Register your personal certificate at h t t p s : / / g o o . g l / N L 2 z B 8
3. Export your certificate from the browser
— see a step-by-step conversion guide at h t t p : / / g o o . g l / G z K j J X
For more information, see h t t p s : / / g o o . g l / f D o s g K
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds M a y 9, 2016 12 / 20
^ Meta
$ oneuser login $USERNAME --x509 --force \
--cert usercert.pem --key userkey.pem
$ onetemplate l i s t
3397 oneadmin metacloud METACLOUD-Ubuntu-Docker-14.04
$ onetemplate instantiate 3397 --name MyPA200Instance
ID: ...
$ onevm show $INSTANCE_ID
VM NICS
## IP address(es) HERE
$ onevm shutdown $INSTANCE_ID --hard
Adding "--xml" to commands gives machine-readable output!
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds M a y 9, 2016 13 / 20
Meta
EGI Federated Cloud
Infrastructure Example
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds M a y 9, 2016 14 / 20
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 15 / 20
Technical Aspects
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds May 9, 2016 1 6 / 2 0
- How To Use Cloud? Boris
Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds M a y 9, 2016 17 / 20
^ Meta
• A u t o m a t e d and repeatable installation (and configuration)
• Possible approaches:
• custom scripting (bash, powershell)
• Ansible - https://www.ansible.com/
• SaltStack - https://saltstack.com/
• Chef - https://docs.chef.io/chef_solo.html
Puppet - https://puppet.com/
• O f t e n used in c o m b i n a t i o n with containers
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds M a y 9, 2016 18 / 20
^ Meta
- T h a t ' s All Folks! Boris
Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds M a y 9, 2016 19 / 20
^ Meta
D o you have any questions?
• ask N O W !
• ask us directly at parak@cesnet.cz or sustr4@cesnet.cz
• send your questions to cloud@metacentrum.cz
Boris Parak, Zdenek Sustr ( C E S N E T ) Scientific Clouds M a y 9, 2016 20 / 20