7. Computer Networks & Multimedia PA159: Net-Centric Computing I. Eva Hladká Slides by: Tomáš Rebok Faculty of Informatics Masaryk University Autumn 2018 Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 1/59 Lecture Overview Q Media, Multimedia Applications • Media Characteristics • Media Classification Q Multimedia Requirements on the Communication Network o Traffic Requirements • Functional Requirements O Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID Q Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 2/59 Media, Multimedia Applications Lecture Overview Q Media, Multimedia Applications • Media Characteristics • Media Classification 0 Multimedia Requirements on the Communication Network • Traffic Requirements • Functional Requirements 0 Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID Q Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 3/59 Multimedia Applications I. • multimedia = the information/data that is composed of a number of different types/forms of media which are integrated together in some way • media = text, images, speech/audio, video, interaction, etc. • multimedia applications • powerful technologies that can enable remote sharing of resources or interactive work collaborations o thus saving both time and money • examples of multimedia applications: • video telephony/conferencing (speech and video) • multimedia electronic mail (text, images, and audio) • radio/television broadcast (audio and video) • electronic commerce (text, images, audio, and video) • web TV, distant learning (text, audio, and video) • real-time interactive and collaborative work environments (audio, video, and interaction) • .. . Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 4/59 Multimedia Applications II. • while text is inherently digital, other media types (sound, visuals) can be analog • ^> these are required to be converted into the digital form using appropriate analog to digital conversion techniques • then, the integrated digital information stream can be stored within a computer and/or transmitted over a network • in order to reduce the volume of information to be transferred, appropriate compression algorithms can be applied • suitable for the particular media type • lossy compression - eliminate redundant information from data and subsequently introduce distortion or noise in the original data • lossless compression - do not loose any information (received data is exactly identical to the original data) • the communication requirements highly differ depending on the type of media/application Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 5/59 Media, Multimedia Applications Media Characteristics • most popular media type • distributed in many forms • FTP (File Transfer Protocol), HTTP (Hyper Text Transfer Protocol), SMTP (Simple Mail Transfer Protocol), etc. • bandwidth requirements depend on its size • can be further easily reduced by compression techniques 9 error characteristics depend largely on the application • some text applications (e.g., file transfer) require text communication to be completely loss/error free —>► TCP should be used 9 some text applications (e.g., instant messaging) may tolerate some errors as well as losses —>► UDP can be used • regarding the delay, text-based applications usually do not have any real-time constraints (such as bounded delay or jitter) • however, applications like instant messaging do require some guarantees on the experienced delay Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 6/59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Text II. Compression Scheme Comments Shannon-F ano C o ding Uses variable length code words, i.e., symbols with higher probability of occurrence are represented by smaller codes-words. Huffman Coding Same as Shannon-F ano Coding. LZW LZW compression replaces strings of characters with single codes. It does not do any analysis of the incoming text. Instead, it just adds every new string of characters it sees to a table of strings. Compression occurs when a single code is output instead of a string of characters. Unix Compress Uses LZW with growing dictionary. Initially the dictionary contains 512 entries, and is subsequently doubled till it reaches the maximum value set by the user. Figure: Text Compression schemes. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 7/5< Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Audio I. = sound/speech converted into digital form using sampling and quantization techniques • sampling - a reduction of a continuous signal to a discrete signal • the samples are scanned in defined time periods (= sample rate) 9 quantization - process of mapping/approximating the samples' signal strenghts by discrete symbols or integer values ^ hodnota hodnota \._ rekonstruovaný skutečná hodnota Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 8/59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Audio II. • bandwidth requirements of digitized audio depend on its dynamic range and/or frequency spectrum • e.g., telephone-grade voice uses dynamic range reduction (logarithmic A-law mechanism) • thus, the linear range of 12 bits is reduced to a nonlinear range of 8 bits only ^> throughput is reduced from 96 kbps to 64 kbps • a number of compression schemes also exist Voice/Audio Codec Used for Bit Rate (Kbps) Pulse code Modulation (G.711) Narrowband speech (300 -3300 Hz) 64 GSM Narrowband speech (300 -3300 Hz) 13 CS-ACELP (G.729) Narrowband speech (300 -3300 Hz) S G.723.3 Narrowband speech (300 -3300 Hz) 6.4 and 5.3 Adaptive differential PCM (G.726) Narrowband speech (300 -3300 Hz) 32 SBC (G.722) Wideband speech (50 - 7000 Hz) 48/56/64 MPEG layer III (MP3) CD-quality music Wideband Audio (10 - 22Khz) 128-112 Kbps Eva Hladká (Fl MU) Figure: Audio Compression schemes. 7. Networks & Multimedia Autumn 2018 9 / Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Audio III. • audio media type has loose requirements on packet loss/errors • it can tolerate up to 1 to 2% packet loss/error without much degradation • most audio applications have further inbuilt mechanisms to deal with the lost packets using advanced interpolation techniques a the real-time/latency requirements strictly depend on the expected interactivity between the involved parties • e.g., Internet telephony/conferencing applications are highly interactive and require shorter response times • the audio media requires strong bounds on end-to-end packet delay/jitter to be of acceptable/decipherable quality • = Real-Time Intolerant (RTI) applications • most RTI applications require the end-to-end delay to be limited by « 100 — 200 msec to get an acceptable performance • on the other side, e.g., Internet webcasts have relatively low interactivity 9 it requires weaker bounds on delay/jitter • = Real-Time Tolerant (RTT) applications Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 10/59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Graphics and Animation I. • includes static media types (e.g., digital images) and dynamic media types (e.g., flash presentations) • an uncompressed, digitally encoded image consists of an array of pixels, with each pixel encoded in a number of bits to represent luminance and color • usually large in size =^> compression schemes are usually used • most modern image compression schemes are progressive this has important implications to transmission over the communication networks • when such an image is received and decompressed, the receiver can display the image in a low-quality format and then improve the display as subsequent image information is received and decompressed • e.g., a pyramid-coding method - images encoded into layers, where early layers are of low resolution and the later layers progressively increase the resolution • usually error-tolerant • provided the application used to render them knows how to handle lost packets • similarly to text files, images do not have any real-time constraints Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 11/59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Graphics and Animation II. Compression scheme Comments Graphics Interchange Format (GIF) Supports a maximum of 256 colors and is best used on images with sharply defined edges and large, flat areas of color like Text and line based drawings. GIF uses LZW (Lempel-Ziv-Welch) compression to make files small. This is a lossless compression scheme. Portable Network Graphics (PNG) Supports any number of colors and works best with almost any type of image. PNG uses the zlib compression scheme, compressing data in blocks dependant on the "filter" of choice (usually adaptive). This is a lossless compression scheme and does not support animation. Joint Photographic Experts Group (JPEG) Best suited for images with subtle and smooth color transitions such as photographs, grayscale/colored images. This compression standard is based on the Huffman and Run-Length encoding of the quantized Discrete Cosine Transform (DCT) coefficients of image blocks. JPEG is a "lossy" compression. Standard JPEG encoding does not allow interlacing, but the Progressive JPEG format does. Progressive JPEGs start out with large blocks of color that gradually become more detailed. JPEG2000 Suitable for a wide range of images ranging from those produced by portable digital cameras through to advanced pre-press, medical imaging. JPEG 2000 is a new image coding system that uses state-of-the-art compression techniques based on wavelet technology that stores its information in a data stream, instead of blocks as in JPEG. This is a scalable lossy compression scheme. JPEG-LS Suitable for continuous-tone images. The standard is based on the LOCO-I algorithm (Low Complexity LOssless Compression for Images) developed by HP. This is a lossless/near-lossless compression standard. Joint Bi-level Image Experts Group (JBIG) Suitable for compressing black and white monochromatic images. Uses multiple arithmetic coding schemes to compress the image. This is a lossless type of compression. Eva Hladkä (Fl MU) 7. Networks & Multimedia Autumn 2018 12/59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Video I. • a sequence of images/frames displayed at a certain rate (e.g., 24 or 30 frames per second) 9 similarly to digitized audio, also transmitted as a stream of discrete packets over the network 9 bandwidth requirements are usually very high, depending whether a compression method is employed or not • uncompressed video - usually large in size, but allows to minimize the end-to-end latency • HD = 1.5Gbps, 2K = 3Gbps, 4K = 6Gbps, etc. • compressed video - smaller in size, but the compression method increases the latency 9 thus not suitable for real-time applications (particularly for the ones with strict real-time constraints) • error requirements and real-time characteristics similar to the audio ones Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 13/59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Video II. Compression scheme Comments MPEG-I Used to produce VCR NTSC (352 x 240) quality video compression to be stored on CD-ROM (CD-I and CD-Video format) using a data rate of 1.2 Mbps. Uses heavy down-sampling of images as well as limits image rate to 24-30 Hz to achieve this goal. MPEG-II More generic standard for a variety of audio-visual coding applications and supports error-resilience for broadcasting. Supports broadcast-quality video compression (DVB) and High Definition Television (HDTV). MPEG-2 supports four resolution levels: low (352 x 240), mam (720 x 480), high-1440 (1440 x 1152), and high (1920 x 1080). The MPEG-2 compressed video data rates are in the range of 3-100 Mbps. MPEG-IV Supports low bandwidth video compression at data rate of 64 Kbps that can be transmitted over a single N-ISDN B channel. MPEG-4 is a genuine multimedia compression standard that supports audio and video as well as synthetic and animated images, text, graphics, texture, and speech synthesis. H.261 Supports video communications over ISDN at data rates of px64 Kbps. It relies on intra and inter-frame coding where integer-pixel accuracy motion estimation is required for inter mode coding H.263 The H.263 standard is aimed at video communications over POTS and wireless networks at very low data rates (as low as 18-64 Kbps). Improvements in this standard are due to the incorporation of several features such as half-pixel motion estimation, overlapping and variable blocks sizes, bi-directional temporal prediction, and improved variable-length coding options. Figure: Video Compression schemes. Eva Hladkä (Fl MU) 7. Networks & Multimedia Autumn 2018 14/59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Summary Application BW Latency Loss Adaptability Distributed computing all 1 /is — 10 s no low Voice low 30 ms low low Video high 300 A77S low middle Haptics very low 10 ms low low WWW middle 100 ms Is no middle Push model low-middle 1 min — 1 day middle high Telemetry low-middle variable no restricted Streaming middle 1 - 10s low high Data backup high 1 hour zero high Email low 1 min — 1 hour no high Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 15/59 Media, Multimedia Applications Media Classification Media Classification (f rom the networking perspective) I. o all media types can be classified as either real-time (RT) or non real-time (NRT) • RT media types require either hard or soft bounds on the end-to-end packet delay/jitter • NRT media types (text, image files, etc.) do not have any strict delay constraints • but may have rigid constraints on error • the RT media can be further classified as discrete media (DM) or continuous media (CM) • depending on whether the data is transmitted in a discrete quantum (e.g., file or message) or continuously (as a stream of messages with inter-message dependency) • the RT-CM media can be further classified as delay tolerant or delay intolerant • depending on whether the media can tolerate higher amounts of delay without significant performance degradation Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 16/59 Media, Multimedia Applications Media Classification Media Classification (f rom the networking perspective) I. WA L Non-Real Time E.g. Text, Data I Media Types E.g. Images Real Time Discrete —i E.g. Text chat, Instant Messaging E.g. Weather Updates 1 í Continuous Delay Intolerant E.g Remote Desktop Applications E.g. Interactive AudioA/ideo 1 Delay Tolerant I E.g. Streaming AudioA/ldeo I Error Intolerant Figure: Network oriented classification of media types. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 17/59 Multimedia Requirements on the Communication Network Lecture Overview Media, Multimedia Applications • Media Characteristics • Media Classification Q Multimedia Requirements on the Communication Network o Traffic Requirements • Functional Requirements 0 Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID Q Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 18/59 Multimedia Requirements on the Communication Network • can be generally divided into: • traffic requirements - limits on real-time parameters (such as delay jitter, bandwidth, and reliability) • can be met only by enhancements to the basic Internet Architecture • functional requirements - support for multimedia services (such as multicasting, security, mobility, and session management) • can be met by introducing newer protocols over the TCP/IP networking stack Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 19 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) I. • real-time traffic enforces strict bounds on end-to-end packet delay and jitter • delay = time taken by the packet to travel from the source to the destination • human beings can tolerate a latency of ~ 100 — 200 msec • jitter = variability in the inter-packet delay at the receiver • the performance improves with decrease in both these quantities • the end-to-end delay is influenced by: • Packet Processing Delay o Packet Transmission Delay • Propagation Delay • Routing and Queuing delay Eva Hladka (Fl MU) 7. Networks & Multimedia Autumn 2018 20/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) II. Packet Processing Delay o a constant amount of delay faced at both the source and the destination • A/D, D/A conversion time and time taken to packetize it through different layers of protocols • usually a characteristic of the operating system and the multimedia application • for a lightly loaded system this can be considered as negligible • however, with increasing load this delay can become significant • its reductions imply software enhancements • including the use of multimedia OSs that provide enhanced process-, resource-, file- and memory-management techniques with real-time scheduling, and • enhancing the application Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 21/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) III. Packet Transmission Delay • time taken by the physical layer at the source to transmit the packets over the link • depends on multiple factors: • number of active sessions - since physical layer processes packets in FIFO order, this may become significant especially if the OS does not support real-time scheduling algorithms supporting multimedia traffic • MAC access delay - the delay influenced by accessing the media by the data sender(s) • collisions lead to delayed service time • widespread Ethernet networks cannot provide any firm guarantees on this access delay (due to the indeterminism of the CSMA/CD approach) • e.g., Isochronous Ethernet (integrated voice data LAN, IEEE 802.9) and demand priority Ethernet (100Base-VG, AnyLAN, IEEE 802.12) can provide QoS • context switch in the OS - sending or receiving a packet involves context switch in the OS • takes a finite time there exists a theoretical maximum at which computer can transmit packets • reduction in this delay require enhancements to the device drivers and _increasing the operating speed of the computer_ Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 22/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) IV. Propagation Delay e the flight time of packets over the transmission link • limited by the speed of the transmission signal (at most, by the light) • shorter distances imply shorter propagation delays • distance of 200 m propagation delay ~ 1 ms • distance of 20000 km ^> propagation delay ^ 100 ms (!!!) • the physical limits which cannot be reduced! (speed of the light) • remember, interactive applications require the response time to be lower than 100 - 200 ms • ^> if the propagation delay becomes greater than this value, no enhancements can improve the quality of the interactive sessions Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 23/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) V. Routing and Queuing Delay • the best-effort Internet treats every packet equally • regardless of whether it is a real-time packet or a non-real-time packet • all intermediate routers make independent routing decisions for every incoming packet • when packets arrive at a queue, they have to wait for a random amount of time before they can be serviced • depends on the current load on the router • this delay is random and thus becomes the major contributor to jitter in the traffic streams a can be reduced by: • Integrated Services (Intserv), Differentiated Services (Diffserv), MPLS, etc. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 24/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) VI. Eva Hladká (Fl MU) Figure: Integrated services model. 7. Networks & Multimedia Autumn 2018 25/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) VII core router: forwarding H2 leaf router: classification, conditioning Figure: Differentiated services model Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 26/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Bandwidth I. 9 multimedia traffic streams have usually high bandwidth demands • the communication network must be able to handle such high bandwidth requirements • without being unfair to other conventional flows • the required bandwidth can be reduced by compression mechanisms • but, these cannot be used for all multimedia transmissions (e.g., interactive real-time video) • the best-effort Internet does not provide any mechanism to reserve network resources to meet such high bandwidth requirements • left to the discretion of the application to dynamically adapt to network congestions • TCP-based applications take advantage of built in congestion control • however, multimedia apps are usually UDP-based - these do not have any mechanism to control congestion (they can create a congestion collapse) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 27/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Bandwidth II. o impossible to remove these shortcomings in the nowadays best-effort Internet • an enhanced Internet service model would require: • Admission control: application must first get a permission to send traffic at a given rate with given traffic characteristics • Bandwidth reservation: if the admission is given, appropriate resources (buffers, bandwidth) will get reserved along the path • Traffic policing mechanisms: to ensure that applications do not send data at a rate higher than what was negotiated cost of the protocol processing operations relates more directly to the packet processing rate than to the bandwidth (in terms of bit rate) • packet processing is largely dependent on number of packets and less on packet size • ^> packet processing rate is an important metric Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 28/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Requirements • different media types have vastly different error requirements 9 (error = lost or damaged packet) • from being completely error-intolerant to being somewhat error-tolerant • the best-effort Internet cannot provide guarantees on the error characteristics because the path a packet follows is not fixed • =>> the sender application has no knowledge of the error characteristics of the network • most common methods for error correction: • Sender-based Repair • active repair - Automatic Retransmission Request (ARQ) mechanisms, suitable for error intolerant applications • passive repair - Forward Error Correction (FEC), Interleaving, etc. • Receiver-based Repair - Error Concealment Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 29/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Recovery - Forward Error Correction (FEC) I. • requires extra information (repair data) to be added to the packet stream • this information is used for recovering lost packets • Media-independent FEC - uses block or algebraic codes to produce additional packets which help in loss recovery • codes take a codeword of n data packets and generate m additional check packets • examples: • Parity Coding - XOR is applied across groups of packets to generate parity packets • Reed-Solomon Coding - based on properties of polynomials over particular number bases • disadvantage: cause additional delay and increase bandwidth usage Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 30/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Recovery - Forward Error Correction (FEC) II. • Media-dependent FEC - exploits media characteristics • each data unit is sent in multiple packets 9 primary encoding + secondary encoding (could be of lower quality/bandwidth than the primary one) • may not be necessary to transmit FEC for every packet due to nature of the media • advantage: low latency (only single packet delay added) ^> suitable for interactive applications • it is advantagenous for an application to know the error characteristics of the communication network • ^> adequate level of FEC can be employed • e.g., wireless vs. wired networks Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 31/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Recovery - Forward Error Correction (FEC) III 1 1 1 2 2 3 31 4 sif 1 1 2 1 výpadek 3 I 4 Figure: A FEC technique - second stream in a lower quality. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 32/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Recovery - Interleaving II. • can be used only when media unit size is smaller than packet size (e.g., audio units) and the end-to-end delay is not important • based on resequencing media units before transmission (originally adjacent units are separated by a guaranteed distance) and returning to the original order at the receiver • disperses the effect of the packet loss • loss of a single packet causes multiple smaller gaps among original media units (easier to deal with) • advantage: does not increase bandwidth usage • disadvantage: increases latency =4> not well suited for interactive applications Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 33/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Requirements - Interleaving II. sír 1 5 9 13] 1 5 9 5 6 7 8 2 6 10fl4| r 2 6 10 14 4 8 12 16 výpadek 4 8 1216 Figure: Interleaving technique. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 34/59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Requirements - Error Concealment • predicting/compensating the lost information from correctly received packets • used by many error-tolerant applications o works well for relatively small loss rates (< 15%) and for small packets (4 - 40 ms) • several types (in increasing order of computational cost and improved performance): • Insertion-based: inserting a fill-in packet that contains silence, noise, or a repitition of adjacent packets • Interpolation-based: some form of pattern matching and interpolation to derive the missing packet • Regeneration-based: derive decoder state from packets surrounding the loss and generate a lost packet from that (= model based recovery) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 35/59 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Multicasting Support I. • multicast = delivery of a message/information to a group of destination computers simultaneously in a single transmission from the source, creating copies automatically in inner network elements (e.g., routers) only when the topology of the network requires it • network property (hop-by-hop, not end-to-end service) • however, not available in many networks • due to problems it usually brings • employed protocols: IGMP, DVMRP, MOSPF, PIM, etc. • ^> can be compensated by the application layer (virtual) multicast • a single (or a set of) special-purpose node(s)/application(s) in the network reflecting the incoming traffic • e.g., UDP packet reflector, (Distributed) Active Element, SIP and H.323 MCUs, etc. • can provide additional features (data processing, access control, moderating, etc.) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 36/59 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Multicasting Support II. sender □ Figure: Native Multicast Distribution Tree (at most one data copy per link) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 37/59 Functional Requirements Multicasting Support III. Figure: Virtual Multicast Distribution Tree (one data copy per host). Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Security & Mobility Security • provides the following aspects to multimedia data: • Integrity = data cannot be changed in mid-flight • Authenticity = data comes from the right source • Encryption = data cannot be deciphered by any third party o ensured by IPSec/IPv6 • Authentication Header (AH) and Encapsulated Security Payload (ESP) protocols Mobility • « mobility in IPv6 Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 39/59 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Session Management I. The session management functionality includes: • Media Description • enables a distributed multimedia application to distribute session information • e.g., media type (audio, video or data) used in the session, media encoding schemes (PCM, MPEG II), session start time, session stop time, IP addresses of involved hosts, etc. • usually essential to describe the session before establishment • e.g., Session Description Protocol (SDP), Session Initiation Protocol Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 40 / 59 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Session Management II. • Session Identification • a multimedia session often consists of multiple media streams (audio, video, text, images, etc.) that need to be separately identified • e.g., sending the audio and video as two separate streams, which the receiver needs to decode synchronously 9 e.g., dividing the quality of the A/V stream into low-bandwidth and high-bandwidth ones • e.g., Real-Time Transport Protocol (RTP) • Session Control • since multimedia session may involve multiple media streams, whose information is often inter-related, the multimedia communication network must guarantee to maintain such relationships at the end user • = Multimedia Synchronization • moreover, many Internet multimedia applications may want to control the playback of continuous media by pausing, playing-back, repositioning playback, etc. • e.g., Real-Time Control Protocol (RTCP), Real-Time Streaming Protocol (RTSP) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 41/59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Session Description Protocol (SDP) I. a developed by IETF 9 a format for describing streaming media parameters (media type, media encoding, etc.) • more of a description syntax than a protocol o media descriptions encoded in text format • SDP message contains a series of lines (called fields) • each field has a = format • tag = a pre-defined single letter abbreviation Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 42/59 Multimedia Requirements on the Communication Network Functional Requirements Session description v= (protocol version) o= (originator and session identifier) s= (session name) i=* (session information) u=* (URI of description) e=* (email address) p=* (phone number) c=* (connection information — not required if included in all media) b=* (zero or more bandwidth information lines) One or more time descriptions ("t=" and "r=" lines; see below) z=* (time zone adjustments) k=* (encryption key) a=* (zero or more session attribute lines) Zero or more media descriptions Time description t= (time the session is active) r=* (zero or more repeat times) Media description, if present m= (media name and transport address) i=* (media title) c=* (connection information — optional if included at session level) b=* (zero or more bandwidth information lines) k=* (encryption key) a=* (zero or more media attribute lines) ion Management Protocols Session Description Protocol (SDP) II. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 43/ Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Session Announcement Protocol (SAP) used for advertising multicast conferences and other multicast sessions • SAP announcer periodically multicasts announcement packets to a well-known multicast address and port (port number 9875) • with the same scope as the session being announced =^> announcements' recipients are also potential recipients of sessions being advertised uses SDP to describe the session(s) contains mechanisms for: 9 ensuring integrity of session announcements 9 authenticating the origin of an announcement • encrypting such announcements Eva Hladkä (Fl MU) 7. Networks & Multimedia Autumn 2018 44/59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Real-Time Transport Protocol (RTP) • runs on top of UDP (usually implemented within the application) • carries chunks of real-time (audio/video) data o provides: • Sequencing: • a sequence number in the RTP header helps to detect lost packets • Pay load identification: • payload identifier (included in each RTP packet) describes encoding of the media • Frame indication: • video and audio are sent in logical units called frames - RTP's frame marker bit indicates the beginning and end of a frame • Source identification: • to identify the originator of a frame in a multicast session, the Synchronization Source (SSRC) identifier is provided • Intra media synchronization: o to compensate for different delay and jitter for packets within the same stream, timestamps are provided 9 additional media information can be inserted using profile headers and extensions • RTP cannot assure any quality of the transmissions, but provides resources to the applications to do that_ Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 45/59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Real-Time Control Protocol (RTCP) • a control protocol that works in conjunction with RTP • the primary function of RTCP is to gather statistics on quality aspects of the media distribution during a session and to transmit this data to the session media source and other session participants • the participants periodically send RTCP packets to give feedback about the quality of the received data • provides out-of-band statistics and control information for an RTP flow • packets sent, lost, jitter, round-trip time, etc. • sources can use them to adjust their data rate (adaptive media encoding) and for detection of transmission faults • enclosed time information can be used for synchronization purposes o other information includes: • email address, phone number, name, etc. • allow users to know the identities of other users in the session Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 46 /59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Real-Time Streaming Protocol (RTSP) • an out-of-band control protocol that allows the media player application to control the transmission of the media stream • including functions such as pause, resume, repositioning, playback, etc. 9 the transmission of streaming data itself is not a task of the RTSP protocol 9 designed to work with RTP (and RSVP) to provide a complete streaming service over the Internet • similar to HTTP in some ways • but, HTTP is stateless, while RTSP is statefull (an identifier is used when needed to track concurrent session) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 47/59 Multimedia Applications @ CESNET/FI MU Lecture Overview Media, Multimedia Applications • Media Characteristics • Media Classification 0 Multimedia Requirements on the Communication Network • Traffic Requirements • Functional Requirements O Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID Q Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 48 / 59 Multimedia Applications @ CESNET/FI ML) State-of-the-Art I. a current research in the area of multimedia networking mainly focuses on video media transmissions 9 the research issues include: • to allow higher resolutions (4K, 8K, etc.) • to provide minimal latency (uncompressed video transmissions) • to lower the required transmission throughputs (JPEG2000, JPEG2000 on GPUs) • (including their combinations) HD (1920 x 1080) 2K (2048x 1080) UHDTV1 (3840x2160) 4K (4096x2160) UHDTV2 (7680x4320) 8K (8192x4320) Eva Hladka (Fl MU) 7. Networks & Multimedia Autumn 2018 49/59 Multimedia Applications @ CESNET/FI MU State-of-the-Art II. • achieved by employing both SW (e.g., Ultragrid) and HW tools/devices (e.g., MVTP-4K) Figure: Modular Video Transfer Platform (MVTP) designed by Cesnet. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 50/59 Multimedia Applications @ CESNET/FI MU SW-based Solutions FLPA177 (High Performance Computing) Figure: FI:PA177 (High Performance Computing) by Ultragrid & UDP packet reflectors: Course taught by prof. Thomas Sterling at Louisiana State University and transferred in realtime to Fl students (uncompressed HD @ 1.5 Gbps). Remotely-involved universities include: Fl MU, University of Arkansas, Louisiana Technical University, MCNC North Carolina, and NortF^Carolin^StateUnive^ Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 51/59 Multimedia Applications @ CESNET/FI MU SW-based Solutions FLPA177 (High Performance Computing) Eva Hladká (Fl MU) Figure: LSU side. 7. Networks & Multimedia Autumn 2018 52/59 FLPA177 (High Performance Computing) Connectivity scheme Figure: FI:PA177 (High Performance Computing) by Ultragrid &. UDP packet reflectors: Connectivity scheme with time axis. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 53/59 Visualisations in Collaborative Environments Figure: High-volume real-time visualisation in collaborative environment: HDTV stream generated in Baton Rouge and transported (by Ultragrid) to Brno and San Diego. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 Multimedia Applications @ CESNET/FI MU SW-based Solutions Collaborative Environments Orchestration - CoUniverse Collaborative environment Network API '-' Encapsulated Encapsulated Encapsulated application 1-' application application k_é CoUniverse Network orchestration Network infrastructure r Application encapsulation and orchestration Application monitoring Media streams transmissions planning Node monitoring Network link monitoring DCN monitoring Figure: CoUniverse architecture overview. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 55 / 5Í Multimedia Applications @ CESNET/FI ML) HW-based Solutions MVTP-4K demo Figure: Cinepost demo (4K video @ 4.5Gbps) by MVTP: real-time (i.e., interactive) movies' post-production. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 56/59 Multimedia Applications @ CESNET/FI MU CineGRID CineGRID • a non-profit international membership organization • promotes research, development, and deployment of new distributed applications of ultra-high performance digital media (sound and picture) over advanced (optical) networks • established in 2007 • members from commerce (Cisco, Sony, JVC, Sharp, etc.) as well as education (many universities and research organization, including CESNET) • *i t G i * ri www.cinegrid.org Eva Hladkä (Fl MU) 7. Networks & Multimedia Autumn 2018 57/ Further Information & Information Sources Lecture Overview Media, Multimedia Applications • Media Characteristics • Media Classification Q Multimedia Requirements on the Communication Network • Traffic Requirements • Functional Requirements 0 Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID Q Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 58/59 Further Information & Information Sources Further Information & Information Sources Fl Courses: 9 PV188: Principles of Multimedia Processing and Transport (doc. Hladká, dr. Liška, Ing. Siler) 9 PV235: Basics of IP telephony (doc. Dočkal, doc. Hladká) Literature: • S. Khanvilkar, F. Bashir, D. Schonfeld, and A. Khokhar: Multimedia Networks and Communication. University of Illinois, Chicago. 2002. • S. M. Rahman: Multimedia networking: technology, management and applications. Idea Group lne (IGI), 2002. • J. N. Hwang: Multimedia Networking: From Theory to Practice. Cambridge University Press, 2009. • relevant RFCs Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2018 59/59