7. Computer Networks & Multimedia PA159: Net-Centric Computing I. Eva Hladká Slides by: Tomáš Rebok Faculty of Informatics Masaryk University Autumn 2012 Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 1/59 Lecture Overview Q Media, Multimedia Applications • Media Characteristics • Media Classification Q Multimedia Requirements on the Communication Network • Traffic Requirements • Functional Requirements Q Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID Q Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 2 / 59 Media, Multimedia Applications Lecture Overview Q Media, Multimedia Applications • Media Characteristics • Media Classification Ql Multimedia Requirements on the Communication Network • Traffic Requirements • Functional Requirements 0 Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID Qi Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 3 / 59 Multimedia Applications I. • multimedia = the information/data that is composed of a number of different types/forms of media which are integrated together in some way • media — text, images, speech/audio, video, interaction, etc. • multimedia applications • powerful technologies that can enable remote sharing of resources or interactive work collaborations • thus saving both time and money • examples of multimedia applications: • video telephony/conferencing (speech and video) • multimedia electronic mail (text, images, and audio) • radio/television broadcast (audio and video) • electronic commerce (text, images, audio, and video) • web TV, distant learning (text, audio, and video) • real-time interactive and collaborative work environments (audio, video, and interaction) a . . . Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 4 / 59 Multimedia Applications II. a while text is inherently digital, other media types (sound, visuals) can be analog • =^> these are required to be converted into the digital form using appropriate analog to digital conversion techniques • then, the integrated digital information stream can be stored within a computer and/or transmitted over a network • in order to reduce the volume of information to be transferred, appropriate compression algorithms can be applied • suitable for the particular media type • lossy compression - eliminate redundant information from data and subsequently introduce distortion or noise in the original data • lossless compression - do not loose any information (received data is exactly identical to the original data) • the communication requirements highly differ depending on the type of media/application Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 5 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Text I. • most popular media type • distributed in many forms • FTP (File Transfer Protocol), HTTP (Hyper Text Transfer Protocol), SMTP (Simple Mail Transfer Protocol), etc. • bandwidth requirements depend on its size • can be further easily reduced by compression techniques • error characteristics depend largely on the application • some text applications (e.g., file transfer) require text communication to be completely loss/error free —>• TCP should be used • some text applications (e.g., instant messaging) may tolerate some errors as well as losses —>• UDP can be used • regarding the delay, text-based applications usually do not have any real-time constraints (such as bounded delay or jitter) • however, applications like instant messaging do require some guarantees on the experienced delay Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 6 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Text II. Compression Scheme Comments Sliannon-Fano Coding Uses variable length code words, i.e., symbols with higher probability of occurrence are represented by smaller codes-words._ Huffman Coding Same as Shannon-Fano Coding. LZW LZW compression replaces strings of characters with single codes. It docs not do any analysis of the incoming text. Instead, it just adds every new string of characters it sees to a table of strings. Compression occurs when a single code is output instead of a string of characters._ Unix Compress Uses LZW with growing dictionary. Initially the dictionary contains 512 entries, and is subsequently doubled till it reaches the maximum value set by the user. Figure: Text Compression schemes. Eva Hladká (Fl MU) 7. Networks &i Multimedia Autumn 2012 7 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Audio I. • = sound/speech converted into digital form using sampling and quantization techniques • sampling - a reduction of a continuous signal to a discrete signal • the samples are scanned in defined time periods (= sample rate) • quantization - process of mapping/approximating the samples' signal strenghts by discrete symbols or integer values Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 8 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Audio II. o bandwidth requirements of digitized audio depend on its dynamic range and/or frequency spectrum o e.g., telephone-grade voice uses dynamic range reduction (logarithmic A-law mechanism) o thus, the linear range of 12 bits is reduced to a nonlinear range of 8 bits only => throughput is reduced from 96 kbps to 64 kbps o a number of compression schemes also exist Voice/Audio Codec Used fur Bit Rate (Kbps) Pulse code Modulation (G.71 1) Narrowband speech (300 -3300 Hz) 64 GSM Narrowband speech (300 - 3300 Hz) 13 CS-ACELP (G.729) Narrowband speech (300 - 3300 Hz) 8 G. 723.3 Narrowband speech (300 - 3300 Hz) 6.4 and 5.3 Adaptive differential PCM (G.726) Narrowband speech (300 - 3300 Hz) 32 SBC (G.722) Wideband speech (50 - 7000 Hz) 48/56/64 MPEG layer III (MP3) CD-quality music Wideband Audio (10 - 22Khz) 128-112 Kbps Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 9 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Audio III. • audio media type has loose requirements on packet loss/errors • it can tolerate up to 1 to 2% packet loss/error without much degradation • most audio applications have further inbuilt mechanisms to deal with the lost packets using advanced interpolation techniques • the real-time/latency requirements strictly depend on the expected interactivity between the involved parties • e.g., Internet telephony/conferencing applications are highly interactive and require shorter response times • the audio media requires strong bounds on end-to-end packet delay/jitter to be of acceptable/decipherable quality » = Real-Time Intolerant (RTI) applications • most RTI applications require the end-to-end delay to be limited by « 100 — 200 msec to get an acceptable performance • on the other side, e.g., Internet webcasts have relatively low interactivity • it requires weaker bounds on delay/jitter • = Real-Time Tolerant (RTT) applications Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 10 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Graphics and Animation I. • includes static media types (e.g., digital images) and dynamic media types (e.g., flash presentations) » an uncompressed, digitally encoded image consists of an array of pixels, with each pixel encoded in a number of bits to represent luminance and color • usually large in size => compression schemes are usually used • most modern image compression schemes are progressive =>• this has important implications to transmission over the communication networks • when such an image is received and decompressed, the receiver can display the image in a low-quality format and then improve the display as subsequent image information is received and decompressed • e.g., a pyramid-coding method - images encoded into layers, where early layers are of low resolution and the later layers progressively increase the resolution • usually error-tolerant • provided the application used to render them knows how to handle lost packets • similarly to text files, images do not have any real-time constraints Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 11 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Graphics and Animation II. (compression scheme Comments Graphics Interchange Format (GIF) Supports a maximum of 256 colors and is best used on images with sharply defined edges and large. Hat areas of color like Text and line based drawings. GIF uses I.ZW (T,empel-7iv-Welch) compression to make files small. This is a lossless compression scheme. Portable Network Graphics (PNG) Supports any number of colors and works best with almost any type of image. PNG uses the zlib compression scheme, compressing data in blocks dependant on the "filter" of choice (usually adaptive). This is a lossless compression scheme and does not support animation. Joint Photographic Experts Group (JPEG) Best suited for images with subtle and smooth color transitions such as photographs, grayscale/colored images. Tins compression standard is based on the Huffman and Run-Length encoding of the quantized Discrete Cosine Transform (OCT) coefficients of image blocks. JPEG is a "lossy" compression. Standard JPEG encoding does not allow interlacing, but the Progressive JPEG format does. Progressive JPEGs start out with large blocks of color that gradually become more detailed. JPEG2000 Suitable for a wide range of images ranging from those produced by portable digital cameras through to advanced prc-prcss, medical imaging. JPEG 2000 is a new image coding system that uses state-of-the-art compression techniques based on wavelet technology lhat stores its information in a data stream, instead of blocks as m JPEG. This is a scalable lossy compression scheme. JPEG-LS Suitable for continuous-tone images. The standard is based on the LOCO-I algorithm (Low Complexity T.Ossless Compression lor Images) developed by TIP. This is a lossless/near-lossless compression standard. Joint Bi-level Image Experts Group (JBIG) Suitable for compressing black and white monochromatic images. Uses multiple arithmetic coding schemes to compress the image. This is a lossless type of compression. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 12 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Video I. • a sequence of images/frames displayed at a certain rate (e.g., 24 or 30 frames per second) • similarly to digitized audio, also transmitted as a stream of discrete packets over the network • bandwidth requirements are usually very high, depending whether a compression method is employed or not • uncompressed video - usually large in size, but allows to minimize the end-to-end latency • HD = 1.5Gbps, 2K = 3Gbps, 4K = 6Gbps, etc. o compressed video - smaller in size, but the compression method increases the latency • thus not suitable for real-time applications (particularly for the ones with strict real-time constraints) • error requirements and real-time characteristics similar to the audio ones Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 13 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Compression scheme Comments Used To produce VCR N'l'SC (352 x 240) quality video compression to be stored on CD-ROM (CD-I and CD-Video format) using a data rate of 1.2 Mbps. Uses heavy down-sampling of images as well as limits image rate to 24-30 Hz to achieve this goal. More generic standard for a variety of audio-visual coding applications and supports error-resilience for broadcasting. Supports broadcast-quality video compression (DVB) and High Definition Television (HDTV). MPEG-2 supports four resolution levels: low (352 x 240), mam (720 x 480). lngh-1440 (1440 x 1152), and high (1920 x 1080), The MPEG-2 compressed video data rales are in the range of 3-100 Mbps. Supports low bandwidth video compression at data rale of 64 Kbps thai can be transmitted over a single N-1SDX B channel. MPEG-4 is a genuine multimedia compression standard that supports audio and video as well as synthetic and animated images, text, graphics, texture, and speech synthesis. Supports video communications over ISDN at data rates ofpx64 Kbps. It relies on intra and inter-frame coding where integer-pixel accuracy motion estimation is required for inter mode coding The H.263 standard is aimed at video communications over POTS and wireless networks at very low data atcs (as low as 18-64 Kbps). Improvements in this standard arc due to the incorporation of several features such as half-pixel motion estimation, overlapping and variable blocks sizes, bi-directional temporal ^^h^o^n^m^^np^^^Uv^^^c^^^^i^oth^g^^^o^s^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Eva Hladká (Fl MU) Compression sehe 7. Networks & Multimedia Autumn 2012 14 / 59 Media, Multimedia Applications Media Characteristics Media Characteristics (From the Networking Point of View) Summary Application BW Latency Loss Adaptability Distributed computing all 1/xs — 10 s no low Voice low 30 ms low low Video high 300 ms low middle Haptics very low 10 ms low low WWW middle 100ms - Is no middle Push model low-middle 1 min — 1 day middle high Telemetry low-middle variable no restricted Streaming middle 1 - 10 s low high Data backup high 1 hour zero high Email low 1 min — 1 hour no high Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 15 / 59 Media, Multimedia Applications Media Classification Media Classification (from the networking perspective) I. • all media types can be classified as either real-time (RT) or non real-time (NRT) • RT media types require either hard or soft bounds on the end-to-end packet delay/jitter • NRT media types (text, image files, etc.) do not have any strict delay constraints • but may have rigid constraints on error • the RT media can be further classified as discrete media (DM) or continuous media (CM) • depending on whether the data is transmitted in a discrete quantum (e.g., file or message) or continuously (as a stream of messages with inter-message dependency) • the RT-CM media can be further classified as delay tolerant or delay intolerant • depending on whether the media can tolerate higher amounts of delay without significant performance degradation Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 16 / 59 Media, Multimedia Applications Media Classification Multimedia Requirements on the Communication Network Lecture Overview 0 Media, Multimedia Applications • Media Characteristics • Media Classification Q Multimedia Requirements on the Communication Network • Traffic Requirements • Functional Requirements 0 Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID 0 Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 18 / 59 Multimedia Requirements on the Communication Network Multimedia Requirements on the Communication Network • can be generally divided into: • traffic requirements - limits on real-time parameters (such as delay, jitter, bandwidth, and reliability) » can be met only by enhancements to the basic Internet Architecture • functional requirements - support for multimedia services (such as multicasting, security, mobility, and session management) • can be met by introducing newer protocols over the TCP/IP networking stack Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 19 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) I. • real-time traffic enforces strict bounds on end-to-end packet delay and jitter • delay — time taken by the packet to travel from the source to the destination » human beings can tolerate a latency of « 100 — 200 msec • jitter — variability in the inter-packet delay at the receiver • the performance improves with decrease in both these quantities • the end-to-end delay is influenced by: • Packet Processing Delay • Packet Transmission Delay • Propagation Delay • Routing and Queuing delay Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 20 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) II. Packet Processing Delay • a constant amount of delay faced at both the source and the destination • A/D, D/A conversion time and time taken to packetize it through different layers of protocols • usually a characteristic of the operating system and the multimedia application • for a lightly loaded system this can be considered as negligible » however, with increasing load this delay can become significant • its reductions imply software enhancements • including the use of multimedia OSs that provide enhanced process-, resource-, file- and memory-management techniques with real-time scheduling, and • enhancing the application Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 21 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) III. Packet Transmission Delay • time taken by the physical layer at the source to transmit the packets over the link • depends on multiple factors: • number of active sessions - since physical layer processes packets in FIFO order, this may become significant especially if the OS does not support real-time scheduling algorithms supporting multimedia traffic • MAC access delay - the delay influenced by accessing the media by the data sender(s) • collisions lead to delayed service time • widespread Ethernet networks cannot provide any firm guarantees on this access delay (due to the indeterminism of the CSMA/CD approach) • e.g., Isochronous Ethernet (integrated voice data LAN, IEEE 802.9) and demand priority Ethernet (100Base-VG, AnyLAN, IEEE 802.12) can provide QoS • context switch in the OS - sending or receiving a packet involves context switch in the OS • takes a finite time => there exists a theoretical maximum at which computer can transmit packets • reduction in this delay require enhancements to the device drivers and increasing the operating speed of the computer Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 22 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) IV. Propagation Delay • the flight time of packets over the transmission link • limited by the speed of the transmission signal (at most, by the light) • shorter distances imply shorter propagation delays • distance of 200 m propagation delay « 1 ms • distance of 20000 km propagation delay « 100 ms (!!!) • the physical limits which cannot be reduced! (speed of the light) • remember, interactive applications require the response time to be lower than 100 - 200 ms • => if the propagation delay becomes greater than this value, no enhancements can improve the quality of the interactive sessions Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 23 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Real-time characteristics (limits on delay and jitter) V. Routing and Queuing Delay • the best-effort Internet treats every packet equally • regardless of whether it is a real-time packet or a non-real-time packet • all intermediate routers make independent routing decisions for every incoming packet • when packets arrive at a queue, they have to wait for a random amount of time before they can be serviced • depends on the current load on the router • this delay is random and thus becomes the major contributor to jitter in the traffic streams • can be reduced by: • Integrated Services (Intserv), Differentiated Services (Diffserv), MPLS, etc. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 24 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Figure: Differentiated services model. Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Bandwidth I. • multimedia traffic streams have usually high bandwidth demands • the communication network must be able to handle such high bandwidth requirements • without being unfair to other conventional flows • the required bandwidth can be reduced by compression mechanisms • but, these cannot be used for all multimedia transmissions (e.g., interactive real-time video) • the best-effort Internet does not provide any mechanism to reserve network resources to meet such high bandwidth requirements • left to the discretion of the application to dynamically adapt to network congestions • TCP-based applications take advantage of built in congestion control • however, multimedia apps are usually UDP-based - these do not have any mechanism to control congestion (they can create a congestion collapse) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 27 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Bandwidth II. • impossible to remove these shortcomings in the nowadays best-effort Internet • an enhanced Internet service model would require: • Admission control: application must first get a permission to send traffic at a given rate with given traffic characteristics • Bandwidth reservation: if the admission is given, appropriate resources (buffers, bandwidth) will get reserved along the path • Traffic policing mechanisms: to ensure that applications do not send data at a rate higher than what was negotiated • cost of the protocol processing operations relates more directly to the packet processing rate than to the bandwidth (in terms of bit rate) • packet processing is largely dependent on number of packets and less on packet size • packet processing rate is an important metric Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 28 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Requirements • different media types have vastly different error requirements • (error — lost or damaged packet) • from being completely error-intolerant to being somewhat error-tolerant » the best-effort Internet cannot provide guarantees on the error characteristics because the path a packet follows is not fixed • => the sender application has no knowledge of the error characteristics of the network • most common methods for error correction: • Sender-based Repair • active repair - Automatic Retransmission Request (ARQ) mechanisms, suitable for error intolerant applications • passive repair - Forward Error Correction (FEC), Interleaving, etc. • Receiver-based Repair - Error Concealment Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 29 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Recovery - Forward Error Correction (FEC) I. a requires extra information (repair data) to be added to the packet stream • this information is used for recovering lost packets • Media-independent FEC- uses block or algebraic codes to produce additional packets which help in loss recovery • codes take a codeword of n data packets and generate m additional check packets • examples: • Parity Coding - XOR is applied across groups of packets to generate parity packets • Reed-Solomon Coding - based on properties of polynomials over particular number bases • disadvantage: cause additional delay and increase bandwidth usage Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 30 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Recovery - Forward Error Correction (FEC) II. • Media-dependent FEC - exploits media characteristics • each data unit is sent in multiple packets • primary encoding + secondary encoding (could be of lower quality/bandwidth than the primary one) • may not be necessary to transmit FEC for every packet due to nature of the media • advantage: low latency (only single packet delay added) =^> suitable for interactive applications • it is advantagenous for an application to know the error characteristics of the communication network • =^> adequate level of FEC can be employed • e.g., wireless vs. wired networks Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 31 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Recovery - Forward Error Correction (FEC) III. sit Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 32 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Recovery - Interleaving II. • can be used only when media unit size is smaller than packet size (e.g., audio units) and the end-to-end delay is not important a based on resequencing media units before transmission (originally adjacent units are separated by a guaranteed distance) and returning to the original order at the receiver • disperses the effect of the packet loss • loss of a single packet causes multiple smaller gaps among original media units (easier to deal with) • advantage: does not increase bandwidth usage • disadvantage: increases latency =4> not well suited for interactive applications Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 33 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Requirements - Interleaving II. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 34 / 59 Multimedia Requirements on the Communication Network Traffic Requirements Traffic Requirements Error Requirements - Error Concealment • predicting/compensating the lost information from correctly received packets • used by many error-tolerant applications • works well for relatively small loss rates (< 15%) and for small packets (4 - 40 ms) • several types (in increasing order of computational cost and improved performance): • Insertion-based: inserting a fill-in packet that contains silence, noise, or a repitition of adjacent packets • Interpolation-based: some form of pattern matching and interpolation to derive the missing packet • Regeneration-based: derive decoder state from packets surrounding the loss and generate a lost packet from that (— model based recovery) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 35 / 59 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Multicasting Support I. • multicast = delivery of a message/information to a group of destination computers simultaneously in a single transmission from the source, creating copies automatically in inner network elements (e.g., routers) only when the topology of the network requires it • network property (hop-by-hop, not end-to-end service) • however, not available in many networks • due to problems it usually brings o employed protocols: IGMP, DVMRP, MOSPF, PIM, etc. • =^> can be compensated by the application layer (virtual) multicast • a single (or a set of) special-purpose node(s)/application(s) in the network reflecting the incoming traffic • e.g., UDP packet reflector, (Distributed) Active Element, SIP and H.323 MCUs, etc. • can provide additional features (data processing, access control, moderating, etc.) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 36 / 59 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Multicasting Support II. sender Figure: Native Multicast Distribution Tree (at most one data copy per link). Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 37 / Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Multicasting Support III. reflector sender Figure: Virtual Multicast Distribution Tree (one data copy per host). Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Security & Mobility Security • provides the following aspects to multimedia data: a Integrity — data cannot be changed in mid-flight • Authenticity — data comes from the right source • Encryption — data cannot be deciphered by any third party • ensured by IPSec/IPv6 • Authentication Header (AH) and Encapsulated Security Payload (ESP) protocols Mobility • mobility in IPv6 Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 39 / 59 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Session Management I. The session management functionality includes: • Media Description • enables a distributed multimedia application to distribute session information • e.g., media type (audio, video or data) used in the session, media encoding schemes (PCM, MPEG II), session start time, session stop time, IP addresses of involved hosts, etc. • usually essential to describe the session before establishment • e.g., Session Description Protocol (SDP), Session Initiation Protocol (SIP) • Session Announcement • allows participants to announce future sessions o e.g., announcing scheduled shows by Internet radio stations • e.g., Session Announcement Protocol (SAP) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 40 / 59 Multimedia Requirements on the Communication Network Functional Requirements Functional Requirements Session Management II. a Session Identification • a multimedia session often consists of multiple media streams (audio, video, text, images, etc.) that need to be separately identified • e.g., sending the audio and video as two separate streams, which the receiver needs to decode synchronously • e.g., dividing the quality of the A/V stream into low-bandwidth and high-bandwidth ones • e.g., Real-Time Transport Protocol (RTP) a Session Control • since multimedia session may involve multiple media streams, whose information is often inter-related, the multimedia communication network must guarantee to maintain such relationships at the end user • = Multimedia Synchronization • moreover, many Internet multimedia applications may want to control the playback of continuous media by pausing, playing-back, repositioning playback, etc. • e.g., Real-Time Control Protocol (RTCP), Real-Time Streaming Protocol (RTSP) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 41 / 59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Session Description Protocol (SDP) I. • developed by IETF • a format for describing streaming media parameters (media type, media encoding, etc.) • more of a description syntax than a protocol • media descriptions encoded in text format • SDP message contains a series of lines (called fields) » each field has a = format • tag = a pre-defined single letter abbreviation Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 42 / 59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Session Description Protocol (SDP) II. Session description v= (protocol version) o= (originator and session identifier) s= (session name) i=* (session information) u=* (URI of description) e=* (email address) p=* (phone number) c=* (connection information — not required if included in all media) b=* (zero or more bandwidth information lines) One or more time descriptions ("t=" and "r=" lines; see below) z=* (time zone adjustments) k=* (encryption key) a=* (zero or more session attribute lines) Zero or more media descriptions Time description t= (time the session is active) r=* (zero or more repeat times) Media description, if present m= (media name and transport address) i=* (media title) c=* (connection information — optional if included at session level) b=* (zero or more bandwidth information lines) k=* (encryption key) a=* (zero or more media attribute lines) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 43 / 59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Session Announcement Protocol (SAP) • used for advertising multicast conferences and other multicast sessions • SAP announcer periodically multicasts announcement packets to a well-known multicast address and port (port number 9875) • with the same scope as the session being announced => announcements' recipients are also potential recipients of sessions being advertised • uses SDP to describe the session(s) • contains mechanisms for: • ensuring integrity of session announcements • authenticating the origin of an announcement • encrypting such announcements Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 44 / 59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Real-Time Transport Protocol (RTP) • runs on top of UDP (usually implemented within the application) • carries chunks of real-time (audio/video) data • provides: • Sequencing: • a sequence number in the RTP header helps to detect lost packets • Payload identification: • payload identifier (included in each RTP packet) describes encoding of the media • Frame indication: • video and audio are sent in logical units called frames — RTP's frame marker bit indicates the beginning and end of a frame • Source identification: • to identify the originator of a frame in a multicast session, the Synchronization Source (SSRC) identifier is provided • Intramedia synchronization: • to compensate for different delay and jitter for packets within the same stream, timestamps are provided • additional media information can be inserted using profile headers and extensions • RTP cannot assure any quality of the transmissions, but provides resources to the applications to do that Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 45 / 59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Real-Time Control Protocol (RTCP) • a control protocol that works in conjunction with RTP • the primary function of RTCP is to gather statistics on quality aspects of the media distribution during a session and to transmit this data to the session media source and other session participants • the participants periodically send RTCP packets to give feedback about the quality of the received data • provides out-of-band statistics and control information for an RTP flow • packets sent, lost, jitter, round-trip time, etc. • sources can use them to adjust their data rate (adaptive media encoding) and for detection of transmission faults » enclosed time information can be used for synchronization purposes • other information includes: o email address, phone number, name, etc. • allow users to know the identities of other users in the session Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 46 / 59 Multimedia Requirements on the Communication Network Functional Requirements Session Management Protocols Real-Time Streaming Protocol (RTSP) • an out-of-band control protocol that allows the media player application to control the transmission of the media stream • including functions such as pause, resume, repositioning, playback, etc. • the transmission of streaming data itself is not a task of the RTSP protocol • designed to work with RTP (and RSVP) to provide a complete streaming service over the Internet • similar to HTTP in some ways • but, HTTP is stateless, while RTSP is statefull (an identifier is used when needed to track concurrent session) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 47 / 59 Multimedia Applications a CESNET/FI MU Lecture Overview Qi Media, Multimedia Applications • Media Characteristics • Media Classification Ql Multimedia Requirements on the Communication Network • Traffic Requirements a Functions! RcQuircmcnts Q Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID Qi Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 48 / 59 Multimedia Applications a CESNET/FI MU State-of-the-Art I. • current research in the area of multimedia networking mainly focuses on video media transmissions • the research issues include: • to allow higher resolutions (4K, 8K, etc.) • to provide minimal latency (uncompressed video transmissions) • to lower the required transmission throughputs (JPEG2000, JPEG2000 on GPUs) • (including their combinations) HD (1920x 1080) 2 K (2048x 1080) UHDTV1 (3840x2160) 4K (4096x2160) UHDTV2 (7680 x 4320) 8K (8192x4320) Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 49 / 59 Multimedia Applications a CESNET/FI MU State-of-the-Art II. • achieved by employing both SW (e.g., Ultragrid) and HW tools/devices (e.g., MVTP-4K) Figure: Modular Video Transfer Platform (MVTP) designed by Cesnet. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 50 / 59 Multimedia Applications @ CESNET/FI ML) SW-based Solutions FI:PA177 (High Performance Computing) Figure: FI.PA177 (High Performance Computing) by Ultragrid &i UDP packet reflectors: Course taught by prof. Thomas Sterling at Louisiana State University and transferred in realtime to Fl students (uncompressed HD 0 1.5 Gbps). Remotely-involved universities include: Fl MU, University of Arkansas, Louisiana Technical University, MCNC North Carolina, and North Carolina State University. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 51 / 59 Eva Hladká (Fl MU) Figure: LSU side. 7. Networks & Multimedia Autumn 2012 52 / 59 Multimedia Applications 0 CESNET/FI MU SW-based Solutions FI:PA177 (High Performance Computing) Connectivity scheme Figure: FI.PA177 (High Performance Computing) by Ultragrid & UDP packet reflectors: Connectivity scheme with time axis. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 Multimedia Applications a CESNET/FI MU SW-based Solutions Visualisations in Collaborative Environments Figure: High-volume real-time visualisation in collaborative environment: HDTV stream generated in Baton Rouge and transported (by Ultragrid) to Brno and San Diego. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 Multimedia Applications 0 CESNET/FI MU SW-based Solutions Multimedia Applications a CESNET/FI MU HW-based Solutions MVTP-4K demo Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 56 / 59 Multimedia Applications 0 CESNET/FI MU CineGRID CineGRID a non-profit international membership organization promotes research, development, and deployment of new distributed applications of ultra-high performance digital media (sound and picture) over advanced (optical) networks established in 2007 members from commerce (Cisco, Sony, JVC, Sharp, etc.) as well as education (many universities and research organization, including CESNET) \ \\ pip Grid Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 57 / Further Information &i Information Sources Lecture Overview Qi Media, Multimedia Applications • Media Characteristics • Media Classification Ql Multimedia Requirements on the Communication Network • Traffic Requirements • Functional Requirements 0 Multimedia Applications @ CESNET/FI MU • SW-based Solutions • HW-based Solutions • CineGRID Q Further Information & Information Sources Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 58 / 59 Further Information & Information Sources Further Information & Information Sources Fl Courses: • PV188: Principles of Multimedia Processing and Transport (doc. Hladká, dr. Liška, Ing. Siler) • PV235: Basics of IP telephony (doc. Dočkal, doc. Hladká) Literature: • S. Khanvilkar, F. Bashir, D. Schonfeld, and A. Khokhar: Multimedia Networks and Communication. University of Illinois, Chicago. 2002. • S. M. Rahman: Multimedia networking: technology, management and applications. Idea Group lne (IGI), 2002. • J. N. Hwang: Multimedia Networking: From Theory to Practice. Cambridge University Press, 2009. • relevant RFCs a . .. Eva Hladká (Fl MU) 7. Networks & Multimedia Autumn 2012 59 / 59