ETSI TR 102 794 V1.1.1 (2010-12)

Media Content Distribution (MCD); 3D Gaming Graphics Delivery Overview

DTR/MCD-00010

General Information

• Status: Published
• Publication Date: 09-Dec-2010
• Technical Committee: MCD - Media Content Distribution
• Current Stage: 12 - Completion
• Due Date: 30-Oct-2010
• Completion Date: 10-Dec-2010

Standards Content (Sample)

Technical Report
Media Content Distribution (MCD);
3D Gaming Graphics Delivery Overview

2 ETSI TR 102 794 V1.1.1 (2010-12)

Reference
DTR/MCD-00010
Keywords
content, transport, protocol
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2010.
All rights reserved.
TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
3 ETSI TR 102 794 V1.1.1 (2010-12)
Contents
Intellectual Property Rights . 4
Foreword . 4
Introduction . 4
1 Scope . 5
2 References . 5
2.1 Normative references . 5
2.2 Informative references . 5
3 Definitions and abbreviations . 7
3.1 Definitions . 7
3.2 Abbreviations . 8
4 Use case(s) and high level requirements . 8
4.1 Primary use case . 8
4.2 High level requirements . 9
4.2.1 Minimize input lag . 9
4.2.2 Maximize image quality and framerate . 10
4.2.3 Allow Scalability . 11
4.2.4 Maximize availability . 11
4.2.5 Distribution of game processing resources . 11
4.2.6 Reuse existing infrastructure. 11
4.2.7 Support for a variety of devices . 11
4.2.8 Authentication and security . 11
5 Solution approaches for delivery of 3D output of games . 12
5.1 Streaming 3D graphics primitives . 12
5.1.1 Main concept . 12
5.1.2 Benefits/Drawbacks of this approach . 12
5.2 Streaming Video . 13
5.2.1 Main concept . 13
5.2.2 Benefits/Drawbacks of this approach . 14
5.3 Transport of 3D models . 14
5.3.1 Main concept . 14
5.3.2 Benefits/Drawbacks of this approach . 15
5.4 Progressive transport of adaptive 3D models . 15
5.4.1 Main concept . 15
5.4.2 Benefits/Drawbacks of this approach . 16
6 Overview of existing standards . 17
7 Conclusions and recommendations . 19
History . 20

ETSI
4 ETSI TR 102 794 V1.1.1 (2010-12)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI Technical Committee Media Content Distribution (MCD).
The main purpose of the present document is to provide an overview of the topic of 3D Gaming Graphics Delivery. The
present document is meant to create an ETSI MCD view on this topic, and includes use cases, high-level requirements,
an overview of potential solutions and a gap analysis.
The present document's conclusions should assist in the definition of new Work Item(s) for actual standardization,
provided this is necessary and ETSI MCD is the right body.
Introduction
By 2010, there will be over 420 million broadband households worldwide. With the standard set for super-high speed,
always-on connectivity, the way people view entertainment has fundamentally changed and created a new standard for
consumption. Consumers no longer expect their internet access to be only from a desktop PC, now they want it through
the TV in the living room or in the palm of their hand, inside the house and on the go.
Video games are officially a mainstream leisure time activity, and consumers are increasingly demanding their
entertainment to be fully networked and available over multiple platforms. Digital distribution of games is growing and
is becoming a common way to acquire games. Several game publishers and game retailers are using this model as a way
to reach consumers. The main deployment model being used at the moment of writing the present document is where an
end consumer buys the game online after which the game executable and (parts of) the game data are downloaded to his
PC or game console, using a run-time environment that protects the game from being copied.
Consumer electronics devices, such as set-top boxes, TVs, mobile phones and Netbook PCs are not considered real
gaming devices, since they lack the necessary CPU power, graphical performance and memory/storage capacity. So a
download distribution model of 3D games is not very likely to be a viable way to reach consumer electronics devices.
Also, there is a large catalogue of existing games that simply are not written with these platforms in mind.
In order to play 3D games on this kind of devices a streaming model is more appropriate. So, instead of downloading
the game to the end user device, the game is executed on a high-performance server, after which the game output is
streamed to the client. Discussion about the nature and content of what is exchanged between servers and clients
constitutes the topic of the present document. Several technical solutions could be possible. The present document will
describe the benefits and drawbacks of several of these technical solutions.
The end goal of this work is to standardize the protocol(s) needed for streaming the game output to a client. The present
document is meant to create an ETSI MCD view on this topic, and includes use cases, high-level requirements, an
overview of potential solutions and a gap analysis to see which actual standardization work needs to be performed.
ETSI
5 ETSI TR 102 794 V1.1.1 (2010-12)
1 Scope
The present document provides an overview of delivery of 3D graphics of games that are running on a
high-performance server to client devices that would otherwise not have the resources to run these games natively. The
present document describes the use cases, high level requirements and different solution approaches, and identifies the
main area(s) where standardization work would be needed. Exploring the nature and content of what is going to be
exchanged between servers and clients constitutes the main topic of the present document
The primary aim for this technology is to enable casual gaming scenarios, and it is not the primary aim to try to satisfy
hardcore gamers, since we have to be realistic about what can be achieved with the current state of networking and
server technology.
Because of the nature of the content and in order to focus on feasible solutions, we may exclude certain type of network
connections to be used, such as using "unreliable" or high-latency wireless connections such as 802.11b/g. We expect to
have a broadband connection between the client and the server, without too many intermediate hops.
In the present document we do not address deployment issues, such as integration into a Content on demand
infrastructure (e.g. user registration, pricing, purchasing), or integration into a QoS framework (e.g. prioritization of
gaming streams compared to other types of streams). Also, use cases and synchronization issues related to multi-player
gaming, whereby different connection speeds/types could result in different response times, are out of scope of the
present document.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are necessary for the application of the present document.
Not applicable.
2.2 Informative references
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] Alf Inge Wang, Martin Jarrett and Eivind Sorteberg: "Experiences from Implementing a Mobile
Multiplayer Real-Time Game for Wireless Networks with High Latency", International Journal of
Computer Games Technology Volume 2009, Hindawi Publishing Corporation.
NOTE: Available at: http://downloads.hindawi.com/journals/ijcgt/2009/530367.pdf.
[i.2] Console Gaming: "The Lag Factor", Eurogamer® Network Ltd., 5 September 2009.
NOTE: Available at: http://www.eurogamer.net/articles/digitalfoundry-lag-factor-article?page=3.
[i.3] OpenGL™ ES: "The Standard for Embedded Accelerated 3D Graphics", Khronos Group.
NOTE: Available at: http://www.khronos.org/opengles/.
ETSI
6 ETSI TR 102 794 V1.1.1 (2010-12)
[i.4] Jurgelionis et al.: "Platform for Distributed 3D Gaming", International Journal of Computer
Games Technology Volume 2009, Article ID 231863.
NOTE: Available at:
http://iphome.hhi.de/fechteler/papers/ijcgt2009_JurgelionisFechtelerEisertBellottiDavidLaulajainenCarmi
chaelPoulopoulosLaikariPeraelaeGloriaBouras.pdf.
[i.5] Philipp Fechteler and Peter Eisert: "Depth Map Enhanced MacroBlock Partitioning for H.264
Video Encoding of Computer Graphics Content", 2009 IEEE International Conference on Image
Processing.
NOTE: Available at: http://iphome.hhi.de/eisert/papers/icip09a.pdf.
[i.6] Hugues Hoppe: "Progressive meshes", SIGGRAPH '96: Proceedings of the 23rd annual
conference on Computer graphics and interactive techniques, Pages: 99 - 108, ACM
Press/Addison-Wesley Publishing Co., 1996.
NOTE: Available at: http://research.microsoft.com/en-us/um/people/hoppe/pm.pdf.
[i.7] "Progressive forest split compression", SIGGRAPH '98: Proceedings of the 25th annual
conference on Computer graphics and interactive techniques, Pages 123-132, ACM
Press/Addison-Wesley Publishing Co., 1998.
NOTE: Available at: http://mesh.brown.edu/taubin/pdfs/taubin-etal-sg98.pdf.
[i.8] ISO/IEC 15444-1 (2004): "Information technology - JPEG 2000 image coding system: Core
coding system, latest revision 2004".
[i.9] ISO/IEC 14496-1 (2010): "Information technology - Coding of audio-visual objects - Part 1:
Systems".
[i.10] ISO/IEC 14496-10 (2009): "Information technology - Coding of audio-visual objects - Part 10:
Advanced Video Coding".
[i.11] ISO/IEC 19775-1.2 (2008): "X3D™ Architecture and base components, Edition 2".
NOTE: Available at: http://www.web3d.org/x3d/specifications/X3DPublicSpecifications.zip.
[i.12] WebGL™ Specification, Working Draft (27 May 2010), Khronos Group.
NOTE: Available at: https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/WebGL-spec.html.
[i.13] Adobe Systems Incorporated: "Adobe Flash™ Player".
NOTE: Available at: http://www.adobe.com/products/flashplayer/.
[i.14] ISO/IEC 14496-11 (2005): "Information technology - Coding of audio-visual objects -
Part 11: description and application engine".
[i.15] OpenGL™ Graphics with the X Window System (GLX), Version 1.4, December 2005.
NOTE: Available at: http://www.opengl.org/documentation/specs/.
[i.16] John Stavrakakis et al.: "Exposing Application Graphics to a Dynamic Heterogeneous Network",
Proceedings of the 14-th International Conferences in Central Europe on Computer Graphics,
Visualization and Computers.
NOTE: Available at: http://wscg.zcu.cz/wscg2006/papers_2006/full/g23-full.pdf.
[i.17] Greg Humphreys et al.: "Chromium: A Stream-Processing Framework for Interactive Rendering
on Clusters", SIGGRAPH 2002.
NOTE: Available at: http://chromium.sourceforge.net/.
ETSI
7 ETSI TR 102 794 V1.1.1 (2010-12)
[i.18] Micah Dowty et al., VMWare™ Inc.: "GPU virtualization on VMware's hosted I/O architecture",
USENIX Workshop on I/O Virtualization 2008, ACM SIGOPS Operating Systems Review ,
Volume 43 Issue 3.
NOTE: Available at: http://www.usenix.org/event/wiov08/tech/full_papers/dowty/dowty.pdf.
[i.19] Gaikai™, Gaikai™ Inc.
NOTE: Available at: http://www.gaikai.com/.
[i.20] T5 Labs Limited.
NOTE: Available at: http://www.t5labs.com/.
[i.21] Onlive™, Onlive™ Inc.
NOTE: Available at: http://www.onlive.com/
[i.22] Tristan Richardson et al.: "Virtual Network Computing (VNC)", IEEE Internet Computing
Volume 2, Number 1, 1998.
NOTE: Available at: http://www.cl.cam.ac.uk/research/dtg/attarchive/pub/docs/att/tr.98.1.pdf.
[i.23] XRT Self-contained Remoting Protocol, ANSI/CEA-2014-A: "Web-based Protocol and
Framework for Remote User Interface on UPnP™ Networks and the Internet (Web4CE),
Annex O".
NOTE: Available at: http://global.ihs.com/search_res.cfm?MID=W097&input_doc_number=CEA-2014.
[i.24] The VirtualGL project.
NOTE: Available at: http://www.virtualgl.org/Main/HomePage.
[i.25] ISO/IEC 14772-1 (1997) and ISO/IEC 14772-2 (2004): "Virtual Reality Modeling
Language (VRML)".
NOTE: Available at: http://www.web3d.org/x3d/specifications/vrml/VRML97Am1/.
[i.26] COLLADA™: "Digital Asset Exchange Schema for Interactive 3D", Khronos Group.
NOTE: Available at: http://www.khronos.org/collada/.
[i.27] Dr. Rémi Arnaud (Sony Computer Entertainment) et al.: "Developing Web Applications with
COLLADA™ and X3D™ Whitepaper", March 25 2007.
NOTE: Available at:
http://www.khronos.org/collada/presentations/Developing_Web_Applications_with_COLLADA_and_X3
D.pdf.
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the following terms and definitions apply:
casual games: games that typically do not require long-term time commitment or special skills to start playing, targeted
at casual consumers, who comes across the game and can get into gameplay almost immediately
NOTE: Casual gaming does not necessarily mean that it has to be offered for free or that the graphics are
primitive.
hardcore gamer: gamer who spends a considerable amount of time and money on games, and who prioritises gaming
over other leisure activities
ETSI
8 ETSI TR 102 794 V1.1.1 (2010-12)
input lag: time between a user action and its response on the screen
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
AVC Advanced Video Coding
BIFS Binary Format for Scenes
COLLADA™ COLLAborative Design Activity
GLX OpenGL™ Extension to the X Window System
GPU Graphics Processing Unit
MCD Media Content Distribution
MMORPG Massive Multiplayer Online Role-Playing Game
OpenGL™ ES OpenGL™ for Embedded Systems
OpenGL™ Open Graphics Library
RTP Realtime Transfer Protocol
SDK Software Development Kit
STB SetTop Box
UDP User Datagram Protocol
VNC Virtual Network Computing
VoD Video on Demand
VRML Virtual Reality Modeling Language
X3D™ eXtensible 3D Graphics
4 Use case(s) and high level requirements
4.1 Primary use case
The primary use case is to offer the ability for a user to play 3D games while part of the game resources (such as the
game graphics, game logic, etc.), are not located on the end user's device but are hosted on a server in the network. The
game's 3D graphics (possibly in conjunction with some game logic) are delivered to the end user's device in a streaming
fashion. This allows an easy and flexible way to access games on a variety of different (thin-client) devices. An
example of such a scenario could be a person on the couch watching TV, that whilst waiting for his favourite show to
start, wants to (spontaneously) play a game to pass the time without having to insert a game DVD or wait for a game
download to finish.
The server device can be located anywhere in a network, for example server in a data center ("cloud server") or a PC at
home. We expect the games to be written for commonly used 3D rendering APIs such as DirectX™ or OpenGL™.
The resulting system's architecture is typically one of the two following ones:

Figure 1: Graphics streaming architecture with game logic running on server
ETSI
9 ETSI TR 102 794 V1.1.1 (2010-12)
In this case, a gaming server loads the game executable (and perhaps some related metadata), and starts running the
game on the server. The graphical output and the audio are both encoded into a streaming format, after which the
encoded content is streamed to the client. The client receives the graphics and audio stream content, decodes and
renders the output. The user input (e.g. from a game controller) is sent back to the server and taken into account by the
game for subsequent rendering. Since the client device does not need to resources to run the game, it enables devices
with limited resources, such as TVs, STBs, Netbook PCs etc. to have access to compelling 3D games that would
otherwise not be able to run.
Figure 2: Graphics streaming architecture with (parts of) game logic running on client
In this second architecture, (parts of) the game logic is delivered to (or already present on) the client and executed on
the client device while graphics and audio data are hosted on the server and streamed to the client on request.
4.2 High level requirements
The main requirement for a streaming and distributed processing system for games is to address the concerns of game
play. As any other gaming platform, streaming games from a server faces strong constraints against which an
appropriate solution is required so that gamers do not simply run away from the platform because their experience is not
up to their expectations. The four main constraints for the solution to take into account are: input lag, image quality,
scalability of the service and availability of the service. See clauses 4.2.1 through 4.2.4 for more details. The list below
also contains a couple of other important high level requirements in clause 4.2.5 onwards.
NOTE: As mentioned in the Scope (clause 1), the primary focus is on the requirements for the 3D gaming
graphics delivery protocol. Deployment issues such as user registration, content selection and purchase
are outside the scope of the present document. Also, inter-destination synchronization of multi-user
games graphics is out of scope of the present document.
4.2.1 Minimize input lag
Input lag, i.e. the time between a user action and its response on the screen, is probably concern #1 for many of the
gamers.
Input lag can be broken down into the following subcomponents:
• input detection time (usually negligible, if we assume the latest wired USB and wireless controllers)
• input information transport time (if information is not handled locally)
• input information management in the game or game engine
• game rendering after input management (actual creation of the output frame)
• transport time of the game rendering data (if not displayed locally)
• display rendering (actual display of the video frame on screen)
Network phenomena, such as latency, jitter, lack of bandwidth or packet loss can introduce additional lag. The solution
needs to take these networking aspects into account and minimize their effects on the gameplay.
ETSI
10 ETSI TR 102 794 V1.1.1 (2010-12)

Figure 3: Graphical illustration of input lag
The effects of input lag on the user depend on the user's own sensitivity to it. A hardcore gamer would certainly not put
up with a minimum amount of lag while an occasional player would probably not even notice it.
In the application of gaming, the type of game being played is another important factor that may increase one's
perception of lag as players' expectations on game play may considerably change depending on the type of the game.
In the present document, we will distinguish:
• Slow paced games (such as chess games and typical MMORPGs such as World of Warcraft™): in slow paced
games, input lag is usually not a problem. Slight delays (up to 500 ms) are usually tolerable (see section 2
of [i.1] for more information).
• Medium paced games (such as platform games like Super Mario™ 64, and sports games): in medium paced
games, delays higher than 150 ms or 200 ms would probably be noticeable by gamers (see section 2 of [i.1] for
more information).
• High paced games (such as fast-action shooters like Unreal Tournament™ or Crysis™, or fighting games): in
high paced games, the reaction time requirement for the user is usually very strict and typically any lag over
100 ms affects the gaming experience (see section 2 of [i.1] for more information).
NOTE 1: Current console games typically have an average lag of about 133 ms (see [i.2]).
NOTE 2: There may not be one unique solution that can fit the constraints imposed by any type of games streamed
on any kind of network with the current state of technology. As a consequence, in the present document,
we may envisage to recommend a different approach in function of game streaming context and its effects
on user's quality of experience.
4.2.2 Maximize image quality and framerate
For some gamers, the image quality of the video game is very important. This is especially true for PC gamers who
have the capability to play their video games with different quality settings. In order to obtain the highest possible
quality; some would regularly update their hardware or even sacrifice framerate (and consequently input lag as well).
However, others prefer a higher framerate over highest possible image quality.
It is perceived that under a certain level of image quality, the gaming experience becomes highly impacted.
Nevertheless, Nintendo™ proved with the Wii™ that image quality is not necessarily #1 criterion for all gamers. For
framerates below a certain level, the gaming experience also becomes highly impacted.
ETSI
11 ETSI TR 102 794 V1.1.1 (2010-12)
So on the one hand we need to maximize image quality and framerate, on the other hand, we should make the solution
flexible enough to allow selection of different quality levels (e.g. level of details rendered by the client).
4.2.3 Allow Scalability
Being able to play a game with the same experience whether there is just one player or thousands of players playing
together is also an important requirement for gamers. For games running on a server, this is a major issue since many
players are sharing the same server even if playing a mono-player video game. While players may understand that the
gaming experience in multi-player games may decrease while the number of players increase, it may be very difficult to
have them understand that the same thing happens with mono-player games too.
Scalability is not only important for gamers, but also for the service providers that distribute the game, since the
scalability of the solution directly affects the cost involved to run the game. The more players that can be served by a
single server, the lower the cost to deploy the game. This affects the price range at which streaming of games can be
offered.
The solutions studied in the present document should take into account the eventuality of adapting quality depending on
the server's resources.
4.2.4 Maximize availability
Availability is not an issue
...