ISO/IEC 23005-1:2016
(Main)Information technology — Media context and control — Part 1: Architecture
Information technology — Media context and control — Part 1: Architecture
ISO/IEC 23005-1:2016 specifies the architecture of MPEG-V (media context and control), its three associated use cases of information adaptation from virtual world to real world, information adaptation from real world to virtual world, and Information exchange between virtual worlds.
Technologies de l'information — Contrôle et contexte de supports — Partie 1: Architecture
General Information
Relations
Buy Standard
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 23005-1
Third edition
2016-07-15
Information technology — Media
context and control —
Part 1:
Architecture
Technologies de l’information — Contrôle et contexte de supports —
Partie 1: Architecture
Reference number
ISO/IEC 23005-1:2016(E)
©
ISO/IEC 2016
---------------------- Page: 1 ----------------------
ISO/IEC 23005-1:2016(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2016, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2016 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 23005-1:2016(E)
Contents Page
1 Scope . 1
2 Terms and definitions . 1
2.1 Device Command . 1
2.2 R V Adaptation . 1
2.3 Sensed Information . 1
2.4 Sensor. 1
2.5 Sensor Adaptation Preferences . 1
2.6 Sensor Capability . 1
2.7 Sensory Device . 2
2.8 Sensory Device Capability. 2
2.9 Sensory Effects . 2
2.10 Sensory Effect Metadata . 2
2.11 User’s Sensory Preferences . 2
2.12 User . 2
2.13 Virtual World . 2
2.14 V R Adaptation . 2
2.15 VW Object Characteristics. 2
3 MPEG-V System Architecture . 2
4 Use cases . 5
4.1 Information adaptation from virtual world to real world . 5
• System Architecture for information adaptation from virtual world to real world . 5
4.2 Information adaptation from real world to virtual world . 6
• System Architecture for information adaptation from real world to virtual world . 6
4.3 Information exchange between virtual worlds . 7
• System Architecture for exchanges between virtual worlds . 7
5 Instantiations . 8
5.1 Instantiation A: Representation of Sensory Effects (RoSE) . 8
• System Architecture for Representation of Sensory Effects . 8
• Instantiation A.1: Multi-sensorial Effects . 9
• Instantiation A.2: Motion effects . 10
5.2 Instantiation B: Natural user interaction with virtual world . 12
• System Architecture for Natural user interaction with virtual world . 12
• Instantiation B.1: Full motion control and navigation of avatar/object with multi-input
sources . 13
• Instantiation B.2: Serious gaming for ambient assisted living. 14
• Instantiation B.3: Gesture recognition using multipoint interaction devices . 15
• Instantiation B.4: Avatar facial expression retargeting using smart camera . 16
• Instantiation B.5: Motion tracking and facial animation with multimodal interaction. 17
• Instantiation B.6: Serious gaming and training with multimodal interaction . 18
• Instantiation B.7: Virtual museum guide with embodied conversational agents . 18
5.3 Instantiation C: Traveling and navigating real and virtual worlds . 19
• System Architecture for traveling and navigating real and virtual worlds . 19
• Instantiation C.1: Virtual travel . 20
• Instantiation C.2: Virtual traces of real places . 20
• Instantiation C.3: Virtual tour guides . 22
• Instantiation C.4: Unmanned aerial vehicle scenario . 23
5.4 Instantiation D: Interoperable virtual worlds . 24
• System Architecture for interoperable virtual worlds . 24
• Instantiation D.1: Avatar appearance . 24
© ISO/IEC 2016 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 23005-1:2016(E)
• Instantiation D.2: Virtual objects . 24
5.5 Instantiation E: Social presence, group decision-making and collaboration within virtual
worlds . 26
• System architecture . 26
• Instantiation E.1: Social presence . 26
• Instantiation E.2: Group decision-making in the context of spatial planning . 27
• Instantiation E.3: Consumer collaboration in product design processes along the supply
chain . 28
5.6 Instantiation F: Interactive haptic sensible media . 30
• System architecture for interactive haptic sensible media . 30
• Instantiation F.1: Internet haptic service - YouTube, online chatting . 30
• Instantiation F.2: Next generation classroom – sensation book . 31
• Instantiation F.3: Immersive broadcasting – home shopping, fishing channels . 32
• Instantiation F.4: Entertainment – game (Second Life, Star Craft), movie theater . 32
• Instantiation F.5: Virtual simulation for training – military task, medical simulations . 33
5.7 Instantiation G: Bio-sensed information in virtual world . 33
• System architecture for bio-sensed information in virtual world . 33
• Instantiation G.1: Interactive games sensitive to user’s conditions . 34
• Instantiation G.2: Virtual hospital and health monitoring . 34
• Instantiation G.3: Mental health for lifestyle management . 35
• Instantiation G.4: Food intake for lifestyle management . 36
• Instantiation G.5: Cardiovascular rehabilitation for health management . 37
• Instantiation G.6: Glucose level / diabetes management for health management . 38
5.8 Instantiation H: Environmental monitoring with sensors. 38
• System architecture for environmental monitoring . 38
• Instantiation H.1: Environmental monitoring system . 39
5.9 Instantiation I: Virtual world interfacing with TV platforms . 40
• System architecture for virtual world interfacing with TV platform . 40
• Instantiation I.1: The TV platform as a virtual worlds I/O device . 41
5.10 Instantiation J: Seamless integration between real and virtual worlds . 42
• System architecture for seamless integration between real and virtual worlds . 42
• Instantiation J.1: Seamless interaction between real and virtual worlds with integrating
virtual and real sensors and actuators . 42
5.11 Instantiation K: Hybrid communication . 44
5.12 Instantiation L: Makeup Avatar . 47
• Spectrum data acquisition . 47
• Spectrum data combination in a virtual world . 48
• Cosmetic color spectrum metamerism . 49
• Color reproduction process for a virtual makeup avatar . 49
• Transformation model generation . 50
• Makeup simulation usage example. 51
5.13 Instantiation M: Usage Scenario for automobile sensors . 53
• Helping auto maintenance/regular inspection . 53
• Monitoring for Eco-friendly driving . 53
Bibliography . 55
iv © ISO/IEC 2016 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 23005-1:2016(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the different types of
document should be noted. This document was drafted in accordance with the editorial rules of the
ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights. Details of any
patent rights identified during the development of the document will be in the Introduction and/or on the ISO
list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity assessment,
as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the
Technical Barriers to Trade (TBT) see the following URL: www.iso.org/iso/foreword.html.
The committee responsible for this document is ISO/IEC JTC 1, Information technology, SC 29, Coding of
audio, picture, multimedia and hypermedia information.
This third edition cancels and replaces the second edition (ISO/IEC 23005-1:2014), which has been technically
revised.
ISO/IEC 23005 consists of the following parts, under the general title Information technology — Media context
and control:
Part 1: Architecture
Part 2: Control information
Part 3: Sensory information
Part 4: Virtual world object characteristics
Part 5: Data formats for interaction devices
Part 6: Common types and tools
Part 7: Conformance and reference software
© ISO/IEC 2016 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC 23005-1:2016(E)
Introduction
The usage of multimedia content is becoming omnipresent in everyday life, in terms of both consumption and
production. On the one hand, professional content is provided to the end user in high-definition quality,
streamed over heterogeneous networks, and consumed on a variety of different devices. On the other hand,
user-generated content overwhelms the Internet with multimedia assets being uploaded to a wide range of
available Web sites. That is, the transparent access to multimedia content, also referred to as Universal
Multimedia Access (UMA), seems to be technically feasible. However, UMA mainly focuses on the end-user
devices and network connectivity issues, but it is the user who ultimately consumes the content. Hence, the
concept of UMA has been extended to take the user into account, which is generally referred to as Universal
Multimedia Experience (UME).
However, the consumption of multimedia assets can also stimulate senses other than vision or audition, e.g.,
olfaction, mechanoreception, equilibrioception, or thermoception. That is, in addition to the audio-visual
content of, for example, a movie, other senses shall also be stimulated giving the user the sensation of being
part of the particular media which shall result in a worthwhile, informative user experience.
This motivates the annotation of the media resources with metadata as defined in this part of ISO/IEC 23005
that steers appropriate devices capable of stimulating these other senses.
ISO/IEC 23005 (MPEG-V) provides an architecture and specifies associated information representations to
enable the interoperability between virtual worlds, for example, digital content provider of a virtual world,
(serious) gaming, simulation, DVD, and with the real world, for example, sensors, actuators, vision and
rendering, robotics (e.g. for revalidation), (support for) independent living, social and welfare systems, banking,
insurance, travel, real estate, rights management and many others.
1)
Virtual worlds (often referred to as 3D3C for 3D visualization & navigation and the 3C's of community,
creation and commerce) integrate existing and emerging (media) technologies (e.g. instant messaging, video,
3D, VR, AI, chat, voice, etc.) that allow for the support of existing and the development of new kinds of social
networks. The emergence of virtual worlds as platforms for social networking is recognized by businesses as
an important issue for at least two reasons:
a) it offers the power to reshape the way companies interact with their environments (markets,
customers, suppliers, creators, stakeholders, etc.) in a fashion comparable to the Internet;
b) it allows for the development of new (breakthrough) business models, services, applications and
devices.
Each virtual world however has a different culture and audience making use of these specific worlds for a
variety of reasons. These differences in existing metaverses permit users to have unique experiences.
Resistance to real-world commercial encroachment still exists in many virtual worlds where users primarily
seek an escape from real life. Hence, marketers should get to know a virtual world beforehand and the rules
that govern each individual universe.
Although realistic experiences have been achieved via devices such as 3-D audio/visual devices, it is hard to
realize sensory effects only with presentation of audiovisual contents. The addition of sensory effects leads to
even more realistic experiences in the consumption of audiovisual contents. This will lead to the application of
new media for enhanced experiences of users in a more realistic sense.
Such new media will benefit from the standardization of a control and sensory information which can include
sensory effect metadata, sensory device (actuator) capabilities/commands, user’s sensory preferences, and
1) Some examples of virtual worlds are: Second Life (http://secondlife.com/), IMVU (http://www.imvu.com/) and Entropia
Universe (http://www.entropiauniverse.com/).
vi © ISO/IEC 2016 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC 23005-1:2016(E)
various delivery formats. The MPEG-V architecture can be applicable for various business models for which
audiovisual contents can be associated with sensory effects that need to be rendered on appropriate sensory
devices (actuators).
Multi-user online virtual worlds, sometimes called Networked Virtual Environments (NVEs) or massively-
multiplayer online games (MMOGs), have reached mainstream popularity. Although most publications tend to
focus on well-known virtual worlds like World of Warcraft, Second Life, and Lineage, there are hundreds of
popular virtual worlds in active use worldwide, most of which are not known to the general public. These can
be quite different from the above-mentioned titles. To understand current trends and developments, it is useful
to keep in mind that there is large variety in virtual worlds and that they are not all variations on Second Life.
The concept of online virtual worlds started in the late 70s with the creation of the text-based Dungeons &
Dragons world MUD. In the eighties, larger-scale graphical virtual worlds followed, and in the late nineties the
first 3D virtual worlds appeared. Many virtual worlds are not considered games (MMOGs) since there is no
clear objective and/or there are no points to score or levels to achieve. In this report we will use “virtual
worlds” as an umbrella term that includes all possible varieties. See the literature for further discussion of the
distinction between gaming/non-gaming worlds. Often, a virtual world which is not considered to be an MMOG
does contain a wide selection of mini-games or quests, in some way embedded into the world. In this manner
a virtual world acts like a combined graphical portal offering games, commerce, social interactions and other
forms of entertainment. Another way to see the difference: games contain mostly pre-authored stories; in
virtual worlds the users more or less create the stories themselves. The current trend in virtual worlds is to
provide a mix of pre-authored and user-generated stories and content, leading to user-modified content.
Current virtual worlds are graphical and rendered in 2D, 2.5 D (isometric view) or 3D, depending on the
intended effect and technical capabilities of the platform: web-browser, gaming PC, average PC, game
console, mobile phone, and so on.
“Would it not be great if the real world economy could be boosted by the exponential growing economy of the
virtual worlds by connecting the virtual - and real world”; in 2007 the Virtual Economy in Second Life alone
was around 400 MEuro, a factor nine growth from 2006. The connected devices and services in the real world
can represent an economy of a multiple of this virtual world economy.
Virtual worlds have entered our lives, our communication patterns, our culture, and our entertainment never to
leave again. It's not only the teenager active in Second Life and World of Warcraft, the average age of a
gamer is 35 years by now, and it increases every year. This does not even include role-play in the
professional context, also known as serious gaming, inevitable when learning practical skills. Virtual worlds
are in use for entertainment, education, training, obtaining information, social interaction, work, virtual tourism,
reliving the past and forms of art. They augment and interact with our real world and form an important part of
people's lives. Many virtual worlds already exist as games, training systems, social networks and virtual cities
and world models. Virtual worlds will change every aspect of our lives: the way we work, interact, play, travel
and learn. Games will be everywhere and their societal need is very big and will lead to many new products
and require many companies.
Technology improvement, both in hardware and software, forms the basis of this. It is envisaged that the most
important developments will occur in the areas of display technology, graphics, animation, (physical)
simulation, behavior and artificial intelligence, loosely distributed systems and network technology.
The figures in this part of ISO/IEC 23005 have been reproduced here with the permission of Samsung, Sharp
Electronics, ETRI, University of Klagenfurt, Institute of Science and Technology, Myongji University, Institut
national des télécommunications and the partners of the ITEA2 project Metaverse1: Philips, Forthnet S.A.,
Alcatel-Lucent Bell N.V., Innovalia, Alcatel-Lucent France, Technicolor, Orange Labs, DevLab, CBT, Nextel,
Carsa, Avantalia, Ceesa, Virtualware, I&IMS, VicomTECH, E-PYME, CIC Tour Gune, Artefacto, Metaverse
Labs, Technical University Eindhoven, Utrecht University, University of Twente, Stg. EPN, VU Economics &
BA, VU CAMeRA, Ellinogermaniki Agogi, IBBT-SMIT, UPF-MTG, CEA List and Loria/Inria Lorraine.
© ISO/IEC 2016 – All rights reserved vii
---------------------- Page: 7 ----------------------
INTERNATIONAL STANDARD ISO/IEC 23005-1:2016(E)
Information technology — Media context and control — Part 1:
Architecture
Part 1:
Architecture
1 Scope
This part of ISO/IEC 23005 specifies the architecture of MPEG-V (media context and control), its three
associated use cases of information adaptation from virtual world to real world, information adaptation from
real world to virtual world, and Information exchange between virtual worlds.
2 Terms and definitions
2.1 Device Command
description of controlling actuators used to generate Sensory Effects
2.2 R V Adaptation
procedure that processes the Sensed Information from the real world in order to be consumed within the
virtual world’s context; takes the Sensed Information with/without the Sensor Capabilities from Sensors, the
Sensor Adaptation Preferences from Users, and/or the Virtual World Object Characteristics from a Virtual
world; controls the Virtual World Object Characteristics or adapts the Sensed Information by adapting the
Sensed Information based on the Sensor Capabilities and/or the Sensor Adaptation Preferences
2.3 Sensed Information
information acquired by sensors
2.4 Sensor
device by which user input or environmental information can be gathered
EXAMPLES Temperature sensor, distance sensor, motion sensor, etc.
2.5 Sensor Adaptation Preferences
description schemes and descriptors to represent (user’s) preferences with respect to adapting sensed
information
2.6 Sensor Capability
description of representing the characteristics of sensors in terms of the capability of the given sensor such as
accuracy, or sensing range
© ISO/IEC 2016 – All rights reserved 1
---------------------- Page: 8 ----------------------
ISO/IEC 23005-1:2016(E)
2.7 Sensory Device
consumer device by which the corresponding sensory effect can be made
NOTE Real world devices can contain any combination of sensors and actuators in one device.
2.8 Sensory Device Capability
description of representing the characteristics of actuators used to generate sensory effects in terms of the
capability of the given device
2.9 Sensory Effects
effects to augment perception by stimulating human senses in a particular scene
EXAMPLES Scent, wind, light, haptic [kinesthetic-force, stiffness, weight, friction, texture, widget (button,
slider, joystick, etc.), tactile: air-jet, suction pressure, thermal, current, vibration, etc. Note that combinations of
tactile display can also provide directional, shape information].
2.10 Sensory Effect Metadata
metadata that defines the description schemes and descriptors to represent sensory effects
2.11 User’s Sensory Preferences
description schemes and descriptors to represent (user’s) preferences with respect to rendering of sensory
effect
2.12 User
the end user of the system.
2.13 Virtual World
digital content, real time or non real time, of various nature ranging from an on-line virtual world, simulation
environment, multi-user game, a broadcasted multimedia production, a peer-to-peer multimedia production or
packaged content like a DVD or game
2.14 V R Adaptation
procedure that processes the Sensory Effects from the Virtual World in order to be consumed within the real
world’s context; takes Sensory Effect Metadata from a Virtual World, Sensory Device (Actuator) Capabilities
from the Sensory Devices (Actuators), the User’s Sensory Preferences from users, and/or the Sensed
Information as well as the Sensor Capabilities from Sensors as inputs; generates the Device Commands by
adapting the Sensory Effects based on the Sensed Information, the Capabilities and/or the Preferences
2.15 VW Object Characteristics
description schemes and descriptors to represent and describe virtual world objects (from the real world into
the virtual world and vice versa)
3 MPEG-V System Architecture
Figure 1 — A strong connection (defined by an architecture that provides interoperability trough
standardization) between the virtual and the real world is needed to reach simultaneous reactions in both
worlds to changes in the environment and human behavior. Efficient, effective, intuitive and entertaining
2 © ISO/IEC 2016 – All rights reserved
-------------
...
FINAL
INTERNATIONAL ISO/IEC
DRAFT
STANDARD FDIS
23005-1
ISO/IEC JTC 1
Information technology — Media context
Secretariat: JISC
and control —
Voting begins on:
2016-04-04
Part 1:
Architecture
Voting terminates on:
2016-06-04
Technologies de l'information — Contrôle et contexte de supports —
Partie 1: Architecture
RECIPIENTS OF THIS DRAFT ARE INVITED TO
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
THEY ARE AWARE AND TO PROVIDE SUPPORT-
ING DOCUMENTATION.
IN ADDITION TO THEIR EVALUATION AS
Reference number
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO-
ISO/IEC FDIS 23005-1:2016(E)
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN-
DARDS TO WHICH REFERENCE MAY BE MADE IN
©
ISO/IEC 2016
NATIONAL REGULATIONS.
---------------------- Page: 1 ----------------------
ISO/IEC FDIS 23005-1:2016(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2016
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form or by any
means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior written permission.
Permission can be requested from either ISO at the address below or ISO’s member body in the country of the requester.
ISO copyright office
Case postale 56 CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2016 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC FDIS 23005-1:2016(E)
Contents Page
1 Scope . 1
2 Terms and definitions . 1
2.1 Device Command . 1
2.2 R V Adaptation . 1
2.3 Sensed Information . 1
2.4 Sensor. 1
2.5 Sensor Adaptation Preferences . 1
2.6 Sensor Capability . 1
2.7 Sensory Device . 2
2.8 Sensory Device Capability . 2
2.9 Sensory Effects . 2
2.10 Sensory Effect Metadata . 2
2.11 User’s Sensory Preferences . 2
2.12 User . 2
2.13 Virtual World . 2
2.14 V R Adaptation . 2
2.15 VW Object Characteristics . 2
3 MPEG-V System Architecture . 2
4 Use cases . 5
4.1 Information adaptation from virtual world to real world . 5
• System Architecture for information adaptation from virtual world to real world . 5
4.2 Information adaptation from real world to virtual world . 6
• System Architecture for information adaptation from real world to virtual world . 6
4.3 Information exchange between virtual worlds . 7
• System Architecture for exchanges between virtual worlds . 7
5 Instantiations . 8
5.1 Instantiation A: Representation of Sensory Effects (RoSE) . 8
• System Architecture for Representation of Sensory Effects. 8
• Instantiation A.1: Multi-sensorial Effects . 9
• Instantiation A.2: Motion effects . 10
5.2 Instantiation B: Natural user interaction with virtual world . 12
• System Architecture for Natural user interaction with virtual world . 12
• Instantiation B.1: Full motion control and navigation of avatar/object with multi-input
sources . 13
• Instantiation B.2: Serious gaming for ambient assisted living . 14
• Instantiation B.3: Gesture recognition using multipoint interaction devices . 15
• Instantiation B.4: Avatar facial expression retargeting using smart camera . 16
• Instantiation B.5: Motion tracking and facial animation with multimodal interaction . 17
• Instantiation B.6: Serious gaming and training with multimodal interaction . 18
• Instantiation B.7: Virtual museum guide with embodied conversational agents . 18
5.3 Instantiation C: Traveling and navigating real and virtual worlds . 19
• System Architecture for traveling and navigating real and virtual worlds . 19
• Instantiation C.1: Virtual travel . 20
• Instantiation C.2: Virtual traces of real places . 20
• Instantiation C.3: Virtual tour guides . 22
• Instantiation C.4: Unmanned aerial vehicle scenario . 23
5.4 Instantiation D: Interoperable virtual worlds . 24
• System Architecture for interoperable virtual worlds . 24
• Instantiation D.1: Avatar appearance . 24
© ISO/IEC 2016 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC FDIS 23005-1:2016(E)
• Instantiation D.2: Virtual objects .24
5.5 Instantiation E: Social presence, group decision-making and collaboration within virtual
worlds .26
• System architecture .26
• Instantiation E.1: Social presence .26
• Instantiation E.2: Group decision-making in the context of spatial planning .27
• Instantiation E.3: Consumer collaboration in product design processes along the supply
chain .28
5.6 Instantiation F: Interactive haptic sensible media .30
• System architecture for interactive haptic sensible media .30
• Instantiation F.1: Internet haptic service - YouTube, online chatting .30
• Instantiation F.2: Next generation classroom – sensation book .31
• Instantiation F.3: Immersive broadcasting – home shopping, fishing channels .32
• Instantiation F.4: Entertainment – game (Second Life, Star Craft), movie theater .32
• Instantiation F.5: Virtual simulation for training – military task, medical simulations .33
5.7 Instantiation G: Bio-sensed information in virtual world .33
• System architecture for bio-sensed information in virtual world .33
• Instantiation G.1: Interactive games sensitive to user’s conditions .34
• Instantiation G.2: Virtual hospital and health monitoring .34
• Instantiation G.3: Mental health for lifestyle management .35
• Instantiation G.4: Food intake for lifestyle management .36
• Instantiation G.5: Cardiovascular rehabilitation for health management .37
• Instantiation G.6: Glucose level / diabetes management for health management .38
5.8 Instantiation H: Environmental monitoring with sensors .38
• System architecture for environmental monitoring .38
• Instantiation H.1: Environmental monitoring system .39
5.9 Instantiation I: Virtual world interfacing with TV platforms .40
• System architecture for virtual world interfacing with TV platform .40
• Instantiation I.1: The TV platform as a virtual worlds I/O device .41
5.10 Instantiation J: Seamless integration between real and virtual worlds .42
• System architecture for seamless integration between real and virtual worlds .42
• Instantiation J.1: Seamless interaction between real and virtual worlds with integrating
virtual and real sensors and actuators .42
5.11 Instantiation K: Hybrid communication .44
5.12 Instantiation L: Makeup Avatar .47
• Spectrum data acquisition .47
• Spectrum data combination in a virtual world .48
• Cosmetic color spectrum metamerism .49
• Color reproduction process for a virtual makeup avatar .49
• Transformation model generation .50
• Makeup simulation usage example.51
5.13 Instantiation M: Usage Scenario for automobile sensors .53
• Helping auto maintenance/regular inspection .53
• Monitoring for Eco-friendly driving.53
Bibliography .55
iv © ISO/IEC 2016 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC FDIS 23005-1:2016(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 23005-1 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information
This third edition cancels and replaces the second edition (ISO/IEC 23005-1:2014), which has been
technically revised.
ISO/IEC 23005 consists of the following parts, under the general title Information technology — Media context
and control:
— Part 1: Architecture
— Part 2: Control information
— Part 3: Sensory information
— Part 4: Virtual world object characteristics
— Part 5: Data formats for interaction devices
— Part 6: Common types and tools
— Part 7: Conformance and reference software
v
© ISO/IEC 2016 – All rights reservedV
---------------------- Page: 5 ----------------------
ISO/IEC FDIS 23005-1:2016(E)
Introduction
The usage of multimedia content is becoming omnipresent in everyday life, in terms of both consumption and
production. On the one hand, professional content is provided to the end user in high-definition quality,
streamed over heterogeneous networks, and consumed on a variety of different devices. On the other hand,
user-generated content overwhelms the Internet with multimedia assets being uploaded to a wide range of
available Web sites. That is, the transparent access to multimedia content, also referred to as Universal
Multimedia Access (UMA), seems to be technically feasible. However, UMA mainly focuses on the end-user
devices and network connectivity issues, but it is the user who ultimately consumes the content. Hence, the
concept of UMA has been extended to take the user into account, which is generally referred to as Universal
Multimedia Experience (UME).
However, the consumption of multimedia assets can also stimulate senses other than vision or audition, e.g.,
olfaction, mechanoreception, equilibrioception, or thermoception. That is, in addition to the audio-visual
content of, for example, a movie, other senses shall also be stimulated giving the user the sensation of being
part of the particular media which shall result in a worthwhile, informative user experience.
This motivates the annotation of the media resources with metadata as defined in this part of ISO/IEC 23005
that steers appropriate devices capable of stimulating these other senses.
ISO/IEC 23005 (MPEG-V) provides an architecture and specifies associated information representations to
enable the interoperability between virtual worlds, for example, digital content provider of a virtual world,
(serious) gaming, simulation, DVD, and with the real world, for example, sensors, actuators, vision and
rendering, robotics (e.g. for revalidation), (support for) independent living, social and welfare systems, banking,
insurance, travel, real estate, rights management and many others.
1)
Virtual worlds (often referred to as 3D3C for 3D visualization & navigation and the 3C's of community,
creation and commerce) integrate existing and emerging (media) technologies (e.g. instant messaging, video,
3D, VR, AI, chat, voice, etc.) that allow for the support of existing and the development of new kinds of social
networks. The emergence of virtual worlds as platforms for social networking is recognized by businesses as
an important issue for at least two reasons:
a) it offers the power to reshape the way companies interact with their environments (markets,
customers, suppliers, creators, stakeholders, etc.) in a fashion comparable to the Internet;
b) it allows for the development of new (breakthrough) business models, services, applications and
devices.
Each virtual world however has a different culture and audience making use of these specific worlds for a
variety of reasons. These differences in existing metaverses permit users to have unique experiences.
Resistance to real-world commercial encroachment still exists in many virtual worlds where users primarily
seek an escape from real life. Hence, marketers should get to know a virtual world beforehand and the rules
that govern each individual universe.
Although realistic experiences have been achieved via devices such as 3-D audio/visual devices, it is hard to
realize sensory effects only with presentation of audiovisual contents. The addition of sensory effects leads to
even more realistic experiences in the consumption of audiovisual contents. This will lead to the application of
new media for enhanced experiences of users in a more realistic sense.
Such new media will benefit from the standardization of a control and sensory information which can include
sensory effect metadata, sensory device (actuator) capabilities/commands, user’s sensory preferences, and
1) Some examples of virtual worlds are: Second Life (http://secondlife.com/), IMVU (http://www.imvu.com/) and Entropia
Universe (http://www.entropiauniverse.com/).
vi © ISO/IEC 2016 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC FDIS 23005-1:2016(E)
various delivery formats. The MPEG-V architecture can be applicable for various business models for which
audiovisual contents can be associated with sensory effects that need to be rendered on appropriate sensory
devices (actuators).
Multi-user online virtual worlds, sometimes called Networked Virtual Environments (NVEs) or massively-
multiplayer online games (MMOGs), have reached mainstream popularity. Although most publications tend to
focus on well-known virtual worlds like World of Warcraft, Second Life, and Lineage, there are hundreds of
popular virtual worlds in active use worldwide, most of which are not known to the general public. These can
be quite different from the above-mentioned titles. To understand current trends and developments, it is useful
to keep in mind that there is large variety in virtual worlds and that they are not all variations on Second Life.
The concept of online virtual worlds started in the late 70s with the creation of the text-based Dungeons &
Dragons world MUD. In the eighties, larger-scale graphical virtual worlds followed, and in the late nineties the
first 3D virtual worlds appeared. Many virtual worlds are not considered games (MMOGs) since there is no
clear objective and/or there are no points to score or levels to achieve. In this report we will use “virtual
worlds” as an umbrella term that includes all possible varieties. See the literature for further discussion of the
distinction between gaming/non-gaming worlds. Often, a virtual world which is not considered to be an MMOG
does contain a wide selection of mini-games or quests, in some way embedded into the world. In this manner
a virtual world acts like a combined graphical portal offering games, commerce, social interactions and other
forms of entertainment. Another way to see the difference: games contain mostly pre-authored stories; in
virtual worlds the users more or less create the stories themselves. The current trend in virtual worlds is to
provide a mix of pre-authored and user-generated stories and content, leading to user-modified content.
Current virtual worlds are graphical and rendered in 2D, 2.5 D (isometric view) or 3D, depending on the
intended effect and technical capabilities of the platform: web-browser, gaming PC, average PC, game
console, mobile phone, and so on.
“Would it not be great if the real world economy could be boosted by the exponential growing economy of the
virtual worlds by connecting the virtual - and real world”; in 2007 the Virtual Economy in Second Life alone
was around 400 MEuro, a factor nine growth from 2006. The connected devices and services in the real world
can represent an economy of a multiple of this virtual world economy.
Virtual worlds have entered our lives, our communication patterns, our culture, and our entertainment never to
leave again. It's not only the teenager active in Second Life and World of Warcraft, the average age of a
gamer is 35 years by now, and it increases every year. This does not even include role-play in the
professional context, also known as serious gaming, inevitable when learning practical skills. Virtual worlds
are in use for entertainment, education, training, obtaining information, social interaction, work, virtual tourism,
reliving the past and forms of art. They augment and interact with our real world and form an important part of
people's lives. Many virtual worlds already exist as games, training systems, social networks and virtual cities
and world models. Virtual worlds will change every aspect of our lives: the way we work, interact, play, travel
and learn. Games will be everywhere and their societal need is very big and will lead to many new products
and require many companies.
Technology improvement, both in hardware and software, forms the basis of this. It is envisaged that the most
important developments will occur in the areas of display technology, graphics, animation, (physical)
simulation, behavior and artificial intelligence, loosely distributed systems and network technology.
The figures in this part of ISO/IEC 23005 have been reproduced here with the permission of Samsung, Sharp
Electronics, ETRI, University of Klagenfurt, Institute of Science and Technology, Myongji University, Institut
national des télécommunications and the partners of the ITEA2 project Metaverse1: Philips, Forthnet S.A.,
Alcatel-Lucent Bell N.V., Innovalia, Alcatel-Lucent France, Technicolor, Orange Labs, DevLab, CBT, Nextel,
Carsa, Avantalia, Ceesa, Virtualware, I&IMS, VicomTECH, E-PYME, CIC Tour Gune, Artefacto, Metaverse
Labs, Technical University Eindhoven, Utrecht University, University of Twente, Stg. EPN, VU Economics &
BA, VU CAMeRA, Ellinogermaniki Agogi, IBBT-SMIT, UPF-MTG, CEA List and Loria/Inria Lorraine.
© ISO/IEC 2016 – All rights reserved vii
---------------------- Page: 7 ----------------------
FINAL DRAFT INTERNATIONAL STANDARD ISO/IEC FDIS 23005-1:2016(E)
Information technology — Media context and control — Part 1:
Architecture
Part 1:
Architecture
1 Scope
This part of ISO/IEC 23005 specifies the architecture of MPEG-V (media context and control), its
three associated use cases of information adaptation from virtual world to real world, information
adaptation from real world to virtual world, and Information exchange between virtual worlds.
2 Terms and definitions
2.1 Device Command
description of controlling actuators used to generate Sensory Effects.
2.2 R V Adaptation
procedure that processes the Sensed Information from the real world in order to be consumed within
the virtual world’s context; takes the Sensed Information with/without the Sensor Capabilities from
Sensors, the Sensor Adaptation Preferences from Users, and/or the Virtual World Object Characteristics
from a Virtual world; controls the Virtual World Object Characteristics or adapts the Sensed Information
by adapting the Sensed Information based on the Sensor Capabilities and/or the Sensor Adaptation
Preferences.
2.3 Sensed Information
information acquired by sensors.
2.4 Sensor
device by which user input or environmental information can be gathered
EXAMPLES Temperature sensor, distance sensor, motion sensor, etc.
2.5 Sensor Adaptation Preferences
description schemes and descriptors to represent (user’s) preferences with respect to adapting
sensed information.
2.6 Sensor Capability
description of representing the characteristics of sensors in terms of the capability of the given sensor such
as accuracy, or sensing range.
© ISO/IEC 2016 – All rights reserved 1
---------------------- Page: 8 ----------------------
ISO/IEC FDIS 23005-1:2016(E)
2.7 Sensory Device
consumer device by which the corresponding sensory effect can be made
NOTE Real world devices can contain any combination of sensors and actuators in one device.
2.8 Sensory Device Capability
description of representing the characteristics of actuators used to generate sensory effects in terms of
the capability of the given device
2.9 Sensory Effects
effects to augment perception by stimulating human senses in a particular scene.
EXAMPLES Scent, wind, light, haptic [kinesthetic-force, stiffness, weight, friction, texture, widget
(button, slider, joystick, etc.), tactile: air-jet, suction pressure, thermal, current, vibration, etc. Note that
combinations of tactile display can also provide directional, shape information].
2.10 Sensory Effect Metadata
metadata that defines the description schemes and descriptors to represent sensory effects
2.11 User’s Sensory Preferences
description schemes and descriptors to represent (user’s) preferences with respect to rendering of
sensory effect.
2.12 User
the end user of the system.
2.13 Virtual World
digital content, real time or non real time, of various nature ranging from an on-line virtual world,
simulation environment, multi-user game, a broadcasted multimedia production, a peer-to-peer multimedia
production or packaged content like a DVD or game.
2.14 V R Adaptation
procedure that processes the Sensory Effects from the Virtual World in order to be consumed within the
real world’s context; takes Sensory Effect Metadata from a Virtual World, Sensory Device (Actuator)
Capabilities from the Sensory Devices (Actuators), the User’s Sensory Preferences from users, and/or
the Sensed Information as well as the Sensor Capabilities from Sensors as inputs; generates the Device
Commands by adapting the Sensory Effects based on the Sensed Information, the Capabilities and/or the
Preferences.
2.15 VW Object Characteristics
description schemes and descriptors to represent and describe virtual world objects (from the real world
into the virtual world and vice verse.)
3 MPEG-V System Architecture
Figure 1 — A strong connection (defined by an architecture that provides interoperability
trough standardization) between the virtual and the real world is needed to reach simultaneous
reactions in both worlds to changes in the environment and human behavior. Efficient, effective,
intuitive and entertaining
2 © ISO/IEC 2016 – All rights reserve
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.