ISO/IEC 23005-3:2019

Information technology - Media context and control - Part 3: Sensory information

The technologies specified in this document are description languages and vocabularies which describe sensorial effects. The adaptation engine is not within the scope of this document (or the ISO/IEC 23005 series). This document specifies syntax and semantics of the tools describing sensory information to enrich audio-visual contents: - Sensory Effect Description Language (SEDL) as an XML schema-based language which enables one to describe a basic structure of sensory information; - Sensory Effect Vocabulary (SEV), an XML representation for describing sensorial effects such as light, wind, fog, vibration, etc. that trigger human senses.

Technologies de l'information — Contrôle et contexte de supports — Partie 3: Information sensorielle

General Information

• Status: Published
• Publication Date: 06-Aug-2019
• ICS: 35.040 - Information coding
• ICS: 35.040.40 - Coding of audio, video, multimedia and hypermedia information
• Technical Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Drafting Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Current Stage: 9060 - Close of review
• Completion Date: 04-Mar-2030

Relations

• Revises: ISO/IEC 23005-3:2016 - Information technology - Media context and control - Part 3: Sensory information
• Effective Date: 02-Apr-2016

Overview

ISO/IEC 23005-3:2019 - Part 3 of the ISO/IEC 23005 series - defines standardized tools to describe sensorial effects that enrich audio‑visual content. It specifies two complementary XML-based technologies:

• Sensory Effect Description Language (SEDL) - an XML schema-based language for describing the basic structure and metadata of sensory information.
• Sensory Effect Vocabulary (SEV) - an XML representation of concrete sensorial effects (light, wind, fog, vibration, temperature, scent, spraying, color correction, rigid body motion, etc.) that can trigger human senses.

The standard covers syntax and semantics, validation rules, binary representation mnemonics and examples. The adaptation engine (the runtime that maps descriptions to devices) is explicitly out of scope.

Key technical topics and requirements

• XML schema definitions: Schema wrappers and base datatypes used to author sensory descriptions.
• Structure and metadata: Root element, description metadata, declarations, parameters, and grouping mechanisms (group of effects, reference effects).
• Syntax and semantics: Formal rules for how each sensory effect (light, flash, wind, vibration, fog, scent, temperature, spraying, color correction, rigid-body motion, etc.) is represented and interpreted.
• Validation: XML validation rules plus additional semantic validation to ensure descriptions are consistent and portable.
• Binary representations: Mnemonics and common header definitions for compact binary encodings where needed.
• Examples and processing guidance: Sample XML and binary examples to support implementation and testing.

Practical applications and users

ISO/IEC 23005-3 is designed for anyone creating or integrating multisensory media systems, including:

• Content creators and producers - to author sensory metadata that accompanies audio/video.
• Interactive-media and VR/AR developers - to deliver synchronized haptic, environmental and sensory cues.
• Device and actuator manufacturers - to implement parsers and execution engines that map standard sensory descriptions to hardware (lights, fans, scent dispensers, vibration motors, fog machines, heaters/coolers).
• System integrators and theme‑park designers - for synchronized multisensory experiences across heterogeneous equipment.
• Accessibility and assistive-technology teams - to enrich media with non-visual sensory cues.

Related standards

• Part of the ISO/IEC 23005 series (media context and control). Note: runtime adaptation engines and device mapping are addressed outside Part 3.

Keywords: ISO/IEC 23005-3, Sensory Effect Description Language (SEDL), Sensory Effect Vocabulary (SEV), XML schema, sensory effects, multisensory media, haptic, VR, AR, validation, binary representation, media context and control.

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 23005-3
Fourth edition
2019-08
Information technology — Media
context and control —
Part 3:
Sensory information
Technologies de l'information — Contrôle et contexte de supports —
Partie 3: Information sensorielle
Reference number
©
ISO/IEC 2019
© ISO/IEC 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Fax: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2019 – All rights reserved

Contents Page
Foreword .vi
Introduction .vii
1 Scope . 1
2 Normative references . 1
3 Terms, definitions and abbreviated terms . 1
3.1 Terms and definitions . 1
3.2 Abbreviated terms . 2
3.3 Schema documents . 2
3.4 Use of prefixes . 3
4 Sensory effects description language . 3
4.1 General . 3
4.2 Validation . 4
4.3 Processing . 4
4.4 Basic building blocks . 4
4.4.1 General. 4
4.4.2 Schema wrapper . 5
4.4.3 Mnemonics for binary representations . . 6
4.4.4 Common header for binary representations . 6
4.4.5 Base datatypes and elements. 6
4.4.6 Root element .12
4.4.7 Description metadata .16
4.4.8 Declarations .17
4.4.9 Group of effects .18
4.4.10 Effect .19
4.4.11 Reference effect .23
4.4.12 Parameters .24
4.4.13 Additional validation rules .26
4.4.14 Examples .28
5 Sensory effect vocabulary .31
5.1 General .31
5.2 Validation .31
5.3 Schema wrapper .31
5.4 Light effect .32
5.4.1 General.32
5.4.2 Syntax .32
5.4.3 Binary representation syntax .32
5.4.4 Semantics .33
5.4.5 Additional validation rules .33
5.4.6 Example .
.....................................................................................................................................................................................................33
5.5 Flash effect .34
5.5.1 General.34
5.5.2 Syntax .34
5.5.3 Binary representation syntax .34
5.5.4 Semantics .34
5.5.5 Example .35
5.6 Temperature effect .35
5.6.1 General.35
5.6.2 Syntax .35
5.6.3 Binary representation syntax .35
5.6.4 Semantics .35
5.6.5 Additional validation rules .36
© ISO/IEC 2019 – All rights reserved iii

5.6.6 Example .36
5.7 Wind effect .36
5.7.1 General.36
5.7.2 Syntax .36
5.7.3 Binary representation syntax .36
5.7.4 Semantics .37
5.7.5 Additional validation rules .37
5.7.6 Example .37
5.8 Vibration effect .37
5.8.1 General.37
5.8.2 Syntax .37
5.8.3 Binary representation syntax .38
5.8.4 Semantics .38
5.8.5 Additional validation rules .38
5.8.6 Example .38
5.9 Spraying effect .39
5.9.1 General.39
5.9.2 Syntax .39
5.9.3 Binary representation syntax .39
5.9.4 Semantics .39
5.9.5 Additional validation rules .40
5.9.6 Example .40
5.10 Scent effect.40
5.10.1 General.40
5.10.2 Syntax .40
5.10.3 Binary representation syntax .40
5.10.4 Semantics .41
5.10.5 Additional validation rules .41
5.10.6 Example .41
5.11 Fog effect .42
5.11.1 General.42
5.11.2 Syntax .42
5.11.3 Binary representation syntax .42
5.11.4 Semantics .42
5.11.5 Additional validation rules .42
5.11.6 Example .43
5.12 Color correction effect .43
5.12.1 General.43
5.12.2 Syntax .43
5.12.3 Binary representation syntax .43
5.12.4 Semantics .44
5.12.5 Additional validation rules .44
5.12.6 Example .44
5.13 Rigid body motion effect .46
5.13.1 General.46
5.13.2 Syntax .46
5.13.3 Binary representation syntax .47
5.13.4 Semantics .52
5.13.5 Example .62
5.14 Passive kinesthetic motion effect .64
5.14.1 General.64
5.14.2 Syntax .64
5.14.3 Binary representation syntax .64
5.14.4 Semantics .65
5.14.5 Additional validation rules .65
5.14.6 Example .65
5.15 Passive kinesthetic force effect .65
5.15.1 General.65
iv © ISO/IEC 2019 – All rights reserved

5.15.2 Syntax .65
5.15.3 Binary representation syntax .66
5.15.4 Semantics .66
5.15.5 Additional validation rules .66
5.15.6 Examples .67
5.16 Active kinesthetic effect .67
5.16.1 General.67
5.16.2 Syntax .67
5.16.3 Binary representation syntax .67
5.16.4 Semantics .68
5.16.5 Examples .69
5.17 Tactile effect .69
5.17.1 General.69
5.17.2 Syntax .69
5.17.3 Binary representation syntax .69
5.17.4 Semantics .70
5.17.5 Example .
.....................................................................................................................................................................................................71
5.18 Parameterized Tactile effect .72
5.18.1 General.72
5.18.2 Syntax .72
5.18.3 Binary representation syntax .73
5.18.4 Semantics .75
5.18.5 Additional validation rules .78
5.18.6 Examples .78
5.19 Bubble Effect .79
5.19.1 General.79
5.19.2 Syntax .79
5.19.3 Binary Representation .79
5.19.4 Semantics .80
5.19.5 Examples .80
5.20 Arrayed Light effect .80
5.20.1 General.80
5.20.2 Syntax .80
5.20.3 Binary Representation Syntax .81
5.20.4 Semantics .81
5.20.5 Examples .82
Annex A (informative) Intended usage of sensory information .84
Annex B (informative) Schema documents .85
Bibliography .86
© ISO/IEC 2019 – All rights reserved v

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the General and/or on
the ISO list of patent declarations received (see www .iso .org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www .iso .org/iso/foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
This fourth edition cancels and replaces the third edition (ISO/IEC 23005-3:2016), which has been
technically revised. The main changes compared to the previous edition are as follows:
— Addition of the arrayed light effect.
A list of all parts in the ISO/IEC 23005 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/members .html.
vi © ISO/IEC 2019 – All rights reserved

Introduction
The ISO/IEC 23005 series provides an architecture and specifies information representation of data
flowing in and out of the real world and virtual worlds.
The data for the real world are communicated through sensors and actuators. The data for virtual
worlds consist of properties of virtual objects and multi-sensorial data embedded in audio-visual
content. MPEG-V specifies data formats for sensors, actuators, virtual objects, and audio-visual content.
Data captured from the real world could need to be adapted for use in a virtual world and data from
virtual worlds could also need to be adapted for use in the real world. This document does not specify
how the adaptation is carried out but only specifies the interfaces.
Data for sensors are sensor capabilities, sensed data, and sensor adaptation preferences.
Data for actuators are sensory device capabilities, sensory device commands, and sensory effect
preferences.
Data for virtual objects are characteristics of avatars and virtual world objects.
Sensory effect could be needed to enrich audio-visual contents.
The system architecture is depicted in Figure 1 and the scope of this document is highlighted. That is
only the information representation that acts as an input to the possible Adaptation VR – as defined in
ISO/IEC 23005-1 – is specified in this document.
Figure 1 — Scope of ISO/IEC 23005-3 marked with a yellow box
© ISO/IEC 2019 – All rights reserved vii

NOTE 1 The actual R→V/V→R Adaptation is deliberately informative and left open for industry competition.
NOTE 2 Additional informative information can be found in Annex A.
This document contains the tools of the sensory information which can stimulate other senses than
vision or audition, e.g. olfaction, mechanoreception, equilibrioception, or thermoception. That is, in
addition to the audio-visual content of, for example, a movie, other senses are also stimulated giving the
user the sensation of being part of the particular media and resulting in a worthwhile, informative user
experience. This document also illustrates some non-normative examples.
The International Organization for Standardization (ISO) and the International Electrotechnical
Commission (IEC) draw attention to the fact that it is claimed that compliance with this document may
involve the use of patents.
ISO and the IEC take no position concerning the evidence, validity and scope of these patent rights.
The holders of these patent rights have assured ISO and IEC that they are willing to negotiate licences
under reasonable and non-discriminatory terms and conditions with applicants throughout the world.
In this respect, the statements of the holders of these patents right are registered with ISO and IEC.
Information may be obtained from the companies listed below.
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights other than those identified in this list. ISO and IEC shall not be held responsible for
identifying any or all such patent rights.
ISO (www .iso .org/patents) and IEC (http: //patents .iec .ch) maintain online databases of patents
relevant to their standards. Users are encouraged to consult the databases for the most up to date
information concerning patents.
Company Address
Samsung Electronics Co.Ltd. 416, Maetan-dong, Yeongtong-gu, Suwon-si, Gyeonggi-do,
152-848, Republic of Korea
Gwangju Institute of Science and Technology 261 Cheomdan-gwagino (Oryong-dong), Buk-gu, Gwangju
500-712, Republic of Korea
Electronics and Telecommunications Research 218 Gajeongro, Yuseong-gu, Daejeon, 305-700, Republic of
Institute (ETRI) Korea
Konkuk University Department of Textile Engineering, 1 Hwayang-dong,
Kwangjin-gu, Seoul 143-701, Republic of Korea
Myongji University 116 Myongji-ro, Cheoin-gu, Yongin, 449-728, Republic of
Korea
viii © ISO/IEC 2019 – All rights reserved

INTERNATIONAL STANDARD ISO/IEC 23005-3:2019(E)
Information technology — Media context and control —
Part 3:
Sensory information
1 Scope
The technologies specified in this document are description languages and vocabularies which describe
sensorial effects.
The adaptation engine is not within the scope of this document (or the ISO/IEC 23005 series).
This document specifies syntax and semantics of the tools describing sensory information to enrich
audio-visual contents:
— Sensory Effect Description Language (SEDL) as an XML schema-based language which enables one
to describe a basic structure of sensory information;
— Sensory Effect Vocabulary (SEV), an XML representation for describing sensorial effects such as
light, wind, fog, vibration, etc. that trigger human senses.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 21000-7:2007, Information technology — Multimedia framework (MPEG-21) — Part 7: Digital
Item Adaptation
ISO/IEC 23005-6:2019, Information technology — Media context and control — Part 6: Common types
and tools
3 Terms, definitions and abbreviated terms
3.1 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 23005-6 and the
following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
3.1.1
digital content provider
entity that acts as the source of digital information of various nature
Note 1 to entry: The digital content may be provided in real-time or non real-time.
© ISO/IEC 2019 – All rights reserved 1

EXAMPLE Digital content from an online virtual world, simulation environment, multi-user game, a
broadcasted multimedia production, a peer-to-peer multimedia production, or packaged content like a DVD
or game.
3.1.2
sensory information
standardized representation format of data flowing in and out of the real world and virtual worlds
Note 1 to entry: This representation format is applicable to ISO/IEC 23005-1:2016, Clause 5, area B.
EXAMPLE Sensory effect metadata, haptic (kinesthetic/tactile) information, emotion information, avatar
information.
3.1.3
sensory effect metadata
definition of the description schemes and descriptors to represent sensory effects (3.1.4)
3.1.4
sensory effect
effect to augment perception by stimulating human senses in a particular scene of a multimedia
application
Note 1 to entry: Combinations of tactile display may provide also directional, shape information.
EXAMPLE Scent, wind, light, haptic (kinesthetic-force, stiffness, weight, friction, texture, widget, button,
slider, joystick), tactile: air-jet, suction pressure, thermal, current, vibration.
3.1.5
adaptation VR
entity that can process the sensory information (3.1.2) in order to be consumed within the real
world’s context
Note 1 to entry: This may include the adaptation or transformation of the sensory information according to
the capabilities of real-world devices or the preferences of the user. A specification of these capabilities and
preferences can be found in ISO/IEC 23005-2.
3.2 Abbreviated terms
CT common types
DIA digital item adaptation (ISO/IEC 21000-7)
SEDL sensory effects description language
SEM sensory effect metadata
SEV sensory effects vocabulary
UMA universal multimedia access
UME universal multimedia experience
XML extensible mark-up language
XSI XML streaming instructions
3.3 Schema documents
In the main text of this document, the syntax of description schemes and descriptors is provided
whenever possible as a single schema document.
2 © ISO/IEC 2019 – All rights reserved

In some cases though, and in particular for Clause 5, the syntax of description schemes and descriptors
is provided as a collection of schema snippets imbricated with other text. In order to form a valid
schema document, these schema components should be gathered in a single document with the schema
wrapper provided at the head of the clause. For better readability, the relevant schema documents are
provided in Annex B.
In all cases, each schema document has a version attribute, the value of which is "ISO/IEC 23005-3".
Furthermore, an informative identifier is given as the value of the id attribute of the schema component.
This identifier is non-normative and used as a convention in this document to reference another schema
document. In particular, it is used for the schemaLocation attribute of the include and import schema
components.
3.4 Use of prefixes
For clarity, throughout this document, consistent namespace prefixes are used.
"xsi:" prefix is not normative. It is a naming convention in this document to refer to an element of the
http://www.w3.org/2001/XMLSchema-instance namespace.
"xml:" and "xmlns:" are normative prefixes defined in Reference [1]. The prefix “xml:” is by definition
bound to "http://www.w3.org/XML/1998/namespace". The prefix “xmlns:” is used only for namespace
bindings and is not itself bound to any namespace name.
All other prefixes used in either the text or examples of this document are not normative, e.g., “sedl:”,
“sev:”, “dia:”, “si:”, “mpeg7:”.
In particular, most of the informative examples in this document are provided as XML fragments
without the normally required XML document declaration and, thus, miss a correct namespace binding
context declaration. In these descriptions fragments, the different prefixes are bound to the namespaces
as given in Table 1.
Table 1 — Mapping of prefixes to namespaces in examples and text
Prefix Corresponding namespace
ct urn:mpeg:mpeg-v:2018:01-CT-NS
sedl urn:mpeg:mpeg-v:2018:01-SEDL-NS
sev urn:mpeg:mpeg-v:2018:01-SEV-NS
dia urn:mpeg:mpeg21:2003:01-DIA-NS
si urn:mpeg:mpeg21:2003:01-DIA-XSI-NS
mpeg7 urn:mpeg:mpeg7:schema:2004
xsi http://www.w3.org/2001/XMLSchema-instance
xsd http://www.w3.org/2001/XMLSchema
Unlike the informative descriptions examples, the normative specification of the syntax of tools in XML
schema follows the namespace binding context defined in the relevant schema declaration such as the
one defined in 5.3.
4 Sensory effects description language
4.1 General
This clause specifies the syntax and semantics of the sensory effects description language (SEDL) which
provides basic building blocks for the authoring of sensory effect metadata.
© ISO/IEC 2019 – All rights reserved 3

4.2 Validation
Validating a document against the SEDL schema (as specified in W3C XML Schema) is necessary, but
not sufficient, to determine its validity with respect to SEDL. After a document is validated against the
SEDL schema, it shall also be subjected to additional validation rules. These additional rules are given
below in the descriptions of the elements to which they pertain.
4.3 Processing
The processing model for the sensory effect metadata is defined as an XML processor (as specified by
W3C XML) and the utilization of the elements and attributes as defined in the subsequent (sub)clauses.
NOTE The processing of the sensory effect metadata can follow existing XML decoding/parsing models such
as the Document Object Model (DOM) or the Simple API for XML (SAX).
The time information that can be associated to sensory effects may be used for the synchronization
with respect to other media assets.
EXAMPLE These other media assets can be video and/or audio.
4.4 Basic building blocks
4.4.1 General
This subclause specifies the syntax and semantics of the basic building blocks for authoring sensory
effect metadata.
This document adopts the XML streaming instructions (XSI) as defined in ISO/IEC 21000-7 for the
purpose of identifying process unit
...