ISO/IEC 23090-14:2023
(Main)Information technology — Coded representation of immersive media — Part 14: Scene description
Information technology — Coded representation of immersive media — Part 14: Scene description
This document specifies extensions to existing scene description formats in order to support MPEG media, in particular immersive media. MPEG media includes but is not limited to media encoded with MPEG codecs, media stored in MPEG containers, MPEG media and application formats as well as media provided through MPEG delivery mechanisms. Extensions include scene description format syntax and semantics and the processing model when using these extensions by a Presentation Engine. It also defines a Media Access Function (MAF) API for communication between the Presentation Engine and the Media Access Function for these extensions. While the extensions defined in this document can be applicable to other scene description formats, they are provided for ISO/IEC 12113.
Technologies de l'information — Représentation codée de média immersifs — Partie 14: Description de scènes
General Information
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 23090-14
First edition
2023-06
Information technology — Coded
representation of immersive media —
Part 14:
Scene description
Technologies de l'information — Représentation codée de média
immersifs —
Partie 14: Description de scènes
Reference number
ISO/IEC 23090-14:2023(E)
© ISO/IEC 2023
---------------------- Page: 1 ----------------------
ISO/IEC 23090-14:2023(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2023
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 23090-14:2023(E)
Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 1
3 Terms, definitions, abbreviated terms, and conventions . 1
3.1 Terms and definitions . 1
3.2 Abbreviated terms . 3
3.3 Conventions . 3
3.3.1 General . 3
3.3.2 Arithmetic operators . 3
3.3.3 Logical operators . 4
3.3.4 Relational operators . 4
3.3.5 Bit-wise operators . 4
3.3.6 Assignment operators . 4
3.3.7 Other operators . 5
3.3.8 Order of operation precedence . 5
3.3.9 Text description of logical operations . 5
4 Overview and architecture .7
4.1 Overview . 7
4.2 Architecture . 7
4.3 Timing model . 11
5 Scene description extensions .11
5.1 General . 11
5.1.1 Overview of extensions . 11
5.1.2 Formatting and typing .12
5.2 Generic extensions . 13
5.2.1 MPEG_media extension . 13
5.2.2 MPEG_accessor_timed extension . 16
5.2.3 MPEG_buffer_circular extension . 19
5.2.4 MPEG_scene_dynamic extensions . 21
5.3 Visual Extensions . 23
5.3.1 MPEG_texture_video extensions . 23
5.3.2 MPEG_mesh_linking extensions . 24
5.4 Audio extensions. 26
5.4.1 MPEG_audio_spatial extensions . 26
5.5 Metadata extensions .29
5.5.1 MPEG_viewport_recommended extensions .29
5.5.2 MPEG_animation_timing extensions.30
6 Media access function and buffer API .31
6.1 General . 31
6.2 Media access function API . 32
6.3 Buffer API . 35
7 Carriage formats .37
7.1 General . 37
7.2 Carriage format for glTF JSON and JSON patch .38
7.2.1 General .38
7.2.2 glTF patch config box .39
7.3 Carriage format for glTF object and glTF source object as non-timed item .39
7.3.1 General .39
7.3.2 glTF Items .40
7.3.3 glTF source items .40
7.4 Carriage format for mesh correspondence values . 41
iii
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/IEC 23090-14:2023(E)
7.4.1 General . 41
7.4.2 Vertices correspondence sample entry . 41
7.4.3 Vertices correspondence sample format . 42
7.5 Carriage format for pose and weight . 42
7.5.1 General . 42
7.5.2 Pose transformation sample entry . 43
7.5.3 Pose transformation sample format . 43
7.6 Carriage format for animation timing .44
7.6.1 General .44
7.6.2 Animation sample entry .44
7.6.3 Animation sample format .44
7.7 Sample redundancies .46
7.8 Brands .46
Annex A (informative) JSON schema reference .47
Annex B (normative) Attribute registry . .49
Annex C (normative) Support for real-time media.50
Annex D (normative) Audio attenuation functions .51
Annex E (informative) Linking a dependent mesh and its associated shadow mesh .53
Annex F (informative) glTF extension usage examples .55
Bibliography .57
iv
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 23090-14:2023(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work.
The procedures used to develop this document and those intended for its further maintenance
are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria
needed for the different types of document should be noted. This document was drafted in
accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or
www.iec.ch/members_experts/refdocs).
ISO and IEC draw attention to the possibility that the implementation of this document may involve
the use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability
of any claimed patent rights in respect thereof. As of the date of publication of this document, ISO and
IEC had received notice of (a) patent(s) which may be required to implement this document. However,
implementers are cautioned that this may not represent the latest information, which may be obtained
from the patent database available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall
not be held responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see
www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO 23090 series can be found on the ISO and IEC websites.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national-committees.
v
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 5 ----------------------
ISO/IEC 23090-14:2023(E)
Introduction
This document defines the MPEG-I Scene Description. It provides an architecture for the MPEG-I Scene
Description, a set of extensions based on ISO/IEC 12113, a set of APIs, and storage formats for scene
description documents and scene description updates documents.
vi
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 23090-14:2023(E)
Information technology — Coded representation of
immersive media —
Part 14:
Scene description
1 Scope
This document specifies extensions to existing scene description formats in order to support MPEG
media, in particular immersive media. MPEG media includes but is not limited to media encoded with
MPEG codecs, media stored in MPEG containers, MPEG media and application formats as well as media
provided through MPEG delivery mechanisms. Extensions include scene description format syntax
and semantics and the processing model when using these extensions by a Presentation Engine. It also
defines a Media Access Function (MAF) API for communication between the Presentation Engine and
the Media Access Function for these extensions. While the extensions defined in this document can be
applicable to other scene description formats, they are provided for ISO/IEC 12113.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 12113, Information technology — Runtime 3D asset delivery format — Khronos glTF™ 2.0
ISO/IEC 14496-12, Information technology — Coding of audio-visual objects — Part 12: ISO base media file
format
ISO/IEC 21778, Information technology — The JSON data interchange syntax
IEEE 754-2019, IEEE Standard for Floating-Point Arithmetic
IETF RFC 6902, JavaScript Object Notation (JSON) Patch
IETF RFC 8259, The JavaScript Object Notation (JSON) Data Interchange Format
3 Terms, definitions, abbreviated terms, and conventions
3.1 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 12113 and the following
apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1.1
asset
3D scene described by a scene description document (3.1.10) together with corresponding scene
description data (3.1.9)
1
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 23090-14:2023(E)
3.1.2
node
element in the scene graph (3.1.12)
3.1.3
media access function
function that retrieves and prepares media for rendering on request by the presentation engine (3.1.7)
3.1.4
media pipeline
chain of media processing components to process media
3.1.5
object
node in a scene description document (3.1.10)
3.1.6
patch document
document that contains update instructions
Note 1 to entry: For example, update instruction can be provided as defined in RFC 6902.
3.1.7
presentation engine
engine that processes and renders the asset (3.1.1)
3.1.8
scene activation time
time on the media timeline at which the scene described by a scene description document (3.1.10) takes
effect in the presentation engine (3.1.7)
3.1.9
scene description data
binary data that is described by scene description document (3.1.10)
3.1.10
scene description document
document describing a 3D scene
Note 1 to entry: For example, scene description document is containing description of node hierarchy, materials,
cameras, as well as description information for meshes, animations, and other constructs.
3.1.11
scene description update
patch document (3.1.6) to a scene description document (3.1.10) or a scene description document (3.1.10)
3.1.12
scene graph
data structure used to represent objects (3.1.5) in a 3D scene and their hierarchical relationships
3.1.13
timed accessor
accessor defined in ISO/IEC 12113 that has an MPEG_accessor_timed extension and is used to describe
access to timed data
3.1.14
timed data
timed media
media, which when decoded results in content, possibly containing internal timing values, to be
presented at a given presentation time and for a certain duration
2
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC 23090-14:2023(E)
3.2 Abbreviated terms
3D Three-Dimensional
3DoF Three Degrees of Freedom
6DoF Six Degrees of Freedom
API Application Programming Interface
AR Augmented Reality
DASH Dynamic Adaptive Streaming over HTTP
dB Decibel
DSR Diffuse to Source Ratio
glTF Graphics Language Transmission Format
HOA Higher Order Ambisonics
ISOBMFF ISO Base Media File Format
JSON JavaScript Object Notation
MAF Media Access Function
MPEG Moving Picture Experts Group
IDL Interface Definition Language
PCM Pulse-Code Modulation
RT60 60 dB Reverberation Time
SDP Session Description Protocol
3.3 Conventions
3.3.1 General
The mathematical operators used in this document are similar to those used in the C programming
language. However, the results of integer division and arithmetic shift operations are defined more
precisely, and additional operations are defined, such as exponentiation and real-valued division.
Numbering and counting conventions generally begin from 0.
3.3.2 Arithmetic operators
+ addition
− subtraction (as a two-argument operator) or negation (as a unary prefix operator)
* multiplication, including matrix multiplication
integer division with truncation of the result toward zero. For example, 7 / 4 and −7 / −4 are
/
truncated to 1 and −7 / 4 and 7 / −4 are truncated to −1.
÷ division in mathematical equations where no truncation or rounding is intended.
3
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 23090-14:2023(E)
3.3.3 Logical operators
! Boolean logical "not".
3.3.4 Relational operators
> Greater than.
>= Greater than or equal to.
< Less than.
<= Less than or equal to.
== Equal to.
!= Not equal to.
3.3.5 Bit-wise operators
~ bit-wise "not".
When operating on integer arguments, operates on a two's complement representation of the
integer value. When operating on a binary argument that contains fewer bits than another
argument, the shorter argument is extended by adding more significant bits equal to 0.
& bit-wise "and".
When operating on integer arguments, operates on a two's complement representation of the
integer value. When operating on a binary argument that contains fewer bits than another
argument, the shorter argument is extended by adding more significant bits equal to 0.
| bit-wise "or".
When operating on integer arguments, operates on a two's complement representation of the
integer value. When operating on a binary argument that contains fewer bits than another
argument, the shorter argument is extended by adding more significant bits equal to 0.
^ bit-wise "exclusive or".
When operating on integer arguments, operates on a two's complement representation of the
integer value. When operating on a binary argument that contains fewer bits than another
argument, the shorter argument is extended by adding more significant bits equal to 0.
x >> y arithmetic right shift of a two's complement integer representation of x by y binary digits.
This function is defined only for non-negative integer values of y. Bits shifted into the MSBs
as a result of the right shift have a value equal to the MSB of x prior to the shift operation.
x << y arithmetic left shift of a two's complement integer representation of x by y binary digits.
This function is defined only for non-negative integer values of y. Bits shifted into the LSBs
as a result of the left shift have a value equal to 0.
3.3.6 Assignment operators
= assignment operator.
++ increment, i.e. x++ is equivalent to x = x + 1; when used in an array index, evaluates to the
value of the variable prior to the increment operation.
-- decrement, i.e. x-- is equivalent to x = x − 1; when used in an array index, evaluates to the
value of the variable prior to the decrement operation.
4
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC 23090-14:2023(E)
+= increment by amount specified, i.e. x += 3 is equivalent to x = x + 3, and x += (−3) is equivalent
to x = x + (−3).
−= decrement by amount specified, i.e. x −= 3 is equivalent to x = x − 3, and x −= (−3) is equivalent
to x = x − (−3).
3.3.7 Other operators
y.z range operator/notation.
This function is defined only for integer values of y and z. When z is larger than or equal
to y, it defines an ordered set of values from y to z in increments of 1. Otherwise, when z is
smaller than y, the output of this function is an empty set. If this operator is used within the
context of a loop, it specifies that any subsequent operations defined are performed using
each element of this set, unless this set is empty.
3.3.8 Order of operation precedence
When order of precedence in an expression is not indicated explicitly by use of parentheses, the
following rules apply:
— Operations of a higher precedence are evaluated before any operation of a lower precedence.
— Operations of the same precedence are evaluated sequentially from left to right.
Table 1 specifies the precedence of operations from highest to lowest; a higher position in the table
indicates a higher precedence.
NOTE For those operators that are also used in the C programming language, the order of precedence used
in this document is the same as used in the C programming language.
Table 1 — Operation precedence from highest (at top of table) to lowest (at bottom of table)
operations (with operands x, y, and z)
"x++", "x--"
"!x", "−x" (as a unary prefix operator)
"x * y", "x / y", "x ÷ y", "x % y"
"x + y", "x − y" (as a two-argument operator)
"x << y", "x >> y"
"x < y", "x <= y", "x > y", "x >= y"
"x == y", "x != y"
"x & y"
"x | y"
"x && y"
"x || y"
"x ? y : z"
"x.y"
"x = y", "x += y", "x −= y"
3.3.9 Text description of logical operations
In the text, a statement of logical operations as would be described mathematically in the following
form:
5
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 23090-14:2023(E)
if( condition 0 )
statement 0
else if( condition 1 )
statement 1
.
else /* informative remark on remaining condition */
statement n
may be described in the following manner:
... as follows / ... the following applies:
— If condition 0, statement 0
— Otherwise, if condition 1, statement 1
— .
— Otherwise (informative remark on remaining condition), statement n
Each "If . Otherwise, if . Otherwise, ." statement in the text is introduced with ". as follows" or ".
the following applies" immediately followed by "If . ". The last condition of the "If . Otherwise, if .
Otherwise, ." is always an "Otherwise, .". Interleaved "If . Otherwise, if . Otherwise, ." statements
can be identified by matching ". as follows" or ". the following applies" with the ending "Otherwise, .".
In the text, a statement of logical operations as would be described mathematically in the following
form:
if( condition 0a && condition 0b )
statement 0
else if( condition 1a || condition 1b )
statement 1
.
else
statement n
may be described in the following manner:
... as follows / ... the following applies:
— If all of the following conditions are true, statement 0:
— condition 0a
— condition 0b
— Otherwise, if one or more of the following conditions are true, statement 1:
— condition 1a
— condition 1b
— .
— Otherwise, statement n
In the text, a statement of logical operations as would be described mathematically in the following
form:
if( condition 0 )
statement 0
if( condition 1 )
statement 1
6
© ISO/IEC 2023 – All rights reserved
---------------------- Page: 12 ----------------------
ISO/IEC 23090-14:2023(E)
may be described in the following manner:
When condition 0, statement 0
When condition 1, statement 1
In addition, a “continue” statement, which is used within loops, is defined as follows:
The “continue” s
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.