ISO/IEC 23001-4:2017
(Main)Information technology — MPEG systems technologies — Part 4: Codec configuration representation
Information technology — MPEG systems technologies — Part 4: Codec configuration representation
ISO/IEC 23001-4:2017 defines the methods and general principles capable of describing codec configurations in the reconfigurable video coding (RVC) framework. It primarily addresses reconfigurable video aspects and will only focus on the description of representation for video codec configurations within the RVC framework. Within the scope of the RVC framework, two languages, namely FNL and RVC-BSDL, are specified normatively. FNL is a language that describes the FND, also known as "network of FUs". RVC-BSDL is a pertinent subset of BSDL defined in ISO/IEC 23001‑5. This RVC-BSDL also includes possibilities for further extensions, which are necessary to provide complete description of video bitstreams.
Technologies de l'information — Technologies des systèmes MPEG — Partie 4: Représentation de configuration codec
General Information
Relations
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 23001-4
Fourth edition
2017-08
Information technology — MPEG
systems technologies —
Part 4:
Codec configuration representation
Technologies de l’information — Technologies des systèmes MPEG —
Partie 4: Représentation de configuration codec
Reference number
ISO/IEC 23001-4:2017(E)
©
ISO/IEC 2017
---------------------- Page: 1 ----------------------
ISO/IEC 23001-4:2017(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2017 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 23001-4:2017(E)
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Functional unit network description . 3
4.1 General . 3
4.2 Specification of an FU network . 6
5 Bitstream syntax description . 6
6 Model instantiation . 6
Annex A (normative) Functional unit network description . 8
Annex B (informative) Examples of FU network description.15
Annex C (normative) Specification of RVC-BSDL .18
Annex D (normative) Specification of the RVC-CAL language .41
Annex E (informative) FU Classification according to their dataflow model of computation
of RVC-CAL .67
Annex F (informative) I/O FUs .73
Annex G (normative) Storage of RMC in MP4 file format .78
Annex H (normative) Carriage of RMC over RTP .79
Annex I (informative) Instantiation of bitstream syntax parser from bitstream
syntax descriptions .80
Annex J (informative) Relation between codec configuration representation and
multimedia middleware (M3W) .89
Bibliography .90
© ISO/IEC 2017 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 23001-4:2017(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO’s adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: w w w . i s o .org/ iso/ foreword .html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
This fourth edition cancels and replaces the third edition (ISO/IEC 23001-4:2014), which constitutes a
minor revision with the following changes:
— addition of citations to Annexes G, H and J in the Introduction;
— addition of a citation to Annex E in Clause 4;
— improvement of the usage description of rvc: port attribute and addition of a citation to
Annex F in Clause 6;
— improvement of the specification of RVC-BSDL in Annex C;
— addition of informative description of a generic bitstream parser in Annex I.
A list of all parts in the ISO/IEC 23001 series can be found on the ISO website.
iv © ISO/IEC 2017 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 23001-4:2017(E)
Introduction
This document defines the methods capable of describing codec configurations in the reconfigurable
video coding (RVC) framework. The objective of RVC is to offer a framework that is capable of
configuring and specifying video codecs as a collection of “higher level” modules by using video coding
tools. The video coding tools are defined in the video tool library. ISO/IEC 23002–4 defines the MPEG
video tool library. The RVC framework principle could also support non-MPEG tool libraries, provided
that their developers have taken care to obey the appropriate rules of operation.
For the purpose of framework deployment, an appropriate description is needed to describe
configurations of decoders composed of or instantiated from a subset of video tools from either one or
more libraries. As illustrated in Figure 1, the configuration information consists of
— bitstream syntax description, and
— network of functional units (FUs) description (also referred to as the decoder configuration)
that together constitute the entire decoder description (DD).
Bitstreams of existing MPEG standards are specified by specific syntax structures and decoders are
composed of various coding tools. Therefore, RVC includes support for bitstream syntax descriptions,
as well as video coding tools. As depicted in Figure 1, a typical RVC decoder requires two types of
information, namely the decoder description and the encoded media (e.g. video bitstreams) data.
Figure 1 — Conceptual diagram of RVC
Figure 2 illustrates a more detailed description of the RVC decoder.
A more detailed description of the RVC decoder is shown in Figure 2, where the decoder description
is required for the configuration of an RVC decoder. The Bitstream Syntax Description (BSD) and FU
Network Description (FND) (which compose the Decoder Description) are used to configure or compose
an abstract decoder model (ADM) which is instantiated through the selection of FUs from tool libraries
optionally with proper parameter assignment. Such an ADM constitutes the behavioural reference
model used in setting up a decoding solution under the RVC framework. The process of yielding a
decoding solution may vary depending on the technologies used for the desired implementations.
Examples of the instantiation of an abstract decoder model and generation of proprietary decoding
solutions are given in Annex I.
© ISO/IEC 2017 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC 23001-4:2017(E)
Figure 2 — Graphical representation of the instantiation process or decoder composition
mechanism for the RVC normative ADM and for the non-normative proprietary compliant
decoder implementation
Within the RVC framework, the decoder description describes a particular decoder configuration and
consists of the FND and the BSD. The FND describes the connectivity of the network of FUs used to
form a decoder whereas the parsing process for the bitstream syntax is implicitly described by the BSD.
These two descriptions are specified using two standard XML-based languages or dialects:
— Functional Unit Network Language (FNL) is a language that describes the FND, known also as
“network of FUs”. The FNL specified normatively within the scope of the RVC framework is provided
in this document;
— Bitstream Syntax Description Language (BSDL), standardized in ISO/IEC 23001-5 (MPEG-B Part
5), describes the bitstream syntax and the parsing rules. A pertinent subset of this BSDL named
RVC-BSDL is defined within the scope of the current RVC framework. This RVC-BSDL also includes
possibilities for further extensions, which are necessary to provide complete description of video
bitstreams. RVC-BSDL specified normatively within the scope of the RVC framework is provided in
this document.
The decoder configuration specified using FNL, together with the specification of the bitstream
syntax using RVC-BSDL fully specifies the ADM and provides an “executable” model of the RVC decoder
description.
The instantiated ADM includes the information about the selected FUs and how they should be
connected. As already mentioned, the FND with the network connection information is expressed by
using FNL. Furthermore, the RVC framework specifies and uses a dataflow-oriented language called
vi © ISO/IEC 2017 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC 23001-4:2017(E)
RVC-CAL for describing FUs’ behaviour. The normative specification of RVC-CAL is provided in this
document. The ADM is the behavioural model that should be referred to in order to implement any RVC
conformant decoder. Any RVC compliant decoding solution/implementation can be achieved by using
proprietary non-normative tools and mechanisms that yield decoders that behave equivalent to the
RVC ADM.
The decoder description, the MPEG video tool library, and the associated instantiation of an ADM are
normative. More precisely, the ADM is intended to be normative in terms of a behavioural model. In
other words, what is normative is the input/output behaviour of the complete ADM, as well as the
input/output behaviour of all the FUs that are included in the ADM.
This document also includes informative technical descriptions to facilitate implementation of the
RVC framework. In Annex G, allocation of the decoder configuration data within MP4 file format
is introduced. In Annex H, carriage of the decoder configuration over RTP is described. Finally, in
Annex J, technical relation between the codec configuration representation and the MPEG multimedia
middleware (M3W) is described.
© ISO/IEC 2017 – All rights reserved vii
---------------------- Page: 7 ----------------------
INTERNATIONAL STANDARD ISO/IEC 23001-4:2017(E)
Information technology — MPEG systems technologies —
Part 4:
Codec configuration representation
1 Scope
This document defines the methods and general principles capable of describing codec configurations
in the reconfigurable video coding (RVC) framework. It primarily addresses reconfigurable video
aspects and will only focus on the description of representation for video codec configurations within
the RVC framework.
Within the scope of the RVC framework, two languages, namely FNL and RVC-BSDL, are specified
normatively. FNL is a language that describes the FND, also known as "network of FUs". RVC-BSDL is
a pertinent subset of BSDL defined in ISO/IEC 23001-5. This RVC-BSDL also includes possibilities for
further extensions, which are necessary to provide complete description of video bitstreams.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 14496-2:2004, Information technology — Coding of audio-visual objects — Part 2: Visual
ISO/IEC 14496-12, Information technology — Coding of audio-visual objects — Part 12: ISO base media
file format
ISO/IEC 23001-5, Information technology — MPEG systems technologies — Part 5: Bitstream Syntax
Description Language (BSDL)
ISO/IEC 23002-4, Information technology — MPEG video technologies — Part 4: Video tool library
IETF RFC 1889, RTP A Transport Protocol for Real-Time Applications, H. Schulzrinne, et. al., January 1996
IETF RFC 2327, SDP: Session Description Protocol, M. Handley, April 1998
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http:// www .electropedia .org/
— ISO Online browsing platform: available at http:// www .iso .org/ obp
3.1
abstract decoder model
ADM
conceptual model of the instantiation of functional units (3.8) from the video tool library (3.16) and their
connection according to the FU network description (3.9)
© ISO/IEC 2017 – All rights reserved 1
---------------------- Page: 8 ----------------------
ISO/IEC 23001-4:2017(E)
3.2
bitstream syntax description
BSD
description containing the bitstream syntax, its implicit parsing rules and possibly tables [e.g. VLD
tables if not already existing in the reconfigurable video coding (3.13) video tool library (3.16)] to define
the parser functional unit (3.8)
Note 1 to entry: The BSD is expressed using reconfigurable video coding-bitstream syntax description
language (3.14).
3.3
bitstream syntax description language
BSDL
description of the bitstream syntax and the parsing rules
Note 1 to entry: Bitstream syntax description language (BSDL) is standardized by ISO/IEC 23001-5.
3.4
connection
link from an output port to an input port of a functional unit (3.8) that enables token exchange
between FUs
3.5
decoder configuration
conceptual configuration of a decoding solution
Note 1 to entry: Using the MPEG video tool library (3.12), decoder configuration can be designed as one of the
following cases:
— a decoding solution of an existing MPEG standard at a specific profile and level;
— a new decoding solution built from tools of an existing MPEG standard;
— a new decoding solution built from tools of an existing MPEG standard and some new MPEG tools included in
the MPEG video tool library;
— a new decoding solution that is composed of new MPEG tools included in the MPEG video tool library.
Note 2 to entry: In summary, an RVC decoder description essentially consists of a list of functional units (3.8)
and of the specification of the FU connections [FU network description (3.9) expressed in FU network language
(3.10)] plus the implicit specification of the parser in terms of bitstream syntax description (3.2) [BSD expressed
in reconfigurable video coding-bitstream syntax description language (3.14)]. In order to be a complete behavioural
model [i.e. abstract decoder model (3.1)] an RVC decoder description (3.6) needs to make reference to the
behaviour of each FU that is provided in terms of I/O behaviour by the MPEG video tool library (3.12) specified in
ISO/IEC 23002-4.
3.6
decoder description
DD
description of a particular decoder configuration, which consists of two parts: FU network description
(3.9) and bitstream syntax description (3.2)
3.7
decoding solution
implementation of the abstract decoder model (3.1)
3.8
functional unit
FU
modular tool which consists of a processing unit characterized by the input/output behaviour
2 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 23001-4:2017(E)
3.9
FU network description
FND
FU (3.8) connections used in forming a decoder which are modelled using FU network language (3.10)
3.10
FU network language
FNL
language that describes the FU network description (3.9), known also as a "network of FUs"
3.11
model instantiation
building of the abstract decoder model (3.1) from the decoder description (3.6) [consisting of the
bitstream syntax description (3.2) and the FU network description (3.9)] and from functional units (3.8)
from the video tool library (3.16)
Note 1 to entry: During the model instantiation, the parser FU is reconfigured according to the BSD or loaded
from VTL.
3.12
MPEG video tool library
MPEG VTL
video tool library (3.16) that contains functional units (3.8) defined by MPEG, that is, drawn from existing
MPEG International Standards
3.13
reconfigurable video coding
RVC
framework defined by MPEG to promote coding standards at tool-level while maintaining
interoperability between solutions from different implementers
3.14
reconfigurable video coding-bitstream syntax description language
RVC-BSDL
pertinent subset of bitstream syntax description language (3.3), which is defined within the scope of the
current reconfigurable video coding (3.13) framework
3.15
token
data entity exchanged between input and output among functional units (3.8)
3.16
video tool library
VTL
collection of functional units (3.8)
4 Functional unit network description
4.1 General
The FUs in MPEG RVC are specified by
— the textual description in ISO/IEC 23002-4, and
— the RVC-CAL reference software.
The RVC-CAL language is formally specified in Annex D, and the classification of FUs according to the
dataflow computation model of RVC-CAL is informatively described in Annex E.
© ISO/IEC 2017 – All rights reserved 3
---------------------- Page: 10 ----------------------
ISO/IEC 23001-4:2017(E)
The Functional Unit Network Language (FNL) is specified in this subclause and is used to describe
networks of FUs. FNL is derived from Extensible Markup Language (XML) which was in turn derived
from SGML (ISO 8879). The ADM consists of a number of FUs with input and output ports, and the
connections between those ports. In addition, the ADM may have input and output ports, which may be
connected to the ports of FUs or to each other.
A decoder can be described as a network of a number of FUs or even only one FU (e.g. Figure 3).
Figure 3 — FU network of one FU
A network of FUs is described in FND. An FND includes the list of selected FUs to form the decoder and
the three types of connections: connections between FUs (type A), connections between decoder inputs
and FU inputs (type B), and connections between FU outputs and decoder outputs (type C), which are
illustrated in Figure 4.
The list of selected FUs (Figure 4) is described in FND according to Table 1. When selecting FUs from
VTL, the IDs and names of FUs defined in ISO/IEC 23002-4 shall be used in the FND. The parameter
assignments in the listed FUs are supported in the FND, but optional.
The connections (type A, type B, and type C shown in Figure 4) are described in FND as shown in
Table 1.
Table 1 — Connection types
Type A
Type B
Type C
4 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 23001-4:2017(E)
Figure 4 — Three types of connections in an FU network
Another example of FU networks with four FUs is illustrated in Figure 5. The textual description of
Figure 5 in FND is described as follows.
Figure 5 — Another example of FU networks
© ISO/IEC 2017 – All rights reserved 5
---------------------- Page: 12 ----------------------
ISO/IEC 23001-4:2017(E)
4.2 Specification of an FU network
The XML structures with names of elements, such as Decl, Network, Package, Expr, etc. are described in
the specification of FNL in Annex A. In addition, attributes that direct an individual element’s features
are also introduced there. Attribute names will be prefixed with "@". For instance, common attribute
names are @id, @name or @kind. In cases where an element name may be qualified by the value of
an attribute, square brackets are used. For instance, in order to express the notion of an Expr element
whose @kind attribute is the string "literal", Expr[@kind="literal"] is written.
By using the RVC-CAL model, FNL also allows FU networks and individual FUs to be parameterized.
In particular, it is possible to pass bounded values for specific parameters into FU and FU networks.
These values are represented by Expr and Decl syntax. Expr and Decl are the syntactical constructs
describing a computation, which may, itself, be dependent upon the values of parameters which are
either global or local variables.
5 Bitstream syntax description
The MPEG video tool library contains FUs that specify MPEG decoding tools. A new decoder configuration
implies new bitstream syntax. The description of the bitstream syntax in RVC is provided using BSDL as
specified in ISO/IEC 23001-5 and the BSDL schema. However, to facilitate the developments of synthesis
tools that are able to generate parsers directly from a BSD (i.e. a BSDL schema), the RVC framework
standardizes a version of BSDL called RVC-BSDL specified by including new RVC specific extensions and
usage restrictions of standard BSDL in ISO/IEC 23001-5. Such extensions and restrictions versus the
MPEG standard BSDL are defined in Annex C. RVC-BSDL contains all information necessary to parse any
bitstream compliant with such syntax. The procedure to instantiate the parser capable of parsing and
decoding any bitstream compliant with the syntax specified by the RVC-BSDL schema is not normative.
Examples of such non-normative procedures are provided in Annex I.
6 Model instantiation
This clause describes the model instantiation process which consists of the selection of Functional Units
(FUs) from the video tool library and instantiation of the FUs with the proper parameter assignments.
The instantiation process requires the following information:
— video tool library;
— FU network description;
— bitstream syntax description.
The instantiation process consists of attaching the source code corresponding to the FUs identified in
the FND in order to build a complete model that can be simulated. The video tool library is a library of
source code of all FUs standardized in ISO/IEC 23002-4. The FND contains only the references (names
of the FUs) to the pieces of code in the VTL. The process outputs the ADM. Figure 6 illustrates the model
instantiation process.
6 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 13 ----------------------
ISO/IEC 23001-4:2017(E)
Figure 6 — Description of the model instantiation process
The FU Network Description (FND) provides the structure of the decoder by giving the names of the
FUs composing the decoder and their respective connections among them. The name of the instance
of the FU in the ADM is contained in the tag . For instance, the attribute rvc:
port (See C.4.4.5) indicates the name of the instance of the FU into the ADM to which this element of
syntax is sent. The tag provides the values of the parameters, which shall be used for
the instantiation of the FU in the ADM.
The Bitstream Syntax Description (BSD) provides the structure of the bitstream. The parser is
generated automatically from the BSD. Informative examples are provided in Annex I for building the
parser. The syntax parser FU of the ADM might use other FUs to parse the bitstream. Thus, a clear
link between identifiers inside the BSD and the FND shall be established. The tag
indicates the name of the instance of the FU into the ADM to which this element of syntax is sent.
NOTE The FND could include instances of FUs that represent input and output to/from the network.
Informative technical descriptions for the input/output FUs can be found in Annex F.
© ISO/IEC 2017 – All rights reserved 7
---------------------- Page: 14 ----------------------
ISO/IEC 23001-4:2017(E)
Annex A
(normative)
Functional unit network description
A.1 Elements of a functional unit network
A.1.1 XDF
An FU network is identified by the root element called XDF, which marks the beginning and end of the
XML description of the network.
— Optional attribute: @name, the name of the network; @version, the version number of the current
network (assumed to be "1.0" if absent).
— Optional children: Package, Decl, Port, Instance, Connection.
.
A.1.2 Package
This element contains a structured representation of a qualified identifier (QID), i.e. a list of identifiers
that are used to specify a locus in a hierarchical namespace. The QID provides the context for the @
name attributed of the enclosing Network element: that name is intended to be valid within the
specified namespace or package.
— Required child: QID, the qualified identifier.
A.1.3 Decl[@kind="Param"]
This element represents the declaration of a parameter of the network.
— Required attribute: @name, the name of the parameter.
— Optional child: Type, the declared type of the parameter.
A.1.4 Decl[@kind="Var"]
This element represents a variable declaration. Variables are used within expressions to compute
parameter values for actors instantiated withi
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.