ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information

This document specifies the framework, concepts, methodology for testing, and criteria to be achieved to claim conformance to the image coding technology Rec. ITU-T T.840.1 | ISO/IEC 6048-1, known as Learning-based image coding (JPEG AI). It lists the conformance testing procedures.

This document specifies a reference software implementation of ISO/IEC 19566-5 [1]. The reference software is accompanied with a reference dataset which provides an extensive list of the various JUMBF data structures specified in ISO/IEC 19566-5 [1].

This document specifies a framework for establishing trust in media. This framework includes aspects of authenticity, provenance, attribution, intellectual property rights, and integrity through secure and reliable annotation of the media assets throughout their life cycle.

This document specifies a set of tests and procedures designed to indicate whether encoders or decoders meet the requirements specified in Rec. ITU-T H.266 | ISO/IEC 23090-3.

This document defines a storage and delivery format for metadata for energy-efficient decoding, encoding, presentation, and selection of media (green metadata) as defined in ISO/IEC 23001-11. The green metadata are timed metadata which can be associated with other tracks in the ISO Base Media File Format. Timed metadata such as power consumption information and their metrics are defined in this document for carriage in files based on ISO/IEC 14496-12. In the context of DASH delivery, the green metadata representations and their association to the media representations are also defined using the signalling mechanisms specified in ISO/IEC 23009-1:2022 and ISO/IEC 23009-3[1]. These metadata can be used for multiple purposes, including optimizing power consumption during playback and supporting dynamic adaptive streaming.

This document specifies the syntax and semantics of description schemes to represent data exchanged by media analysers for distributed artificial intelligence (AI) processing. Moreover, it specifies the application programming interfaces (APIs) to exchange these data between media things. This document does not specify how analysis is carried out, but defines the interfaces between the media things.

This document specifies low latency, low complexity light detection and ranging (LiDAR) coding.

This document specifies the syntax and semantics of description schemes for representing data exchanged by media things (e.g. media sensors, media actuators, and media storages). Moreover, it specifies the APIs to exchange these data between media things. The following interfaces are under the scope of this document: APIs for accessing media sensors, actuators, storage, managers, controllers, and aggregators; structured data formats (XML) representing media thing’s base data types, including sensors, actuators, storage, managers, aggregators, controllers, and their elements; structured data formats (XML) representing the media sensor’s output data; and, structured data formats (XML) representing media actuator’s commands. This document does not specify how sensing and actuating are carried out but defines the interfaces between the media things.

This document specifies syntax, semantics, and decoding for video based dynamic mesh coding (V-DMC), basemesh coding, MPEG edgebreaker static mesh coding, and arithmetic coded displacement. Furthermore, this document specifies processes that can be used for reconstruction of visual volumetric media and also include additional processes such as post-decoding, pre-reconstruction, post-reconstruction, and adaptation.

This document specifies system level functionalities for the communication of interactive audio-visual scenes, i.e. the coded representation of information related to the management of data streams (synchronization, identification, description and association of stream content).

This document specifies the Open Font Format (OFF) specification, including the TrueType and Compact Font Format (CFF) outline formats. Many references to both TrueType and PostScript exist throughout this document, as Open Font Format fonts combine the two technologies. The document defines data structures for various font tables and provides the necessary details for developers to build a font rendering and text layout/shaping engines in compliance with this document.

This document primarily specifies formats for the Media Presentation Description and Segments for dynamic adaptive streaming delivery of MPEG media over HTTP. It is applicable to streaming services over the Internet.

This document presents the role of artificial intelligence (AI) technologies, including neural networks (NN), in multimedia coding and processing activities. It describes the current perspectives on AI for multimedia and identifies working assumptions and technical challenges expected from working with AI and NN-based technologies. This document highlights a variety of multimedia coding activities, key scenarios and gaps that are to be addressed by further standardization efforts.

This document specifies a subset of the syntax specified in Rec. ITU-T T.840.1 | ISO/IEC 6048-1 via one Main stream profile and three decoder tool subsets as three decoder profiles. It also defines higher bounds on tile and picture sizes via levels of a decoder profile that a conforming decoder implementation shall support, and specifies profiles, each being tailored to certain application domains, and defines the levels of the profiles.

This document specifies the syntax, semantics, and decoding for visual volumetric media using video‑based coding methods. Furthermore, this document specifies processes that can be needed for reconstruction of visual volumetric media, and can also include additional processes such as post‑decoding, pre-reconstruction, post‑reconstruction, and adaptation.

This document provides information about optimizations for encoders and receiving systems for conducting machine analysis tasks on coded video content. It provides a concept-level overview of recent practices and provides comments on technical aspects and cautions to be taken when interpreting the results. This document describes technologies that have recently been studied and have demonstrated benefits to coding efficiency for some machine analysis tasks.

This document specifies container file formats for JPEG AI codestreams as specified in Rec. ITU-T T.840.1 | ISO/IEC 6048-1 and Rec. ITU-T T.840.2 | ISO/IEC 6048-2. It defines file formats for working with image and motion sequence files on computer platforms, allowing Internet-based and other communications. This document uses already existing specifications for file formats and extends them for the embedding of JPEG AI codestreams.

This document specifies the ISO base media file format, which is a general format forming the basis for a number of other more specific file formats. This format contains the timing, structure, and media information for timed sequences of media data, such as audio-visual presentations.

This document specifies the transport and container formats for JPEG XL codestreams as specified in ISO/IEC 18181-1. This document specifies how to add metadata and extensions to JPEG XL codestreams. A file as described by this document is called a JPEG XL file.

This document specifies an architecture and data model for interchange of scene-based media for use in systems and applications that employ renderers for the presentation of immersive media, including audio, visual, tactile and other representations of media types. The architecture and data model specified in this document applies to use cases where there is a need for interchange of scene-based media for presentation by systems with 3D render-based technologies. One specific application of this document is immersive display technologies that utilize a real-time renderer of 3D media to create their visual presentations, as opposed to a video decoder and renderer of 2D raster-based media. Another application of this document is for the distribution of media assets for use across a large variety of applications comprising the Metaverse.

This document specifies technology that supports the efficient transmission of immersive audio signals and flexible rendering for the playback of immersive audio in a wide variety of listening scenarios. These include home theatre setups with 3D loudspeaker configurations, 22.2 loudspeaker systems, automotive entertainment systems and playback over headphones connected to a tablet or smartphone.

This document specifies a subjective image quality assessment methodology that covers a range from good quality up to mathematically lossless. This document is applicable to the assessment of distortions due to image coding (i.e. lossy compression) and not necessarily other kinds of distortions (e.g. capture, sensor or rendering artefacts).

This document specifies omnidirectional/360-degree image and motion contents using Rec. ITU-T T.81 | ISO/IEC 10918‑1, Rec. ITU-T T.800 (11/2015) | ISO/IEC 15444‑1, and ISO/IEC 18477-3.

This document specifies a set of tests and procedures designed to indicate whether encoders or decoders meet the requirements specified in ISO/IEC 23090-12.

This document specifies the abstract class of a media thing (MThing), a basic component to construct the Internet of media things. The MThing class contains the basic APIs to: — discover other MThing(s) in the network; — connect/disconnect MThing(s); and, — support transactions (e.g. payments) using media tokens between MThings.

This document provides reference software for Rec. ITU-T H.266 | ISO/IEC 23090-3. The reference software includes both encoder and decoder functionality. Reference software is useful in aiding users of a video coding standard to establish and test conformance and interoperability, and to educate users and demonstrate the capabilities of the standard. For these purposes, the accompanying software is provided as an aid for the study and implementation of Rec. ITU-T H.266 | ISO/IEC 23090-3.

This document describes the architecture of systems for the internet of media things. It also includes a comprehensive set of use cases that can be deployed on such an architecture.

This document specifies extensions to existing scene description formats in order to support MPEG media, in particular immersive media. MPEG media includes but is not limited to media encoded with MPEG codecs, media stored in MPEG containers, MPEG media and application formats as well as media provided through MPEG delivery mechanisms. Extensions include scene description format syntax and semantics and the processing model when using these extensions by a Presentation Engine. It also defines a media access function (MAF) API for communication between the Presentation Engine and the media access function for these extensions. While the extensions defined in this document can be applicable to other scene description formats, they are provided for ISO/IEC 12113.

The document specifies a set of tests and procedures designed to indicate whether encoders or decoders meet the requirements specified in ISO/IEC 23090-31.

This document contains simulation software for the MPEG-Immersive Audio standard as defined in ISO/IEC 23090-4.

This document specifies technology that supports the real-time interactive rendering of an immersive virtual or augmented reality audio presentation while permitting the user to have 6DoF movement in the audio scene. It defines metadata to support this rendering and a bitstream syntax that enables efficient storage and streaming of immersive audio content.

This document contains simulation software for the MPEG-H 3D audio standard as defined in ISO/IEC 23008-3.

This document contains the reference software of the ISO/IEC 21122 series. It acts as guidance for implementation of the ISO/IEC 21122 series and as a reference for conformance testing.

This document specifies the syntax, semantics and decoding processes for MPEG immersive video (MIV), as an extension of ISO/IEC 23090-5. It provides support for playback of a three-dimensional (3D) scene within a limited range of viewing positions and orientations, with 6 Degrees of Freedom (6DoF).

This document specifies an image coding technology known as JPEG AI learning-based image coding (JPEG AI) comprising an image coding technology that facilitates the compression and processing of images for both human and machine vision. The scope of the JPEG AI document is the creation of a learning-based image coding standard offering a single-stream, compact compressed domain representation, targeting both human visualization, with significant compression efficiency improvement over image coding standards in common use at equivalent subjective quality, and effective performance for image processing and computer vision tasks. The core coding system describes JPEG AI standard for the human vision reconstruction task and thus specifies the JPEG AI specifies the parsing coded stream and the image reconstruction process. Only the syntax format, semantics, and associated decoding process requirements are specified, while other matters such as pre-processing, the encoding process, system signalling and multiplexing, data loss recovery, post-processing, and video display are considered to be outside the scope of this document. This document is designed to be generic in the sense that it serves a wide range of applications, bit rates, resolutions, qualities and services. In the course of creating this document, several requirements from typical applications have been considered, necessary algorithmic elements have been developed, and these have been integrated into a single syntax. Hence, this document is designed to facilitate image data interchange among different applications and services.

This document specifies carriage of haptic media in ISO base media files.

This document specifies the system layer of the coding. It was developed principally to support the combination of the video and audio coding methods defined in Parts 2 and 3 of ISO/IEC 13818. The system layer supports six basic functions: 1) the synchronization of multiple compressed streams on decoding; 2) the interleaving of multiple compressed streams into a single stream; 3) the initialization of buffering for decoding start up; 4) continuous buffer management; 5) time identification; 6) multiplexing and signalling of various components in a system stream. A Rec. ITU-T H.222.0 | ISO/IEC 13818-1 multiplexed bit stream is either a transport stream or a program stream. Both streams are constructed from PES packets and packets containing other necessary information. Both stream types support multiplexing of video and audio compressed streams from one program with a common time base. The transport stream additionally supports the multiplexing of video and audio compressed streams from multiple programs with independent time bases. For almost error-free environments the program stream is generally more appropriate, supporting software processing of program information. The transport stream is more suitable for use in environments where errors are likely. A Rec. ITU-T H.222.0 | ISO/IEC 13818-1 multiplexed bit stream, whether a transport stream or a program stream, is constructed in two layers: the outermost layer is the system layer, and the innermost is the compression layer. The system layer provides the functions necessary for using one or more compressed data streams in a system. The video and audio parts of this Specification define the compression coding layer for audio and video data. Coding of other types of data is not defined by this Specification, but is supported by the system layer provided that the other types of data adhere to the constraints defined in 2.7.

This document specifies: — How to uniquely identify Digital Items (and parts thereof); — How to uniquely identify IP related to the Digital Items (and parts thereof), for example abstractions; — How to express the relationship between the two above identifiers; — How to deal with varying levels of functional granularity for Digital Item identifiers; — How to uniquely identify description schemes; — The relationship between Digital Items (and parts thereof) and existing identification systems. Annex C contains a list of relevant identification systems. This is not an exhaustive list and is subject to change over time; — How to express the relationship between two Digital Items. This document does not specify: — New identification systems for the content elements for which identification and description schemes already exist and are in use (e.g. this document does not attempt to replace the ISRC, as defined in ISO 3901, for sound recordings); — Normative description schemes for describing content.

This document describes the desired joint behaviour of MPEG-4 Systems (MPEG-4 File Format) and MPEG-4 Audio codecs. It is desired that MPEG-4 Audio encoders and decoders permit finite length signals to be encoded to a file (particularly MPEG-4 files) and decoded again to obtain the identical signal, subject to codec distortions. This enables the use of audio in systems implementations (particularly MPEG-4 Systems), perhaps with other media such as video, in a deterministic fashion. Most importantly, the decoded signal has nothing “extra” at the beginning or “missing” at the end. This permits: a) an exact "round trip" from raw audio to encoded file back to raw audio (excepting encoding artefacts); b) predictable synchronization between audio and other media such as video; c) correct behaviour when performing random access as well as when starting at the beginning of a stream; d) identical behaviour when edits are applied in the raw domain and the encoded domain (excepting encoding artefacts). It is also expected that there be predictable interoperability between encoders (as represented by files) and decoders. There are two kinds of audio "offsets" (or "delay" in the context of transmission): those that are result from the encoding process, and those that are result from the decoding process. This document is primarily concerned with the latter. These issues are resolved by the following: — The handling of composition time stamps for audio composition units is specified. Special care is taken in the case of compressed data, like HE-AAC coded audio, that can be decoded in a backward compatible fashion as well as in an enhanced fashion. — Examples are given that show how a finite length signals can be encoded to an MPEG-4 file and decoded again to obtain the identical signal, excepting codec distortions. Most importantly, the decoded signal has nothing “extra” at the beginning or “missing” at the end.

This document specifies the format of the Session-Based Description document and the media presentation description's (MPD) extension to be used in session-based operations with ISO/IEC 23009-1 (MPEG DASH).