IEC TR 63669:2026

Alignment of heterogeneous media data streams

IEC TR 63669:2026 describes multimedia services that combine heterogeneous media data and proposes the concept of alignment of heterogeneous media streams to address the common challenges these services face. Based on the conceptual model and the analysis of relevant technologies and standards, this document also proposes items for future standardization.

General Information

• Status: Published
• Publication Date: 27-May-2026
• ICS: 33.160.60 - Multimedia systems and teleconferencing equipment
• Technical Committee: TA 18 - Multimedia home systems and applications for end-user networks
• Drafting Committee: MT 62680-1-2 - TC 100/TA 18/MT 62680-1-2
• Current Stage: PPUB - Publication issued
• Start Date: 28-May-2026
• Completion Date: 19-Jun-2026

Overview

IEC TR 63669:2026 "Alignment of heterogeneous media data streams" is a technical report from the International Electrotechnical Commission (IEC) that addresses the challenges encountered when combining diverse media data types in modern multimedia services. As digital technologies advance, the integration of new media types-such as sensor signals, haptic feedback, 360° video, and 3D audio-has outpaced existing international standards, which were traditionally designed for video and audio integration. This lack of standardization often leads to interoperability issues, limited scalability, and the formation of data silos.

IEC TR 63669:2026 examines current technologies and relevant standards, proposes a conceptual model for aligning heterogeneous media streams, and identifies key items for future standardization. By enabling seamless alignment and integration, this document lays the foundation for next-generation multimedia services and applications.

Key Topics

• Heterogeneous Media Data Streams

  • Inclusion of conventional (video, audio, text) and emerging media types (sensor data, haptic data, geospatial information).
  • Importance of synchronizing and integrating data streams from various sources in real time.

• Three-layer Model for Data Alignment

  • Input Layer: Receives diverse data streams (video, audio, sensors), each with its specific format and metadata.
  • Process Layer: Aligns different streams via order (timestamp, sequence), attaches metadata, and manages relationships.
  • Output Layer: Delivers the aligned multimedia data, supporting flexible querying and replay (by time or space).

• Challenges Identified

  • Lack of standardized data formats for new media types.
  • Absence of unified methods for time and space alignment across streams.
  • Scarcity of protocols and schemas for dynamic, multi-modal data integration.

• Potential Standardization Areas

  • Stream alignment methods (sequence, timestamp, spatial coordinates).
  • Schema and metadata definitions for heterogeneous streams.
  • Formats and protocols for flexible output and cross-platform interoperability.

Applications

The standard enhances multimedia services by addressing the alignment and integration of heterogeneous data. Typical application areas include:

• Photorealistic Virtual Worlds

  • Example: Virtual city exploration or cultural heritage tours using integrated 360° video, geospatial data, and AR overlays.
  • Value: Offers immersive, interactive user experiences that go beyond time-based navigation, enabling spatial queries and custom interactions.

• Interactive 3D Multimedia Content

  • Example: Virtual music events recorded with multiple 360° cameras and object-based 3D audio.
  • Value: Allows for highly immersive environments where users can change viewpoints, navigate space, and interact with multimedia elements seamlessly.

• Remote Collaboration Systems

  • Example: Remote work guidance using video, audio, and real-time sensor data (e.g., hand gesture tracking, orientation).
  • Value: Facilitates real-time, synchronized collaboration and support, enhancing user experience and operational effectiveness.

Related Standards

IEC TR 63669:2026 maps its standardization focus to existing and emerging international frameworks, establishing continuity and synergy with:

• Conventional Multimedia Standards

  • File/container formats: MP4 (ISO/IEC 14496-14), MPEG-DASH, HLS, RTP/RTCP.
  • Audio/video coding: H.264/AVC, H.265/HEVC.

• Emerging Media Standards

  • 360° media: ISO/IEC 23090-2 (OMAF), MPEG-H 3D Audio.
  • Sensor/haptic data: IEC TS 63528 (Haptics stimuli descriptors), IEC 63430 (Wearable sensor data containers).
  • Time sync: IEEE 1588 (Precision Time Protocol), SmartBAN (IEC 63203-801 series).

• Schema and Metadata Standards

  • Resource Description Framework (RDF)
  • Ontology for Media Resource (W3C)
  • SQL/MM Spatial (ISO/IEC 13249), Database languages SQL (ISO/IEC 9075)

By bridging gaps between established and emerging standards, IEC TR 63669:2026 aims to enable scalable, interoperable, and future-proof multimedia services that leverage the full spectrum of heterogeneous media data.

Keywords: heterogeneous media, multimedia data alignment, IEC TR 63669, 360° video, 3D audio, haptic data, sensor integration, media synchronization, multimedia standards, virtual reality, remote collaboration, media metadata, data interoperability.

Standards Content (Sample)

IEC TR 63669 ®
Edition 1.0 2026-05
TECHNICAL
REPORT
Alignment of heterogeneous media data streams

ICS 33.160.60  ISBN 978-2-8327-1250-4

All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or
by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from either
IEC or IEC's member National Committee in the country of the requester. If you have any questions about IEC copyright
or have an enquiry about obtaining additional rights to this publication, please contact the address below or your local
IEC member National Committee for further information.

IEC Secretariat Tel.: [phone]
3, rue de Varembé [e-mail]
CH-1211 Geneva 20 www.iec.ch
Switzerland
About the IEC
The International Electrotechnical Commission (IEC) is the leading global organization that prepares and publishes
International Standards for all electrical, electronic and related technologies.

About IEC publications
The technical content of IEC publications is kept under constant review by the IEC. Please make sure that you have the
latest edition, a corrigendum or an amendment might have been published.

IEC publications search - IEC Products & Services Portal - products.iec.ch
webstore.iec.ch/advsearchform Discover our powerful search engine and read freely all the
The advanced search enables to find IEC publications by a publications previews, graphical symbols and the glossary.
variety of criteria (reference number, text, technical With a subscription you will always have access to up to date
committee, …). It also gives information on projects, content tailored to your needs.
replaced and withdrawn publications.
Electropedia - www.electropedia.org
The world's leading online dictionary on electrotechnology,
IEC Just Published - webstore.iec.ch/justpublished
Stay up to date on all new IEC publications. Just Published containing more than 22 500 terminological entries in English
details all new publications released. Available online and and French, with equivalent terms in 25 additional languages.
once a month by email. Also known as the International Electrotechnical Vocabulary
(IEV) online.
IEC Customer Service Centre - webstore.iec.ch/csc
If you wish to give us your feedback on this publication or
need further assistance, please contact the Customer
Service Centre: [e-mail].
CONTENTS
FOREWORD . 3
INTRODUCTION . 5
1 Scope . 6
2 Normative references . 6
3 Terms, definitions, and abbreviated terms . 6
3.1 Terms and definitions. 6
3.2 Abbreviated terms . 6
4 Overview . 6
4.1 Purpose . 6
4.2 Combination of media data . 7
4.2.1 General . 7
4.2.2 Photorealistic virtual world . 8
4.2.3 Interactive 3D multimedia content . 8
4.2.4 Remote collaboration system . 9
4.3 Classification of media data . 9
4.4 Concept of alignment of heterogeneous media data streams. 10
4.4.1 General . 10
4.4.2 Input layer . 11
4.4.3 Process layer. 11
4.4.4 Output layer . 12
4.5 Items of standardization . 12
4.5.1 Potential items of standardization . 12
4.5.2 Standards gap analysis of related technologies . 13
4.5.3 Focus items for standardization . 15
5 Case study . 15
5.1 Photorealistic virtual world . 15
5.1.1 Virtual city exploration . 15
5.1.2 Virtual tour . 15
5.2 Interactive 3D multimedia content - Virtual music events experience . 16
5.3 Remote collaboration system . 16
5.3.1 Remote guided tours . 16
5.3.2 Remote work guidance . 16
5.3.3 VR meeting . 16
Bibliography . 17

Figure 1 – Abstraction of services by heterogeneous media data . 7
Figure 2 – Combination of media data. 7
Figure 3 – Example of MovieMap . 8
Figure 4 – Example of T-Leap . 9
Figure 5 – Sequential alignment of video and audio streams in multimedia . 10
Figure 6 – Concept of alignment of heterogeneous media data streams . 11
Figure 7 – Alignment process . 12
Figure 8 – Flexible alignment of multimedia streams . 12

Table 1 – Media types . 9
Table 2 – Potential items of standardization . 13
Table 3 – Mapping of potential standardization items to existing standards . 13
Table 4 – Mapping of potential standardization items to emerging standards . 14

INTERNATIONAL ELECTROTECHNICAL COMMISSION
____________
Alignment of heterogeneous media data streams

FOREWORD
1) The International Electrotechnical Commission (IEC) is a worldwide organization for
standardization comprising all national electrotechnical committees (IEC National Committees).
The object of IEC is to promote international co-operation on all questions concerning
standardization in the electrical and electronic fields. To this end and in addition to other
activities, IEC publishes International Standards, Technical Specifications, Technical Reports,
Publicly Available Specifications (PAS) and Guides (hereafter referred to as "IEC
Publication(s)"). Their preparation is entrusted to technical committees; any IEC National
Committee interested in the subject dealt with may participate in this preparatory work.
International, governmental and non-governmental organizations liaising with the IEC also
participate in this preparation. IEC collaborates closely with the International Organization for
Standardization (ISO) in accordance with conditions determined by agreement between the two
organizations.
2) The formal decisions or agreements of IEC on technical matters express, as nearly as
possible, an international consensus of opinion on the relevant subjects since each technical
committee has representation from all interested IEC National Committees.
3) IEC Publications have the form of recommendations for international use and are accepted
by IEC National Committees in that sense. While all reasonable efforts are made to ensure that
the technical content of IEC Publications is accurate, IEC cannot be held responsible for the
way in which they are used or for any misinterpretation by any end user.
4) In order to promote international uniformity, IEC National Committees undertake to apply IEC
Publications transparently to the maximum extent possible in their national and regional
publications. Any divergence between any IEC Publication and the corresponding national or
regional publication shall be clearly indicated in the latter.
5) IEC itself does not provide any attestation of conformity. Independent certification bodies
provide conformity assessment services and, in some areas, access to IEC marks of conformity.
IEC is not responsible for any services carried out by independent certification bodies.
6) All users should ensure that they have the latest edition of this publication.
7) No liability shall attach to IEC or its directors, employees, servants or agents including
individual experts and members of its technical committees and IEC National Committees for
any personal injury, property damage or other damage of any nature whatsoever, whether direct
or indirect, or for costs (including legal fees) and expenses arising out of the publication, use
of, or reliance upon, this IEC Publication or any other IEC Publications.
8) Attention is drawn to the Normative references cited in this publication. Use of the referenced
publications is indispensable for the correct application of this publication.
9) IEC draws attention to the possibility that the implementation of this document may involve
the use of (a) patent(s). IEC takes no position concerning the evidence, validity or applicability
of any claimed patent rights in respect thereof. As of the date of publication of this document,
IEC had not received notice of (a) patent(s), which may be required to implement this document.
However, implementers are cautioned that this may not represent the latest information, which
may be obtained from the patent database available at https://patents.iec.ch. IEC shall not be
held responsible for identifying any or all such patent rights.
IEC TR 63669 has been prepared by IEC technical committee 100: Audio, video and multimedia
systems and equipment. It is a Technical Report.
The text of this Technical Report is based on the following documents:
Draft Report on voting
100/4460/DTR 100/4491/RVDTR
Full information on the voting for its approval can be found in the report on voting indicated in
the above table.
The language used for the development of this Technical Report is English.
This document was drafted in accordance with ISO/IEC Directives, Part 2, and developed in
accordance with ISO/IEC Directives, Part 1 and ISO/IEC Directives, IEC Supplement, available
at www.iec.ch/members_experts/refdocs. The main document types developed by IEC are
described in greater detail at www.iec.ch/publications.
The committee has decided that the contents of this document will remain unchanged until the
stability date indicated on the IEC website under webstore.iec.ch in the data related to the
specific document. At this date, the document will be
– reconfirmed,
– withdrawn, or
– revised.
INTRODUCTION
Driven by advances in digital technologies, new multimedia services have emerged. These
services combine advanced media types-such as sensor signals and haptic data – alongside
enhanced versions of video and audio-such as 360° video and 3D audio. While standards exist
for combining conventional media data (e.g. video, audio) into "multimedia data", currently there
is no standard covering these emerging media data. Consequently, each service independently
implements its own approach for combining these media data. This leads to problems such as
limited interoperability, reduced reusability, scalability concerns, and data silos. This document
clarifies multimedia services that combine emerging heterogeneous media data and the concept
of alignment of heterogeneous media streams to address the common challenges these
services face. Based on the conceptual model and the analysis of relevant technologies and
standards, this document also proposes items for future standardization.
1 Scope
This document describes multimedia services that combine heterogeneous media data and
proposes the concept of alignment of heterogeneous media streams to address the common
challenges these services face. Based on the conceptual model and the analysis of relevant
technologies and standards, this document also proposes items for future standardization.
2 Normative references
There are no normative references in this document.
3 Terms, definitions, and abbreviated terms
3.1 Terms and definitions
No terms and definitions are listed in this document.
ISO and IEC maintain terminology databases for use in standardization at the following
addresses:
– IEC Electropedia: available at https://www.electropedia.org/
– ISO Online browsing platform: available at https://www.iso.org/obp
3.2 Abbreviated terms
For the purposes of this document, the following abbreviated terms apply:
DDL Data definition language
MPEG Moving Picture Experts Group
MPEG-DASH MPEG Dynamic adaptive streaming over HTTP
MP4 MPEG-4 Part 14 file format
RTCP Real-time control protocol
RTP Real-time transport protocol
SDO Standard Development Organization
SQL Structured Query Language
VR Virtual reality
4 Overview
4.1 Purpose
Driven by advances in digital technologies, new multimedia services have rapidly emerged.
These services combine advanced media types-such as sensor signals and haptic data-
alongside enhanced versions of video and audio-such as 360° video and 3D audio. While
standards exist for combining conventional media data (e.g. video, audio) into "multimedia
data", currently there is no standard covering these emerging media data. This leads to issues
such as limited interoperability, reduced reusability, scalability concerns, and data silos.
The purpose of standardization is to establish a standardized framework that can combine
heterogeneous media data into multimedia data and facilitate the development and widespread
adoption of new multimedia products and services.
See Figure 1 for a diagram of abstraction of services by heterogeneous media data.
Figure 1 – Abstraction of services by heterogeneous media data
4.2 Combination of media data
4.2.1 General
This Subclause 4.2 summarizes examples of combinations of media data and their key
characteristics based on the results of an investigation into services that combine
heterogeneous media data:
– Photorealistic virtual world
– Interactive 3D multimedia experiences
– Remote collaboration system
See Figure 2 for an example of combination of media data. Specific service examples are
introduced in Clause 5.
Figure 2 – Combination of media data

4.2.2 Photorealistic virtual world
As shown in the example given in Figure 3, each video frame in a 360° video of a real-world
scene is associated with geospatial information from the time of recording through calculations
or manual processes. Based on this integrated video-frame-geospatial data set, a realistic
virtual world can be created that provides an interactive experience for the user. Conventional
methods of building virtual spaces with 3D-CG modelling software are extremely costly. For this
reason, there is growing demand for computational approaches that generate realistic virtual
worlds from video or images. However, there is currently no standard framework to integrate
geospatial information with videos. This has led to increased costs in the integration process,
and interoperability across different systems remains limited.
A key characteristic of this combination of media data (integrated video-frame-geospatial data
set) is that time is not the only reference for rendering or for replay. For instance, instead of
searching by time, users can render or replay the scene by location or other axes. Therefore,
combined media data must support flexible re-indexing and querying by different references
(e.g. time, space), which can require rearranging the data sequence according to the user's
chosen axis.
Figure 3 – Example of MovieMap
4.2.3 Interactive 3D multimedia content
In this example, an event such as a music concert is recorded using multiple 360° cameras and
microphones. These multiple sets of multimedia data are combined to form a virtual space that
offers users an interactive, highly immersive experience. Conventional video-and-audio-based
experiences tend to be passive and provide limited immersion and presence. Consequently,
there is an increasing need for more interactive, more immersive multimedia experiences.
However, there is no standardized framework for aligning multiple recorded data sets in both
time and space. As a result, generating these data sets can be costly. Moreover, they are often
tied to a specific application, which limits reusability.
A key characteristic of this combination is that multiple video and audio data sets are integrated
spatially for rendering. For instance, each set is placed in three-dimensional space exactly as
it was when originally captured.

4.2.4 Remote collaboration system
In this example, multiple media data streams (e.g. video, audio, and sensor data about a
device's orientation or motion) are transmitted and rendered in real time to provide a highly
immersive collaborative experience for remote collaborators. Conventional voice or video chat
alone has limitations, making it increasingly important to develop systems that allow real-time,
comprehensive understanding of remote conditions and seamless interaction with remote
collaborators. In many cases, existing systems transmit multiple multimedia data streams
separately and integrate and render them in the application. As a result, data consistency and
reusability remain challenges. See Figure 4 for an example of T-leap.
A key characteristic of this combination (of multiple media data streams, such as video, audio,
and sensor data related to a device’s orientation or motion) is that heterogeneous media data
are synchronized and integrated in real time, and dynamic data rendering is provided according
to the user's position and viewpoint.

Figure 4 – Example of T-Leap
4.3 Classification of media data
Table 1 summarizes the classification of key media data types used in services with
heterogeneous media data, along with their representative formats.
Table 1 – Media types
Media type Representative formats
Video H.264|AVC, H.265|HEVC
Image JPEG, PNG
Audio MP3, AAC, Opus
Text ASCII, Unicode ®
Animation
BVH, FBX ®
3D model
glTF™, OBJ, FBX
Point clouds LAS, E57, PCD, PCC
Haptic data No widely adopted standard is currently established.
Motion and orientation data (e.g. No widely adopted standard is currently established.
accelerometer, gyro)
Geospatial data (location, orientation) No widely adopted standard is currently established.
International standards exist for media types that make up conventional multimedia data, such
as video and audio. However, in recent years, advances and diversification in sensor and
display technologies have led to emerging media data — such as sensor-signal data (e.g.
...