ISO/IEC 15938-6:2020

Information technology - Multimedia content description interface - Part 6: Reference software

The ISO/IEC 15938 series operates on and generates conformant bitstreams. This document provides a specific implementation that behaves in a conformant manner. NOTE 1 Other implementations that conform to the ISO/IEC 15938 series are possible, which do not necessarily use the algorithms or the programming techniques of the reference software. The software contained in this document is known as eXperimentation Model (XM) and is divided into five categories: a) Binary format for MPEG-7 (BiM). This software converts DDL (XML) based descriptions to binary format and vice versa as explained in Clause 5. b) DDL parser and DDL validation parser. The components of this software module are specified in Clause 6. c) Visual descriptors. This software creates standard visual descriptions from associated (visual) media content as explained in Clause 7. NOTE 2 The techniques used for extracting descriptors are informative, and the quality and complexity of these extraction tools has not been optimized. d) Audio descriptors. This software creates standard descriptions from associated (audio) media content as explained in Clause 8. NOTE 3 The techniques used for extracting descriptors are informative, and the quality and complexity of these extraction tools has not been optimized. e) Multimedia description schemes. This software modules provide standard descriptions of multimedia description schemes as specified in Clause 9.

Technologies de l'information — Interface de description du contenu multimédia — Partie 6: Logiciel de référence

General Information

• Status: Published
• Publication Date: 03-Jun-2020
• ICS: 35.040 - Information coding
• ICS: 35.040.40 - Coding of audio, video, multimedia and hypermedia information
• Technical Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Drafting Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Current Stage: 9060 - Close of review
• Completion Date: 02-Dec-2030

Relations

• Revises: ISO/IEC 15938-6:2003/Amd 4:2011 - Information technology - Multimedia content description interface - Part 6: Reference software - Amendment 4: Reference software for video signature tools
• Effective Date: 05-Nov-2015
• Revises: ISO/IEC 15938-6:2003/Amd 3:2010 - Information technology - Multimedia content description interface - Part 6: Reference software - Amendment 3: Reference software for image signature tools
• Effective Date: 05-Nov-2015
• Revises: ISO/IEC 15938-6:2003/Amd 2:2007 - Information technology - Multimedia content description interface - Part 6: Reference software - Amendment 2: Reference software of perceptual 3D shape descriptor
• Effective Date: 05-Nov-2015
• Revises: ISO/IEC 15938-6:2003/Amd 1:2006 - Information technology - Multimedia content description interface - Part 6: Reference software - Amendment 1: Reference software extensions
• Effective Date: 04-Nov-2015
• Revises: ISO/IEC 15938-6:2003 - Information technology - Multimedia content description interface - Part 6: Reference software
• Effective Date: 04-Nov-2015

Overview

ISO/IEC 15938-6:2020 is the Part 6 reference software component of the ISO/IEC 15938 series (commonly known as MPEG-7). It publishes a standardized, conformance‑oriented implementation - the eXperimentation Model (XM) - that generates and operates on conformant multimedia description bitstreams. The standard provides source modules, example implementations and tools to produce, validate and transcode MPEG-7 descriptions for multimedia content.

Key keywords: ISO/IEC 15938-6:2020, MPEG-7, reference software, XM, multimedia content description, BiM, DDL, descriptors.

Key Topics and Technical Requirements

• Reference implementation (XM): The document supplies an implementation that behaves in a conformant manner. Other conformant implementations can differ in algorithms and optimizations.
• Five software categories:
  • Binary format for MPEG-7 (BiM): Converts DDL/XML‑based descriptions to an efficient binary bitstream and back.
  • DDL parser and DDL validation: Tools to parse MPEG-7 Description Definition Language (DDL) and validate description instances.
  • Visual descriptors: Example extractors that produce standard visual descriptors from video/image media (informative; not optimized for performance).
  • Audio descriptors: Example extractors for audio features and descriptors (informative; not optimized).
  • Multimedia description schemes (MDS): Software modules providing standard descriptionscheme implementations and sample DS instances.
• XM software architecture: Modular components such as media databases, AV decoders, extraction tools, descriptors/DSs, coding schemes, search & matching tools, transcoders and applications are described to support typical use cases.
• Systems tools and conformance: Includes tools for synchronization, multiplexing, access unit navigation and packaging to produce conformant MPEG-7 bitstreams.
• Legal and reuse notes: Source code includes copyright disclaimers; ISO flags possible patent claims and provides links to patent declarations.

Applications and Who Uses This Standard

• Developers and implementers building MPEG-7 metadata toolchains, converters, or delivery systems.
• Digital asset management and media archival systems that require standard multimedia descriptors for indexing and retrieval.
• Search & retrieval and content‑based multimedia retrieval researchers and vendors using reference extractors and evaluation baselines.
• Broadcast, streaming and synchronization engineers needing BiM binary packaging and synchronization examples.
• Standards testers and interoperability labs using XM to validate conformance of their products against the MPEG-7 suite.

Practical benefits: rapid prototyping, conformance testing, converting XML descriptions to efficient delivery formats, and a common baseline for descriptor extraction and DS usage.

Related Standards

• ISO/IEC 15938 series (MPEG-7): Parts 1–5 cover Systems, Description Definition Language (DDL), Visual, Audio and Multimedia Description Schemes; ISO/IEC 15938-6 provides the corresponding reference software.
• Common terms: BiM, DDL, DS, descriptor, XM, MPEG-7.

For official source code and downloads see: https://standards.iso.org/iso-iec/15938/-6/ed-2/en.

Standards Content (Sample)

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 15938-6
ISO/IEC JTC 1/SC 29 Secretariat: JISC
Voting begins on: Voting terminates on:
2015-10-29 2016-01-29
Information technology — Multimedia content
description interface —
Part 6:
Reference software
Technologies de l’information — Interface de description du contenu multimédia —
Partie 6: Logiciel de référence
ICS: 35.040
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 15938-6:2015(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
©
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2015

ISO/IEC DIS 15938-6:2015(E)
© ISO/IEC 2015, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. [phone]
Fax [phone]
[e-mail]
www.iso.org
ii © ISO/IEC 2015 – All rights reserved

ISO/IEC DIS 15938-6
Contents Page
Foreword . iv
Introduction . v
1 Scope . 1
2 Symbols and abbreviated terms . 1
3 Copyright disclaimer for software modules . 2
4 XM software architecture . 2
4.1 Block diagrams . 2
4.1.1 Media database . 4
4.1.2 AV decoders . 4
4.1.3 Media data . 5
4.1.4 Extraction tools . 5
4.1.5 Descriptors (Ds) and description schemes (DSs) . 6
4.1.6 Coding schemes (CSs) . 8
4.1.7 Search & matching tools . 9
4.1.8 Media transcoders . 11
4.1.9 Applications . 11
4.1.10 Interface structure . 12
5 Systems reference software (BiM) . 12
5.1 BiM reference software . 12
5.1.1 Introduction . 12
5.1.2 Software overview . 13
5.2 Access unit navigator . 14
5.2.1 Introduction . 14
5.2.2 Textual access unit encoder module . 14
5.2.3 Textual access unit decoder module . 15
6 Systems reference software (DDL) . 16
7 Video reference software . 16
8 Audio reference software . 17
9 Multimedia description scheme reference software . 19
10 Compilation of the reference software . 21
11 Usage information for individual descriptors and description schemes . 21
Annex A (informative) Providers of reference software . 22
Annex B (informative) Patent statements . 24

© ISO/IEC 2015 – All rights reserved iii

ISO/IEC DIS 15938-6
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 15938-6 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
This second edition cancels and replaces the first edition of which has been technically revised.
ISO/IEC 15938 consists of the following parts, under the general title Information technology — Multimedia
content description interface:
⎯ Part 1: Systems
⎯ Part 2: Description definition language
⎯ Part 3: Visual
⎯ Part 4: Audio
⎯ Part 5: Multimedia description schemes
⎯ Part 6: Reference software
⎯ Part 7: Conformance testing
⎯ Part 8: Extraction and use of MPEG-7 descriptions
⎯ Part 9: Profiles and levels
⎯ Part 10: Schema definition
⎯ Part 11: MPEG-7 profile schemas
⎯ Part 12: Query format
⎯ Part 13: Compact descriptors for visual search
iv © ISO/IEC 2015 – All rights reserved

ISO/IEC DIS 15938-6
Introduction
This International Standard, also known as "Multimedia Content Description Interface," provides a
standardized set of technologies for describing multimedia content. It addresses a broad spectrum of
multimedia applications and requirements by providing a metadata system for describing the features of
multimedia content.
The following are specified in this International Standard:
Description schemes (DS) describe entities or relationships pertaining to multimedia content. Description
schemes specify the structure and semantics of their components, which may be description schemes,
descriptors, or datatypes.
Descriptors (D) describe features, attributes, or groups of attributes of multimedia content.
Datatypes are the basic reusable datatypes employed by description schemes and descriptors
Systems tools support delivery of descriptions, multiplexing of descriptions with multimedia content,
synchronization, file format, and so forth.
This International Standard is subdivided into 13 parts:
Part 1 – Systems: specifies the tools for preparing descriptions for efficient transport and storage,
compressing descriptions, and allowing synchronization between content and descriptions.
Part 2 – Description definition language: specifies the language for defining the International Standard set
of description tools (DSs, Ds, and datatypes) and for defining new description tools.
Part 3 – Visual: specifies the description tools pertaining to visual content.
Part 4 – Audio: specifies the description tools pertaining to audio content.
Part 5 – Multimedia description schemes: specifies the generic description tools pertaining to multimedia
including audio and visual content.
Part 6 – Reference software: provides a software implementation of the International Standard.
Part 7 – Conformance testing: specifies the guidelines and procedures for testing conformance of
implementations of the International Standard.
Part 8 – Extraction and use of MPEG-7 descriptions: provides guidelines and examples of the extraction
and use of descriptions.
Part 9 – Profiles and levels: provides guidelines and standard profiles.
Part 10 – Schema definition: specifies the schema using description definition language.
Part 11 – Profile Schemas: listing of profile schemas using description definition language.
Part 12 – Query format: contains the tools of the MPEG Query Format (MPQF)
Part 13 – Compact descriptors for visual search: specifies an image description tool for visual search
applications
© ISO/IEC 2015 – All rights reserved v

ISO/IEC DIS 15938-6
This part of ISO/IEC 15938 contains simulation software for tools defined in parts 1, 2, 3, 4 and 5 of ISO/IEC
15938. This software has been derived from the verification models used in the process of developing the
International Standard.
Where multimedia content extraction or multimedia content description software is provided, attention is called
to the fact that these software modules are provided for the purpose of creating bit streams of descriptors and
description schemes with normative syntax. The performance of these software tools should not be taken as
indicative of that which can be obtained from implementations where quality and computational optimization
are given priority. The techniques used for extracting descriptors or deriving description schemes are not
specified by this document. This information can be found in the corresponding sections of part 1-5.

vi © ISO/IEC 2015 – All rights reserved

DRAFT INTERNATIONAL STANDARD ISO/IEC DIS 15938-6

Information technology — Multimedia content description
interface — Part 6: Reference software
1 Scope
This International Standard operates on and generates conformant bitstreams. This International Standard
provides a specific implementation that behaves in a conformant manner. In general, other implementations
that conform to ISO/IEC 15938 are possible that do not necessarily use the algorithms or the programming
techniques of the reference software.
The software contained in this part of ISO/IEC 15938 is known as eXperimentation Model (XM) and is divided
into five categories:
a) Binary format for MPEG-7 (BiM). This software converts DDL (XML) based descriptions to binary format
and vice versa as explained in Clause 5 of this document.
b) DDL parser and DDL validation parser. The components of this software module are specified in Clause 6
of this document.
c) Visual descriptors. This software creates standard visual descriptions from associated (visual) media
content as explained in Clause 7 of this document. The techniques used for extracting descriptors are
informative, and the quality and complexity of these extraction tools has not been optimized.
d) Audio descriptors. This software creates standard descriptions from associated (audio) media content as
explained in Clause 8 of this document. The techniques used for extracting descriptors are informative,
and the quality and complexity of these extraction tools has not been optimized.
e) Multimedia description schemes. This software modules provide standard descriptions of multimedia
description schemes as specified in Clause 9 of this document.
2 Symbols and abbreviated terms
For the purposes of this part of ISO/IEC 15938, the following symbols and abbreviated terms apply:
AV:  Audio-visual
CS:  Coding Scheme
D:  Descriptor
Ds:  Descriptors
DCT:  Discrete Cosine Transform
DDL:  Description Definition Language
DS:  Description Scheme
DSs: Description Schemes
ISO: International Organization for Standardization
MDS:  Multimedia Description Schemes
MPEG:  Moving Picture Experts Group
MPEG-7: Multimedia Content Description Interface Standard (see ISO/IEC 15938)
XML:  Extensible Markup Language
© ISO/IEC 2015 – All rights reserved 1

ISO/IEC DIS 15938-6
3 Copyright disclaimer for software modules
Each source code module in this specification contains a copyright disclaimer which shall not be removed
from the source code module.
In the text of each copyright disclaimer, is replaced with a reference to its associated
specification, e.g. MPEG-7 Systems (ISO/IEC 15938-1), MPEG-7 Visual (ISO/IEC 15938-3), MPEG-7 Audio
(ISO/IEC 15938-4), MPEG-7 Multimedia description schemes (ISO/IEC 15938-5).
"This software module was originally developed by () and edited by
(), (), in the course of development of the . This software
module is an implementation of a part of one or more tools as specified by the standard>. ISO/IEC gives users of the free license to this software module or modifications
thereof for use in hardware or software products claiming conformance to the . Those
intending to use this software module in hardware or software products are advised that its use may infringe
existing patents. The original developer of this software module and his/her company, the subsequent editors
and their companies, and ISO/IEC have no liability for use of this software module or modifications thereof in
an implementation. Copyright is not released for non conforming products. retains
full right to use the code for his/her own purpose, assign or donate the code to a third party and to inhibit third
parties from using the code for non conforming products. This copyright notice must be
20xx-20xx".
included in all copies or derivative works. Copyright
=First Name, =Last Name, =Company Name
4 XM software architecture
4.1 Block diagrams
This section provides information about the XM software architecture. The block diagrams give short
overviews, and introduce individual components of the XM software. This section also provides the directory
locations for the different modules.
The composing elements of the MPEG-7 Reference Software are characterized by their functionality and by
their interfaces. They can be configured according to what here is referred as "Key Applications". We can
distinguish from the functional point of view:
• "Extraction Applications" (a description data base is built from a media data base)
• "Search and Retrieval Applications" (a description is compared with the descriptions in a database to
find the one with the lowest distance)
• "Transcoding Applications" (a media data base is converted into another media data base based on
its description)
2 © ISO/IEC 2015 – All rights reserved

ISO/IEC DIS 15938-6
Figure 1 — Schematic diagram of an "Extraction Application" using the XM reference software
modules, with boxes representing procedural parts and circles representing data structures.

Figure 2 — Schematic diagram of a "Search and Retrieval Application" using the XM reference
software modules, with boxes representing procedural parts and circles representing data structures.

© ISO/IEC 2015 – All rights reserved 3

ISO/IEC DIS 15938-6
Figure 3 — Schematic diagram of a "Transcoding Application" using the XM reference software
modules, with boxes representing procedural parts and circles representing data structures.

In the following, the blocks of the "Key Applications" are distinguished. For elements that are related to
specific descriptors or description schemes, the interface is given using a DummyType example. This
represents the XM integration template and not a normative descriptor or description scheme.
4.1.1 Media database
The media database contains media files, which are supported as input files by the AV decoders. The
database file is read from a file and contains one media filename per line. From this media filename all
additional input and output filenames can be derived.
4.1.2 AV decoders
The XM supports the following AV decoders:
• Still image decoders: ImageMagick (Ver. 6.8.8 linked as external library, not included in the XM
reference software distribution).
• MPEG-1, MPEG-2 video decoders: (XM directory: Decoders/MPEG2Dec).
• MPEG-1 video motion vector extractor: (XM directory: Decoders/MPEG2Dec). This extracts images
and motion vectors.
• 3D Objects: (XM directory: Media). This reads a 3D object for 3D shape descriptors.
4 © ISO/IEC 2015 – All rights reserved

ISO/IEC DIS 15938-6
• Key Points: (XM directory: Media). This reads in a list of key points from a file.
• Audio decoders: (XM directory: Media). They read audio files using external library AFsp version v5r2
from McGill University.
4.1.3 Media data
This is the internal XM representation of the raw media data (one class with different structures depending on
the media content type). The class description for media data can be found in the Media XM directory.
4.1.4 Extraction tools
Extraction tools are specific extraction methods defined for each descriptor and description scheme. All the
source files are available in the ExtractionUtilities XM directory. The extraction tools extract the descriptions
from media data. Because media data can be of significant size, the extraction is performed on time entities of
the media, i.e., if the media is a video the extraction is done frame by frame. Some of the extraction tools may
need OpenCV linked with XM as an external library. The interface of the DummyType extraction tool
(implementation template) is given below:

//=============================================================================
class DummyTypeExtractionTool: public DescriptorExtractor
{
friend DummyTypeExtractionInterface;
public:
// Null constructor
DummyTypeExtractionTool();
// Also connects the Descriptor (result memory) to the extraction
// If set to "0" it automatically creates the descriptor
DummyTypeExtractionTool(DummyTypeDescriptorInterfaceABC
*DummyType);
// ID of object type
virtual const UUID& GetObjectID(void);
// Object type name
virtual const char *GetName(void);

// This informs the extractor where the source data comes from
virtual int SetSourceMedia(MultiMediaInterfaceABC* media);

// Pointer where the description is stored
virtual DummyTypeDescriptorInterfaceABC*
GetDescriptorInterface(void);
virtual int SetDescriptorInterface(DummyTypeDescriptorInterfaceABC
*aDummyTypeDescriptorInterface);

// initililaize descriptor and extraction process (input media must be known)
virtual unsigned long InitExtracting(void);

// performs extraction form input media frame by input media frame
virtual unsigned long StartExtracting(void);

// collects descriptor data after all input media frames were processed
virtual unsigned long PostExtracting(void);

© ISO/IEC 2015 – All rights reserved 5

ISO/IEC DIS 15938-6
// Extraction object must no be used, only its interface is allowd to
// to be used. This function is to get the interface
virtual DummyTypeExtractionInterfaceABC *GetInterface(void);

// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
private:
// Destructor is private to allow creation of
// object only by using "new"
virtual ~DummyTypeExtractionTool();

DummyTypeExtractionInterface m_Interface;
DummyTypeDescriptorInterfaceABC *m_DescriptorInterface;
MultiMediaInterfaceABC* m_Media;

// only used in this dummy type to show extraction function
unsigned long m_FrameCnt;
#ifdef __HasSubTypes /*include this section if sub descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeAExtractionInterfaceABC *m_SubDummyTypeAExtraction;
SubDummyTypeBExtractionInterfaceABC *m_SubDummyTypeBExtraction;
#endif /* __HasSubTypes*/
int m_DummyExtractionParameter;
}; // End class
//=============================================================================

4.1.5 Descriptors (Ds) and description schemes (DSs)
These modules implement the data structure of normative descriptors and description schemes. Low level
Video descriptors are using a dedicated C++ class. This class provides methods to access the elements of the
normative descriptions. The source files are located in the Descriptors directory. All other normative Ds and
DSs are using the GenericDS class located in the DescriptionSchemes directory. The GenericDS class does
not implement the data structure in a dedicated way, but it is an interface to the XML parser library which
controls the memory for the tree structure of the instantiated D or DS. The interface of the descriptors class is
given for the DummyType descriptor (implementation template) below:

/=============================================================================
class DummyTypeDescriptor: public Descriptor
{
friend DummyTypeDescriptorInterface;
public:
DummyTypeDescriptor();
#ifdef __HasSubTypes /*include this section if sub descriptors exist,
remove this section if no sub-descriptors exist*/
// constructor which also constructs and/or connects the descriptor object
DummyTypeDescriptor(SubDummyTypeADecriptorInterfaceABC *aSubDummyTypeA,
SubDummyTypeBDescriptorInterfaceABC *aSubDummyTypeB);
6 © ISO/IEC 2015 – All rights reserved

ISO/IEC DIS 15938-6
#endif /* __HasSubTypes*/
virtual const UUID& GetValueID(void);
virtual const char* GetValueName(void);

virtual const UUID& GetObjectID(void);
virtual const char *GetName(void);

// for reference counting
virtual void addref();
virtual void release();
#ifdef __HasSubTypes /*include this section if sub descriptors exist,
remove this section if no sub-descriptors exist*/
/* only needed for manual connection with sub components*/
virtual SubDummyTypeADescriptorInterfaceABC
*GetSubDummyTypeADescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeADescriptorInterface(SubDummyTypeADescriptorInterfaceABC
*aSubDummyTypeADescriptorInterface);

virtual SubDummyTypeBDescriptorInterfaceABC
*GetSubDummyTypeBDescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeBDescriptorInterface(SubDummyTypeBDescriptorInterfaceABC
*aSubDummyTypeBDescriptorInterface);
#endif /* __HasSubTypes*/
// actual descriptor methods, only in this dummy type example
virtual long GetDummyContents(void);
virtual void SetDummyContents(const long val);

// transformation to GenericDS object (MDS implementaion style)
virtual unsigned long
ExportDDL(GenericDSInterfaceABC *aParentDescription);
virtual unsigned long ImportDDL(GenericDSInterfaceABC *aDescription);

// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
virtual DummyTypeDescriptorInterfaceABC *GetInterface(void);

private:
// private destructor to force reference counting mechanism
virtual ~DummyTypeDescriptor();

// reference counter
unsigned long m_refcount;
DummyTypeDescriptorInterface m_Interface;

const bool m_isProprietary;
static const char * valName;
© ISO/IEC 2015 – All rights reserved 7

ISO/IEC DIS 15938-6
static const UUID valID;
#ifdef __HasSubTypes /*include this section if sub descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeADescriptorInterfaceABC *m_SubDummyTypeADescriptorInterface;
SubDummyTypeBDescriptorInterfaceABC *m_SubDummyTypeBDescriptorInterface;
#endif /* __HasSubTypes*/
// This is the actual data the D/DSType stores. In this particular
// dummy example it's just a signed long called m_DummyContents
long m_DummyContents;
};
/=============================================================================

4.1.6 Coding schemes (CSs)
Coding schemes are specific coding and decoding methods defined for individual descriptors (Ds) and
description schemes (DSs). All the source files are available in the CodingSchemes directory. If an individual
coding scheme is available, it represents a normative part of the standard. Coding schemes are available for
the visual descriptors to encode or to decode a description into its binary representation. The alternative
implementation of the coding scheme box, which is available for all Ds and DSs, allows encoding and
decoding of descriptors into its DDL representation using the GenericDSCS which is an interface to the "write
to file"- and "read from file"-functions of XML Xerces-C++ external parser library. The interface of the coding
schemes is given below on the example of the DummyType coding scheme (implementation template):

//=============================================================================
class DummyTypeCS: public DescriptionCodingEngine
{
friend DummyTypeCSInterface;
public:
DummyTypeCS();
// constructor which also constructs and/or connects the descriptor object
DummyTypeCS(DummyTypeDescriptorInterfaceABC
*DummyType);
virtual const UUID& GetValueID(void);
virtual const char* GetValueName(void);

virtual const UUID& GetObjectID(void);
virtual const char *GetName(void);

// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
virtual DummyTypeCSInterfaceABC *GetInterface(void);

// accessor methods
virtual EncoderFileIO *GetEncoderStreamBuffer(void);
8 © ISO/IEC 2015 – All rights reserved

ISO/IEC DIS 15938-6
virtual int SetEncoderStreamBuffer(EncoderFileIO *aBuffer);
virtual DecoderFileIO *GetDecoderStreamBuffer(void);
virtual int SetDecoderStreamBuffer(DecoderFileIO *aBuffer);

virtual DummyTypeDescriptorInterfaceABC
*GetDescriptorInterface(void);
virtual int SetDescriptorInterface(DummyTypeDescriptorInterfaceABC
*aDummyTypeDescriptorInterface);

//this function writes the description via the encoder buffer to a file
virtual int StartEncode();
//this function reads the description via the decoder buffer from a file
virtual int StartDecode();
pr
...

INTERNATIONAL ISO/IEC
STANDARD 15938-6
Second edition
2020-06
Information technology — Multimedia
content description interface —
Part 6:
Reference software
Technologies de l'information — Interface de description du contenu
multimédia —
Partie 6: Logiciel de référence
Reference number
©
ISO/IEC 2020
© ISO/IEC 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Fax: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 Copyright disclaimer for software modules . 2
6 XM software architecture . 2
6.1 Block diagrams . 2
6.2 Block descriptions . 4
6.2.1 General. 4
6.2.2 Media database . 4
6.2.3 AV decoders . 4
6.2.4 Media data . 5
6.2.5 Extraction tools . 5
6.2.6 Descriptors (Ds) and description schemes (DSs) . 6
6.2.7 Coding schemes (CSs) . . 7
6.2.8 Search and matching tools. 8
6.2.9 Media transcoders .10
6.2.10 Applications .10
6.2.11 Interface structure.10
7 Systems reference software (BiM) .11
7.1 General .11
7.2 BiM reference software .11
7.2.1 General.11
7.2.2 Package content .11
7.2.3 Installation and execution .11
7.2.4 Software overview .12
7.2.5 Main packages .12
7.3 Access unit navigator .12
7.3.1 General.12
7.3.2 Textual access unit encoder module .13
7.3.3 Textual access unit decoder module .13
8 Description definition language (DDL) reference software .14
9 Video reference software .14
10 Audio reference software .15
11 Multimedia description scheme reference software .17
12 Compilation of the reference software .18
13 Usage information for individual descriptors and description schemes .18
© ISO/IEC 2020 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see http:// patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
This second edition cancels and replaces the first edition (ISO/IEC 15938-6:2003), which
has been technically revised. It also incorporates the Amendments ISO/IEC 15938-6:2003/
Amd.1:2006, ISO/IEC 15938-6:2003/Amd.1:2006/Cor.1:2007, ISO/IEC 15938-6:2003/Amd.2:2007,
ISO/IEC 15938-6:2003/Amd.3:2010 and ISO/IEC 15938-6:2003/Amd.4:2011. The main changes
compared to the previous edition are as follows:
— all previous Amendments have been incorporated and the electronic attachments have been
updated;
— minor editorial corrections have been made throughout the document to fully align with ISO/IEC
Directives.
A list of all parts in the ISO/IEC 15938 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO/IEC 2020 – All rights reserved

Introduction
This document provides a standardized set of technologies for describing multimedia content. It
addresses a broad spectrum of multimedia applications and requirements by providing a metadata
system for describing the features of multimedia content.
The software is available at: https:// standards .iso .org/ iso -iec/ 15938/ -6/ ed -2/ en.
The following are specified in this document:
Description schemes (DS) describe entities or relationships pertaining to multimedia content.
Description schemes specify the structure and semantics of their components, which can be description
schemes, descriptors, or datatypes.
Descriptors (D) describe features, attributes, or groups of attributes of multimedia content.
Datatypes are the basic reusable datatypes employed by description schemes and descriptors.
Systems tools support delivery of descriptions, multiplexing of descriptions with multimedia content,
synchronization, file format, and so forth.
This document contains simulation software for tools defined in ISO/IEC 15938-1, ISO/IEC 15938-2,
ISO/IEC 15938-3, ISO/IEC 15938-4 and ISO/IEC 15938-5. This software has been derived from the
verification models used in the process of developing this series.
Where multimedia content extraction or multimedia content description software is provided, attention
is called to the fact that these software modules are provided for the purpose of creating bitstreams of
descriptors and description schemes with normative syntax. The performance of these software tools
is not indicative of that which can be obtained from implementations where quality and computational
optimization are given priority. The techniques used for extracting descriptors or deriving description
schemes are not specified by this document. This information can be found in the corresponding Part of
the ISO/IEC 15938 series.
The International Organization for Standardization (ISO) and International Electrotechnical
Commission (IEC) draw attention to the fact that it is claimed that compliance with this document may
involve the use of patents.
ISO and IEC take no position concerning the evidence, validity and scope of these patent rights.
The holders of these patent rights have assured ISO and IEC that they are willing to negotiate licences
under reasonable and non-discriminatory terms and conditions with applicants throughout the world.
In this respect, the statements of the holders of these patent rights are registered with ISO and IEC.
Information may be obtained from the patent database available at www .iso .org/ patents.
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights other than those in the patent database. ISO and IEC shall not be held responsible for
identifying any or all such patent rights.
© ISO/IEC 2020 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 15938-6:2020(E)
Information technology — Multimedia content description
interface —
Part 6:
Reference software
1 Scope
The ISO/IEC 15938 series operates on and generates conformant bitstreams. This document provides a
specific implementation that behaves in a conformant manner.
NOTE 1 Other implementations that conform to the ISO/IEC 15938 series are possible, which do not necessarily
use the algorithms or the programming techniques of the reference software.
The software contained in this document is known as eXperimentation Model (XM) and is divided into
five categories:
a) Binary format for MPEG-7 (BiM). This software converts DDL (XML) based descriptions to binary
format and vice versa as explained in Clause 5.
b) DDL parser and DDL validation parser. The components of this software module are specified in
Clause 6.
c) Visual descriptors. This software creates standard visual descriptions from associated (visual)
media content as explained in Clause 7.
NOTE 2 The techniques used for extracting descriptors are informative, and the quality and complexity
of these extraction tools has not been optimized.
d) Audio descriptors. This software creates standard descriptions from associated (audio) media
content as explained in Clause 8.
NOTE 3 The techniques used for extracting descriptors are informative, and the quality and complexity
of these extraction tools has not been optimized.
e) Multimedia description schemes. This software modules provide standard descriptions of
multimedia description schemes as specified in Clause 9.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
No terms and definitions are listed in this document.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
© ISO/IEC 2020 – All rights reserved 1

4 Abbreviated terms
AV audio-visual
CS coding scheme
D descriptor
Ds descriptors
DCT discrete cosine transform
DDL description definition language
DS description scheme
DSs description schemes
MDS multimedia description schemes
MPEG moving picture experts group
MPEG-7 multimedia content description interface standard (the ISO/IEC 15938 series)
XML eXtensible Markup Language
5 Copyright disclaimer for software modules
The source code for this document can be found at https:// standards .iso .org/ iso -iec/ 15938/ -6/ ed -2/ en/ .
Each source code module in this document contains a copyright disclaimer which shall not be removed
from the source code module.
In the text of each copyright disclaimer, is replaced with a reference to its associated
specification, e.g. MPEG-7 Systems (ISO/IEC 15938-1), MPEG-7 Visual (ISO/IEC 15938-3), MPEG-7 Audio
(ISO/IEC 15938-4), MPEG-7 Multimedia description schemes (ISO/IEC 15938-5).
"This software module was originally developed by () and edited by
(), (), in the course of development of the . This software
module is an implementation of a part of one or more tools as specified by the
. ISO/IEC gives users of the free license to this software module
or modifications thereof for use in hardware or software products claiming conformance to the standard>. Those intending to use this software module in hardware or software products are advised
that its use may infringe existing patents. The original developer of this software module and his/her
company, the subsequent editors and their companies, and ISO/IEC have no liability for use of this software
module or modifications thereof in an implementation. Copyright is not released for non
conforming products. retains full right to use the code for his/her own purpose, assign or donate the
code to a third party and to inhibit third parties from using the code for non conforming
products. This copyright notice must be included in all copies or derivative works. Copyright ©20xx-20xx".
=First Name, =Last Name, =Company Name
6 XM software architecture
6.1 Block diagrams
This clause provides information about the XM software architecture. The block diagrams give short
overviews and introduce individual components of the XM software.
2 © ISO/IEC 2020 – All rights reserved

The composing elements of the MPEG-7 reference software are characterized by their functionality and
by their interfaces. They can be configured according to what here is referred as "key applications".
From a functional point of view, they can be distinguished as follows:
— "extraction applications" (a description data base is built from a media database) as illustrated in
Figure 1;
— "search and retrieval applications" (a description is compared with the descriptions in a database to
find the one with the lowest distance) as illustrated in Figure 2;
— "transcoding applications" (a media data base is converted into another media database based on
its description) as illustrated in Figure 3.
Figure 1 — An "extraction application" using the XM reference software modules, with boxes
representing procedural parts and circles representing data structures
Figure 2 — A "search and retrieval application" using the XM reference software modules, with
boxes representing procedural parts and circles representing data structures
© ISO/IEC 2020 – All rights reserved 3

Figure 3 — A "transcoding application" using the XM reference software modules, with boxes
representing procedural parts and circles representing data structures
6.2 Block descriptions
6.2.1 General
In subclauses 6.2.2 through 6.2.11, the blocks of the "key applications" are distinguished. For
elements that are related to specific descriptors or description schemes, the interface is given using a
DummyType example. This represents the XM integration template and not a normative descriptor or
description scheme.
6.2.2 Media database
The media database contains media files, which are supported as input files by the AV decoders. The
database file is read from a file and contains one media filename per line. From this media filename all
additional input and output filenames can be derived.
6.2.3 AV decoders
The XM supports the following AV decoders:
1)
— Still image decoders: ImageMagick® (Ver. 6.8.8 linked as external library, not included in the XM
reference software distribution).
— MPEG-1, MPEG-2 video decoders: (XM directory: Decoders/MPEG2Dec).
1) ImageMagick® is the trademark of a product supplied by ImageMagick Studio LLC. This information is given
for the convenience of users of this document and does not constitute an endorsement by ISO or IEC of the product
named. Equivalent products may be used if they can be shown to lead to the same results.
4 © ISO/IEC 2020 – All rights reserved

— MPEG-1 video motion vector extractor: (XM directory: Decoders/MPEG2Dec). This extracts images
and motion vectors.
— 3D Objects: (XM directory: Media). This reads a 3D object for 3D shape descriptors.
— Key Points: (XM directory: Media). This reads in a list of key points from a file.
— Audio decoders: (XM directory: Media). They read audio files.
6.2.4 Media data
This is the internal XM representation of the raw media data (one class with different structures
depending on the media content type). The class description for media data can be found in the Media
XM directory.
6.2.5 Extraction tools
Extraction tools are specific extraction methods defined for each descriptor and description scheme.
All the source files are available in the ExtractionUtilities XM directory. The extraction tools extract the
descriptions from media data. Because media data can be of significant size, the extraction is performed
on time entities of the media, i.e., if the media is a video the extraction is done frame by frame. Some
of the extraction tools may need OpenCV linked with XM as an external library. The interface of the
DummyType extraction tool (implementation template) is:
//=============================================================================
class DummyTypeExtractionTool: public DescriptorExtractor
{
friend DummyTypeExtractionInterface;
public:
// Null constructor
DummyTypeExtractionTool();
// Also connects the Descriptor (result memory) to the extraction
// If set to "0" it automatically creates the descriptor
DummyTypeExtractionTool(DummyTypeDescriptorInterfaceABC
*DummyType);
// ID of object type
virtual const UUID& GetObjectID(void);
// Object type name
virtual const char *GetName(void);

// This informs the extractor where the source data comes from
virtual int SetSourceMedia(MultiMediaInterfaceABC* media);

// Pointer where the description is stored
virtual DummyTypeDescriptorInterfaceABC*
GetDescriptorInterface(void);
virtual int SetDescriptorInterface(DummyTypeDescriptorInterfaceABC
*aDummyTypeDescriptorInterface);

// initialize descriptor and extraction process (input media must be known)
virtual unsigned long InitExtracting(void);

// performs extraction from input media frame by input media frame
virtual unsigned long StartExtracting(void);

// collects descriptor data after all input media frames were processed
virtual unsigned long PostExtracting(void);

// Extraction object must not be used, only its interface is allowed
// to be used. This function is to get the interface
virtual DummyTypeExtractionInterfaceABC *GetInterface(void);

// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
© ISO/IEC 2020 – All rights reserved 5

// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
private:
// Destructor is private to allow creation of
// object only by using "new"
virtual ~DummyTypeExtractionTool();

DummyTypeExtractionInterface m_Interface;
DummyTypeDescriptorInterfaceABC *m_DescriptorInterface;
MultiMediaInterfaceABC* m_Media;

// only used in this dummy type to show extraction function
unsigned long m_FrameCnt;
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeAExtractionInterfaceABC *m_SubDummyTypeAExtraction;
SubDummyTypeBExtractionInterfaceABC *m_SubDummyTypeBExtraction;
#endif /* __HasSubTypes*/
int m_DummyExtractionParameter;
}; // End class
//=============================================================================

6.2.6 Descriptors (Ds) and description schemes (DSs)
These modules implement the data structure of normative descriptors and description schemes. Low level
video descriptors are using a dedicated C++ class. This class provides methods to access the elements of
the normative descriptions. The source files are located in the descriptors directory. All other normative
Ds and DSs are using the GenericDS class located in the DescriptionSchemes directory. The GenericDS
class does not implement the data structure in a dedicated way, but it is an interface to the XML parser
library which controls the memory for the tree structure of the instantiated D or DS. The interface of the
descriptors class is given for the DummyType descriptor (implementation template) below:
/=============================================================================
class DummyTypeDescriptor: public Descriptor
{
friend DummyTypeDescriptorInterface;
public:
DummyTypeDescriptor();
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
// constructor which also constructs and/or connects the descriptor object
DummyTypeDescriptor(SubDummyTypeADecriptorInterfaceABC *aSubDummyTypeA,
SubDummyTypeBDescriptorInterfaceABC *aSubDummyTypeB);
#endif /* __HasSubTypes*/
virtual const UUID& GetValueID(void);
virtual const char* GetValueName(void);

virtual const UUID& GetObjectID(void);
virtual const char *GetName(void);

// for reference counting
virtual void addref();
virtual void release();
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
/* only needed for manual connection with sub components*/
virtual SubDummyTypeADescriptorInterfaceABC
*GetSubDummyTypeADescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeADescriptorInterface(SubDummyTypeADescriptorInterfaceABC
*aSubDummyTypeADescriptorInterface);

virtual SubDummyTypeBDescriptorInterfaceABC
6 © ISO/IEC 2020 – All rights reserved

*GetSubDummyTypeBDescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeBDescriptorInterface(SubDummyTypeBDescriptorInterfaceABC
*aSubDummyTypeBDescriptorInterface);
#endif /* __HasSubTypes*/
// actual descriptor methods, only in this dummy type example
virtual long GetDummyContents(void);
virtual void SetDummyContents(const long val);

// transformation to GenericDS object (MDS implementaion style)
virtual unsigned long
ExportDDL(GenericDSInterfaceABC *aParentDescription);
virtual unsigned long ImportDDL(GenericDSInterfaceABC *aDescription);

// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
virtual DummyTypeDescriptorInterfaceABC *GetInterface(void);

private:
// private destructor to force reference counting mechanism
virtual ~DummyTypeDescriptor();

// reference counter
unsigned long m_refcount;
DummyTypeDescriptorInterface m_Interface;

const bool m_isProprietary;
static const char * valName;
static const UUID valID;
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeADescriptorInterfaceABC *m_SubDummyTypeADescriptorInterface;
SubDummyTypeBDescriptorInterfaceABC *m_SubDummyTypeBDescriptorInterface;
#endif /* __HasSubTypes*/
// This is the actual data the D/DSType stores. In this particular
// dummy example it's just a signed long called m_DummyContents
long m_DummyContents;
};
/=============================================================================

6.2.7 Coding schemes (CSs)
Coding schemes are specific coding and decoding methods defined for individual descriptors (Ds)
and description schemes (DSs). All the source files are available in the CodingSchemes directory. All
normative coding schemes are located in the CodingSchemes directory. Coding schemes are available
for the visual descriptors to encode or to decode a description into its binary representation. The
alternative implementation of the coding scheme box, which is available for all Ds and DSs, allows
encoding and decoding of descriptors into its DDL representation using the GenericDSCS which is an
interface to the "write to file" and "read from
...