Information technology — Multimedia content description interface — Part 6: Reference software

The ISO/IEC 15938 series operates on and generates conformant bitstreams. This document provides a specific implementation that behaves in a conformant manner. NOTE 1 Other implementations that conform to the ISO/IEC 15938 series are possible, which do not necessarily use the algorithms or the programming techniques of the reference software. The software contained in this document is known as eXperimentation Model (XM) and is divided into five categories: a) Binary format for MPEG-7 (BiM). This software converts DDL (XML) based descriptions to binary format and vice versa as explained in Clause 5. b) DDL parser and DDL validation parser. The components of this software module are specified in Clause 6. c) Visual descriptors. This software creates standard visual descriptions from associated (visual) media content as explained in Clause 7. NOTE 2 The techniques used for extracting descriptors are informative, and the quality and complexity of these extraction tools has not been optimized. d) Audio descriptors. This software creates standard descriptions from associated (audio) media content as explained in Clause 8. NOTE 3 The techniques used for extracting descriptors are informative, and the quality and complexity of these extraction tools has not been optimized. e) Multimedia description schemes. This software modules provide standard descriptions of multimedia description schemes as specified in Clause 9.

Technologies de l'information — Interface de description du contenu multimédia — Partie 6: Logiciel de référence

General Information

Status
Published
Publication Date
03-Jun-2020
Current Stage
6060 - International Standard published
Completion Date
04-Jun-2020
Ref Project

RELATIONS

Buy Standard

Standard
ISO/IEC 15938-6:2020 - Information technology -- Multimedia content description interface
English language
19 pages
sale 15% off
Preview
sale 15% off
Preview
Standard
ISO/IEC 15938-6:2020 - Information technology -- Multimedia content description interface
English language
19 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

INTERNATIONAL ISO/IEC
STANDARD 15938-6
Second edition
2020-06
Information technology — Multimedia
content description interface —
Part 6:
Reference software
Technologies de l'information — Interface de description du contenu
multimédia —
Partie 6: Logiciel de référence
Reference number
ISO/IEC 15938-6:2020(E)
ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC 15938-6:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 15938-6:2020(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction ..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Abbreviated terms .............................................................................................................................................................................................. 2

5 Copyright disclaimer for software modules ............................................................................................................................ 2

6 XM software architecture ............................................................................................................................................................................ 2

6.1 Block diagrams ........................................................................................................................................................................................ 2

6.2 Block descriptions ................................................................................................................................................................................ 4

6.2.1 General...................................................................................................................................................................................... 4

6.2.2 Media database ................................................................................................................................................................. 4

6.2.3 AV decoders .......................................................................................................................................................................... 4

6.2.4 Media data ............................................................................................................................................................................. 5

6.2.5 Extraction tools ................................................................................................................................................................. 5

6.2.6 Descriptors (Ds) and description schemes (DSs) ................................................................................ 6

6.2.7 Coding schemes (CSs) ........................................................................................................................................... ....... 7

6.2.8 Search and matching tools....................................................................................................................................... 8

6.2.9 Media transcoders .......................................................................................................................................................10

6.2.10 Applications ......................................................................................................................................................................10

6.2.11 Interface structure.......................................................................................................................................................10

7 Systems reference software (BiM) ..................................................................................................................................................11

7.1 General ........................................................................................................................................................................................................11

7.2 BiM reference software .................................................................................................................................................................11

7.2.1 General...................................................................................................................................................................................11

7.2.2 Package content .............................................................................................................................................................11

7.2.3 Installation and execution ....................................................................................................................................11

7.2.4 Software overview .......................................................................................................................................................12

7.2.5 Main packages .................................................................................................................................................................12

7.3 Access unit navigator ......................................................................................................................................................................12

7.3.1 General...................................................................................................................................................................................12

7.3.2 Textual access unit encoder module ............................................................................................................13

7.3.3 Textual access unit decoder module ............................................................................................................13

8 Description definition language (DDL) reference software .................................................................................14

9 Video reference software ..........................................................................................................................................................................14

10 Audio reference software .........................................................................................................................................................................15

11 Multimedia description scheme reference software ...................................................................................................17

12 Compilation of the reference software .......................................................................................................................................18

13 Usage information for individual descriptors and description schemes ...............................................18

© ISO/IEC 2020 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 15938-6:2020(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that

are members of ISO or IEC participate in the development of International Standards through

technical committees established by the respective organization to deal with particular fields of

technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other

international organizations, governmental and non-governmental, in liaison with ISO and IEC, also

take part in the work.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for

the different types of document should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

list of patent declarations received (see http:// patents .iec .ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/

iso/ foreword .html.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.

This second edition cancels and replaces the first edition (ISO/IEC 15938-6:2003), which

has been technically revised. It also incorporates the Amendments ISO/IEC 15938-6:2003/

Amd.1:2006, ISO/IEC 15938-6:2003/Amd.1:2006/Cor.1:2007, ISO/IEC 15938-6:2003/Amd.2:2007,

ISO/IEC 15938-6:2003/Amd.3:2010 and ISO/IEC 15938-6:2003/Amd.4:2011. The main changes

compared to the previous edition are as follows:

— all previous Amendments have been incorporated and the electronic attachments have been

updated;

— minor editorial corrections have been made throughout the document to fully align with ISO/IEC

Directives.
A list of all parts in the ISO/IEC 15938 series can be found on the ISO website.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO/IEC 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 15938-6:2020(E)
Introduction

This document provides a standardized set of technologies for describing multimedia content. It

addresses a broad spectrum of multimedia applications and requirements by providing a metadata

system for describing the features of multimedia content.

The software is available at: https:// standards .iso .org/ iso -iec/ 15938/ -6/ ed -2/ en.

The following are specified in this document:

Description schemes (DS) describe entities or relationships pertaining to multimedia content.

Description schemes specify the structure and semantics of their components, which can be description

schemes, descriptors, or datatypes.

Descriptors (D) describe features, attributes, or groups of attributes of multimedia content.

Datatypes are the basic reusable datatypes employed by description schemes and descriptors.

Systems tools support delivery of descriptions, multiplexing of descriptions with multimedia content,

synchronization, file format, and so forth.

This document contains simulation software for tools defined in ISO/IEC 15938-1, ISO/IEC 15938-2,

ISO/IEC 15938-3, ISO/IEC 15938-4 and ISO/IEC 15938-5. This software has been derived from the

verification models used in the process of developing this series.

Where multimedia content extraction or multimedia content description software is provided, attention

is called to the fact that these software modules are provided for the purpose of creating bitstreams of

descriptors and description schemes with normative syntax. The performance of these software tools

is not indicative of that which can be obtained from implementations where quality and computational

optimization are given priority. The techniques used for extracting descriptors or deriving description

schemes are not specified by this document. This information can be found in the corresponding Part of

the ISO/IEC 15938 series.

The International Organization for Standardization (ISO) and International Electrotechnical

Commission (IEC) draw attention to the fact that it is claimed that compliance with this document may

involve the use of patents.

ISO and IEC take no position concerning the evidence, validity and scope of these patent rights.

The holders of these patent rights have assured ISO and IEC that they are willing to negotiate licences

under reasonable and non-discriminatory terms and conditions with applicants throughout the world.

In this respect, the statements of the holders of these patent rights are registered with ISO and IEC.

Information may be obtained from the patent database available at www .iso .org/ patents.

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights other than those in the patent database. ISO and IEC shall not be held responsible for

identifying any or all such patent rights.
© ISO/IEC 2020 – All rights reserved v
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 15938-6:2020(E)
Information technology — Multimedia content description
interface —
Part 6:
Reference software
1 Scope

The ISO/IEC 15938 series operates on and generates conformant bitstreams. This document provides a

specific implementation that behaves in a conformant manner.

NOTE 1 Other implementations that conform to the ISO/IEC 15938 series are possible, which do not necessarily

use the algorithms or the programming techniques of the reference software.

The software contained in this document is known as eXperimentation Model (XM) and is divided into

five categories:

a) Binary format for MPEG-7 (BiM). This software converts DDL (XML) based descriptions to binary

format and vice versa as explained in Clause 5.

b) DDL parser and DDL validation parser. The components of this software module are specified in

Clause 6.

c) Visual descriptors. This software creates standard visual descriptions from associated (visual)

media content as explained in Clause 7.

NOTE 2 The techniques used for extracting descriptors are informative, and the quality and complexity

of these extraction tools has not been optimized.

d) Audio descriptors. This software creates standard descriptions from associated (audio) media

content as explained in Clause 8.

NOTE 3 The techniques used for extracting descriptors are informative, and the quality and complexity

of these extraction tools has not been optimized.

e) Multimedia description schemes. This software modules provide standard descriptions of

multimedia description schemes as specified in Clause 9.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
No terms and definitions are listed in this document.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
© ISO/IEC 2020 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC 15938-6:2020(E)
4 Abbreviated terms
AV audio-visual
CS coding scheme
D descriptor
Ds descriptors
DCT discrete cosine transform
DDL description definition language
DS description scheme
DSs description schemes
MDS multimedia description schemes
MPEG moving picture experts group

MPEG-7 multimedia content description interface standard (the ISO/IEC 15938 series)

XML eXtensible Markup Language
5 Copyright disclaimer for software modules

The source code for this document can be found at https:// standards .iso .org/ iso -iec/ 15938/ -6/ ed -2/ en/ .

Each source code module in this document contains a copyright disclaimer which shall not be removed

from the source code module.

In the text of each copyright disclaimer, is replaced with a reference to its associated

specification, e.g. MPEG-7 Systems (ISO/IEC 15938-1), MPEG-7 Visual (ISO/IEC 15938-3), MPEG-7 Audio

(ISO/IEC 15938-4), MPEG-7 Multimedia description schemes (ISO/IEC 15938-5).

"This software module was originally developed by () and edited by

(), (), in the course of development of the . This software

module is an implementation of a part of one or more tools as specified by the

. ISO/IEC gives users of the free license to this software module

or modifications thereof for use in hardware or software products claiming conformance to the

standard>. Those intending to use this software module in hardware or software products are advised

that its use may infringe existing patents. The original developer of this software module and his/her

company, the subsequent editors and their companies, and ISO/IEC have no liability for use of this software

module or modifications thereof in an implementation. Copyright is not released for non

conforming products. retains full right to use the code for his/her own purpose, assign or donate the

code to a third party and to inhibit third parties from using the code for non conforming

products. This copyright notice must be included in all copies or derivative works. Copyright ©20xx-20xx".

=First Name, =Last Name, =Company Name
6 XM software architecture
6.1 Block diagrams

This clause provides information about the XM software architecture. The block diagrams give short

overviews and introduce individual components of the XM software.
2 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 15938-6:2020(E)

The composing elements of the MPEG-7 reference software are characterized by their functionality and

by their interfaces. They can be configured according to what here is referred as "key applications".

From a functional point of view, they can be distinguished as follows:

— "extraction applications" (a description data base is built from a media database) as illustrated in

Figure 1;

— "search and retrieval applications" (a description is compared with the descriptions in a database to

find the one with the lowest distance) as illustrated in Figure 2;

— "transcoding applications" (a media data base is converted into another media database based on

its description) as illustrated in Figure 3.

Figure 1 — An "extraction application" using the XM reference software modules, with boxes

representing procedural parts and circles representing data structures

Figure 2 — A "search and retrieval application" using the XM reference software modules, with

boxes representing procedural parts and circles representing data structures
© ISO/IEC 2020 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC 15938-6:2020(E)

Figure 3 — A "transcoding application" using the XM reference software modules, with boxes

representing procedural parts and circles representing data structures
6.2 Block descriptions
6.2.1 General

In subclauses 6.2.2 through 6.2.11, the blocks of the "key applications" are distinguished. For

elements that are related to specific descriptors or description schemes, the interface is given using a

DummyType example. This represents the XM integration template and not a normative descriptor or

description scheme.
6.2.2 Media database

The media database contains media files, which are supported as input files by the AV decoders. The

database file is read from a file and contains one media filename per line. From this media filename all

additional input and output filenames can be derived.
6.2.3 AV decoders
The XM supports the following AV decoders:

— Still image decoders: ImageMagick® (Ver. 6.8.8 linked as external library, not included in the XM

reference software distribution).
— MPEG-1, MPEG-2 video decoders: (XM directory: Decoders/MPEG2Dec).

1) ImageMagick® is the trademark of a product supplied by ImageMagick Studio LLC. This information is given

for the convenience of users of this document and does not constitute an endorsement by ISO or IEC of the product

named. Equivalent products may be used if they can be shown to lead to the same results.

4 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 15938-6:2020(E)

— MPEG-1 video motion vector extractor: (XM directory: Decoders/MPEG2Dec). This extracts images

and motion vectors.

— 3D Objects: (XM directory: Media). This reads a 3D object for 3D shape descriptors.

— Key Points: (XM directory: Media). This reads in a list of key points from a file.

— Audio decoders: (XM directory: Media). They read audio files.
6.2.4 Media data

This is the internal XM representation of the raw media data (one class with different structures

depending on the media content type). The class description for media data can be found in the Media

XM directory.
6.2.5 Extraction tools

Extraction tools are specific extraction methods defined for each descriptor and description scheme.

All the source files are available in the ExtractionUtilities XM directory. The extraction tools extract the

descriptions from media data. Because media data can be of significant size, the extraction is performed

on time entities of the media, i.e., if the media is a video the extraction is done frame by frame. Some

of the extraction tools may need OpenCV linked with XM as an external library. The interface of the

DummyType extraction tool (implementation template) is:
//=============================================================================
class DummyTypeExtractionTool: public DescriptorExtractor
friend DummyTypeExtractionInterface;
public:
// Null constructor
DummyTypeExtractionTool();
// Also connects the Descriptor (result memory) to the extraction
// If set to "0" it automatically creates the descriptor
DummyTypeExtractionTool(DummyTypeDescriptorInterfaceABC
*DummyType);
// ID of object type
virtual const UUID& GetObjectID(void);
// Object type name
virtual const char *GetName(void);
// This informs the extractor where the source data comes from
virtual int SetSourceMedia(MultiMediaInterfaceABC* media);
// Pointer where the description is stored
virtual DummyTypeDescriptorInterfaceABC*
GetDescriptorInterface(void);
virtual int SetDescriptorInterface(DummyTypeDescriptorInterfaceABC
*aDummyTypeDescriptorInterface);
// initialize descriptor and extraction process (input media must be known)
virtual unsigned long InitExtracting(void);
// performs extraction from input media frame by input media frame
virtual unsigned long StartExtracting(void);
// collects descriptor data after all input media frames were processed
virtual unsigned long PostExtracting(void);
// Extraction object must not be used, only its interface is allowed
// to be used. This function is to get the interface
virtual DummyTypeExtractionInterfaceABC *GetInterface(void);
// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
© ISO/IEC 2020 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC 15938-6:2020(E)
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
private:
// Destructor is private to allow creation of
// object only by using "new"
virtual ~DummyTypeExtractionTool();
DummyTypeExtractionInterface m_Interface;
DummyTypeDescriptorInterfaceABC *m_DescriptorInterface;
MultiMediaInterfaceABC* m_Media;
// only used in this dummy type to show extraction function
unsigned long m_FrameCnt;
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeAExtractionInterfaceABC *m_SubDummyTypeAExtraction;
SubDummyTypeBExtractionInterfaceABC *m_SubDummyTypeBExtraction;
#endif /* __HasSubTypes*/
int m_DummyExtractionParameter;
}; // End class
//=============================================================================
6.2.6 Descriptors (Ds) and description schemes (DSs)

These modules implement the data structure of normative descriptors and description schemes. Low level

video descriptors are using a dedicated C++ class. This class provides methods to access the elements of

the normative descriptions. The source files are located in the descriptors directory. All other normative

Ds and DSs are using the GenericDS class located in the DescriptionSchemes directory. The GenericDS

class does not implement the data structure in a dedicated way, but it is an interface to the XML parser

library which controls the memory for the tree structure of the instantiated D or DS. The interface of the

descriptors class is given for the DummyType descriptor (implementation template) below:

/=============================================================================
class DummyTypeDescriptor: public Descriptor
friend DummyTypeDescriptorInterface;
public:
DummyTypeDescriptor();
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
// constructor which also constructs and/or connects the descriptor object
DummyTypeDescriptor(SubDummyTypeADecriptorInterfaceABC *aSubDummyTypeA,
SubDummyTypeBDescriptorInterfaceABC *aSubDummyTypeB);
#endif /* __HasSubTypes*/
virtual const UUID& GetValueID(void);
virtual const char* GetValueName(void);
virtual const UUID& GetObjectID(void);
virtual const char *GetName(void);
// for reference counting
virtual void addref();
virtual void release();
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
/* only needed for manual connection with sub components*/
virtual SubDummyTypeADescriptorInterfaceABC
*GetSubDummyTypeADescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeADescriptorInterface(SubDummyTypeADescriptorInterfaceABC
*aSubDummyTypeADescriptorInterface);
virtual SubDummyTypeBDescriptorInterfaceABC
6 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 15938-6:2020(E)
*GetSubDummyTypeBDescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeBDescriptorInterface(SubDummyTypeBDescriptorInterfaceABC
*aSubDummyTypeBDescriptorInterface);
#endif /* __HasSubTypes*/
// actual descriptor methods, only in this dummy type example
virtual long GetDummyContents(void);
virtual void SetDummyContents(const long val);
// transformation to GenericDS object (MDS implementaion style)
virtual unsigned long
ExportDDL(GenericDSInterfaceABC *aParentDescription);
virtual unsigned long ImportDDL(GenericDSInterfaceABC *aDescription);
// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
virtual DummyTypeDescriptorInterfaceABC *GetInterface(void);
private:
// private destructor to force reference counting mechanism
virtual ~DummyTypeDescriptor();
// reference counter
unsigned long m_refcount;
DummyTypeDescriptorInterface m_Interface;
const bool m_isProprietary;
static const char * valName;
static const UUID valID;
#ifdef __HasSubTypes /*include this section if sub-descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeADescriptorInterfaceABC *m_SubDummyTypeADescriptorInterface;
SubDummyTypeBDescriptorInterfaceABC *m_SubDummyTypeBDescriptorInterface;
#endif /* __HasSubTypes*/
// This is the actual data the D/DSType stores. In this particu
...

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 15938-6
ISO/IEC JTC 1/SC 29 Secretariat: JISC
Voting begins on: Voting terminates on:
2015-10-29 2016-01-29
Information technology — Multimedia content
description interface —
Part 6:
Reference software
Technologies de l’information — Interface de description du contenu multimédia —
Partie 6: Logiciel de référence
ICS: 35.040
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 15938-6:2015(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2015
---------------------- Page: 1 ----------------------
ISO/IEC DIS 15938-6:2015(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2015, Published in Switzerland

All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form

or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior

written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of

the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2015 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC DIS 15938-6
Contents Page

Foreword ............................................................................................................................................................ iv

Introduction ......................................................................................................................................................... v

1 Scope ...................................................................................................................................................... 1

2 Symbols and abbreviated terms .......................................................................................................... 1

3 Copyright disclaimer for software modules ....................................................................................... 2

4 XM software architecture ...................................................................................................................... 2

4.1 Block diagrams ...................................................................................................................................... 2

4.1.1 Media database ...................................................................................................................................... 4

4.1.2 AV decoders ........................................................................................................................................... 4

4.1.3 Media data .............................................................................................................................................. 5

4.1.4 Extraction tools ..................................................................................................................................... 5

4.1.5 Descriptors (Ds) and description schemes (DSs) .............................................................................. 6

4.1.6 Coding schemes (CSs) ......................................................................................................................... 8

4.1.7 Search & matching tools ...................................................................................................................... 9

4.1.8 Media transcoders ............................................................................................................................... 11

4.1.9 Applications ......................................................................................................................................... 11

4.1.10 Interface structure ............................................................................................................................... 12

5 Systems reference software (BiM) ..................................................................................................... 12

5.1 BiM reference software ....................................................................................................................... 12

5.1.1 Introduction .......................................................................................................................................... 12

5.1.2 Software overview ............................................................................................................................... 13

5.2 Access unit navigator ......................................................................................................................... 14

5.2.1 Introduction .......................................................................................................................................... 14

5.2.2 Textual access unit encoder module ................................................................................................ 14

5.2.3 Textual access unit decoder module ................................................................................................ 15

6 Systems reference software (DDL) .................................................................................................... 16

7 Video reference software .................................................................................................................... 16

8 Audio reference software ................................................................................................................... 17

9 Multimedia description scheme reference software ........................................................................ 19

10 Compilation of the reference software .............................................................................................. 21

11 Usage information for individual descriptors and description schemes ...................................... 21

Annex A (informative) Providers of reference software ................................................................................ 22

Annex B (informative) Patent statements ....................................................................................................... 24

© ISO/IEC 2015 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC DIS 15938-6
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are members of

ISO or IEC participate in the development of International Standards through technical committees

established by the respective organization to deal with particular fields of technical activity. ISO and IEC

technical committees collaborate in fields of mutual interest. Other international organizations, governmental

and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information

technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.

International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.

The main task of the joint technical committee is to prepare International Standards. Draft International

Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as

an International Standard requires approval by at least 75 % of the national bodies casting a vote.

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent

rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.

ISO/IEC 15938-6 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.

This second edition cancels and replaces the first edition of which has been technically revised.

ISO/IEC 15938 consists of the following parts, under the general title Information technology — Multimedia

content description interface:
⎯ Part 1: Systems
⎯ Part 2: Description definition language
⎯ Part 3: Visual
⎯ Part 4: Audio
⎯ Part 5: Multimedia description schemes
⎯ Part 6: Reference software
⎯ Part 7: Conformance testing
⎯ Part 8: Extraction and use of MPEG-7 descriptions
⎯ Part 9: Profiles and levels
⎯ Part 10: Schema definition
⎯ Part 11: MPEG-7 profile schemas
⎯ Part 12: Query format
⎯ Part 13: Compact descriptors for visual search
iv © ISO/IEC 2015 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC DIS 15938-6
Introduction

This International Standard, also known as "Multimedia Content Description Interface," provides a

standardized set of technologies for describing multimedia content. It addresses a broad spectrum of

multimedia applications and requirements by providing a metadata system for describing the features of

multimedia content.
The following are specified in this International Standard:

Description schemes (DS) describe entities or relationships pertaining to multimedia content. Description

schemes specify the structure and semantics of their components, which may be description schemes,

descriptors, or datatypes.

Descriptors (D) describe features, attributes, or groups of attributes of multimedia content.

Datatypes are the basic reusable datatypes employed by description schemes and descriptors

Systems tools support delivery of descriptions, multiplexing of descriptions with multimedia content,

synchronization, file format, and so forth.
This International Standard is subdivided into 13 parts:

Part 1 – Systems: specifies the tools for preparing descriptions for efficient transport and storage,

compressing descriptions, and allowing synchronization between content and descriptions.

Part 2 – Description definition language: specifies the language for defining the International Standard set

of description tools (DSs, Ds, and datatypes) and for defining new description tools.

Part 3 – Visual: specifies the description tools pertaining to visual content.
Part 4 – Audio: specifies the description tools pertaining to audio content.

Part 5 – Multimedia description schemes: specifies the generic description tools pertaining to multimedia

including audio and visual content.

Part 6 – Reference software: provides a software implementation of the International Standard.

Part 7 – Conformance testing: specifies the guidelines and procedures for testing conformance of

implementations of the International Standard.

Part 8 – Extraction and use of MPEG-7 descriptions: provides guidelines and examples of the extraction

and use of descriptions.
Part 9 – Profiles and levels: provides guidelines and standard profiles.

Part 10 – Schema definition: specifies the schema using description definition language.

Part 11 – Profile Schemas: listing of profile schemas using description definition language.

Part 12 – Query format: contains the tools of the MPEG Query Format (MPQF)

Part 13 – Compact descriptors for visual search: specifies an image description tool for visual search

applications
© ISO/IEC 2015 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC DIS 15938-6

This part of ISO/IEC 15938 contains simulation software for tools defined in parts 1, 2, 3, 4 and 5 of ISO/IEC

15938. This software has been derived from the verification models used in the process of developing the

International Standard.

Where multimedia content extraction or multimedia content description software is provided, attention is called

to the fact that these software modules are provided for the purpose of creating bit streams of descriptors and

description schemes with normative syntax. The performance of these software tools should not be taken as

indicative of that which can be obtained from implementations where quality and computational optimization

are given priority. The techniques used for extracting descriptors or deriving description schemes are not

specified by this document. This information can be found in the corresponding sections of part 1-5.

vi © ISO/IEC 2015 – All rights reserved
---------------------- Page: 6 ----------------------
DRAFT INTERNATIONAL STANDARD ISO/IEC DIS 15938-6
Information technology — Multimedia content description
interface — Part 6: Reference software
1 Scope

This International Standard operates on and generates conformant bitstreams. This International Standard

provides a specific implementation that behaves in a conformant manner. In general, other implementations

that conform to ISO/IEC 15938 are possible that do not necessarily use the algorithms or the programming

techniques of the reference software.

The software contained in this part of ISO/IEC 15938 is known as eXperimentation Model (XM) and is divided

into five categories:

a) Binary format for MPEG-7 (BiM). This software converts DDL (XML) based descriptions to binary format

and vice versa as explained in Clause 5 of this document.

b) DDL parser and DDL validation parser. The components of this software module are specified in Clause 6

of this document.

c) Visual descriptors. This software creates standard visual descriptions from associated (visual) media

content as explained in Clause 7 of this document. The techniques used for extracting descriptors are

informative, and the quality and complexity of these extraction tools has not been optimized.

d) Audio descriptors. This software creates standard descriptions from associated (audio) media content as

explained in Clause 8 of this document. The techniques used for extracting descriptors are informative,

and the quality and complexity of these extraction tools has not been optimized.

e) Multimedia description schemes. This software modules provide standard descriptions of multimedia

description schemes as specified in Clause 9 of this document.
2 Symbols and abbreviated terms

For the purposes of this part of ISO/IEC 15938, the following symbols and abbreviated terms apply:

AV: Audio-visual
CS: Coding Scheme
D: Descriptor
Ds: Descriptors
DCT: Discrete Cosine Transform
DDL: Description Definition Language
DS: Description Scheme
DSs: Description Schemes
ISO: International Organization for Standardization
MDS: Multimedia Description Schemes
MPEG: Moving Picture Experts Group
MPEG-7: Multimedia Content Description Interface Standard (see ISO/IEC 15938)
XML: Extensible Markup Language
© ISO/IEC 2015 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC DIS 15938-6
3 Copyright disclaimer for software modules

Each source code module in this specification contains a copyright disclaimer which shall not be removed

from the source code module.

In the text of each copyright disclaimer, is replaced with a reference to its associated

specification, e.g. MPEG-7 Systems (ISO/IEC 15938-1), MPEG-7 Visual (ISO/IEC 15938-3), MPEG-7 Audio

(ISO/IEC 15938-4), MPEG-7 Multimedia description schemes (ISO/IEC 15938-5).

"This software module was originally developed by () and edited by

(), (), in the course of development of the . This software

module is an implementation of a part of one or more tools as specified by the

standard>. ISO/IEC gives users of the free license to this software module or modifications

thereof for use in hardware or software products claiming conformance to the . Those

intending to use this software module in hardware or software products are advised that its use may infringe

existing patents. The original developer of this software module and his/her company, the subsequent editors

and their companies, and ISO/IEC have no liability for use of this software module or modifications thereof in

an implementation. Copyright is not released for non conforming products. retains

full right to use the code for his/her own purpose, assign or donate the code to a third party and to inhibit third

parties from using the code for non conforming products. This copyright notice must be

20xx-20xx".
included in all copies or derivative works. Copyright
=First Name, =Last Name, =Company Name
4 XM software architecture
4.1 Block diagrams

This section provides information about the XM software architecture. The block diagrams give short

overviews, and introduce individual components of the XM software. This section also provides the directory

locations for the different modules.

The composing elements of the MPEG-7 Reference Software are characterized by their functionality and by

their interfaces. They can be configured according to what here is referred as "Key Applications". We can

distinguish from the functional point of view:

• "Extraction Applications" (a description data base is built from a media data base)

• "Search and Retrieval Applications" (a description is compared with the descriptions in a database to

find the one with the lowest distance)

• "Transcoding Applications" (a media data base is converted into another media data base based on

its description)
2 © ISO/IEC 2015 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC DIS 15938-6

Figure 1 — Schematic diagram of an "Extraction Application" using the XM reference software

modules, with boxes representing procedural parts and circles representing data structures.

Figure 2 — Schematic diagram of a "Search and Retrieval Application" using the XM reference

software modules, with boxes representing procedural parts and circles representing data structures.

© ISO/IEC 2015 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/IEC DIS 15938-6

Figure 3 — Schematic diagram of a "Transcoding Application" using the XM reference software

modules, with boxes representing procedural parts and circles representing data structures.

In the following, the blocks of the "Key Applications" are distinguished. For elements that are related to

specific descriptors or description schemes, the interface is given using a DummyType example. This

represents the XM integration template and not a normative descriptor or description scheme.

4.1.1 Media database

The media database contains media files, which are supported as input files by the AV decoders. The

database file is read from a file and contains one media filename per line. From this media filename all

additional input and output filenames can be derived.
4.1.2 AV decoders
The XM supports the following AV decoders:

• Still image decoders: ImageMagick (Ver. 6.8.8 linked as external library, not included in the XM

reference software distribution).
• MPEG-1, MPEG-2 video decoders: (XM directory: Decoders/MPEG2Dec).

• MPEG-1 video motion vector extractor: (XM directory: Decoders/MPEG2Dec). This extracts images

and motion vectors.

• 3D Objects: (XM directory: Media). This reads a 3D object for 3D shape descriptors.

4 © ISO/IEC 2015 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC DIS 15938-6

• Key Points: (XM directory: Media). This reads in a list of key points from a file.

• Audio decoders: (XM directory: Media). They read audio files using external library AFsp version v5r2

from McGill University.
4.1.3 Media data

This is the internal XM representation of the raw media data (one class with different structures depending on

the media content type). The class description for media data can be found in the Media XM directory.

4.1.4 Extraction tools

Extraction tools are specific extraction methods defined for each descriptor and description scheme. All the

source files are available in the ExtractionUtilities XM directory. The extraction tools extract the descriptions

from media data. Because media data can be of significant size, the extraction is performed on time entities of

the media, i.e., if the media is a video the extraction is done frame by frame. Some of the extraction tools may

need OpenCV linked with XM as an external library. The interface of the DummyType extraction tool

(implementation template) is given below:
//=============================================================================
class DummyTypeExtractionTool: public DescriptorExtractor
friend DummyTypeExtractionInterface;
public:
// Null constructor
DummyTypeExtractionTool();
// Also connects the Descriptor (result memory) to the extraction
// If set to "0" it automatically creates the descriptor
DummyTypeExtractionTool(DummyTypeDescriptorInterfaceABC
*DummyType);
// ID of object type
virtual const UUID& GetObjectID(void);
// Object type name
virtual const char *GetName(void);
// This informs the extractor where the source data comes from
virtual int SetSourceMedia(MultiMediaInterfaceABC* media);
// Pointer where the description is stored
virtual DummyTypeDescriptorInterfaceABC*
GetDescriptorInterface(void);
virtual int SetDescriptorInterface(DummyTypeDescriptorInterfaceABC
*aDummyTypeDescriptorInterface);
// initililaize descriptor and extraction process (input media must be known)
virtual unsigned long InitExtracting(void);
// performs extraction form input media frame by input media frame
virtual unsigned long StartExtracting(void);
// collects descriptor data after all input media frames were processed
virtual unsigned long PostExtracting(void);
© ISO/IEC 2015 – All rights reserved 5
---------------------- Page: 11 ----------------------
ISO/IEC DIS 15938-6
// Extraction object must no be used, only its interface is allowd to
// to be used. This function is to get the interface
virtual DummyTypeExtractionInterfaceABC *GetInterface(void);
// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
private:
// Destructor is private to allow creation of
// object only by using "new"
virtual ~DummyTypeExtractionTool();
DummyTypeExtractionInterface m_Interface;
DummyTypeDescriptorInterfaceABC *m_DescriptorInterface;
MultiMediaInterfaceABC* m_Media;
// only used in this dummy type to show extraction function
unsigned long m_FrameCnt;
#ifdef __HasSubTypes /*include this section if sub descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeAExtractionInterfaceABC *m_SubDummyTypeAExtraction;
SubDummyTypeBExtractionInterfaceABC *m_SubDummyTypeBExtraction;
#endif /* __HasSubTypes*/
int m_DummyExtractionParameter;
}; // End class
//=============================================================================
4.1.5 Descriptors (Ds) and description schemes (DSs)

These modules implement the data structure of normative descriptors and description schemes. Low level

Video descriptors are using a dedicated C++ class. This class provides methods to access the elements of the

normative descriptions. The source files are located in the Descriptors directory. All other normative Ds and

DSs are using the GenericDS class located in the DescriptionSchemes directory. The GenericDS class does

not implement the data structure in a dedicated way, but it is an interface to the XML parser library which

controls the memory for the tree structure of the instantiated D or DS. The interface of the descriptors class is

given for the DummyType descriptor (implementation template) below:
/=============================================================================
class DummyTypeDescriptor: public Descriptor
friend DummyTypeDescriptorInterface;
public:
DummyTypeDescriptor();
#ifdef __HasSubTypes /*include this section if sub descriptors exist,
remove this section if no sub-descriptors exist*/
// constructor which also constructs and/or connects the descriptor object
DummyTypeDescriptor(SubDummyTypeADecriptorInterfaceABC *aSubDummyTypeA,
SubDummyTypeBDescriptorInterfaceABC *aSubDummyTypeB);
6 © ISO/IEC 2015 – All rights reserved
---------------------- Page: 12 ----------------------
ISO/IEC DIS 15938-6
#endif /* __HasSubTypes*/
virtual const UUID& GetValueID(void);
virtual const char* GetValueName(void);
virtual const UUID& GetObjectID(void);
virtual const char *GetName(void);
// for reference counting
virtual void addref();
virtual void release();
#ifdef __HasSubTypes /*include this section if sub descriptors exist,
remove this section if no sub-descriptors exist*/
/* only needed for manual connection with sub components*/
virtual SubDummyTypeADescriptorInterfaceABC
*GetSubDummyTypeADescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeADescriptorInterface(SubDummyTypeADescriptorInterfaceABC
*aSubDummyTypeADescriptorInterface);
virtual SubDummyTypeBDescriptorInterfaceABC
*GetSubDummyTypeBDescriptorInterface(void);
virtual unsigned long
SetSubDummyTypeBDescriptorInterface(SubDummyTypeBDescriptorInterfaceABC
*aSubDummyTypeBDescriptorInterface);
#endif /* __HasSubTypes*/
// actual descriptor methods, only in this dummy type example
virtual long GetDummyContents(void);
virtual void SetDummyContents(const long val);
// transformation to GenericDS object (MDS implementaion style)
virtual unsigned long
ExportDDL(GenericDSInterfaceABC *aParentDescription);
virtual unsigned long ImportDDL(GenericDSInterfaceABC *aDescription);
// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
virtual DummyTypeDescriptorInterfaceABC *GetInterface(void);
private:
// private destructor to force reference counting mechanism
virtual ~DummyTypeDescriptor();
// reference counter
unsigned long m_refcount;
DummyTypeDescriptorInterface m_Interface;
const bool m_isProprietary;
static const char * valName;
© ISO/IEC 2015 – All rights reserved 7
---------------------- Page: 13 ----------------------
ISO/IEC DIS 15938-6
static const UUID valID;
#ifdef __HasSubTypes /*include this section if sub descriptors exist,
remove this section if no sub-descriptors exist*/
SubDummyTypeADescriptorInterfaceABC *m_SubDummyTypeADescriptorInterface;
SubDummyTypeBDescriptorInterfaceABC *m_SubDummyTypeBDescriptorInterface;
#endif /* __HasSubTypes*/
// This is the actual data the D/DSType stores. In this particular
// dummy example it's just a signed long called m_DummyContents
long m_DummyContents;
/=============================================================================
4.1.6 Coding schemes (CSs)

Coding schemes are specific coding and decoding methods defined for individual descriptors (Ds) and

description schemes (DSs). All the source files are available in the CodingSchemes directory. If an individual

coding scheme is available, it represents a normative part of the standard. Coding schemes are available for

the visual descriptors to encode or to decode a description into its binary representation. The alternative

implementation of the coding scheme box, which is available for all Ds and DSs, allows encoding and

decoding of descriptors into its DDL representation using the GenericDSCS which is an interface to the "write

to file"- and "read from file"-functions of XML Xerces-C++ external parser library. The interface of the coding

schemes is given below on the example of the DummyType coding scheme (implementation template):

//=============================================================================
class DummyTypeCS: public DescriptionCodingEngine
friend DummyTypeCSInterface;
public:
DummyTypeCS();
// constructor which also constructs and/or connects the descriptor object
DummyTypeCS(DummyTypeDescriptorInterfaceABC
*DummyType);
virtual const UUID& GetValueID(void);
virtual const char* GetValueName(void);
virtual const UUID& GetObjectID(void);
virtual const char *GetName(void);
// access is allowed only by class factories for this
// object. This avoids having to duplicate the
// ID definition in multiple locations. In the future, we may
// have to do this. PLEASE DO NOT USE THESE UNLESS YOU ARE
// IMPLEMENTING A CLASS FACTORY GENERATING THIS OBJECT
static const UUID myID;
static const char * myName;
virtual DummyTypeCSInterfaceABC *GetInterface(void);
// accessor meth
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.