ISO/IEC 15938-12:2012

Information technology — Multimedia content description interface — Part 12: Query format

ISO/IEC 15938-12:2012 describes the query format tools which may be used independently or in combination with other parts of ISO/IEC 15938. Each query format tool is described in two normative sections: a) Syntax: Normative specification of the query and management format. b) Semantic: Normative definition of the semantics of all the components of the corresponding query format specification. In some instances the query format level tool is also described using either one or two informative sections: a) Examples:Optionally, an informative section dealing with examples is appended. b) Definitions:Optionally, an informative section dealing with definitions is appended. The query format provides a standardized interface for multimedia content information retrieval systems (e.g. MPEG-7 databases) in three aspects which are input query format, output query format, and query managements. The input query format specifies the interface through which the users can describe their search criteria with a set of precise input parameters in addition to a set of preferred output parameters to depict the return result sets. The output query format specifies the interface format for the result set. The query management provides means for selecting services (e.g. MPEG-7 database) or aggregated services (e.g. service provider that administers a set of different services) based on service properties (e.g. supported query format).

Technologies de l'information — Interface de description du contenu multimédia — Partie 12: Format de requête

General Information

• Status: Published
• Publication Date: 22-Oct-2012
• ICS: 35.040 - Information coding
• ICS: 35.040.40 - Coding of audio, video, multimedia and hypermedia information
• Technical Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Drafting Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Current Stage: 9060 - Close of review
• Start Date: 02-Dec-2027

Relations

• Revises: ISO/IEC 15938-12:2008 - Information technology — Multimedia content description interface — Part 12: Query format
• Effective Date: 24-Dec-2011
• Revises: ISO/IEC 15938-12:2008/Amd 2:2011 - Information technology — Multimedia content description interface — Part 12: Query format — Amendment 2: Semantic enhancement
• Effective Date: 24-Dec-2011
• Revises: ISO/IEC 15938-12:2008/Amd 1:2011 - Information technology — Multimedia content description interface — Part 12: Query format — Amendment 1: Reference software and flat metadata output
• Effective Date: 24-Dec-2011

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 15938-12
Second edition
2012-11-01

Information technology — Multimedia
content description interface —
Part 12:
Query format
Technologies de l'information — Interface de description du contenu
multimédia —
Partie 12: Format de requête




Reference number
ISO/IEC 15938-12:2012(E)
©
ISO/IEC 2012

---------------------- Page: 1 ----------------------
ISO/IEC 15938-12:2012(E)

COPYRIGHT PROTECTED DOCUMENT


©  ISO/IEC 2012
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56  CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail [e-mail]
Web www.iso.org
Published in Switzerland

ii © ISO/IEC 2012 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/IEC 15938-12:2012(E)
Contents Page
Foreword . v
Introduction . vi
1  Scope . 1
1.1  Organization of the document . 1
1.2  Overview of the Query Format . 1
2  Normative references . 1
3  Terms, definitions, abbreviated terms and conventions . 2
3.1  Terms and definitions . 2
3.2  Abbreviated terms . 3
3.3  Conventions . 3
4  Structure and Data Model . 5
4.1  Structure . 5
4.2  Data Model . 8
5  Root Element . 8
5.1  Introduction . 8
5.2  Syntax . 9
5.3  Semantics . 10
5.4  Example . 10
6  Datatypes . 11
6.1  MediaLocatorType . 11
6.2  MimeType . 13
6.3  XPathType . 13
6.4  ZeroToOneType . 14
6.5  TermType . 14
6.6  ClassificationScheme . 15
6.7  RelationType . 16
6.8  SemanticFieldType . 17
7  Input Query Format . 18
7.1  Introduction . 18
7.2  Syntax . 19
7.3  Semantics . 19
7.4  Example . 20
8  QFDeclaration . 21
8.1  Introduction . 21
8.2  Syntax . 21
8.3  Semantics . 22
8.4  Example . 24
9  Output Description . 25
9.1  Introduction . 25
9.2  Syntax . 25
9.3  Semantics . 26
9.4  Example . 28
9.5  SortBy . 30
10  Query Condition . 33
10.1  Introduction . 33
10.2  Syntax . 33
10.3  Semantics . 33
© ISO/IEC 2012 – All rights reserved iii

---------------------- Page: 3 ----------------------
ISO/IEC 15938-12:2012(E)
10.4  Example .34
11  Expression Types .35
11.1  Introduction .35
11.2  Syntax .35
11.3  Semantics .35
11.4  Operands .36
11.5  Boolean Expression Types .38
11.6  Comparison Expression Types .42
11.7  Arithmetic Expression Types .47
11.8  Aggregate Expression Types .50
11.9  String Expression Types .52
11.10  Semantic Expression Types .54
12  Query Types .60
12.1  Introduction .60
12.2  Syntax .60
12.3  Semantics .60
12.4  QueryByMedia .60
12.5  QueryByDescription .62
12.6  QueryByFreeText .63
12.7  QueryByFeatureRange .65
12.8  SpatialQuery .69
12.9  TemporalQuery .72
12.10  Query By XQuery .74
12.11  Query By Relevance Feedback .75
12.12  Query By ROI .76
12.13  Join Query .80
12.14  QueryBySPARQL .82
13  Output Query Format .84
13.1  Output .84
13.2  ResultItem .86
13.3  SystemMessageType .89
14  Query Management Tools .90
14.1  Introduction .90
14.2  InputManagementType .90
14.3  OutputManagementType .95
15  MPEG Query Format Reference Software .98
15.1  Introduction .98
15.2  MPQF Reference Software specific terms, definitions and conventions .98
15.3  Overview of the architecture of the 15938-12 reference software .99
15.4  MPQF Validator . 100
15.5  MPQF Parser . 107
15.6  Basic Interpreter . 112
15.7  MPQF Semantic Enhancement . 118
Annex A (normative) System Messages . 122
Annex B (normative) Classification Scheme . 124
Bibliography . 153

iv © ISO/IEC 2012 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/IEC 15938-12:2012(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 15938-12 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
This second edition cancels and replaces the first edition (ISO/IEC 15938-12:2008), which has been
technically revised.
It also incorporates ISO/IEC 15938-12:2008/Cor.1:2009, ISO/IEC 15938-12:2008/Cor.2:2010,
ISO/IEC 15938-12:2008:Amd.1:2011 and ISO/IEC 15938-12:2008:Amd.2:2011.
ISO/IEC 15938 consists of the following parts, under the general title Information technology — Multimedia
content description interface:
 Part 1: Systems
 Part 2: Description definition language
 Part 3: Visual
 Part 4: Audio
 Part 5: Multimedia description schemes
 Part 6: Reference software
 Part 7: Conformance testing
 Part 8: Extraction and use of MPEG-7 descriptions
 Part 9: Profiles and levels
 Part 10: Schema definition
 Part 11: MPEG-7 Profile schemas
 Part 12: Query format
© ISO/IEC 2012 – All rights reserved v

---------------------- Page: 5 ----------------------
ISO/IEC 15938-12:2012(E)
Introduction
The MPEG-7 standard, also known as the "Multimedia Content Description Interface", aims at providing
standardized core technologies allowing the description of audiovisual data content in multimedia
environments. This is a challenging task given the broad spectrum of requirements and targeted multimedia
applications, and the broad number of audiovisual features of importance in such a context. In order to
achieve this broad goal, MPEG-7 standardizes:
 Datatypes that are description elements not specific to the audiovisual domain that corresponds to
reusable basic types or structures employed by multiple Descriptors and Description Schemes.
 Descriptors (D) to represent Features. Descriptors define the syntax and the semantics of each feature
representation. A Feature is a distinctive characteristic of the data, which signifies something to somebody.
It is possible to have several descriptors representing a single feature, i.e. to address different relevant
requirements. A Descriptor does not participate in many-to-one relationships with other description
elements.
 Description Schemes (DS) to specify the structure and semantics of the relationships between their
components, which may be both Ds and DSs. A Description Scheme shall have descriptive information
and may participate in many-to-one relationships with other description elements.
 A Description Definition Language (DDL) to allow the creation of new DSs and, possibly, Ds and to allow
the extension and modification of existing DSs.
 Systems tools to support multiplexing of descriptions or description and data, synchronization issues,
transmission mechanisms, file format, etc.
The standard is subdivided into twelve parts:
1. Systems: Architecture of the standard, tools that are needed to prepare MPEG-7 Descriptions for efficient
transport and storage, and to allow synchronization between content and descriptions. Also tools related
to managing and protecting intellectual property.
2. Description Definition Language: Language for defining new DSs and eventually also new Ds, binary
representation of DDL expressions.
3. Visual: Visual description tools (Ds and DSs).
4. Audio: Audio description tools (Ds and DSs).
5. Multimedia Description Schemes: Description tools (Ds and DSs) that are generic, i.e. neither purely
visual nor purely audio.
6. Reference Software: Software implementation of relevant parts of the MPEG-7 Standard.
7. Conformance: Guidelines and procedures for testing conformance of MPEG-7 implementations.
8. Extraction and use of MPEG-7 descriptions.
9. Profiles and Levels.
10. Schema Definition.
vi © ISO/IEC 2012 – All rights reserved

---------------------- Page: 6 ----------------------
ISO/IEC 15938-12:2012(E)
11. MPEG-7 Profile Schemas.
12. Query Format.
This part of ISO/IEC 15938 contains the tools of the MPEG Query Format (MPQF). It addresses the normative
aspects of the MPQF and also illustrates some non-normative examples. The syntax of the Query Format is
defined using the guidelines of DDL ISO/IEC 15938-2.

© ISO/IEC 2012 – All rights reserved vii

---------------------- Page: 7 ----------------------
INTERNATIONAL STANDARD ISO/IEC 15938-12:2012(E)

Information technology — Multimedia content description
interface —
Part 12:
Query format
1 Scope
1.1 Organization of the document
This part of ISO/IEC 15938 describes the query format tools which may be used independently or in
combination with other parts of ISO/IEC 15938. Each query format tool is described in two normative sections:
 Syntax:  Normative specification of the query and management format.
 Semantic:  Normative definition of the semantics of all the components of the corresponding query
format specification.
In some instances the query format level tool is also described using either one or two informative sections:
 Examples: Optionally an informative section dealing with examples is appended.
 Definitions: Optionally an informative section dealing with definitions is appended.
1.2 Overview of the Query Format
The query format provides a standardized interface for multimedia content information retrieval systems (e.g.
MPEG-7 databases) in three aspects which are input query format, output query format, and query
managements. The input query format specifies the interface through which the users can describe their
search criteria with a set of precise input parameters in addition to a set of preferred output parameters to
depict the return result sets. The output query format specifies the interface format for the result set. The
query management provides means for selecting services (e.g. MPEG-7 database) or aggregated services
(e.g. service provider that administers a set of different services) based on service properties (e.g. supported
query format).
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
XQuery 1.0 and XPath 2.0 Data Model (XDM). W3C Recommendation, 23 January
2007. http://www.w3.org/TR/xpath-datamodel/
XML Path Language (XPath) 2.0. W3C Recommendation, 23 January 2007. http://www.w3.org/TR/xpath20/
© ISO/IEC 2012 – All rights reserved 1

---------------------- Page: 8 ----------------------
ISO/IEC 15938-12:2012(E)
3 Terms, definitions, abbreviated terms and conventions
For the purposes of this document, the following terms and definitions, abbreviated terms and conventions
apply.
3.1 Terms and definitions
3.1.1
content
data and the associated metadata
3.1.2
evaluation item
EI
unit, against which the query condition is tested
NOTE By default, an EI is a multimedia content of the multimedia repository, but other types of EI are also possible.
An EI can be:
1) A multimedia content;
2) A segment of a multimedia resource;
3) An XPath-item related to the multimedia content’s metadata XML tree.
3.1.3
input query format
interface format going from a requester to one or more responders with two functionalities
 functionality providing a combination of syntax and semantics of the interface, through which the
requester assigns search criteria and associated data;
 functionality providing syntax and semantics of the interface, through which the requester specifies the
format of the result data
NOTE The second functionality of the input query format provides tools by which the requester can express desired
output format that should be conformant to the specification of output query format.
3.1.4
metadata
data expressed as a schema valid XML instance to carry additional information describing a multimedia
resource, where the schema defines the information model of the data
3.1.5
multimedia content
coded representation of the information contained in or related to a multimedia resource in a formalized
manner suitable for interpretation by human means
3.1.6
multimedia resource
URI identifiable portion of raw data of a video, an image, an audio or text in any format, that is associated with
a MIME Content-Type
3.1.7
output query format
interface format going from the responder to the requester as a response to the request specified by the input
query format
NOTE Output query format defines all possible structures of return from responder to the requester. The structure of
an actual return shall be decided by OutputDescription element in input query format.
2 © ISO/IEC 2012 – All rights reserved

---------------------- Page: 9 ----------------------
ISO/IEC 15938-12:2012(E)
3.1.8
query management tools
tools to support the functionality required to manage the query transaction between the requesters and the
responders
NOTE The query management tools do not include tools that are supported by network protocols. The query
management tools intend to be network agnostic and media agnostic.
3.1.9
segment
spatial, temporal, or spatio-temporal unit of multimedia, for example, a temporal segment of video, or a spatial
segment of an image
3.1.10
XPath-item
either a node from the multimedia content’s metadata XML tree or an atomic value
NOTE Details about the different types of nodes and atomic values can be found in the W3C Recommendation on
XQuery 1.0 and XPath 2.0 Data Model. An XPath-item of a multimedia content’s metadata may or may not be related to
a multimedia content’s segment. Also, a multimedia content or a multimedia content’s segment may or may not be
related to XML metadata. Within MPQF queries, XPath can be used to select a sequence of multimedia content’s
segments and/or metadata XPath-items. According to the W3C Recommendation on XQuery 1.0 and XPath 2.0 Data
Model, a sequence is an ordered collection of zero or more XPath-items.
3.2 Abbreviated terms
MPEG: Moving Picture Experts Group
MPEG-7: ISO/IEC 15938
MPQF: MPEG Query Format, ISO/IEC 15938-12
URI: Uniform Resource Identifier (IETF Standard is RFC 2396)
URL: Uniform Resource Locator (IETF Standard is RFC 2396)
XML: Extensible Markup Language (W3C, http://www.w3.org/XML/)
RDF: Resource Description Framework (W3C, http://www.w3.org/RDF/)
SPARQL: SPARQL Query Language for RDF. W3C Recommendation 15 January
2008. http://www.w3.org/TR/rdf-sparql-query/
3.3 Conventions
3.3.1 Query and query management tools
This part of ISO/IEC 15938 specifies the format for query and query management tools using XML-Schema.
 Query – A set of tools supporting the definition of the query request as well as the query response defined
for the MPEG query format. The structure defined for the Query element provides a container for input
query format or output query format.
o Input query format – Input and FetchResult elements are defined for the query. The input
query format structure provides a container for a query request. Such a request should contain a
set of conditions and/or the output description which specifies the structure and content of the
output query format and/or a set of declarations.
© ISO/IEC 2012 – All rights reserved 3

---------------------- Page: 10 ----------------------
ISO/IEC 15938-12:2012(E)
o Output query format – The Output element is defined for the query. It provides a container for all
the responses from a responder to a requester. It may contain not only query results but also any
messages such as error and exceptions.
 Management tools – A set of tools for the query management defined for the MPEG query format
including service discovery, querying service capability, and service capability description. The structure
defined for the management tools provides a container for input management tools or the output
management tools.
o Input management tools – An Input element is defined for the management tools intended to be
sent from a requester to one or more responders.
o Output management tools – An Output element is defined for the management tools intended to
be sent from a responder to one or more requesters.
3.3.2 Naming convention
r to specify tools for the query format, this part of ISO/IEC 15938 uses constructs provided by the
In orde
language specified in ISO/IEC 15938-2 [1], such as "element", "attribute", "simpleType" and "complexType".
The names associated to these constructs are created on the basis of the following conventions:
 If the name is composed of various words, the first letter of each word is capitalized. The rule for the
capitalization of the first word depends on the
...