ISO/IEC TR 15938-8:2002/Amd 6:2011

Information technology — Multimedia content description interface — Part 8: Extraction and use of MPEG-7 descriptions — Amendment 6: Extraction and matching of video signature tools

Technologies de l'information — Interface de description du contenu multimédia — Partie 8: Extraction et utilisation des descriptions MPEG-7 — Amendement 6: Extraction et correspondance des outils de signature vidéo

General Information

• Status: Published
• Publication Date: 30-Oct-2011
• ICS: 35.040 - Information coding
• ICS: 35.040.40 - Coding of audio, video, multimedia and hypermedia information
• Technical Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Drafting Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Current Stage: 6060 - International Standard published
• Start Date: 31-Oct-2011
• Due Date: 28-Feb-2014
• Completion Date: 28-Feb-2014

Relations

• Amends: ISO/IEC TR 15938-8:2002 - Information technology — Multimedia content description interface — Part 8: Extraction and use of MPEG-7 descriptions
• Effective Date: 23-Apr-2020

Overview

ISO/IEC TR 15938-8:2002/Amd 6:2011 is an amendment to the multimedia content description interface standard focusing on the extraction and matching of video signature tools. This technical report supplements the MPEG-7 framework, which is used for describing multimedia content. It specifically addresses video signature extraction and matching techniques to identify duplicate video content accurately, even after common video editing operations. Developed by ISO/IEC Joint Technical Committee 1 (JTC 1), Subcommittee SC 29, this amendment enhances video retrieval and duplicate detection capabilities using robust visual content descriptors.

Key Topics

• Video Signature Concept
  The video signature comprises three main components:

  • Frame signature
  • Compact summary frame signatures ("words")
  • Group-of-frames representation called a "bag of words" for temporal segmentation
    This signature uniquely identifies video content and is robust to editing alterations.

• Extraction Process
  The report references ISO/IEC 15938-3:2002 for detailed procedures on extracting video signatures from multimedia content.

• Matching Methodology
  Matching video signatures involves a three-stage process designed to optimize speed and accuracy:

  1. Segment Matching with Bag of Words – Uses Jaccard distance metrics to assess similarity between temporal segments.
  2. Frame Rate Ratio and Time Shift Estimation – Utilizes a Hough transform to estimate temporal alignment between segments by analyzing frame pairs.
  3. Frame-by-Frame Matching – Extends temporal intervals using L1 distance between frame signatures to confirm matches and filter false positives.

• Fast Matching with Index Tables

  • Employs index tables generated from the "words" in video signatures to pre-filter potential matches.
  • Separate index tables for query and reference videos allow rapid identification of candidate frame pairs.
  • Keyframe selection methods optimize index table creation for query videos, enhancing matching speed.

• Thresholds and Confidence Measures
  Detailed criteria are provided for thresholding distance metrics and frame confidence to reject weak matches and reduce false positives.

Applications

• Duplicate Video Detection
  The video signature tools enable efficient detection of duplicate or near-duplicate video content, even after typical editing such as cropping, scaling, or insertion of additional frames.

• Multimedia Content Retrieval
  By facilitating precise matching of video segments, the standard supports robust search and retrieval of video content based on visual similarity.

• Content Management Systems
  Improved video signature extraction and matching empowers content providers to organize and manage large multimedia libraries, ensuring copyright enforcement and content authenticity.

• Video Monitoring and Surveillance
  The temporal matching stages can identify matching video segments across different footage, useful in surveillance and monitoring scenarios.

• Metadata Generation and Enhancement
  Complementing MPEG-7 descriptors, video signatures add a layer of detailed content description aiding in automated indexing and cataloging.

Related Standards

• ISO/IEC 15938-3:2002 – Specifies core visual content descriptors and foundational extraction techniques critical for video signature creation.
• ISO/IEC 15938-6:2003 – Provides exemplary implementations and source code for matching algorithms referenced in this amendment.
• MPEG-7 Framework – The entire family of multimedia content description standards which ISO/IEC TR 15938-8 extends with enhanced video signature methodologies.
• ISO/IEC JTC 1/SC 29 – The subcommittee responsible for audiovisual coding standards including MPEG, which governs this technical work.

Summary

ISO/IEC TR 15938-8:2002/Amd 6:2011 delivers essential guidelines and methodologies for video signature extraction and matching within the MPEG-7 framework. By introducing a robust three-stage matching system and fast indexing mechanisms, this amendment significantly improves the automated detection of video duplicates and similar content. It is a vital resource for developers, content managers, and service providers managing large volumes of multimedia data, ensuring efficient, accurate, and scalable video content description and retrieval aligned with international standards.

Keywords: MPEG-7, video signature, multimedia content description, duplicate video detection, video matching, video retrieval, ISO/IEC 15938, content indexing, frame signature, bag of words, multimedia standards.

Standards Content (Sample)

TECHNICAL ISO/IEC
REPORT TR
15938-8
First edition
2002-12-15
AMENDMENT 6
2011-11-01
Information technology — Multimedia
content description interface —
Part 8:
Extraction and use of MPEG-7
descriptions
AMENDMENT 6: Extraction and matching of
video signature tools
Technologies de l'information — Interface de description du contenu
multimédia —
Partie 8: Extraction et utilisation des descriptions MPEG-7
AMENDEMENT 6: Extraction et correspondance des outils de signature
vidéo
Reference number
ISO/IEC TR 15938-8:2002/Amd.6:2011(E)
©
ISO/IEC 2011
ISO/IEC TR 15938-8:2002/Amd.6:2011(E)

©  ISO/IEC 2011
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56  CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail [e-mail]
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2011 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.6:2011(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
In exceptional circumstances, when the joint technical committee has collected data of a different kind from
that which is normally published as an International Standard (“state of the art”, for example), it may decide to
publish a Technical Report. A Technical Report is entirely informative in nature and shall be subject to review
every five years in the same manner as an International Standard.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
Amendment 6 to ISO/IEC TR 15938-8:2002 was prepared jointly by Joint Technical Committee
ISO/IEC JTC 1, Information technology, Subcommittee SC 29, Coding of audio, picture, multimedia and
hypermedia information.
© ISO/IEC 2011 – All rights reserved iii

ISO/IEC TR 15938-8:2002/Amd.6:2011(E)

Information technology — Multimedia content description
interface —
Part 8:
Extraction and use of MPEG-7 descriptions
AMENDMENT 6: Extraction and matching of video signature tools
After 4.9.1, add:
4.9.2 Video Signature
The visual content descriptors in Clauses 6-9 of ISO/IEC 15938-3:2002 are very useful when trying to find
videos with similar content. These descriptors are intended to be general and were found to be unsuitable for
the task of finding duplicate content. The video signature descriptor is designed to identify duplicate video
content. This descriptor is robust to a wide range of common video editing operations, but is sufficiently
different for every original content to identify it reliably.
The video signature is composed of three main elements: a frame signature, a set of compact summary frame
signatures, referred to as words, and a group-of-frames representation for a temporal segment, referred to as
a bag of words.
4.9.2.1 Extraction
11.4.5 to 11.4.8 of ISO/IEC 15938-3:2002 describe the extraction of the video signature.
4.9.2.2 Matching
A Video Signature is composed of multiple temporal segments, each represented by a BagOfWords element,
and multiple frames, each represented by a FrameSignature element and a FrameConfidence element.
1 2
The matching between two Video Signatures v and v is carried out in three stages, designed to maximize
matching speed and true positives and to minimize false positives. The first stage uses the BagOfWords
element to identify candidate matching segments. The second stage uses the FrameSignature element to
identify candidates of frame rate ratio and temporal offset between the candidate matching segments. The
third stage performs frame-by-frame matching to determine candidate matching intervals using the
FrameSignature and FrameConfidence elements, and then determines the best match between the Video
1 2
Signatures v and v . These matching stages are explained in more detail below.
Stage 1 (Segment matching with BagOfWords)
All of the temporal segments of Video Signature v are compared with all of the temporal segments of Video
Signature v .
© ISO/IEC 2011 – All rights reserved 1

ISO/IEC TR 15938-8:2002/Amd.6:2011(E)
1 2
For two segments f and f , their similarity is assessed by comparing the bag-of-words representation for
each vocabulary j and merging the results to reach a decision. More specifically, for BagOfWords[]j and
BagOfWords[ j] , their distance is measured by the Jaccard distance metric given by
#(BagOfWords[]j BagOfWords[]j)
D BagOfWords[]j, BagOfWords []j 1

j
#(BagOfWords[]j BagOfWords[]j)
J
1 2
where # denotes the number of elements in a set. This measures the distance of the segments f and f in
a given vocabulary as a function of the distinct words they have in common and all the distinct words that they
contain jointly.
For Q 5 vocabularies, we have Jaccard distances D , D , …,. D . These distances are fused to give
0 1 Q1
J J
J
the composite distance D as
J
Q1
D D
 k
J
J
k0
Then a decision on the similarity of the segments is reached by thresholding each of Jaccard distances D ,
J
1 2
D , …,. D , and the composite distance D . That is, the segments f and f are passed to stage 2 of
1 Q1 i j
J
J
J
matching if more than half of the Q Jaccard distances D , D , …,. D are less than a threshold T and
0 1 Q1 1
J J
J
the composite distance D is less than another threshold T , otherwise they are declared not matching.
J
Stage 2 (Frame rate ratio & time shift estimation using Hough transform)
For the segment pairs passed to this stage, a Hough transform is used to estimate the temporal parameter
differences, i.e. frame
...