ISO/IEC TR 24722:2007
(Main)Information technology — Biometrics— Multimodal and other multibiometric fusion
Information technology — Biometrics— Multimodal and other multibiometric fusion
ISO/IEC TR 24722:2007 provides a description of and analysis of current practice on multimodal and other multibiometric fusion, including (as appropriate) reference to a more detailed description. It also discusses the need for, and possible routes to, standardization to support multibiometric systems.
Technologies de l'information — Biométrie — Fusion multimodale et autre fusion multibiométrique
General Information
Relations
Standards Content (Sample)
TECHNICAL ISO/IEC
REPORT TR
24722
First edition
2007-07-01
Information technology — Biometrics —
Multimodal and other multibiometric
fusion
Technologies de l'information — Biométrie — Fusion multimodale et
autre fusion multibiométrique
Reference number
ISO/IEC TR 24722:2007(E)
©
ISO/IEC 2007
---------------------- Page: 1 ----------------------
ISO/IEC TR 24722:2007(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2007
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2007 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC TR 24722:2007(E)
Contents Page
1 Scope.1
2 Terminology issues.1
3 Terms and definitions .3
4 Overview of multimodal and other multibiometric systems.5
4.1 General .5
4.2 Simultaneous and sequential presentation.7
4.2.1 General multibiometric system model .7
4.2.2 Simultaneous presentation .8
4.2.3 Sequential presentation.8
4.3 Correlation .8
5 Levels of combination.9
5.1 Overview.9
5.2 Decision-level fusion.12
5.2.1 Simple decision-level fusion .12
5.2.2 Advanced decision-level fusion.13
5.2.2.1 General model.13
5.2.2.2 Layered system.15
5.2.2.3 Cascaded system.15
5.3 Score-level fusion.15
5.3.1 Overview.15
5.3.2 Score normalisation.16
5.3.3 Score fusion methods.19
5.4 Feature-level fusion.21
6 Characterisation data for multibiometric systems .21
6.1 Overview.21
6.2 Use of characterisation data in normalisation and fusion.21
7 Scope and options for standardisation.22
7.1 Introduction.22
7.2 Implementation areas.22
7.3 Interoperability requirements.23
7.4 Possible standardisation activity .23
7.4.1 On record format standardisation .23
7.4.2 On framework standardisation .25
7.4.3 On application profile standardisation.27
7.4.4 On compliance standardisation.27
7.4.5 On multimodal testing standardisation.28
7.5 Summary .28
Bibliography.29
© ISO/IEC 2007 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC TR 24722:2007(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
In exceptional circumstances, the joint technical committee may propose the publication of a Technical Report
of one of the following types:
⎯ type 1, when the required support cannot be obtained for the publication of an International Standard,
despite repeated efforts;
⎯ type 2, when the subject is still under technical development or where for any other reason there is the
future but not immediate possibility of an agreement on an International Standard;
⎯ type 3, when the joint technical committee has collected data of a different kind from that which is
normally published as an International Standard (“state of the art”, for example).
Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether
they can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to
be reviewed until the data they provide are considered to be no longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC TR 24722, which is a Technical Report of type 2, was prepared by Joint Technical Committee
ISO/IEC JTC 1, Information technology, Subcommittee SC 37, Biometrics.
iv © ISO/IEC 2007 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC TR 24722:2007(E)
Introduction
Some applications of biometrics require a level of technical performance that is difficult to obtain with a single
biometric measure. Such applications include prevention of multiple applications for national identity cards and
security checks for air travel. In addition, provision is needed for people who are unable to give a reliable
biometric sample for some biometric modalities.
Use of multiple biometric measurements from substantially independent biometric sensors, algorithms or
modalities typically gives improved technical performance and reduces risk. This includes an improved level of
performance where not all biometric measurements are available such that decisions can be made from any
number of biometric measurements within an overall policy on accept/reject thresholds.
Of the various forms of multibiometric systems, the potential for multimodal biometric systems, each using an
independent measure, has been discussed in the technical literature since at least 1974 [22, 49]. Advanced
methods for combining measures at the score level have been discussed [15, 16]. At the current level of
understanding, combining results at the score level typically requires knowledge of both genuine and impostor
distributions. All of these measures are highly application-dependent and generally unknown in any real system.
Research on the methods not requiring previous knowledge of the score distributions is continuing and research
on fusion at both the image and feature levels is still progressing.
Given the current state of research into those questions and the highly application-dependent and generally
unavailable data required for proper fusion at the score level, work on multimodal and other multibiometric fusion
was considered not sufficiently mature to initiate an International Standard on the subject. Instead, it was
considered appropriate to publish a Technical Report on the subject. This Technical Report is meant to provide
information for future development of standards on multibiometric systems, in particular regarding the various
aspects of fusion. It will also provide a reference on multibiometric fusion for developers of other biometric
standards and implementers.
© ISO/IEC 2007 – All rights reserved v
---------------------- Page: 5 ----------------------
TECHNICAL REPORT ISO/IEC TR 24722:2007(E)
Information technology — Biometrics — Multimodal and other
multibiometric fusion
1 Scope
This Technical Report contains descriptions of and analyses of current practices on multimodal and other
multibiometric fusion, including (as appropriate) references to more detailed descriptions. It also discusses the
need for, and possible routes to, standardisation to support multibiometric systems.
This Technical Report contains descriptions and explanations of high-level multibiometric concepts to aid in the
explanation of multibiometric fusion approaches including multimodal, multiinstance, multisensorial,
multialgorithmic, decision-level and score-level logic.
2 Terminology issues
The primary motivation in addressing the terms and definitions in Clause 3 is to draw a distinction between
“multibiometric” and “multimodal” terms that appeared to be used in the literature interchangeably. To support
defining this terminology, the term “modality” is a key, and Table 1 provides a listing of modalities based on
CBEFF [30]. The distinction between conventional and unconventional categories is subjective, and based on
past and current biometric products.
© ISO/IEC 2007 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC TR 24722:2007(E)
Table 1 — Terms for biometric modalities or data types
Category Biometric Type
Other No Value Available
Multiple Multiple Biometric Types
Conventional Face
Voice
Finger
Iris
Retina
Hand Geometry
Signature or Sign
Unconventional Keystroke
Lip Movement
Gait
Vein
DNA
Ear
Foot
Scent
(Source: ISO/IEC 19785-1: 2006, Information technology — Common Biometric Exchange Formats Framework —
Part 1: Data element specification,Table 1 — Abstract values for BDB_biometric_type.)
2 © ISO/IEC 2007 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC TR 24722:2007(E)
3 Terms and definitions
Note: Two categories of terms are defined here:
⎯ terms that are specific to multimodal and multibiometric systems;
⎯ terms that are not specific to multimodal and multibiometric systems, but are required to define the terms
in the first category and not defined in the latest revision of ISO/IEC JTC 1/SC 37 Standing Document 2 [33].
For definitions of other terms in the subject field of biometrics, refer to ISO/IEC JTC 1/SC 37 Standing Document 2
[33].
For the purposes of this document, the following terms and definitions apply.
3.1
biometric characteristic
biometric (deprecated)
biological and behavioural characteristic of an individual that can be detected and from which distinguishing,
repeatable biometric features can be extracted for the purpose of automated recognition of individuals
Note 1: Biological and behavioural characteristics are physical properties of body parts, physiological and
behavioural processes created by the body and combinations of any of these.
Note 2: Distinguishing does not necessarily imply individualization.
Examples: Galton ridge structure, face topography, facial skin texture, hand topography, finger topography, iris
structure, vein structure of the hand, ridge structure of the palm, and retinal pattern.
3.2
biometric modality
the biometric characteristic which is used in a biometric process
3.3
biometric process
automated process using one or more biometric characteristics of a single individual for the purpose of
enrollment, verification or identification
3.4
biometric fusion
combination of information from multiple sources, i.e. sensors, modalities, algorithms, instances or
presentations
3.5
cascaded system
system where pass/fail thresholds of biometric samples are used to determine if additional biometric samples are
required to reach an overall system decision
3.6
layered system
system where individual biometric scores are used to determine the pass/fail thresholds of other biometric data
processing
3.7
multialgorithmic
using multiple algorithms for processing the same biometric sample
© ISO/IEC 2007 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC TR 24722:2007(E)
3.8
multibiometric
pertaining to multibiometrics
Note: Multibiometric has five distinct subcategories: multimodal, multiinstance, multisensorial, multialgorithmic and
multipresentation
3.9
multibiometric process
biometric process involving the use of biometric fusion
3.10
multibiometrics
automated recognition of individuals based on their biological or behavioral characteristics and involving the use
of biometric fusion
3.11
multiinstance
using multiple biometric instances within one biometric modality
Examples: Iris (left) + Iris (right), Fingerprint (left index) + Fingerprint (right index).
3.12
multimodal
using multiple different biometric modalities
Example: Fingerprint + Face.
3.13
multipresentation
using either multiple presentation samples of one instance of a biometric characteristic or a single presentation
that results in the capture of multiple samples
Example: Several frames from video camera capture of a face image (possibly but not necessarily consecutive).
Note: Multipresentation biometrics is considered a form of multibiometrics, if fusion techniques are employed.
Many fusion and normalisation techniques are appropriate to the integration of information from multiple
presentations of the same biometric instance.
3.14
multisensorial
using multiple sensors for capturing samples of one biometric instance
Examples: For face: infrared spectrum, visible spectrum, 2-D image and 3-D image. For fingerprint: optical,
electrostatic and acoustic sensors.
3.15
sequential presentation
capturing biometric samples in separate capture events to be used for biometric fusion
3.16
simultaneous presentation
capturing biometric samples in a single capture event to be used for biometric fusion
4 © ISO/IEC 2007 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC TR 24722:2007(E)
4 Overview of multimodal and other multibiometric systems
4.1 General
In general, the use of the terms multimodal or multibiometric indicates the presence and use of more than one
modality, sensor, instance, and/or algorithm in some form of combined use for making a specific biometric
identification or verification decision. The methods of combining multiple samples, matching scores or matching
decisions can be very simple or mathematically complex. For the purpose of this document, any method of
combination will be considered a form of “fusion”. Combination techniques will be covered in Clause 5 of this
document.
Multimodal biometrics were first proposed, implemented and tested in the 1970s. Combining measures was seen
as a necessary future requirement for biometric systems. It was widely thought that combining multiple measures
could increase either security by decreasing the false acceptance rate or user convenience by decreasing the
false rejection rate. These systems did not seem to advance into practical applications.
The use of fusion and related methods has been a key tool in the successful implementation of large scale
automated fingerprint identification systems (AFISs), starting in the 1980s. Until recently, multiple modalities have
not been used in AFIS; however, most methods of fusion discussed elsewhere in this report have been
successfully implemented using fingerprints alone. Some of the ways that fusion has been implemented in AFISs
include:
⎯ Image (AKA sample) fusion in creating a single “rolled” image from a series of plain impressions on a livescan
device;
⎯ Template fusion in the use of multiple feature extraction algorithms on each fingerprint image;
⎯ Multiinstance fusion in the use of fingerprints from all ten fingers;
⎯ Multipresentation fusion in the use of rolled and slap (plain) fingerprints;
⎯ Algorithm fusion for the purpose of efficiency (cost, computational complexity, and throughput rate); generally
matchers are used as a series of filters in order of increasing computational complexity. These are generally
implemented as a mix of decision and score-level fusion;
⎯ Algorithm fusion for the purpose of accuracy (decreasing false accept rate and/or false reject rate, lessening
sensitivity to poor-quality data); matchers are used in parallel, with fusion of resulting scores.
The use of fusion has made AFISs possible, because of fusion’s increase in both accuracy and efficiency.
Most work to date on multibiometrics has focused only on improving false acceptance and false rejection error
rates. Work at University of Kent, on project IAMBIC (Intelligent Agents for Multimodal Biometric Identification and
Control) is notable as it considers the use of multibiometrics to flexibly improve usability, security or accuracy [65].
To further the understanding of the distinction among the multibiometric categories, Table 2 illustrates the basic
distinctions among categories of multibiometric implementation. The key aspect of the category that makes it
multi-“something” is shown in boldface.
© ISO/IEC 2007 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC TR 24722:2007(E)
Table 2 — Multibiometric categories illustrated by the simplest case of using 2 of something
Biometric characteristic
Category Modality Algorithm Sensor
(e.g., body part)
2
2 2 2
Multimodal
b
(always) (always) (always) (usually)
1 2 1 1
Multialgorithmic
(always)
(always) (always) (always)
2 instances of 1 characteristic 1
1 1
Multiinstance
c
(always) (always) (always) (usually)
1 1 1 2
Multisensorial
a
(always) (usually) (always, and same instance) (always)
Multipresentation 1 1 1 1
a It is possible that two samples from separate sensors could be processed by separate “feature extraction”
algorithms, and then through a common comparison algorithm, making this “1.5 algorithms”, or two completely different
algorithms.
b Exception: a multimodal system with a single sensor used to capture two different modalities. For example a high
resolution image used to extract face and iris or face and skin texture.
c Exception may be the use of two individual sensors to each capture one instance, for example possibly a two-finger
fingerprint sensor.
Multimodal biometric systems take input from single or multiple sensors that capture two or more different
modalities of biometric characteristics. For example, a single system combining face and iris information for
biometric recognition would be considered a “multimodal” system regardless of whether face and iris images were
captured by different imaging devices or the same device. It is not required that the various measures be
mathematically combined in anyway. For example, a system with fingerprint and voice recognition would be
considered “multimodal” even if the “OR” rule was being applied, allowing users to be verified using either of the
modalities.
Multialgorithmic biometric systems receive a single sample from a single sensor and process that sample with
two or more algorithms. This technique could be applied to any modality. Maximum benefit (theoretically) would
be derived from algorithms that are based on distinctly different and independent principles (such algorithms may
be called “orthogonal”).
Multiinstance biometric systems use one (or possibly multiple) sensor(s) to capture samples of two or more
different instances of the same biometric characteristic. For example, systems capturing images from multiple
fingers are considered to be multiinstance rather than multimodal. However, systems capturing, for example,
sequential frames of facial or iris images are considered to be multipresentation rather than multiinstance.
Multisensorial biometric systems sample the same instance of a biometric characteristic with two or more
distinctly different sensors. Processing of the multiple samples can be done with one algorithm, or some
combination of multiple algorithms. For example, a face recognition application could use both a visible light
camera and an infrared camera coupled with a specific frequency (or several frequencies) of infrared illumination.
For a specific application in an operational environment, there are numerous system design considerations, and
trade-offs that must be made, among factors such as improved performance (e.g., identification or verification
6 © ISO/IEC 2007 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC TR 24722:2007(E)
accuracy, system speed and throughput, robustness, and resource requirements), acceptability, circumvention,
ease of use, operational cost, environmental flexibility, and population flexibility [44].
Especially for a large-scale human identification system, there are additional system design considerations such
as operation and maintenance, reliability, system acquisition cost, life cycle cost, and planned system response to
identified susceptible means of attack, all of which will affect the overall deployability of the system [44].
4.2 Simultaneous and sequential presentation
4.2.1 General multibiometric system model
A general multibiometric system model is shown in Figure 1. For explanatory purposes, this model uses three
biometric samples (P , P , P ) from 3 unique biometric modalities, except for where specified differently. At the
1 2 3
topmost level a subject presents their biometric characteristic(s) to the system. Dependent upon the system
design, there are two methods of presenting characteristics for acquisition by the system: 1) simultaneous and 2)
sequential.
Note: the presentation method (simultaneous or sequential) is distinct from the fusion process itself. The purpose of including this information
is to illustrate considerations that may influence multibiometric system design.
Figure 1 — Multibiometric system model
© ISO/IEC 2007 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/IEC TR 24722:2007(E)
4.2.2 Simultaneous presentation
Simultaneous presentation (with successful capture) provides biometric sample(s) from multiple modalities in a
single event (e.g., a face and iris taken from the same camera). System designs that utilize simultaneous
acquisition would tend toward high throughput applications at the expense of possible added complexity (to
synchronize sample collection) or difficulty of use (dual sensor interaction, user multi-tasking).
4.2.3 Sequential presentation
Sequential capture acquires biometric sample(s) from one or multiple modalities in separate events. Sequential
capture may be utilized in three concepts discussed in the literature. The first is multiinstance, which is the use of
two or more instances within one modality for a subject, i.e. Fingerprint (left index) + Fingerprint (right index). In
this example, one single digit fingerprint reader is used twice in sequence. The second concept is multimodal,
which is the use of multiple different biometric modalities captured from one or more sensors for a subject, i.e.
Hand + Face in sequence. The third concept is multisensorial, which is the use of two or more distinct sensors for
capturing the same biometric feature(s) for a subject, but not at the same time. To avoid confusion with
multimodal, which may also capture biometric feature(s) from two or more distinct sensors, multisensorial can be
clarified as “unimodal multisensorial”. Examples for face recognition are: infrared spectrum, visible spectrum, 2-D
image, and 3-D image; for fingerprint recognition: optical, electrostatic and acoustic sensors.
4.3 Correlation
In multimodal biometric systems the information being fused may be correlated at several different levels [57] as
illustrated in the following examples.
⎯ Correlation between modalities: This refers to biometric samples that are physically related such as the
speech and lip movement of a user.
⎯ Correlation due to identical biometric samples: This is the case in multialgorithmic systems where the same
biometric sample (e.g., a fingerprint image) or sub-sets of the biometric sample (e.g., voice, where an entire
sample may be used by one algorithm and part of the sample by another) is subjected to different feature
extraction and matching algorithms (e.g., a minutiae-based matcher and a texture-based matcher).
⎯ Correlation between feature values: A subset of feature values constituting the feature vectors of different
modalities may be correlated. For example, the area of a user’s palm (hand geometry) may be correlated with
the width of the face.
⎯ Correlation among instances due to common operating procedures (e.g., common capture device and
operator training).
⎯ Correlation among instances due to subject behaviour (e.g., coloured contact lenses on both eyes).
However, in order to determine the extent of correlation it is necessary to examine the match scores (or the
ACCEPT/REJECT decision) pertaining to the matchers involved in the fusion scheme. In the multiple classifier
system literature, it has been demonst
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.