ISO/IEC 29794-1:2009
(Main)Information technology — Biometric sample quality — Part 1: Framework
Information technology — Biometric sample quality — Part 1: Framework
ISO/IEC 29794-1:2009 specifies the derivation, expression and interpretation of biometric sample quality scores and data, and interchange of these scores and data via the multipart ISO/IEC 19794, Information technology — Biometric data interchange formats.
Technologies de l'information — Qualité d'échantillon biométrique — Partie 1: Cadre
General Information
Relations
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 29794-1
First edition
2009-08-01
Information technology — Biometric
sample quality —
Part 1:
Framework
Technologies de l'information — Qualité d'échantillon biométrique —
Partie 1: Cadre
Reference number
ISO/IEC 29794-1:2009(E)
©
ISO/IEC 2009
---------------------- Page: 1 ----------------------
ISO/IEC 29794-1:2009(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2009
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2009 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 29794-1:2009(E)
Contents Page
Foreword. iv
Introduction . v
1 Scope .1
2 Conformance.1
3 Normative references .1
4 Terms and definitions .2
5 Acronyms and abbreviated terms.5
6 General biometric system.5
7 Biometric sample quality criteria .5
7.1 Reference model.5
7.2 Quality components: character, fidelity, utility .6
7.3 Usefulness of quality data .7
7.3.1 Real-time quality assessment .7
7.3.2 Use in different applications.7
7.3.3 Use as a survey statistic .7
7.3.4 Accumulation of relevant statistics .8
7.3.5 Reference dataset improvement .8
7.3.6 Quality-based conditional processing .8
7.3.7 Interchange of quality data by disparate systems .8
8 Data interchange format field definition.9
8.1 Data fields.9
8.2 Quality score .9
8.2.1 Purpose.9
8.2.2 Data transformation considerations.10
8.2.3 Failure modes.10
8.2.4 Resolution .10
8.2.5 Summarization .10
8.3 Quality algorithm identification (QAID) .10
8.3.1 Overview.10
8.3.2 Methodology.10
8.4 Standardized exchange of quality algorithm results .11
9 Normalization .12
Annex A (informative) Procedures to construct a quality score normalization dataset.13
Annex B (informative) Example - standardized exchange of quality algorithm results.19
Annex C (informative) Procedures for aggregation of utility-based quality scores.21
Bibliography .23
© ISO/IEC 2009 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 29794-1:2009(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 29794-1 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 37, Biometrics.
ISO/IEC 29794 consists of the following parts, under the general title Information technology — Biometric
sample quality:
⎯ Part 1: Framework
⎯ Part 4: Finger image data [Technical Report]
⎯ Part 5: Face image data [Technical Report]
Future parts of ISO/IEC 29794 will address other modalities specified by ISO/IEC 19794, with part numbers
and titles aligned appropriately. However, as ISO/IEC 29794-1 is intended for use by all modalities, a modality
does not necessarily need a modality-specific part in order to make use of quality scores.
It is anticipated that a future version of each part of ISO/IEC 19794 will normatively reference
ISO/IEC 29794-1, and their respective data fields will be updated as required.
iv © ISO/IEC 2009 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 29794-1:2009(E)
Introduction
Quality metrics are useful for several applications in the field of biometrics. ISO/IEC 19784-1 specifies a
structure and gives guidelines for quality score categorization, and ISO/IEC 29794 defines and specifies
methodologies for objective, quantitative quality score expression, interpretation, and interchange. This part of
ISO/IEC 29794 is intended to add value to a broad spectrum of applications in a manner that
a) encourages competition, innovation, interoperability and performance improvements; and
b) avoids bias towards particular applications, modalities, or techniques.
This part of ISO/IEC 29794 presents several biometric sample quality scoring tools, the use of which is
generally optional but can be determined to be mandatory by particular application profiles or specific
implementations.
A number of applications can benefit from the use of biometric sample quality data; an example is the use of
real-time quality feedback upon enrolment to improve the operational efficiency and performance of a
biometric system. The association of quality data with biometric samples is an important component of quality
metric standardization. Quality fields as specified in 8.1 will be incorporated into data interchange formats. If a
CBEFF header is present, then CBEFF_BDB_quality can additionally be used to express quality data. Useful
analyses can be performed using quality data along with other data in order to improve the performance of a
biometric system. For example, correlating quality data to other system metrics can be used to diagnose
problems and highlight potential areas of performance improvement.
© ISO/IEC 2009 – All rights reserved v
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 29794-1:2009(E)
Information technology — Biometric sample quality —
Part 1:
Framework
1 Scope
For any or all biometric sample types as necessary, this part of ISO/IEC 29794
1. establishes terms and definitions that are useful in the specification, and use of quality metrics;
2. recommends the purpose and interpretation of biometric quality scores;
3. defines the format and placement of quality data fields in biometric data interchange formats;
4. suggests methods for developing biometric sample datasets for the purpose of quality score
normalization; and
5. suggests a format for exchange of quality algorithm results.
Outside the scope are the following:
1. the specification of minimum requirements for sample, module, or system quality scores;
2. performance assessment of quality algorithms; and
3. standardization of quality algorithms.
2 Conformance
A block of quality data is in conformity with this part of ISO/IEC 29794 if it conforms to the normative
requirements of Clause 8.
3 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
19794-1:2006, Information technology — Biometric data interchange formats — Part 1: Framework
19785-2:2006, Information technology — Common Biometric Exchange Formats Framework — Part 2:
Procedures for the operation of the Biometric Registration Authority
© ISO/IEC 2009 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC 29794-1:2009(E)
4 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
4.1
acquisition fidelity
fidelity of a sample attributed to the acquisition process
4.2
biometric failure to enrol
failure of the biometric system to store a usable biometric reference due to deficiencies in the biometric data
during an enrolment application
NOTE 1 Deficiencies in the biometric data can result from failure to capture an adequate or usable biometric sample,
failure to extract adequate or usable biometric features from the sample, or failure to generate an adequate or usable
biometric reference from the biometric features.
NOTE 2 See SC 37 N SD2 for most recent definition.
4.3
biometric failure to enrol rate
proportion of biometric enrolment sessions that resulted in a biometric failure to enrol for other than non-
biometric reasons
NOTE 1 Basing the denominator on the number of biometric enrolment sessions can result in a higher value than
basing it on the number of biometric capture subjects.
NOTE 2 The proportion denominator is the number of biometric enrolment sessions, excluding those sessions that
failed to complete for non-biometric reasons.
NOTE 3 See SC 37 N SD2 for most recent definition.
4.4
character
contributor to quality of a sample attributable to inherent features of the source
4.5
environment
physical surroundings and conditions where biometric capture occurs, including operational factors such as
operator skill and enrolee cooperation level
4.6
extraction fidelity
component of the fidelity of a sample attributed to the biometric feature extraction process
4.7
extrinsic
〈quality score〉 requiring reference to an external source, such as a standard, register, or technical
specifications, for full interpretation and normalization
4.8
fidelity
expression of how accurately a biometric sample represents its source biometric characteristic
NOTE The fidelity of a sample comprises components attributable to one or more of the processing steps:
acquisition, extraction, signal processing.
2 © ISO/IEC 2009 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 29794-1:2009(E)
4.9
intrinsic
〈quality score〉 conveying fully interpreted, normalized data without the requirement for additional extrinsic
information for quality score normalization
4.10
interpretation
process of analyzing a quality score along with other data in order to give that score contextual, relative
meaning
4.11
failure to acquire rate
proportion of the biometric application attempts that resulted in failure to acquire an adequate or usable
biometric sample, for other than non-biometric reasons
NOTE 1 The proportion denominator is the number of biometric enrolment attempts, excluding those attempts that
failed to complete for non-biometric reasons.
NOTE 2 See SC 37 N SD2 for most recent definition.
4.12
false match rate
FMR
proportion of the completed biometric non-match trials that result in a false match
NOTE 1 The value computed for the false match rate will depend on thresholds, and other parameters of the
comparison process, and the protocol defining the biometric non-match trials. In particular, treatment of comparisons
between
⎯ identical twins,
⎯ completely different biometric characteristics of different individuals, such as face topography and Galton ridges, and
⎯ different but related biometric characteristics from the same individual, such as left and right hand topography,
will need proper consideration. See ISO 19795-1.
NOTE 2 “Completed” refers to the computational processes required to make a comparison decision, i.e. failures to
decide are excluded.
NOTE 3 See SC 37 N SD 2 for most recent definition.
4.13
false non-match rate
FNMR
proportion of the completed biometric match trials that result in a false non-match
NOTE 1 The value computed for the false non-match rate will depend on thresholds and other parameters of the
comparison process, and the protocol defining the biometric match trials.
NOTE 2 “Completed” refers to the computational processes required to make a comparison decision, i.e. failures to
decide are excluded.
NOTE 3 See SC 37 N SD2 for most recent definition.
4.14
operator
individual who processes a user in a biometric system, performing or supervising capture and recapture
4.15
performance
assessment of the FMR, FNMR, failure to enrol rate and failure to acquire rate of a biometric system
© ISO/IEC 2009 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC 29794-1:2009(E)
4.16
quality
degree to which a biometric sample fulfils specified requirements for a targeted application
NOTE Specified quality requirements can address aspects of quality such as focus, resolution, etc. Implicit quality
requirements address the likelihood of achieving a correct matching result.
4.17
quality score
quantitative expression of quality
4.18
quality score normalization
rescaling of quality scores to improve consistency in scale and interpretation
4.19
quality score normalization dataset
QSND
dataset of biometric samples annotated with quality scores for use in quality score normalization
NOTE Target quality scores can be assigned on the basis of performance outcomes using the sample in question, or
can be based on quality factors recorded in acquisition of the dataset.
4.20
quality score percentile rank
QSPR
percentile rank of the quality score of a biometric sample, derived from its own utility score and those of other
samples in an identified control dataset
cf. quality score normalization dataset
4.21
raw quality score
quality score that has not been interpreted, either by the creator or recipient of the score, and alone can not
intrinsically provide contextual information
4.22
sample
image, signal, or pattern based interpretation of a physical human feature used for identification or verification
using biometric techniques
4.23
source
physical body part or function represented by a biometric sample
4.24
utility
observed performance of a biometric sample or set of samples in one or more biometric systems
NOTE 1 The character of the sample source and the fidelity of the processed samples contribute to – or similarly
detract from – the utility of the sample.
NOTE 2 Utility can combine performance measures such as FMR, FNMR, failure to enrol rate, and failure to acquire
rate.
4 © ISO/IEC 2009 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 29794-1:2009(E)
5 Acronyms and abbreviated terms
BDB biometric data block
BIR biometric information record
CBEFF common biometric exchange formats framework (ISO/IEC 19785)
FERET facial recognition technology database
FMR false match rate
FNMR false non-match rate
QAID quality algorithm identification
QSND quality score normalization dataset
QSPR quality score percentile rank
XML extensible markup language
6 General biometric system
A general biometric system is described in Standing Document 11, ISO/IEC JTC 1/SC 37 Part 1 Overview
Standards Harmonization Document (SC 37 N-SD11).
7 Biometric sample quality criteria
7.1 Reference model
In biometrics, the term “quality” is used to describe several different aspects of a biometric sample that
contribute to the overall performance of a biometric system. For the purposes of standardization, this
document defines terms, definitions, and a reference model for distinguishing between these different aspects
of quality, illustrated in Figure 1. Figure 2 illustrates the relationship between character, fidelity, quality, utility,
and system performance.
Source Image-based Feature-based
Processed
image feature
acquisition Sample
Sample
Sample
processing extraction
fidelity
fidelity fidelity
resolution
downsampling
feature quality
lighting
cropping
extraction algorithm
behavior
rotation
compression
character fidelity
Quality = Function [character, fidelity components]
Utility reflects the impact of the quality of a single sample on system performance
Figure 1 — Quality reference model illustration
© ISO/IEC 2009 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC 29794-1:2009(E)
character, fidelity (sample)
The correlation
quality scoring algorithm between predicted
All else equal, the
utility and observed
observed utility of a
utility of each sample
sample reflects its
is indicative of the
quality score (sample)
impact on the
effectiveness of the
performance of the
quality algorithm
correlation
system
predicted utility (sample) observed utility (sample)
correlation
A quality algorithm
should convey the
predicted utility of
the sample
matching
observed performance (system)
algorithm
The performance of a biometric
system is a function of the
matching algorithm
performance and the utility of
all samples in the system
Figure 2 — Relationship between quality and system performance
7.2 Quality components: character, fidelity, utility
The term “quality” as it is currently used in the field of biometrics has several connotations, depending on
context. Three prevalent uses are to subjectively reflect:
1. the character of a sample. An expression of quality based on the inherent features of the source from
which the biometric sample is derived. For example, a scarred fingerprint has poor character, and
blepharoptosis (droopy eyelid) causes poor iris character;
2. the fidelity of a sample to the source from which it is derived. An expression of quality based on
fidelity reflects the degree of its similarity to its source. Sample fidelity is comprised of fidelity
components contributed by different processes;
3. the utility of a sample within a biometric system. An expression of quality based on utility reflects the
predicted positive or negative contribution of an individual sample to the overall performance of a
biometric system. Utility-based quality is dependent on both the character and fidelity of a sample.
Utility –based quality is intended to be more predictive of system performance, e.g. in terms of FMR,
FNMR, failure to enrol rate, and failure to acquire rate, than measures of quality based on character or
fidelity alone. (See Table 1)
The term “quality” should not be solely attributable to the acquisition settings of the sample, such as image
resolution, dimensions in pixels, grayscale/color bit depth, or number of features. Though such factors may
affect sample utility and could contribute to the overall quality score.
Note that the character and utility of an acquired sample depend on the features to be considered by the
comparator. For instance, the same finger image may be of low character and utility with respect to minutiae
recognition (because of too few minutiae), but of high character and utility with respect to spectral pattern
recognition.
6 © ISO/IEC 2009 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 29794-1:2009(E)
Table 1 — Illustration of relationship between fidelity, utility, and character
Fidelity
Low High
Low Low fidelity and low character High fidelity and low character
results in low utility. Recapture results in low utility. Recapture will
Character
might improve utility. However, if not improve utility. Use of other
possible use of other biometric biometric characteristics is
characteristics is recommended. recommended.
High Samples with high character and Samples with high character and
low fidelity typically will not high fidelity indicate capture of
demonstrate high utility. Utility useful sample. High utility is
can be improved upon recapture expected.
or image enhancement
techniques.
7.3 Usefulness of quality data
7.3.1 Real-time quality assessment
Real-time quality data can be used by an operator, automated system, or user to help improve the average
quality of biometric samples submitted upon enrolment. This feedback might indicate the character, fidelity,
utility, and improvability of a sample. In this way, operational efficiency and overall system performance can
be improved by assisting an operator, or augmenting an automated quality control system, in decisions to a)
accept the sample, b) reject the sample, c) reattempt a capture, or d) declare a failure to acquire or failure to
enroll. Quality data can be retained for later use in, for example, determining whether an enrolment sample
should be replaced when the next sample is captured.
7.3.2 Use in different applications
A particular biometric sample might be used for several different applications and therefore its associated
quality data should be applicable to all of these. This would include both one-to-one and one-to-many
comparisons involving the use of comparison algorithms from different vendors that would interpret sample
features differently and yield different comparison scores. The challenge in establishing a universal quality
standard is in defining a metric that is sufficiently adaptable for use by all applications for which a particular
sample might be used given that metrics of utility vary greatly between applications. Therefore, it should be
recognized that it is a technical challenge to define a singular metric that accurately conveys the utility of a
biometric sample for all applications for which it may be used, and this should be taken into consideration in
defining quality standards. Thus a quality metric—ideally predicting performance for a comparator or class of
comparators —will likely be designed to capture only some of the failure modes and sensitivities of only a
limited number of biometric systems. It may be useful to apply more than one quality metric in order to
improve predictability of various failure modes.
It is useful for recipients of quality score data to be able to differentiate between scores generated by different
quality algorithms and capture equipment. This data may be used to enable recipient software to be
configured so that different thresholds or quality classifications can be applied to scores generated by different
algorithms. In addition, by differentiating between scores from different algorithms, a recipient may isolate
results from different algorithms and use the data to optimize thresholds accordingly.
7.3.3 Use as a survey statistic
Quality scores may be used to monitor operational quality. For example aggregated quality scores could be
compared with preset limits or monitored against an operational requirement. If, for example, quality scores
are generated from biometric samples collected at many sites, or over different time periods, then they may be
used to identify anomalous operation. For example, if face image quality is computed at the license issuance
© ISO/IEC 2009 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/IEC 29794-1:2009(E)
desks at a Department of Motor Vehicles, then a ranked list of aggregated quality scores could be used to
identify desks that exhibit a lower than average quality, or to monitor trends over weeks or months.
7.3.4 Accumulation of relevant statistics
Reliable quality scores may be used to survey users and transactions to accumulate statistics giving
conditional probabilities of the kind “given a quality X sample on finger A, what is the likelihood of a quality Y
sample from finger A (or finger B)". This will inform the system and/or operators over whether a higher quality
sample is likely if another capture is attempted.
7.3.5 Reference dataset improvement
The association of quality data with a sample that is to be entered into a reference dataset is important for the
maintenance and improvement of the reference dataset quality. The tracking of sample quality can lead to
detection of potential deterioration of operator training or it may indicate deterioration in the performance of
the sample capture equipment. Tracking of the sample quality data should be an important part of the
biometric system’s operating procedures. The quality data may also be used to improve the quality of the
reference file, and hence the performance of the biometric system. Improvement can be made by the
replacement or possible augmentation of the stored information so as to make use of the best available quality
data. Typically, the replacement decisions are linked to the comparator performance of the system processing
the data.
7.3.6 Quality-based conditional processing
Biometric samples can be processed differently based on quality met
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.