ISO/IEC 30116:2016

Information technology - Automatic identification and data capture techniques - Optical Character Recognition (OCR) quality testing

ISO/IEC 30116:2016 - specifies the methodology for the measurement of specific attributes of OCR-B character strings, - defines a method for evaluating these measurements and deriving an overall assessment of character string quality, - defines a reference decode algorithm for OCR-B, and - gives information on possible causes of deviation from optimum grades to assist users in taking appropriate corrective action. ISO/IEC 30116:2016 applies to OCR-B as defined in ISO 1073‑2, but its methodology can be applied partially or wholly to other OCR fonts.

Technologies de l'information — Techniques automatiques d'identification et de capture des données — Essais de qualité des caractères pour reconnaissance optique

General Information

• Status: Published
• Publication Date: 04-Oct-2016
• ICS: 35.040 - Information coding
• ICS: 35.040.50 - Automatic identification and data capture techniques
• Technical Committee: ISO/IEC JTC 1/SC 31 - Automatic identification and data capture techniques
• Drafting Committee: ISO/IEC JTC 1/SC 31/WG 1 - Data carrier
• Current Stage: 9093 - International Standard confirmed
• Start Date: 11-Apr-2022
• Completion Date: 30-Oct-2025

Overview

ISO/IEC 30116:2016 - "Information technology - Automatic identification and data capture techniques - Optical Character Recognition (OCR) quality testing" defines a standardized methodology for measuring and assessing the quality of OCR-B character strings. The standard specifies how to capture and process images of OCR symbols, how to quantify key attributes, and how to derive an overall quality grade. It also provides a reference decode algorithm for OCR-B and guidance on likely causes of deviations from optimum grades to support corrective action.

Key Topics and Technical Requirements

• Scope: Applies to OCR‑B (ISO 1073‑2) but the methodology can be applied partially or wholly to other OCR fonts.
• Image acquisition: Capture a high-resolution raw image under controlled, uniform illumination and best focus. Effective resolution should ensure character stroke widths span at least ten image pixels.
• Reference grey-scale image: Derived from the raw image by convolving pixel values with a synthetic circular aperture of 0.2 mm.
• Binarized image: Produced from the reference grey-scale using the thresholding algorithm defined in Annex B.
• Reflectance measurements: Calibrated reflectance values expressed as percentages (100% = barium sulphate or magnesium oxide reference). Use LED illumination at 890 nm and 940 nm; illumination elements ≤ 25 mm.
• Measurement parameters: Includes Best‑Fit (character placement), PCS/contrast, Position, Background Noise, Stroke Width Template (SWT) and Character Evaluation Value (CEV).
• Quality grading: Each parameter is graded as Recommended, Needs attention, or Not recommended. The lowest parameter grade determines the overall scan grade.
• Normative artifacts: Annex A (character centreline coordinates), Annex B (threshold determination), Annex C (reference decode algorithm), and worked examples for CEV calculations.
• Environmental/control notes: Ambient temperature guidance (20–25 °C) and IR absorption requirements for surrounding materials.

Applications and Who Uses It

• Identity document issuers and designers (passports, ID cards, driving licences) to validate OCR readability of MRZs.
• Test laboratories and quality assurance teams performing OCR quality testing and compliance verification.
• Manufacturers of document scanners, OCR engines and AIDC (automatic identification and data capture) devices for product validation and benchmarking.
• Border control and Automated Border Control (ABC) system integrators to ensure reliable MRZ recognition and secure access (e.g., BAC/SAC in e‑passports).

Related Standards

• ISO 1073‑2 (OCR‑B font specification)
• ICAO Doc 9303 (Machine Readable Travel Documents / MRZ)
• ISO/IEC 7501 and ISO/IEC 18013 (document formats)
• ISO/IEC 19762 (AIDC harmonized vocabulary)

ISO/IEC 30116:2016 provides a practical, reproducible framework for assessing OCR quality, helping organizations improve OCR reliability and interoperability across document inspection and automated reading systems.

Standards Content (Sample)

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 30116
ISO/IEC JTC 1/SC 31 Secretariat: ANSI
Voting begins on: Voting terminates on:
2015-09-30 2015-12-30
Information technology — Automatic identification and
data capture techniques — Optical Character Recognition
(OCR) quality testing
Titre manque
ICS: 35.040
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 30116:2015(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
©
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2015

ISO/IEC DIS 30116:2015(E)
© ISO/IEC 2015, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. [phone]
Fax [phone]
[e-mail]
www.iso.org
ii © ISO/IEC 2015 – All rights reserved

ISO/IEC WD 30116
Contents Page
Foreword .v
Introduction .vi
1 Scope .1
2 Normative references .1
3 Terms and definitions .1
3.1 binarized image .1
3.2 document reference edge .1
3.3 inspection area .2
3.4 COL .2
3.5 pixel.2
3.6 raw image .2
3.7 reference grey-scale image .2
3.8 scan grade .2
3.9 stroke width .2
3.10 SWT .2
3.11 symbol .3
3.12 X-Tolerance.3
3.13 Y-Tolerance.3
4 Abbreviated terms .3
5 Quality grading .3
6 Measurement methodology for OCR-B .3
6.1 Overview of methodology .3
6.2 Obtaining the test image .3
6.2.1 Measurement conditions.3
6.2.2 Raw image .4
6.2.3 Reference grey-scale image.4
6.2.4 Binarized image .4
6.3 Reference reflectivity measurements .4
6.3.1 General requirements.4
6.3.2 Light sources .4
6.3.3 Effective resolution .4
6.3.4 Optical geometry .4
6.3.5 Inspection area .7
6.4 Basis of symbol grading .8
6.5 Capture the raw image .8
6.6 Image assessment parameters and grading .8
6.6.1 Determining the document horizontal axis .8
6.6.2 Character best-fit algorithm .8
6.6.3 Position of a character .10
6.6.4 Character Evaluation Value (CEV) in the best-fit location .10
6.6.5 Background noise .11
6.6.6 Contrast PCS of the characters .11
7 Reporting the Grade .11
Annex A OCR-B Character Centreline Coordinates (Normative) .13
Annex B Threshold Determination Method (Normative) .21
B.1 Algorithm description .21
B.2 Example .21
Annex C OCR Reference Decode Algorithm (Normative) .25
Annex D Example calculation of Character Evaluation Value (CEV)

© ISO/IEC 2011 – All rights reserved iii

ISO/IEC WD 30116
Bibliography
iv © ISO/IEC 2011 – All rights reserved

ISO/IEC WD 30116
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the International
Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 30116 was prepared by Technical Committee ISO/TC , , Subcommittee SC 31, Automatic identification
and data capture techniques.
This second/third/. edition cancels and replaces the first/second/. edition (), [clause(s) / subclause(s) / table(s)
/ figure(s) / annex(es)] of which [has / have] been technically revised.
© ISO/IEC 2011 – All rights reserved v

ISO/IEC WD 30116
Introduction
For the inspection of ID documents, i.e. MRTDs (Machine Readable Travel Documents) according to ISO/IEC
7501 and driving licences according to ISO/IEC 18013, a reliable and ergonomic document inspection
technology is essential. Considering RFID interoperability, strong improvement has been reached introducing
mechanisms for interoperability evaluation and testing of MRTDs and reader devices. Similar standards for
optical reading would improve the reliability of OCR. This is especially important because OCR of the
document’s MRZ (Machine Readable Zone) is essential for accessing BAC (Basic Access Control) and/or SAC
(Supplementary Access Control) protected passports.
Thus, reliable OCR makes the performance of Automated Border Control systems as well as of many other
applications more predictable. Furthermore, the evaluation of document reader products can be done much
easier. This standardization project defines test methods to evaluate OCR document quality. Furthermore, it
defines requirements ensuring the compliance to the applicable OCR standards. The project applies
experiences from other domains such as bar code reading and possibly other test methods for OCR. Where
conflicts in the specification work between MRTDs and driving licenses may arise, satisfying the definitions for
MRTDs shall be given preference.
vi © ISO/IEC 2011 – All rights reserved

DIS ISO/IEC WD 30116
Information technology — Automatic identification and data
capture techniques — Optical Character Recognition (OCR)
quality testing
1 Scope
This International Standard
-specifies the methodology for the measurement of specific attributes of OCR-B character strings;
-defines a method for evaluating these measurements and deriving an overall assessment of character string
quality;
-defines a reference decode algorithm for OCR-B;
-gives information on possible causes of deviation from optimum grades to assist users in taking appropriate
corrective action.
This International Standard applies to OCR-B as defined in ISO 1073-2, but its methodology can be applied
partially or wholly to other OCR fonts.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced document
(including any amendments) applies.
ISO/IEC 19762 Information technology -- Automatic identification and data capture (AIDC) techniques --
Harmonized vocabulary
3 Terms and definitions
3.1
binarized image
binary (black/white) image created by applying the Global Threshold to the pixel values in the reference grey-
scale image
3.2
document reference edge
physical (i.e. mechanical) end of the surface with the MRZ whose position is determined by putting a black
background under the surface with the MRZ and sliding the document up against a physical stop
© ISO/IEC 2011 – All rights reserved 1

ISO/IEC WD 30116
3.3
inspection area
rectangular area which contains the entire symbol to be tested inclusive of its quiet zones
3.4
character outline limits
outlines of an ideal printed image of a character
Note: this is a qualitative evaluation utilized in ISO 1836 that is replaced in this standard with SWT
3.5
pixel
individual light-sensitive element in a light-sensitive array (e.g. CCD (charge coupled device) or CMOS
(complementary metal oxide semiconductor) device)
3.6
raw image
matrix of the reflectance values in x and y coordinates across a two-dimensional image, derived from the
discrete reflectance values of each pixel of the light-sensitive array
3.7
reference grey-scale image
raw image convolved with a synthesised circular aperture
3.8
scan grade
result of the assessment of a single scan of an OCR symbol, derived by taking the lowest grade achieved for
any measured parameter of the reference grey-scale and binarized images
3.9
stroke width
nominal dimension perpendicular to the direction of the line making up an OCR character
3.10
stroke width template
the inner and outer character boundaries defined by circles whose centres follow the line created by the
character centreline coordinates defined in Annex A
2 © ISO/IEC 2011 – All rights reserved

ISO/IEC WD 30116
3.11
symbol
group of OCR characters comprising the entire machine-readable entity (e.g. Machine Readable Zone (MRZ)
as specified in ICAO Doc 9303, sizes ID-1, ID-2, and ID-3) including quiet zones and the document reference
edge
3.12
X-Tolerance
0,08 mm for Size I with a nominal stroke width of 0,35 mm
Note: 0,08 mm for Size I with a nominal stroke width of 0,35 mm was originally defined in ISO 1831 Table 2
3.13
Y-Tolerance
0,15 mm for Size I with a nominal stroke width of 0,35 mm
Note: 0,15 mm for Size I with a nominal stroke width of 0,35 mm was originally defined in ISO 1831 Table 2
4 Abbreviated terms
COL = character outline limits
CEV = character evaluation value
MRZ = Machine Readable Zone
SWT = Stroke width template
5 Quality grading
Quality grades for Best-Fit, PCS, Position and Background Noise are determined as one of three levels,
Recommended, Needs attention and Not recommended. The parameter with the lowest grade is the grade of
the symbol.
6 Measurement methodology for OCR-B
6.1 Overview of methodology
The basis of the measurement methodology is the evaluation of reflectance from the symbol. This methodology
is also intended to correlate with conditions encountered in OCR scanning systems. The method starts by
obtaining the raw image, which is a high-resolution grey-scale image of the symbol captured under controlled
illumination and viewing conditions.
6.2 Obtaining the test image
6.2.1 Measurement conditions
A test image of the symbol shall be obtained in a configuration that mimics the typical scanning situation for that
symbol, but with substantially higher resolution (see 6.3.3), uniform illumination, and at best focus. The
reference optical arrangement is defined in 6.3.4. Alternative optical arrangements may be used provided that
the measurements obtained with them can be correlated with the use of the reference optical arrangement.
© ISO/IEC 2011 – All rights reserved 3

ISO/IEC WD 30116
Ambient light levels shall be controlled in order not to influence the measurement results. Whenever possible,
measurements shall be made on the symbol in its final configuration, i.e. the configuration in which it is intended
to be scanned. For MRTD evaluation, optically personalized samples shall be used. This includes that all layers
available at a document including laminations, security features and protective layers shall be present.
Two principles govern the design of the optical set-up. First, the test image’s grey-scale shall be nominally linear
and not be enhanced in any way. Second, the image resolution shall be adequate to produce consistent
readings, which generally requires that the character stroke-widths span at least ten image pixels.
6.2.2 Raw image
The raw image is a matrix of the actual reflectance values for each pixel of the light-sensitive array, from which
are derived the reference grey-scale image and the binarized image which are evaluated for the assessment of
symbol quality.
6.2.3 Reference grey-scale image
The reference grey-scale image is obtained from the raw image by processing the individual pixel reflectance
values through a synthetic circular aperture equal to 0,2 mm.
6.2.4 Binarized image
The binarized image is obtained from the reference grey-scale image by applying the algorithm defined in Annex
B.
6.3 Reference reflectivity measurements
6.3.1 General requirements
Equipment for assessing the quality of symbols in accordance with this clause shall comprise a means of
measuring and analysing the variations in the reflectivity of a symbol on its substrate over an inspection area
which shall cover the full height and width of the symbol.
The measured reflectance values shall be expressed in percentage terms by means of calibration and reference
to recognised national standards laboratories, where 100 per cent should correspond to the reflectance of a
barium sulphate or magnesium oxide reference sample.
It should be ensured that all materials visible to the camera or close to the optical path are reflection free, at
least in IR illumination. In particular, the background the symbol is attached to shall be IR absorbing. The
environment temperature shall be between 20 and 25°C.
6.3.2 Light sources
Measurements shall be made using LED light sources at 890 and 940 nm wavelengths.
All illumination elements shall have a diameter of 25 mm or less and may be shaped as circles, squares or
similar.
6.3.3 Effective resolution
The effective resolution of an instrument that implements this international standard shall be sufficient to ensure
that the parameter grading results are consistent irrespective of the rotation of the symbol. The effective
resolution is the product of the resolution of the light-sensitive array and of the magnification of the associated
optical system and effected by distortions introduced by the optical system. The reference optical arrangement
requires an effective resolution of not less than ten pixels per stroke width.
6.3.4 Optical geometry
A reference optical geometry is defined for reflectivity measurements and consists of:
4 © ISO/IEC 2011 – All rights reserved

ISO/IEC WD 30116
 flood incident illumination, uniform across the inspection area, from a set of four light sources arranged at
90 degree intervals around a circle concentric with the inspection area and in a plane parallel to that of the
inspection area, at a height which will allow incident light to fall on the centre of the inspection area at an
angle of 45° to its plane, and
 a light collection device, the optical axis of which is perpendicular to the inspection area and passes through
its centre, and which focuses an image of the test symbol on a light-sensitive array.
The light reflected from the inspection area shall be collected and focussed on the light-sensitive array.
Implementations may use alternative optical geometries and components, provided that their performance can
be correlated with that of the reference optical arrangement defined in this section. Figures 1 and 2 illustrate
the principle of the optical arrangement, but are not intended to represent actual devices; in particular the
magnification of the device is likely to differ from 1:1. For example, it is possible to use a 10MP industrial camera
without IR cut filter with a sensor size of ½”. The image could be captured from a distance of approx. 350 mm
and the lens chosen appropriately. The resulting magnification then would be 1:21.

A
4 4
B


1 – Light sensing element
2 – Lens providing 1:1 magnification (measurement A = measurement B)
3 – Inspection area
4 – Light sources
 - Angle of incidence of light relative to plane of symbol = 45°
Figure 1 — Reference optical arrangement – side view

© ISO/IEC 2011 – All rights reserved 5

ISO/IEC WD 30116
Figure 2 — Reference optical arrangement – top view

Figure 3 — Reference optical arrangement – angles and tolerances

6 © ISO/IEC 2011 – All rights reserved

ISO/IEC WD 30116
When setting up a reference optical arrangement, the following considerations shall be made. The Symbol (e.g.,
the MRZ of an ID-3 MRTD) has a size of approx. 24 mm x 125 mm (marked as a bar in the bottom of Figure 3).
For other document sizes (ID-1, ID-2) the same size of the object to be captured shall be used. All areas not
covered by the travel document but visible to the camera shall be made of IR absorbing material. The small
rectangles at the left and right border of Figure 3 represent the illumination elements.
The nominal illumination angle shall be 45° as given in Figure 1. This angle is measured in the middle axis of
the MRZ zone. An angle of 45° directly determines that the horizontal and vertical distance of the illumination
from the centre of the symbol is identical. This distance should be 200 mm as depicted in Figure 3.
For the small side of the Symbol (24 mm) the minimal and maximal illumination angles at the symbol borders
are 43,3° and 46,8°, as shown in Table 1.
The difference between the nominal angle (in the centre) and the angles at the borders is much higher for the
long side of the symbol; 37,3° and 55,5°.
Table 1 — Dark pixel portion for threshold of 4.5
Angle Short side (24 mm) Long side (125 mm)
α 45° 45°
α’ 46,8° 55,5°
α’’ 43,3° 37,3°
Figure 4 — Reference optical arrangement – reflection considerations

Figure 4 shows the position where the direct reflection of the illumination at the symbol (i.e. caused by plain
lamination) will not visible to the camera. The distance between the camera and the symbol should be
approximately 350 mm.
6.3.5 Inspection area
The inspection area within which all measurements shall be a rectangular area framing the complete symbol.
The centre of the inspection area shall be as close as practicable to the centre of the field of view. For
example, the MRZ in a passport shall be placed accordingly.
© ISO/IEC 2011 – All rights reserved 7

ISO/IEC WD 30116
6.4 Basis of symbol grading
OCR symbol quality assessment shall be based on the measurement and grading of parameters of the
reference grey-scale image, the binarized image derived from it, and the application of the reference decode
algorithm to these. Quality grading of these parameters shall be used to provide a relative measure of symbol
quality under the measurement conditions used.
6.5 Capture the raw image
Centre the symbol in the field of view and align the average bottom edge of the characters with the sensor as
precisely as possible, but always with less than +/- 5 degrees deviation.
Find and replace the brightest and darkest 0,005% pixels in the overall image with the median of the nine pixels
consisting of itself and its eight immediate neighbours.
Apply the aperture defined in 6.2.3 to the raw image to create a reference grey-scale image.
6.6 Image assessment parameters and grading
6.6.1 Determining the document horizontal axis
The application shall define the MRZ region in relation to the Document Reference Edge.
6.6.2 Character best-fit algorithm
These steps should be followed in order to find the best-fit position of the character outline limit (SWT) gauges
on a character image captured from a machine readable character string in order to find the location of the
characters.
Using the binarized image, determine four corner positions that bound the character image. From these points,
establish four more points that are further away from the character by the nominal stroke width. These four
points define the range over which the SWT gauges will be moved in order to find the best-fit position.
Horizontal range
Vertical range
Figure 5 — Sample corner positions

Create the SWT for each defined character in Annex A by moving a circle of radius of the appropriate tolerance
value around the centreline of the character and saving the outermost points (note that this is equivalent to
finding the points perpendicular to the centre line at a distance equal to the radius of the circle). An example of
this process is in Figure 6 below.
8 © ISO/IEC 2011 – All rights reserved

ISO/IEC WD 30116
140 140
120 120 120
100 100 100
80 80 80
60 60
40 40
20 20
0 0
-50 -40 -30 -20 -10 0 10 20 30 40 -50 -40 -30 -20 -10 0 10 20 30 40 -50 -40 -30 -20 -10 0 10 20 30 40 50

Figure 6 — SWT creation example

40 40
20 20
0 0
-60 -40 -20 0 20 40 60
-60 -40 -20 0 20 40 60
-20 -20
© ISO/IEC 2011 – All rights reserved 9

ISO/IEC WD 30116
Figure 7 — Left: Radius 6,75 “sq” inside 10,75 “sq” outside (Note: this is the X-tolerance from ISO
1831 Table 2). Right: Radius 5 “sq” inside 12,5 “sq” outside (Note: this is the Y-tolerance from ISO
1831 Table 2). “Sq” is the count of squares from the original drawings (See Annex A) where each
square equals 1/50 mm.
Overlay the original captured image with the SWT Y-tolerance gauges such that the horizontal axis of the
gauges is parallel with the Document Reference Edge. Starting with the four extreme positions, move the SWT
Y-tolerance gauges right and left, and up and down relative to the Document Reference Edge to each test
position.
At each test position, sum up the reflectance values of each SWT Y-tolerance gauge resolution pixel within the
region defined by the minimum gauge. The test position with the lowest sum is used to determine the position
of the character. If there is more than one test position with the same lowest sum (e.g. the inside of the minimum
gauge is all black), then for each equivalent test position, sum up the reflectance values of each gauge resolution
pixel outside the maximum gauge. The equivalent test position with the highest sum is used to determine the
position of the character. If there is more than one test position with the same highest sum, then compute the
average of these positions and use it to determine the position of the character.
6.6.3 Position of a character
Using the test position determined in 6.6.2, the location of a character is the origin of the SWT, where the
origin is defined as (0,0) for every character in Annex A.
The position of every character is determined and graded according to ???. The character with the lowest grade
determines the Position Grade.
6.6.4 Character Evaluation Value (CEV) in the best-fit location
A pixel is outside or inside a border if more than 50% of the pixel area is outside or inside the border respectively.
For each tolerance template on every character, use the optimal position of the template placed over the
binarized character in the image. The following lists the variables to be used to calculate the CEV grades.
CEV_X_Inside – Number of white pixels inside the X-tolerance inner boundary.
CEV_X_Outside – Number of black pixels outside the X-tolerance outer boundary.
CEV_Y_Inside – Number of white pixels inside the Y-tolerance inner boundary.
CEV_Y_Outside – Number of black pixels outside the Y-tolerance outer boundary.
Y_Boundary_Area – Total number of image pixels of any color inside the Y-tolerance outer boundary.
Y_Inside_Total – Total number of image pixels of any color inside the Y-tolerance inner boundary.
Character_Region_Total – The total number of image pixels in the rectangular area defined by the Y-tolerance
outer template of the chosen character plus a one-stroke width boundary. The Character Region Total is
computed as the product of the width of the Y-tolerance outer boundary plus two times the nominal stroke width
rounded to the nearest number of pixels and the height of the Y-tolerance outer boundary plus the two times
the nominal stroke width rounded to the nearest number of pixels.

From these measurements, compute the following graded parameters.
Character_Inside_Fit = CEV_Y_Inside / Y_Inside_Total
Character_Outside_Fit = CEV_Y_Outside / (Character_Region_Total - Y_Boundary_Area)
10 © ISO/IEC 2011 – All rights reserved

ISO/IEC WD 30116
Grade the total results as follows.
Character_Inside_Fit > 10% need attention
Character_Inside_Fit > 20% not recommended
Character_Outside_Fit > 1% need attention
Character_Outside_Fit > 2% not recommended
The character with the lowest grade determines the Character Evaluation Value (CEV) Grade of the entire MRZ.
Note: CEV_X_Inside and CEV_X_Outside are useful for process control.
See Annex D for an example calculation of Character Evaluation Value (CEV).
6.6.5 Background noise
Find the background noise, N.
Measure the maximum and minimum reflectance levels inside a rectangular area equal to the height and
width of a character (1,4 x 2,4 mm) centred vertically between the two
...

INTERNATIONAL ISO/IEC
STANDARD 30116
First edition
2016-10-01
Information technology — Automatic
identification and data capture
techniques — Optical Character
Recognition (OCR) quality testing
Technologies de l’information — Techniques automatiques
d’identification et de capture des données — Essais de qualité des
caractères pour reconnaissance optique
Reference number
©
ISO/IEC 2016
© ISO/IEC 2016, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. [phone]
Fax [phone]
[e-mail]
www.iso.org
ii © ISO/IEC 2016 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 3
5 Quality grading . 3
6 Measurement methodology for OCR-B . 3
6.1 Overview of methodology . 3
6.2 Obtaining the test image . 3
6.2.1 Measurement conditions . 3
6.2.2 Raw image . 3
6.2.3 Reference grey-scale image . 3
6.2.4 Binarized image . 4
6.3 Reference reflectivity measurements . 4
6.3.1 General requirements . 4
6.3.2 Light sources . 4
6.3.3 Effective resolution . 4
6.3.4 Optical geometry . 4
6.3.5 Inspection area . 8
6.4 Basis of symbol grading . 8
6.5 Capture the raw image . 9
6.6 Image assessment parameters and grading . 9
6.6.1 Determining the document horizontal axis . 9
6.6.2 Character best-fit algorithm . . 9
6.6.3 Position of a character .11
6.6.4 Character evaluation value (CEV) in the best-fit location .11
6.6.5 Background noise . .12
6.6.6 Contrast PCS of the characters .12
7 Reporting the grade .12
Annex A (normative) OCR-B character centreline coordinates .14
Annex B (normative) Threshold determination method .20
Annex C (normative) OCR reference decode algorithm .24
Annex D (informative) Example calculation of character evaluation value (CEV) .25
Bibliography .29
© ISO/IEC 2016 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity assessment,
as well as information about ISO’s adherence to the World Trade Organization (WTO) principles in the
Technical Barriers to Trade (TBT) see the following URL: www.iso.org/iso/foreword.html.
The committee responsible for this document is ISO/JTC 1, Information technology, Subcommittee SC 31,
Automatic identification and data capture techniques.
iv © ISO/IEC 2016 – All rights reserved

Introduction
For the inspection of ID documents, i.e. MRTDs (Machine Readable Travel Documents) according to
ISO/IEC 7501 (all parts)/ICAO Doc 9303 (all parts) and driving licences according to ISO/IEC 18013
(all parts), a reliable and ergonomic document inspection technology is essential. Considering RFID
interoperability, strong improvement has been reached introducing mechanisms for interoperability
evaluation and testing of MRTDs and reader devices. Similar standards for optical reading would
improve the reliability of OCR. This is especially important because OCR of the document’s MRZ (Machine
Readable Zone) is essential for accessing BAC (Basic Access Control) and/or SAC (Supplementary Access
Control) protected passports.
Thus, reliable OCR makes the performance of automated border control systems, as well as of many
other applications, more predictable. Furthermore, the evaluation of document reader products can be
done much easier. This standardization project defines test methods to evaluate OCR document quality.
Furthermore, it defines requirements ensuring the compliance to the applicable OCR standards. The
project applies experiences from other domains such as bar code reading and possibly other test
methods for OCR. Where conflicts in the specification work between MRTDs and driving licenses may
arise, satisfying the definitions for MRTDs is given preference.
© ISO/IEC 2016 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 30116:2016(E)
Information technology — Automatic identification and
data capture techniques — Optical Character Recognition
(OCR) quality testing
1 Scope
This document
— specifies the methodology for the measurement of specific attributes of OCR-B character strings,
— defines a method for evaluating these measurements and deriving an overall assessment of character
string quality,
— defines a reference decode algorithm for OCR-B, and
— gives information on possible causes of deviation from optimum grades to assist users in taking
appropriate corrective action.
This document applies to OCR-B as defined in ISO 1073-2, but its methodology can be applied partially
or wholly to other OCR fonts.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1
binarized image
binary (black/white) image created by applying the global threshold to the pixel (3.5) values in the
reference grey-scale image
3.2
document reference edge
physical (i.e. mechanical) end of the surface with the MRZ whose position is determined by putting a
black background under the surface with the MRZ and sliding the document up against a physical stop
3.3
inspection area
rectangular area which contains the entire symbol (3.11) to be tested inclusive of its quiet zones
3.4
character outline limits
outlines of an ideal printed image of a character
Note 1 to entry: This is a qualitative evaluation utilized in ISO 1831 that is replaced in this document with SWT.
© ISO/IEC 2016 – All rights reserved 1

3.5
pixel
individual light-sensitive element in a light-sensitive array
Note 1 to entry: Examples of light-sensitive array are CCD (charge coupled device) or CMOS (complementary
metal oxide semiconductor) device.
3.6
raw image
matrix of the reflectance values in x and y coordinates across a two-dimensional image, derived from
the discrete reflectance values of each pixel (3.5) of the light-sensitive array
3.7
reference grey-scale image
raw image (3.6) convolved with a synthesized circular aperture
3.8
scan grade
result of the assessment of a single scan of an OCR symbol, derived by taking the lowest grade achieved
for any measured parameter of the reference grey-scale and binarized images (3.1)
3.9
stroke width
nominal dimension perpendicular to the direction of the line making up an OCR character
3.10
stroke width template
inner and outer character boundaries defined by circles whose centres follow the line created by the
character centreline coordinates defined in Annex A
3.11
symbol
group of OCR characters comprising the entire machine-readable entity (e.g. Machine Readable Zone
(MRZ) as specified in ICAO 9303, sizes ID-1, ID-2 and ID-3) including quiet zones and the document
reference edge (3.2)
Note 1 to entry: Document sizes are defined in ISO/IEC 7501 (all parts) (ICAO 9303) as TD1, TD2 and TD2,
whereas the same sizes are defined in ISO/IEC 7810 as ID-1, ID-1 and ID-3. In this document, we use the terms
ID-1, ID-2 and ID-3.
3.12
X-tolerance
0,08 mm for Size I with a nominal stroke width (3.9) of 0,35 mm
Note 1 to entry: 0,08 mm for Size I with a nominal stroke width of 0,35 mm was originally defined in
ISO 1831:1980, Table 2.
3.13
Y-tolerance
0,15 mm for Size I with a nominal stroke width (3.9) of 0,35 mm
Note 1 to entry: 0,15 mm for Size I with a nominal stroke width of 0,35 mm was originally defined in
ISO 1831:1980, Table 2.
2 © ISO/IEC 2016 – All rights reserved

4 Abbreviated terms
COL character outline limits
CEV character evaluation value
MRZ machine readable zone
SWT stroke width template
5 Quality grading
Quality grades for best-fit, PCS, position and background noise are determined as one of three levels:
recommended, needs attention and not recommended. The parameter with the lowest grade is the
grade of the symbol.
6 Measurement methodology for OCR-B
6.1 Overview of methodology
The basis of the measurement methodology is the evaluation of reflectance from the symbol. This
methodology is also intended to correlate with conditions encountered in OCR scanning systems. The
method starts by obtaining the raw image, which is a high-resolution grey-scale image of the symbol
captured under controlled illumination and viewing conditions.
6.2 Obtaining the test image
6.2.1 Measurement conditions
A test image of the symbol shall be obtained in a configuration that mimics the typical scanning
situation for that symbol, but with substantially higher resolution (see 6.3.3), uniform illumination and
at best focus. The reference optical arrangement is defined in 6.3.4. Alternative optical arrangements
may be used provided that the measurements obtained with them can be correlated with the use of the
reference optical arrangement.
Ambient light levels shall be controlled in order not to influence the measurement results. Whenever
possible, measurements shall be made on the symbol in its final configuration, i.e. the configuration
in which it is intended to be scanned. For MRTD evaluation, optically personalized samples shall be
used. This includes that all layers available at a document including laminations, security features and
protective layers shall be present.
Two principles govern the design of the optical set-up. First, the test image’s grey-scale shall be
nominally linear and not be enhanced in any way. Second, the image resolution shall be adequate to
produce consistent readings, which generally requires that the character stroke-widths span at least
10 image pixels.
6.2.2 Raw image
The raw image is a matrix of the actual reflectance values for each pixel of the light-sensitive array,
from which are derived the reference grey-scale image and the binarized image which are evaluated for
the assessment of symbol quality.
6.2.3 Reference grey-scale image
The reference grey-scale image is obtained from the raw image by processing the individual pixel
reflectance values through a synthetic circular aperture equal to 0,2 mm.
© ISO/IEC 2016 – All rights reserved 3

6.2.4 Binarized image
The binarized image is obtained from the reference grey-scale image by applying the algorithm defined
in Annex B.
6.3 Reference reflectivity measurements
6.3.1 General requirements
Equipment for assessing the quality of symbols in accordance with this subclause shall comprise a
means of measuring and analysing the variations in the reflectivity of a symbol on its substrate over an
inspection area which shall cover the full height and width of the symbol.
The measured reflectance values shall be expressed in percentage terms by means of calibration and
reference to recognized national standards laboratories, where 100 per cent should correspond to the
reflectance of a barium sulphate or magnesium oxide reference sample.
It should be ensured that all materials visible to the camera or close to the optical path are reflection-
free, at least in IR illumination. In particular, the background the symbol is attached to shall be IR
absorbing. The environment temperature shall be between 20°C and 25°C.
6.3.2 Light sources
Measurements shall be made using light emitting diode (LED) light sources at 890 nm and 940 nm
wavelengths.
All illumination elements shall have a diameter of 25 mm or less and may be shaped as circles, squares
or similar.
6.3.3 Effective resolution
The effective resolution of an instrument that implements this document shall be sufficient to ensure
that the parameter grading results are consistent irrespective of the rotation of the symbol. The effective
resolution is the product of the resolution of the light-sensitive array and of the magnification of the
associated optical system and effected by distortions introduced by the optical system. The reference
optical arrangement requires an effective resolution of not less than 10 pixels per stroke width.
6.3.4 Optical geometry
A reference optical geometry is defined for reflectivity measurements and consists of
— flood incident illumination, uniform across the inspection area, from a set of four light sources
arranged at 90-degree intervals around a circle concentric with the inspection area and in a plane
parallel to that of the inspection area, at a height which will allow incident light to fall on the centre
of the inspection area at an angle of 45° to its plane, and
— a light collection device, the optical axis of which is perpendicular to the inspection area and passes
through its centre, and which focuses an image of the test symbol on a light-sensitive array.
The light reflected from the inspection area shall be collected and focused on the light-sensitive array.
Implementations may use alternative optical geometries and components, provided that their
performance can be correlated with that of the reference optical arrangement defined in this subclause.
Figure 1 and Figure 2 illustrate the principle of the optical arrangement, but are not intended to
represent actual devices; in particular, the magnification of the device is likely to differ from 1:1. For
example, it is possible to use a 10 MP industrial camera without IR cut filter with a sensor size of ½”. The
image could be captured from a distance of approximately 350 mm and the lens chosen appropriately.
The resulting magnification then would be 1:21.
4 © ISO/IEC 2016 – All rights reserved

Key
1 light-sensing element
2 lens providing 1:1 magnification (measurement A = measurement B)
3 inspection area
4 light sources
ϑ angle of incidence of light relative to plane of symbol = 45°
Figure 1 — Reference optical arrangement — Side view
© ISO/IEC 2016 – All rights reserved 5

Key
1 light source
2 symbol
Figure 2 — Reference optical arrangement — Top view
6 © ISO/IEC 2016 – All rights reserved

Figure 3 — Reference optical arrangement — Angles and tolerances
When setting up a reference optical arrangement, the following considerations shall be made. The
symbol (e.g. the MRZ of an ID-3 MRTD) has a size of approximately 24 mm × 125 mm (marked as a
bar in the bottom of Figure 3). For other document sizes (ID-1, ID-2), the same size of the object to be
captured shall be used. All areas not covered by the travel document but visible to the camera shall be
made of IR-absorbing material. The small rectangles at the left and right border of Figure 3 represent
the illumination elements.
The nominal illumination angle shall be 45° as given in Figure 1. This angle is measured in the middle
axis of the MRZ zone. An angle of 45° directly determines that the horizontal and vertical distance of
the illumination from the centre of the symbol is identical. This distance should be 200 mm as depicted
in Figure 3.
© ISO/IEC 2016 – All rights reserved 7

For the small side of the symbol (24 mm), the minimal and maximal illumination angles at the symbol
borders are 43,3° and 46,8°, as shown in Table 1.
The difference between the nominal angle (in the centre) and the angles at the borders is much higher
for the long side of the symbol; 37,3° and 55,5°.
Table 1 — Dark pixel portion for threshold of 4.5
Angle Short side (24 mm) Long side (125 mm)
α 45° 45°
α′ 46,8° 55,5°
α″ 43,3° 37,3°
Figure 4 — Reference optical arrangement — Reflection considerations
Figure 4 shows the position where the direct reflection of the illumination at the symbol (i.e. caused by
plain lamination) will not be visible to the camera. The distance between the camera and the symbol
should be approximately 350 mm.
6.3.5 Inspection area
The inspection area within which all measurements shall be a rectangular area framing the complete
symbol. The centre of the inspection area shall be as close as practicable to the centre of the field of
view. For example, the MRZ in a passport shall be placed accordingly.
6.4 Basis of symbol grading
OCR symbol quality assessment shall be based on the measurement and grading of parameters of the
reference grey-scale image, the binarized image derived from it and the application of the reference
decode algorithm to these. Quality grading of these parameters shall be used to provide a relative
measure of symbol quality under the measurement conditions used.
8 © ISO/IEC 2016 – All rights reserved

6.5 Capture the raw image
Centre the symbol in the field of view and align the average bottom edge of the characters with the
sensor as precisely as possible, but always with less than +/−5° deviation.
Find and replace the brightest and darkest 0,005 % pixels in the overall image with the median of the
9 pixels consisting of itself and its 8 immediate neighbours.
Apply the aperture defined in 6.2.3 to the raw image to create a reference grey-scale image.
6.6 Image assessment parameters and grading
6.6.1 Determining the document horizontal axis
The application shall define the MRZ region in relation to the document reference edge.
6.6.2 Character best-fit algorithm
These steps should be followed in order to find the best-fit position of the character outline limit (SWT)
gauges on a character image captured from a machine-readable character string in order to find the
location of the characters.
Using the binarized image, determine four corner positions that bound the character image. From these
points, establish four more points that are further away from the character by the nominal stroke
width. These four points define the range over which the SWT gauges will be moved in order to find the
best-fit position. An example is shown in Figure 5.
Key
1 vertical range
2 horizontal range
Figure 5 — Sample corner positions
Create the SWT for each defined character in Annex A by moving a circle of radius of the appropriate
tolerance value around the centreline of the character and saving the outermost points (note that this is
equivalent to finding the points perpendicular to the centre line at a distance equal to the radius of the
circle). An example of this process is in Figure 6. Figure 7 illustrates the tolerances of the SWT boxes as
derived from ISO 1831.
© ISO/IEC 2016 – All rights reserved 9

Figure 6 — SWT creation example
a)  X-tolerance. Radius 6,75 “sq” inside 10,75 b)  Y-tolerance. Radius 5 “sq” inside 12,5 “sq”
“sq” outside outside
NOTE 1 These are the tolerances from ISO 1831:1980, Table 2.
NOTE 2 “Sq” is the count of squares from the original drawings (see Annex A) where each square equals
1/50 mm.
Figure 7 — SWT tolerance gauges
10 © ISO/IEC 2016 – All rights reserved

Overlay the original captured image with the SWT Y-tolerance gauges such that the horizontal axis of
the gauges is parallel with the document reference edge. Starting with the four extreme positions, move
the SWT Y-tolerance gauges right and left and up and down relative to the document reference edge to
each test position.
At each test position, sum up the reflectance values of each SWT Y-tolerance gauge resolution pixel
within the region defined by the minimum gauge. The test position with the lowest sum is used to
determine the position of the character. If there is more than one test position with the same lowest
sum (e.g. the inside of the minimum gauge is all black), then for each equivalent test position, sum up
the reflectance values of each gauge resolution pixel outside the maximum gauge. The equivalent test
position with the highest sum is used to determine the position of the character. If there is more than
one test position with the same highest sum, then compute the average of these positions and use it to
determine the position of the character.
6.6.3 Position of a character
Using the test position determined in 6.6.2, the location of a character is the origin of the SWT, where
the origin is defined as (0,0) for every character in Annex A.
The position of every character is determined and graded according to an application specific profile.
The character with the lowest grade determines the position grade.
6.6.4 Character evaluation value (CEV) in the best-fit location
A pixel is outside or inside a border if more than 50 % of the pixel area is outside or inside the border,
respectively.
For each tolerance template on every character, use the optimal position of the template placed over the
binarized character in the image. The following lists the variables to be used to calculate the CEV grades:
a) CEV_X_Inside — number of white pixels inside the X-tolerance inner boundary;
b) CEV_X_Outside — number of black pixels outside the X-tolerance outer boundary;
c) CEV_Y_Inside — number of white pixels inside the Y-tolerance inner boundary;
d) CEV_Y_Outside — umber of black pixels outside the Y-tolerance outer boundary;
e) Y_Boundary_Area — total number of image pixels of any color inside the Y-tolerance outer
boundary;
f) Y_Inside_Total — total number of image pixels of any color inside the Y-tolerance inner boundary;
g) Character_Region_Total — total number of image pixels in the rectangular area defined by
the Y-tolerance outer template of the chosen character plus a one-stroke width boundary. The
Character Region Total is computed as the product of the width of the Y-tolerance outer boundary
plus two times the nominal stroke width rounded to the nearest number of pixels and the height of
the Y-tolerance outer boundary plus the two times the nominal stroke width rounded to the nearest
number of pixels.
From these measurements, compute the following graded parameters:
a) Character_Inside_Fit = CEV_Y_Inside / Y_Inside_Total
b) Character_Outside_Fit = CEV_Y_Outside / (Character_Region_Total − Y_Boundary_Area)
Grade the total results as follows.
a) Character_Inside_Fit > 10 % need attention
b) Character_Inside_Fit > 20 % not recommended
© ISO/IEC 2016 – All rights reserved 11

c) Character_Outside_Fit> 1 % need attention
d) Character_Outside_Fit > 2 % not recommended
The character with the lowest grade determines the character evaluation value (CEV) Grade of the
entire MRZ.
NOTE CEV_X_Inside and CEV_X_Outside are useful for process control.
See Annex C for the OCR reference decode algorithm. See Annex D for an example calculation of
character evaluation value (CEV).
6.6.5 Background noise
Find the background noise, N.
Measure the maximum and minimum reflectance levels inside a rectangular area equal to the
height and width of a character (1,4 mm × 2,4 mm) centred vertically between the two lines of text
in the MRZ starting at the left end of the MRZ. Calculate N (noise) as the difference of the maximum
reflectance minus the minimum reflectance all divided by the maximum reflectance within the box
[(Rmax − Rmin) / Rmax] and save. Move the box a half character width and repeat. Continue to the end
of the MRZ. Find the largest N.
It is recommended that the backgro
...