ISO/IEC 18477-4:2017
(Main)Information technology — Scalable compression and coding of continuous-tone still images — Part 4: Conformance testing
Information technology — Scalable compression and coding of continuous-tone still images — Part 4: Conformance testing
ISO/IEC 18477-4:2017 specifies the framework, concepts, methodology for testing, and criteria to be achieved to claim conformance to one or multiple parts of ISO/IEC 18477 as listed below. It provides a framework for specifying abstract test suites and for defining the procedures to be followed during conformance testing. ISO/IEC 18477-4:2017 - specifies conformance testing procedures for decoding of ISO/IEC 18477‑1, 18477-2, ISO/IEC 18477‑6, ISO/IEC 18477‑7, ISO/IEC 18477‑8 and ISO/IEC 18477‑9, - specifies conformance testing procedures for codestreams to the above International Standards, - specifies codestreams, decoded images, and error metrics to be used within the decoder testing procedures, and - specifies abstract test suites.
Technologies de l'information — Compression échelonnable et codage d'images plates en ton continu — Partie 4: Essai de conformité
General Information
Relations
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 18477-4
First edition
2017-10
Information technology — Scalable
compression and coding of
continuous-tone still images —
Part 4:
Conformance testing
Technologies de l'information — Compression échelonnable et codage
d'images plates en ton continu —
Partie 4: Essai de conformité
Reference number
ISO/IEC 18477-4:2017(E)
©
ISO/IEC 2017
---------------------- Page: 1 ----------------------
ISO/IEC 18477-4:2017(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2017, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org
ii © ISO/IEC 2017 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 18477-4:2017(E)
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 2
3 Terms, definitions, abbreviated terms and symbols . 2
3.1 Terms and definitions . 2
3.2 Symbols . 8
3.3 Abbreviated terms . 8
4 Conventions . 8
4.1 Conformance language . 8
4.2 Operators . 9
4.2.1 Arithmetic operators . 9
4.2.2 Logical operators . 9
4.2.3 Relational operators . 9
4.2.4 Precedence order of operators . 9
4.2.5 Mathematical functions .10
5 Conventions .10
6 General description .10
6.1 Overview .10
6.2 Parts and profiles .10
6.3 Decoders .11
6.4 Implementation conformance statement .11
6.5 Abstract test suites .11
6.6 Decoder conformance testing procedures .11
7 Copyright .11
8 Conformance files availability and updates .11
Annex A (normative) Decoder conformance testing procedures .12
Annex B (normative) Decoder conformance tests .18
Annex C (normative) Codestream conformance .29
Bibliography .31
© ISO/IEC 2017 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 18477-4:2017(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 18477 series can be found on the ISO website.
iv © ISO/IEC 2017 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 18477-4:2017(E)
Introduction
The ISO/IEC 18477 series, also known under the term JPEG XT, specifies lossy and lossless codestream
formats for storage of continous-tone high and low dynamic range photographic content. All parts
of the ISO/IEC 18477 series are compatible to the Recommendation ITU-T T.81 | ISO/IEC 10918, also
commonly known as JPEG. That is, any decoder conforming to the latter standard will be able to
reconstruct codestreams from the ISO/IEC 18477 series to an 8 bits/sample image. Additional features
offered by ISO/IEC 18477, such as representation of intermediate or high-dynamic range images,
or lossless reconstruction require an extended decoder that implements, in addition to the Rec.
ITU-T T.81 | ISO/IEC 10918-1, also one or multiple members of ISO/IEC 18477.
This document provides the framework, concepts and methodology for testing codestreams and
implementations, and the criteria to be achieved to claim conformance to the parts and profiles
of ISO/IEC 18477. The objective of this document is to promote interoperability between JPEG XT
decoders, and to test these systems for conformance to one or multiple specifications that are part
of the JPEG XT. Conformance testing is the testing of a candidate implementation for the existence of
specific characteristics required by a standard. It involves testing the capabilities of an implementation
against the conformance requirements in the relevant standard.
The purpose of this document is to define a common test methodology, to provide a framework for
specific abstract test suites (ATS) and to define the procedures to be followed during conformance
testing.
Any organization contemplating the use of the test methods defined in this document should carefully
consider the constraints on their applicability. Conformance testing does not include robustness testing,
acceptance testing, and performance testing, all of which are outside the scope of this text.
© ISO/IEC 2017 – All rights reserved v
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 18477-4:2017(E)
Information technology — Scalable compression and
coding of continuous-tone still images —
Part 4:
Conformance testing
1 Scope
This document specifies the framework, concepts, methodology for testing, and criteria to be achieved
to claim conformance to one or multiple parts of ISO/IEC 18477 as listed below. It provides a framework
for specifying abstract test suites and for defining the procedures to be followed during conformance
testing.
This document
— specifies conformance testing procedures for decoding of ISO/IEC 18477-1, 18477-2, ISO/IEC 18477-
6, ISO/IEC 18477-7, ISO/IEC 18477-8 and ISO/IEC 18477-9,
— specifies conformance testing procedures for codestreams to the above International Standards,
— specifies codestreams, decoded images, and error metrics to be used within the decoder testing
procedures, and
— specifies abstract test suites.
This document does not include the following tests:
— testing decoders for conformance to ISO/IEC 18477-3 only. ISO/IEC 18477-6, ISO/IEC 18477-
7, ISO/IEC 18477-8 and ISO/IEC 18477-9 are extensions of ISO/IEC 18477-3 and the required
functionality of ISO/IEC 18477-3 is tested as part of the former standards. Testing codestreams for
conformance to ISO/IEC 18477-3 is specified in C.2;
— testing codestreams for conformance to ISO/IEC 18477-7 beyond testing them for conformance
to individual profiles of this document. Testing such codestreams ("full profile codestreams") for
syntactical correctness is, however, covered by testing them for conformance to ISO/IEC 18477-3;
— testing of the composition of background and foreground for images reconstructed from
ISO/IEC 18477-9 codestreams as this operation is application dependent;
— acceptance testing: the process of determining whether an implementation satisfies acceptance
criteria and enables the user to determine whether or not to accept the implementation. This
includes the planning and execution of several kinds of tests (e.g. functionality, quality, and speed
performance testing) that demonstrate that the implementation satisfies the user requirements;
— performance testing: measures the performance characteristics of an implementation under test
(IUT) such as its throughput, responsiveness, etc. under various conditions.
— robustness testing: the process of determining how well an implementation process data which
contains errors.
The ISO/IEC 18477 series consists of multiple parts, each of which defines one or multiple profiles.
A given IUT (implementation under test) may claim to implement various parts and profiles of
ISO/IEC 18477 at once. To test such implementations, they have to be tested with the Abstract Test
Suites of each part and profile they claim to conform to.
© ISO/IEC 2017 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC 18477-4:2017(E)
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 18477-1, Information technology — Scalable compression and coding of continuous-tone still
images — Part 1: Scalable compression and coding of continuous-tone still images
ISO/IEC 18477-2, Information technology — Scalable compression and coding of continuous-tone still
images — Part 2: Coding of high dynamic range images
ISO/IEC 18477-3, Information technology — Scalable compression and coding of continuous-tone still
images — Part 3: Box file format
ISO/IEC 18477-6, Information technology — Scalable compression and coding of continuous-tone still
images — Part 6: IDR Integer Coding
ISO/IEC 18477-7, Information technology: Scalable compression and coding of continuous-tone still images,
HDR floating point coding
ISO/IEC 18477-8, Information technology — Scalable compression and coding of continuous-tone still
images — Part 8: Lossless and near-lossless coding
ISO/IEC 18477-9, Information technology — Scalable compression and coding of continuous-tone still
images — Part 9: Alpha channel coding
ISO/IEC 10918-1, Information technology — Digital compression and coding of continuous tone still
images — Requirements and guidelines
3 Terms, definitions, abbreviated terms and symbols
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1 Terms and definitions
3.1.1
abstract test suite
generic conformance testing concepts and procedures for a given requirement
3.1.2
ASCII
binary encoding of 7-bit characters defined by ISO/IEC 646
3.1.3
base decoding path
process of decoding legacy codestream (3.1.43) and refinement data to the base image (3.1.4), jointly
with all further steps until residual data is added to the values obtained from the residual codestream
3.1.4
base image
collection of sample values obtained by entropy decoding the DCT coefficients of the legacy codestream
(3.1.43) and the refinement codestream, and inversely DCT transforming them jointly
2 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 18477-4:2017(E)
3.1.5
big endian
order of bytes with the most significant byte first
3.1.6
bit
unit of information representing a single yes/no choice represented by a one or a zero
3.1.7
binary decision
choice between two alternatives
3.1.8
bitstream
partially encoded or decoded sequence of bits (3.1.6) comprising an entropy-coded segment
3.1.9
block
8 × 8 array of samples (3.1.62) or an 8 × 8 array of DCT coefficient values of one component (3.1.19)
3.1.10
box
structured collection of data describing the image or the image decoding process embedded into one or
multiple APP marker segments
11
Note 1 to entry: See ISO/IEC 18477-3:2015, Annex B for the definition of boxes.
3.1.11
byte
group of 8 bits
3.1.12
coder
embodiment of a coding process (3.1.14)
3.1.13
coding model
procedure used to convert input data into symbols to be coded
3.1.14
coding process
general term for referring to an encoding process, a decoding process, or both
3.1.15
coefficient
values that are the result of a discrete cosine transformation
3.1.16
conformance
fulfillment of the specified requirements, as defined in this document, for a given profile and part of
ISO/IEC 18477
3.1.17
conformance test procedure
process of assessing conformance (3.1.16)
3.1.18
compression
reduction in the number of bits (3.1.6) used to represent source image data
© ISO/IEC 2017 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC 18477-4:2017(E)
3.1.19
component
two-dimensional array of samples (3.1.62) having the same designation in the output or display device
3.1.20
continuous-tone image
image whose components (3.1.19) have more than one bit (3.1.6) per sample (3.1.62)
3.1.21
decoder
embodiment of a decoding process (3.1.22)
3.1.22
decoding process
process which takes as its input compressed image data and outputs a continuous-tone image (3.1.20)
3.1.23
dequantization
inverse procedure to quantization by which the decoder (3.1.21) recovers a representation of the DCT
coefficients
3.1.24
downsampling
procedure by which the spatial resolution of a component (3.1.19) is reduced
3.1.25
encoder
embodiment of an encoding process (3.1.26)
3.1.26
encoding process
process which takes as its input a continuous-tone image (3.1.20) and outputs compressed image data
3.1.27
entropy-coded (data) segment
independently decodable sequence of entropy encoded bytes (3.1.11) of compressed image data
3.1.28
entropy decoder
embodiment of an entropy decoding (3.1.29) procedure
3.1.29
entropy decoding
lossless procedure which recovers the sequence of symbols from the sequence of bits (3.1.6) produced
by the entropy encoder (3.1.30)
3.1.30
entropy encoder
embodiment of an entropy encoding (3.1.31) procedure
3.1.31
entropy encoding
lossless procedure which converts a sequence of input symbols into a sequence of bits (3.1.6) such that
the average number of bits (3.1.6) per symbol approaches the entropy of the input symbols
3.1.32
extension image
synonym for residual image (3.1.61)
4 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 18477-4:2017(E)
3.1.33
grayscale image
continuous-tone image (3.1.20) that has only one component (3.1.19)
3.1.34
high dynamic range
image or image data comprised of more than 8 bits per sample (3.1.62)
3.1.35
Huffman decoder
embodiment of a Huffman decoding (3.1.36) procedure
3.1.36
Huffman decoding
entropy decoding (3.1.29) procedure which recovers the symbol from each variable length code
produced by the Huffman encoder (3.1.37)
3.1.37
Huffman encoder
embodiment of a Huffman encoding (3.1.38) procedure
3.1.38
Huffman encoding
entropy encoding (3.1.31) procedure which assigns a variable length code to each input symbol
3.1.39
implementation
realization of a specification
3.1.40
implementation under test
IUT
implementation (3.1.39) that is being evaluated for conformance (3.1.16)
3.1.41
intermediate dynamic range
image or image data comprised of more than 8 bits per sample (3.1.62)
3.1.42
joint photographic experts group
JPEG
informal name of the committee which created this document
Note 1 to entry: The “joint” comes from the ITU-T and ISO/IEC collaboration.
3.1.43
legacy codestream
collection of markers (3.1.51) and syntax elements defined by ISO/IEC 10918-1 bare any additional
syntax elements defined by the ISO/IEC 18477 standard, i.e. the legacy codestream consists of the
collection of all markers except those APP markers that describe JPEG XT boxes by the syntax defined
11
in ISO/IEC 18477-3:2015, Annex A
3.1.44
legacy decoding path
collection of operations to be performed on the entropy coded data as described by ISO/IEC 10918-1
jointly with the Legacy Refinement scans before this data is merged with the residual data to form the
final output image
© ISO/IEC 2017 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC 18477-4:2017(E)
3.1.45
legacy decoder
embodiment of a decoding process (3.1.20) conforming to ISO/IEC 10918-1, confined to the lossy DCT
process and the baseline, sequential or progressive modes, decoding at most four components to 8 bits
per component
3.1.46
legacy image
arrangement of sample values as described by applying the decoding process (3.1.20) described by
ISO/IEC 10918-1 on the entropy coded data as defined by the said standard
3.1.47
lossless
descriptive term for encoding and decoding processes and procedures in which the output of the
decoding procedure(s) is identical to the input to the encoding procedure(s)
3.1.48
lossless coding
mode of operation which refers to any one of the coding processes (3.1.14) defined in ISO/IEC 18477-8 in
which all of the procedures are lossless (3.1.47)
Note 1 to entry: See ISO/IEC 18477-8:2016, Annex H.
3.1.49
lossy
descriptive term for encoding and decoding processes which are not lossless (3.1.47)
3.1.50
low dynamic range
image or image data comprised of data with no more than 8 bits per sample (3.1.62)
3.1.51
marker
two-byte code in which the first byte is hexadecimal FF and the second byte is a value between 1 and
hexadecimal FE
3.1.52
marker segment
marker (3.1.51) together with its associated set of parameters
3.1.53
pixel
collection of sample values in the spatial image domain having all the same sample coordinates
EXAMPLE A pixel may consist of three samples describing its red, green and blue value.
3.1.54
precision
number of bits (3.1.6) allocated to a particular sample (3.1.62) or DCT coefficient
3.1.55
procedure
set of steps which accomplishes one of the tasks which comprise an encoding or decoding process
3.1.56
quantization value
integer value used in the quantization procedure
3.1.57
quantize
act of performing the quantization procedure for a DCT coefficient
6 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 18477-4:2017(E)
3.1.58
residual decoding path
collection of operations applied to the entropy coded data contained in the residual data box and
residual refinement scan boxes up to the point where this data is merged with the base image to form
the final output image
3.1.59
residual image
sample values as reconstructed by inverse quantization and inverse DCT transformation applied to the
entropy-decoded coefficients described by the residual scan (3.1.60) and residual refinement scans
3.1.60
residual scan
additional pass over the image data invisible to legacy decoders (3.1.45) which provides additive and/or
multiplicative correction data of the legacy scans to allow reproduction of high dynamic range (3.1.34)
or wide colour gamut data
3.1.61
refinement scan
additional pass over the image data invisible to legacy decoders (3.1.45) which provides additional least
significant bits to extend the precision (3.1.54) of the DCT transformed coefficients (3.1.15)
3.1.62
sample
one element in the two-dimensional image array which comprises a component (3.1.19)
3.1.63
sample grid
common coordinate system for all samples (3.1.62) of an image
Note 1 to entry: The samples at the top left edge of the image have the coordinates (0,0); the first coordinate
increases towards the right, the second towards the bottom.
3.1.64
scan
single pass through the data for one or more of the components (3.1.19) in an image
3.1.65
scan header
marker segment (3.1.52) that contains a start-of-scan marker and associated scan parameters that are
coded at the beginning of a scan (3.1.64)
3.1.66
source image
image used as input to an encoder (3.1.25)
3.1.67
superbox
box (3.1.10) that carries other boxes as payload data
3.1.68
table specification data
coded representation from which the tables used in the encoder (3.1.24) and decoder (3.1.21) are
generated and their destinations specified
3.1.69
testing
process of evaluating conformance (3.1.16)
© ISO/IEC 2017 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/IEC 18477-4:2017(E)
3.1.70
uniform quantization
method of reducing the precision (3.1.54) of DCT coefficients to enable more efficient entropy
coding of them
3.1.71
upsampling
procedure (3.1.55) by which the spatial resolution of a component (3.1.19) is increased
3.2 Symbols
Nf Number of components in an image
R Additional bits in the HDR image. 8 + R is the sample precision of the reconstructed HDR or IDR image.
b b
3.3 Abbreviated terms
ASCII American standard code for information interchange
ATS Abstract test suite
DCT Discrete cosine transformation
ETS Executable test suite
HDR High dynamic range
IDR Intermediate dynamic range
IUT Implementation under test
JPEG Joint photographic experts group
LDR Low dynamic range
LSB Least significant bit
MRSE Mean relative square error
MSE Mean square error
MSB Most significant bit
PSNR Peak signal to noise ratio
TMO Tone mapping operator
TCS Test codestream
4 Conventions
4.1 Conformance language
This document consists of normative and informative text.
Normative text is that text which expresses mandatory requirements. The word "shall" is used to
express mandatory requirements strictly to be followed in order to conform to this document and
from which no deviation is permitted. A conforming implementation is one that fulfils all mandatory
requirements.
8 © ISO/IEC 2017 – All rights reserved
---------------------- Page: 13 ----------------------
ISO/IEC 18477-4:2017(E)
Informative text is text that is potentially helpful to the user, but not indispensable and can be removed,
changed or added editorially without affecting interoperability. All text in this document is normative,
with the following exceptions: the Introduction, any parts of the text that are explicitly labelled as
"informative", and statements appearing with the preamble "NOTE" and behaviour described using the
word "should". The word "should" is used to describe behaviour that is encouraged but is not required
for conformance to this document.
The keywords "may" and "need not" indicate a course of action that is permissible in a conforming
implementation.
The keyword "reserved" indicates a provision that is not specified at this time, shall not be used, and
may be specified in the future. The keyword "forbidden" indicates "reserved" and in addition indicates
that the provision will never be specified in the future.
4.2 Operators
NOTE Many of the operators used in this document are similar to those used in the C programming language.
4.2.1 Arithmetic operators
+ Addition
− Subtraction (as a binary operator) or negation (as a unary prefix operator)
* Multiplication
/ Division without truncation or rounding
4.2.2 Logical operators
|| Logical OR
&& Logical AND
! Logical NOT
4.2.3 Relational operators
> Greater than
>= Greater than or equal to
< Less than
<= Less than or equal to
== Equal to
!= Not equal to
4.2.4 Precedence order of operators
Operators are listed below in descending order of precedence. If several operators appear in the same
line, they have equal precedence. When several operators of equal precedence appear at the same level
in an expression, evaluation proceeds according to the associativity of the operator either from right to
left or from left to right.
© ISO/IEC 2017 – All rights reserved 9
---------------------- Page: 14 ----------------------
ISO/IEC 18477-4:2017(E)
Operators Type of operation Associativity
(), [ ], . Expression Left to Right
− Unary negation
*, / Multiplication Left to Right
+, − Addition and Subtraction Left to Right
< , >, <=, >= Relational Left to Right
4.2.5 Mathematical functions
⎾x⏋ Ceil of x. Returns the smallest integer that is greater than or equal to x.
⎿x⏌ Floor of x. Returns the largest integer that is lesser than or equal to x.
|x| Absolute value is –x for x < 0, otherwise x.
sign(x) Sign of x, 0 if x is zero, +1 if x is positive, −1 if x is negati
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.