ISO/IEC 21122-4:2025
(Main)Information technology — JPEG XS low-latency lightweight image coding system — Part 4: Conformance testing
Information technology — JPEG XS low-latency lightweight image coding system — Part 4: Conformance testing
This document specifies the framework, concepts, methodology for testing, and criteria to be achieved to claim conformance to multiple parts of the ISO/IEC 21122 series. It lists the conformance testing procedures.
Technologies de l'information — Système de codage d'images léger à faible latence JPEG XS — Partie 4: Essais de conformité
General Information
Relations
Standards Content (Sample)
International
Standard
ISO/IEC 21122-4
Third edition
Information technology — JPEG
2025-04
XS low-latency lightweight image
coding system —
Part 4:
Conformance testing
Technologies de l'information — Système de codage d'images
léger à faible latence JPEG XS —
Partie 4: Essais de conformité
Reference number
© ISO/IEC 2025
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
© ISO/IEC 2025 – All rights reserved
ii
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 2
5 General description . 2
5.1 Overview .2
5.2 Codestream syntax testing.2
5.3 Test procedures to test decoders for conformance to ISO/IEC 21122-1 .2
5.4 Test procedures to test decoders for conformance to ISO/IEC 21122-3 .3
5.5 File format syntax testing .3
5.6 ISO/IEC 21122-2 Profile, level, sublevel and frame buffer level conformance testing .3
5.7 ISO/IEC 21122-2 Buffer model conformance testing .3
5.8 ISO/IEC 21122-2 Frame buffer model conformance testing .3
5.9 Electronic attachments .3
6 Conformance files availability and updates . 3
Annex A (normative) Codestream syntax testing procedures . 4
Annex B (normative) Decoder testing procedures. 5
Annex C (normative) Decoder conformance tests .11
Annex D (normative) File format syntax testing procedures . 17
Annex E (normative) Buffer model conformance testing .18
Annex F (normative) Buffer model conformance testing .20
Annex G (informative) Attached software .22
Bibliography .24
© ISO/IEC 2025 – All rights reserved
iii
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical activity.
ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations,
governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types
of document should be noted. This document was drafted in accordance with the editorial rules of the ISO/
IEC Directives, Part 2 (see www.iso.org/directives or www.iec.ch/members_experts/refdocs).
ISO and IEC draw attention to the possibility that the implementation of this document may involve the
use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of any
claimed patent rights in respect thereof. As of the date of publication of this document, ISO and IEC had not
received notice of (a) patent(s) which may be required to implement this document. However, implementers
are cautioned that this may not represent the latest information, which may be obtained from the patent
database available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall not be held
responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO's adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www.iso.org/iso/foreword.html.
In the IEC, see www.iec.ch/understanding-standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
This third edition cancels and replaces the second edition (ISO 21122-4:2022), which has been technically
revised.
The main changes are as follows:
— reference test streams have been revised;
— a test protocol for testing sequences of codestreams was added;
— additional test streams for the TDC 444.12 and TDC MLS 444.12 profiles were added;
— a test protocol for testing the frame buffer bandwidth model was added.
A list of all parts in the ISO/IEC 21122 series can be found on the ISO and IEC websites.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at https://www.iso.org/members.html and
https://www.iec.ch/national-committees.
© ISO/IEC 2025 – All rights reserved
iv
Introduction
This document is part of a series of standards for a low-latency lightweight image coding system, denoted
JPEG XS. It provides the conformance testing procedures for ISO/IEC 21122-1, ISO/IEC 21122-2 and
ISO/IEC 21122-3.
This document specifies:
— conformance testing procedures for decoders implementing ISO/IEC 21122-1;
— tests to check which conformance point an ISO/IEC 21122-1 decoder conforms to, that is, whether a
decoder satisfies the error bounds required for strict or relaxed conformance;
— conformance testing procedures for decoders implementing ISO/IEC 21122-3;
— tests to check codestreams for conformance to ISO/IEC 21122-1. As such, it provides means to test
whether encoder implementations generate syntactically correct codestreams, and whether codestreams
generated by such implementations follow the requirements of a particular profile, level, sublevel, frame
buffer level and the buffer model implied by them;
— tests to check files for conformance to ISO/IEC 21122-3;
— conformance testing procedures that allow testing whether codestreams conform to any of the profiles
specified in ISO/IEC 21122-2;
— conformance testing procedures that allow testing whether codestreams conform to the buffer model
specified in ISO/IEC 21122-2 as part of a profile, level, sublevel and frame buffer level;
— codestreams, decoded images, and error metrics to be used within the decoder testing procedures;
— a buffer model test;
— abstract test suites.
This document does not specify:
— testing the reconstruction of a full resolution image from a subsampled image format. In particular,
upsampling from 4:2:2 or 4:2:0 to 4:4:4 sampling is a non-normative extension and as such its testing is
beyond the scope of this document;
— testing the conversion of the sample values reconstructed by an ISO/IEC 21122-3 decoder to the target
colour space by means of the colour specification box of ISO/IEC 21122-3;
— testing of the composition of background and foreground for images reconstructed from ISO/IEC 21122-3
files or codestreams that contain auxiliary channels carrying opacity information;
— testing of the interpolation of a colour filter array image to a full scale colour image; this process is not
normatively defined and beyond the scope of this document;
— acceptance testing: the process of determining whether an implementation satisfies acceptance criteria
and enables the user to determine whether or not to accept the implementation. This includes the planning
and execution of several kinds of tests (e.g. functionality, quality, and speed performance testing) that
demonstrate that the implementation satisfies the user requirements;
— performance testing: measures the performance characteristics of an implementation under test (IUT)
such as its throughput, responsiveness, etc. under various conditions;
— robustness testing: the process of determining how well an implementation is able to conceal problems
from attempting to reconstruct an image from an ill-formed codestream.
© ISO/IEC 2025 – All rights reserved
v
International Standard ISO/IEC 21122-4:2025(en)
Information technology — JPEG XS low-latency lightweight
image coding system —
Part 4:
Conformance testing
1 Scope
This document specifies the framework, concepts, methodology for testing, and criteria to be achieved to
claim conformance to multiple parts of the ISO/IEC 21122 series. It lists the conformance testing procedures.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content constitutes
requirements of this document. For dated references, only the edition cited applies. For undated references,
the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 21122-1, Information technology — JPEG XS low-latency lightweight image coding system — Part 1:
Core coding system
ISO/IEC 21122-2, Information technology — JPEG XS low-latency lightweight image coding system — Part 2:
Profiles and buffer models
ISO/IEC 21122-3, Information technology — JPEG XS low-latency lightweight image coding system — Part 3:
Transport and container formats
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 21122-1, ISO/IEC 21122-2,
ISO/IEC 21122-3 and the following apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org
3.1
codestream sequence
contiguously enumerated set of codestreams representing a temporally consistent video signal
3.2
executable test suite
set of executable test cases in the form of codestreams that are input into an implementation under test
3.3
interlace
scan method scanning each frame as two fields consisting of the even and odd lines of the frame
© ISO/IEC 2025 – All rights reserved
3.4
JPEG XS file
sequence of bytes encoding an image in the JXS file format
Note 1 to entry: This format is specified in ISO/IEC 21122-3.
3.5
procedure
set of steps which accomplishes one of the tasks which comprise an encoding or decoding process
3.6
progressive
scan method scanning each frame as a single scan, including both the even and odd lines
3.7
test codestream set
(sub-)set of codestreams that are input to an implementation under test for a particular test purpose
4 Abbreviated terms
ASCII American Standard Code for Information Interchange
ETS executable test suite
IUT implementation under test
PSNR peak signal to noise ratio
TCS test codestream set
5 General description
5.1 Overview
The ISO/IEC 21122 series, also known as JPEG XS, consists of multiple parts. ISO/IEC 21122-1 describes the
core coding system and the syntax of a codestream. ISO/IEC 21122-2 describes profiles and buffer models.
ISO/IEC 21122-3 describes transport and container formats. This document defines test suites for decoder
conformance tests for ISO/IEC 21122-1, ISO/IEC 21122-2 and ISO/IEC 21122-3.
The conformance files, including codestreams, reference decoded images, descriptive files and auxiliary
software to facilitate testing are provided at https:// standards .iso .org/ iso -iec/ 21122/ -4/ ed -3/ en/
, in compressed form. This compressed file is referred to as the electronic attachment from here on. File
locations given in this document are expressed relative to the top-level directory tree within this electronic
attachment. A Unix-style file structure and delimiters are assumed.
This document contains instructions for the use of these files.
5.2 Codestream syntax testing
The procedures of Annex A shall be used for testing codestreams or sequences of codestreams for syntactical
correctness. They depend on an auxiliary codestream syntax parsing tool whose source code is provided
within the electronic attachment.
5.3 Test procedures to test decoders for conformance to ISO/IEC 21122-1
The procedures defined in Annex B and the subset of ETS defined in Annex C, subclauses C.2 to C.15 that
correspond to the profiles, levels, sublevels and frame buffer levels supported by the decoder, shall be used
© ISO/IEC 2025 – All rights reserved
for testing decoders for compliance to ISO/IEC 21122-1. These procedures and ETS allow an IUT to evaluate
conformance to ISO/IEC 21122-1 only. Annex B defines two conformance points:
— a strict conformance point that requires decoders to reconstruct codestreams into image data that is
bitwise identical to the reference data, and
— a relaxed conformance point that offers additional freedom to implementations.
5.4 Test procedures to test decoders for conformance to ISO/IEC 21122-3
The procedures defined in Annex B and the subset of ETS defined in Annex C, subclause C.16 that
correspond to the profiles, levels, sublevels and frame buffer levels supported by the decoder shall be used
for testing decoders for compliance to ISO/IEC 21122-3. These procedures and ETS allow an IUT to evaluate
conformance to ISO/IEC 21122-1 and ISO/IEC 21122-3.
5.5 File format syntax testing
The procedures that shall be used for testing JPEG XS files for compliance to the file format specified in
ISO/IEC 21122-3 are defined in Annex D. These procedures depend on a codestream syntax parsing tool
whose source code is provided within the electronic attachment.
5.6 ISO/IEC 21122-2 Profile, level, sublevel and frame buffer level conformance testing
Annex A also specifies a test procedure that shall be used for testing whether a codestream is conforming to
a particular profile, level, sublevel and frame buffer level, and whether, in particular, its coding parameters
are within the constraints of the profile, level, sublevel and frame buffer level indicated in the codestream.
The test in Annex A depends on a program that is included in the electronic attachment.
5.7 ISO/IEC 21122-2 Buffer model conformance testing
Annex E specifies a test procedures that shall be used for testing whether a codestream is conforming to a
buffer model implied by the profile, level, sublevel and frame buffer level indicated in the codestream. The
tests in Annex E depend on a program that is included in the electronic attachment.
5.8 ISO/IEC 21122-2 Frame buffer model conformance testing
Annex F specifies test a procedures that shall be used for testing whether a sequence of codestreams
conforms to the frame buffer bandwidth model specified in ISO/IEC 21122-2. The tests in Annex F depend
on a program that is included in the electronic attachment.
5.9 Electronic attachments
Annex G lists the electronic attachments to this document and describes how to compile and use them.
6 Conformance files availability and updates
The conformance test images, streams and conformance test software released with this document are the
latest tested versions available at the date at which this document is published.
© ISO/IEC 2025 – All rights reserved
Annex A
(normative)
Codestream syntax testing procedures
A.1 General
This annex defines a procedure that shall be followed for determining whether a codestream or a sequence
of codestreams is syntactically well-formed and follows the syntactical requirements of ISO/IEC 21122-1.
The test procedure also checks whether the coding parameters of a given codestream are consistent
with the profile, level, sublevel and frame buffer level indicators that are part of the picture header (see
ISO/IEC 21122-1). A typical use case for this test is to check whether a given encoder generates profile, level,
1)
sublevel and frame buffer level information correctly. To this end, a Python test script jxscodestream.py
that performs a syntax analysis of a given codestream is included in the electronic attachment.
A.2 Installation
The test tool requires installation of a Python 2.7 interpreter on the computer system to be used for
performing a test. Python is available for multiple operating systems at https:// docs .python .org.
A.3 Usage of the syntax test tool
For testing a particular codestream for syntactical correctness and correct indication of profile, level,
sublevel and frame buffer level, the syntax analyser jxscodestream.py tool shall be run on a command line
as follows:
jxscodestream.py codestream.jxs
where codestream.jxs is the codestream to be tested. A codestream is not conforming in case the above tool
reports any error.
For testing a sequence of codestreams for syntactical correctness and correct indication of profile, level,
sublevel and frame buffer level, each member of the sequence of codestreams shall be tested by the above tool.
The lack of detection of any conformance violation by the syntax test tool should not be considered as a
definite proof that the codestream under testing conforms to all constraints required for conformance to
ISO/IEC 21122-1 and ISO/IEC 21122-2. ISO/IEC 21122-5 provides additional means for testing a codestream
for conformance to the ISO/IEC 21122 series by feeding it into the reference software implementation.
1) Python is a trademark of the Python Software Foundation. This information is given for the convenience of users of
this document and does not constitute an endorsement by ISO/IEC of the product named. Equivalent products may be
used if they can be shown to lead to the same results.
© ISO/IEC 2025 – All rights reserved
Annex B
(normative)
Decoder testing procedures
B.1 General
This annex defines procedures that shall be followed for determining whether a JPEG XS decoder
implementation is conforming to a particular profile, level, sublevel and frame buffer level.
It defines two conformance points:
— strict conformance, and
— relaxed conformance.
B.2 Decoder test procedure
The following steps shall be performed for the purpose of testing a decoder implementation.
— Select to test for ISO/IEC 21122-1 only or ISO/IEC 21122-1 and ISO/IEC 21122-3.
— Select a profile, level, framebuffer level, sublevel and frame buffer level to test against.
— The profile, level, sublevel and frame buffer level define a test codestream set (TCS), which consists of
codestreams, sequences of codestreams or files within the test suite that conform to
— the codestream syntax specified in ISO/IEC 21122-1 or the file format specified in ISO/IEC 21122-3,
— the selected profile,
— a level whose sample count and sample rate is smaller than or equal to the sample count and sample
rate of the selected level,
— to a sublevel whose bitrate is smaller or equal than the bitrate of the selected sublevel, and
— to a frame buffer level whose bitrate is smaller or equal than the bitrate of the selected frame buffer
level, if the temporal differential coding is under test.
— Iterate over the TCS and perform the following operations for each element the TCS; each element
provides a codestream, a file or sequence of codestreams along with a reference image or a reference
decoded image sequence.
— Depending on whether the TCS element consists of a single image or a sequence of images, perform the
tests in subclause B.3 for images or subclause B.4 for sequences.
— If an IUT fails to conform to a single element of the TCS, it fails the entire test for the profile, level,
sublevel and frame buffer level.
— If an IUT fails to conform to the strict conformance point for a single TCS element, but passes the
tests for relaxed conformance point for all elements in the TCS, the decoder conforms to the relaxed
conformance point.
— If an IUT conforms to the strict conformance point for all elements in a TCS, it conforms to the strict
conformance point.
© ISO/IEC 2025 – All rights reserved
The entire set of test codestreams along with PSNR bounds is defined in more detail in Annex C. Instructions
how to measure PSNR are specified in subclause B.11.
B.3 Test procedures for individual images
The following steps shall be performed to test an IUT against a TCS element that consists of a single reference
image and a single reference codestream or reference file.
— Decode the TCS element, consisting of a codestream by the IUT (implementation under test).
— The decoded output shall be format converted if necessary, see B.9 for details.
— If the dimension of the decoded output is different from the reference image, the IUT fails to conform to
this TCS element and thus to the entire TCS.
— If the reference and the decoded image are identical, the IUT conforms to the strict conformance point
for this TCS element.
— The PSNR between the decoded output and reference output shall be computed, B.11 provides the details.
— If the PNSR between the decoded output and the reference output is larger or equal to the PSNR bound
for the TCS element listed in Annex C, then the IUT conforms to the relaxed conformance point for this
TCS element.
— If the PSNR between the decoded output and the reference output is smaller than the PSNR bound for the
TCS element listed in Annex C, then the IUT fails to conform to the TCS element and thus fails the entire
test for the profile, level, sublevel and frame buffer level.
B.4 Test procedures for image sequences
The following steps shall be performed to test an IUT against a TCS element that consists of a sequence of
reference images and a sequence of reference codestreams.
— Decode the TCS element, consisting of a codestream sequence by the IUT (implementation under test).
— The decoded outputs shall be format converted if necessary, see B.9 for details.
— If the number of images in the decoded output is different from the number of images in the reference
codestream sequence, or the dimensions of the images in the decoded output is different from the
dimensions of the images in the reference output sequence, the IUT fails to conform to this TCS element
and thus to the entire TCS.
— If the images within the decoded output sequence are not pairwise identical to the images in the reference
output sequence, the IUT does not conform to the strict conformance point for this TCS element.
— The PSNR between each pair of decoded output image in the output sequence and reference output image
of the reference sequence shall be computed, and the minimum of all PSNR values obtained in this way
shall be formed.
— If this minimum PSNR value is smaller than the PSNR bound from the TCS element listed in Annex C, then
the IUT does not conform to this TCS element and thus for the entire TCS.
— Depending on the column “Test Refresh” of the TCS element as listed in Annex C, testing this TCS element
is at this point either complete, or continues with a second test.
— The codestream or codestreams encoding the first frame, as indicated by the frame index in its file
name, shall be removed from the TCS element; if this TCS element indicates that its contents is interlaced,
see B.5, then the first two codestreams shall be removed, otherwise only the first codestream shall be
removed.
© ISO/IEC 2025 – All rights reserved
— The TCS sequence, minus the first or the first two codestreams shall be decoded by the IUT. If decoding
fails, the IUT fails this TCS element and therefore the entire TCS. Note that this test implies an elementary
test for robustness as the all further codestreams in the codestream sequence forming the TCS element
use temporal differential coding, though the reference image or images established by the first frame are
missing.
— If the last decoded image or pair of images, as identified by its frame index, are not identical to the last
reference output image, or last pair of reference images, then the IUT does not conform to the strict
conformance point for this TCS element.
— If the TCS element is progressive, the PSNR between the last decoded frame and the last reference output
image, i.e. the images with the highest frame index shall be computed.
— If the TCS element is interlaced, two PSNR values shall be computed, namely those between corresponding
fields of the last frame of the decoded image sequence and the last frame of the reference output image.
— If this PSNR, or the minimum PSNR of the two fields for interlaced sequences is smaller than the PSNR
bound listed in Annex C, then the IUT fails the test for this TCS element and thus the entire TCS.
B.5 Files for testing
A particular ETS defines the input codestreams, files and codestream sequences along with their decoded
reference output images and their PSNR bounds. These are specified in Annex C for all profiles.
Files are named as follows.
— If a TCS element consists of a single image, then the file name of the codestream is given by the column
labelled “TCS” in Annex C, and the corresponding reference output image is given by replacing the file
extensions “.jxs” or “.jxc” by “.pgx”.
— If a TCS element consists of an image sequence, then “TCS” column in Annex C provides a directory name
that contains all individual codestreams and reference images.
— In such a directory, the files are enumerated by an index providing a contiguous frame number, an
optional field indicator, a dot (.) and a file extension.
— For interlaced sequences, the field indicator is t for the top field and b for the bottom field.
— The file name of a reference output image is found by replacing the file extension “.jxc” by “.pgx”.
B.6 Testing of temporal differential decoding
Some members of an ETS consist of codestream sequences and decoded image sequences whose elements
are consecutively enumerated by a frame index and an optional field indicator that is indicated as part of the
file name. These sequences test decoders for correct implementation of temporal differential coding (TDC)
and thus require decoders to maintain a temporal state, including a frame buffer, between frames.
While decoding image sequences from codestream sequences is not a normative requirement of a decoder in
order to be compliant to ISO/IEC 21122-1, an IUT shall be equipped for the purpose of testing its support for
temporal differential coding with the ability to accept sequences of codestreams as input and generate from
that sequences of images as output. It is not necessary to equip IUTs with the ability to decode codestream
sequences if conformance with temporal differential coding is not claimed.
For the purpose of testing interlaced content, the codestream sequences representing the top and bottom
fields form two independent subsequences. They are decoded independently from each other, requiring two
independent frame buffers.
For the purpose of a first test, the elements of such sequences shall be made available to the IUT one by one,
in the order of increasing frame index, and the decoder shall emit for each input codestream one decoded
image, thus forming a sequence of images. Each of these decoded images correspond to one reference image
© ISO/IEC 2025 – All rights reserved
in the sequence of reference images, which are also enumerated by an optional field indicator and a frame
index as part of the file name.
For the purpose of a second test, the elements of the test sequence, except those representing the first
frame, shall be made available to the IUT one by one, in the order of increasing frame index, and the decoder
shall emit for each input codestream one decoded image, thus forming a sequence of images. Each of these
decoded images correspond to one reference image in the sequence of reference images, excluding those
representing the first frame of the reference images.
This second test includes a robustness test as it generates wavelet coefficient differentials in the decoder
that are relative to a reference frame in the decoder framebuffer (or framebuffers for interlaced content)
that were not made available to the decoder through the initial frame. There is no requirement to initialize
the frame buffer of the IUT in any particular way as only the last image or last pair of images of a sequence
are compared, and at this frame, all coefficients in the frame buffer underwent a refresh.
A particular ETS defines in the case of testing temporal differential decoding input codestream sequences
along with a minimum PNSR bound for all the frames it consists of, and in addition a PSNR bound for the last
frame if the first frame, consisting of the first two fields for interlace, are eluded from decoding. The first
minimum PSNR bound applies to the first test, the second PSNR bound on the last image to the second test.
These PSNR bounds are listed in Annex C.
B.7 Decoder settings
Decoders may have mechanisms for supporting various decompression settings. These may be set in the
most advantageous way to achieve strict or relaxed conformance. For example, a decoder with a “fast mode”
and an “accurate mode” may be set to the “accurate mode” to determine the level of conformance. These
settings should be noted in any statement of conformance. The same user-controlled settings for accuracy
shall be used for all test codestreams of a TCS.
B.8 Output file format conversion
The reference decoded images or decoded image sequences are provided in a specific file format defined
in subclause B.10. In order to compare the output of the decoder under test with these images, several
conversions may be necessary. These conversions may be done as post-processing steps outside of the
decoder solely for determining conformance. There is no requirement for a conforming decoder to perform
these processes as part of its normal operation. These conversions shall not introduce a quality change
(either loss or gain).
B.9 Sample format conversion
Image sample values are always unsigned integers. For the purpose of testing, it is of advantage to represent
the integer output of the IUT in the PGX format defined in subclause B.10 as ISO/IEC provides tools to measure
on such files directly. Any up-sampling or colour space format conversions shall be disabled for the purpose
of testing should a decoder offer these as optional features.
There is no requirement that a conforming decoder has to generate output to the specified format, and
the representation of the output of the IUT in this format only facilitates the testing process. That is,
it is acceptable to include additional lossless format conversions in the testing procedure provided the
comparison between reconstructed and reference images is performed in the normative way.
Conversion from YCbCr to RGB, colour space conversions, up-sampling or interpolation of components to
populate the entire sample grid may be provided as optional features of a decoder implementation and shall
be disabled for testing purposes.
The conversion between colour spaces described in the above paragraph is not related to the RCT or Star-
Tetrix transformations which are normatively defined in ISO/IEC 21122-1. Both shall be executed for the
purpose of reference testing if enabled in the codestream. Conformance testing applies to the reconstructed
sample values only, bare any interpretation relative to a colour space, and no colour space conversions
© ISO/IEC 2025 – All rights reserved
except those implied by the RCT and Star-Tetrix transformation shall be made for the purpose of testing. In
particular, colour filter array images are understood as four-component images consisting of one red, two
green and one blue component, consistent with their interpretation in ISO/IEC 21122-1. The width of their
sampling grid is half the width of the sensor array, and the height of their sensor grid is half the height of the
sensor array used to create these images.
B.10 Reference components file format (PGX)
B.10.1 General
This subclause specifies the file format, called PGX, of the reference images used for comparison with the
output of the decoder under test. The decoder under test is not required to produce this particular file
format, though it is advantageous to perform a conversion to this file format for testing purposes as ISO/IEC
provides test tools that are able to decode this format. Any necessary lossless conversion to this format, as
specified by subclause B.5, may be applied as part of the test procedure.
The format consists of a directory file, one header file per component, and one raw data file per component.
NOTE The difftest_ng tool, provided as part of the electronic attachment, can convert the .pgx file format to
many other formats. It is therefore not necessary to support the file format listed here directly.
B.10.2 Directory file format
The directory file is the file that is provided as input file to the comparison tool (.pgx file extension). It
consists, for each component, of the file name of the raw data file encoded in ASCII (see ISO/IEC 646), relative
to the path where the directory file is located. Each raw data file name is terminated by a single line feed
character (ASCII 10=0x0a).
NOTE The PGX directory file does not separate lines by CR/LF pairs, i.e. the ASCII 13=0x0d, 10=0x0a code
sequence.
B.10.3 Header file format
The header file is derived from the file name of the raw data file listed in the directory file by removing
the extension .raw and replacing it by the extension .h. It describes the format in which the raw format is
encoded. There is one separate header file per component.
Each header file consists of a single line, terminated by a single Line Feed character (ASCII 10=0x0a)
describing the format. It consists of the following fields, where angle brackets < and > indicate parameters
described below and SPC indicates a blank space (ASCII 32=0x20):
PSPCSPCSPCSPC
where:
P
identifies the header file and shall be present. It has no particular meaning beyond format
identification;
identifies the sample format. ISO/IEC 21122 stores integer samples only, which are indicated
by the single character G as dataformat;
identifies the endianness of the encoded data. The character sequence ML indicates big endian
encoding, i.e. most significant byte first, the character sequence LM little endian encoding,
i.e. least significant byte first;
indicates whether sample values are signed or unsigned. ISO/IEC 21122 covers only unsigned
samples, indicated by the character + in this field;
© ISO/IEC 2025 – All rights reserved
indicates the bit depth of the sample values in the component described by this header file.
The bit depth is represented as ASCII encoded decimal number;
is the number of samples per line for this component, represented as ASCII encoded decimal
number;
is the number of lines of the image, represented as ASCII encoded decimal number.
B.10.4 Data file format
Each data file contains the sample values themselves in raster scan order, left to right, top to bottom. If
the precision of the component is 8 bits or below, each sample is represented in 8 bits, right aligned to the
entire byte, i.e. unused bits remain 0 and make up the most-significant bits of the byte. If the precision of
the component is larger than 8 bits, each sample is represented by two bytes, encoded in the order indicated
by the header, i.e. either most significant byte first if the endianness field is ML, or with the least significant
byte first if the endianness field is LM. The data bits are right aligned into the two bytes, most significant bits
remain 0 if necessary.
B.11 Comparison of decoded and formatted components with reference components
The image reconstructed from a TCS and the reference test image for the TCS need to be identical sample by
sample in order to match the strict conformance point. To test for the relaxed conformance, the Peak Signal
to Noise Ratio (PSNR) between the reconstructed image and reference image shall be computed according
to Formula (B.1):
W −1 H −1
ff 2
χ xy,,cD⋅ xy,,cR− xy,,c
() ()() (()
∑∑
1 N −1
x=0 y=0
c
PSNlR =−10⋅⋅og (B.1)
10 ∑
c=0 2
W −1 H −1
N
Bc[] ff
c
21− ⋅ χ()xy,,c
()
∑∑
x=0 y=0
where
N is the number of components in the image;
c
W is the width of the sensor grid;
f
H the height of the sensor grid;
f
χ xy,,c the indicator function that is 1 if the position xy, of the sampling grid is populated by com-
() ()
ponent c , and is 0 otherwise;
is the sample value of component c of the decoded image output by the IUT at sample grid
Dx(),,yc
position ()xy, ;
Rx(),,yc the sample value of component c of the reference image at sample grid position ()xy, ;
the bit-precision of component c as found in the PGX header file.
Bc
[]
NOTE 1 By this formula, the PSNR is computed from the mean square error over all populated sample grid positions.
NOTE 2 The difftest_ng test tool available as an electronic attachment to this document can be used to implement
such a test. difftest_ng reports a peak signal to noise ratio (PSNR) of INF in the case that decoded and reference
image are identical. Otherwise, it reports the PSNR value as computed by Formula (B.1).
NOTE 3 The reference output image or reference output image sequence is the expected output of a strictly
conforming decoder implementation; thus, the PSNR bound is a bound of the distortion between the output of a
strictly conforming decoder, and a decoder conforming to the relaxed conformance point. In particular, do not confuse
the PSNR bound with the PSNR between an original and a reconstructed image.
© ISO/IEC 2025 – All rights reserved
Annex C
(normative)
Decoder conformance tests
C.1 General
This annex specifies the abstract test suites and executable test suites that shall be used in the conformance
test procedures from Annex B. References reconstructed images are given by substituting the suffix .jxc by
.pgx. If the column indicating the PSNR bound indicates a dash (-), the corresponding codestream shall not
be included for testing an IUT for the relaxed conformance point. If the PSNR bound indicates INF, a decoder
is required to match the reference output bitwise-exact, regardless of the conformance point being tested.
If there is a column denoted as Test Refresh, then this test provides codestream and reference image
sequences, and it indicates whether testing aborts after comparing the fully decoded sequence or continues
testing the refresh mechanism by removing the first, or first two input codestreams, decode these reduced
sequences again, and compare the last or last pair of decoded images with the reference image or reference
image pairs. Subclause B.4 provides further details how to implement this test. If this column states yes, this
second test shall be performed.
C.2 Reference codestreams for the Light subline 422.10 profile
Table C.1 lists the codestreams for testing implementations against the Light subline 422.10 profile.
Table C.1 — Light subline 422.10 profile test streams
TCS Level Sublevel PSNR bound
2.jxc 4k-2 Sublev3bpp
48.jxc 4k-2 Sublev3bpp
20.jxc Unrestricted Sublev9bpp
28.jxc 4k-1 Unrestricted 60
50.jxc 4k-1 Unrestricted
36.jxc 2k-1 Unrestricted
C.3 Reference codestreams for the Light 422.10 profile
Table C.2 lists the codestreams for testing implementations against the Light 422.10 profile.
Table C.2 — Light 422.10 profile test streams
TCS Level Sublevel PSNR bound
3.jxc 4k-2 Sublev6bpp
49.jxc 4k-2 Sublev3bpp 60
12.jxc 4k-2 Sublev3bpp
51.jxc 4k-2 Sublev6bpp
21.jxc Unrestricted Sublev12bpp
29.jxc 4k-1 Sublev12bpp
37.jxc 2k-1 Sublev9bpp
© ISO/IEC 2025 – All rights reserved
C.4 Reference code
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.
Loading comments...