ISO/IEC 21794-3:2021

Information technology - Plenoptic image coding system (JPEG Pleno) - Part 3: Conformance testing

This document provides the conformance testing of the ISO/IEC 21794 series, also known as JPEG Pleno.

Technologies de l'information — Système de codage d'images plénoptiques (JPEG Pleno) — Partie 3: Titre manque

General Information

• Status: Published
• Publication Date: 19-Dec-2021
• ICS: 35.040.30 - Coding of graphical and photographical information
• Technical Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Drafting Committee: ISO/IEC JTC 1/SC 29/WG 1 - JPEG Coding of digital representations of images
• Current Stage: 6060 - International Standard published
• Start Date: 20-Dec-2021
• Due Date: 25-Aug-2021
• Completion Date: 20-Dec-2021

Overview - ISO/IEC 21794-3:2021 (JPEG Pleno Part 3)

ISO/IEC 21794-3:2021 specifies conformance testing for the ISO/IEC 21794 series (JPEG Pleno). JPEG Pleno is a framework for coding advanced plenoptic imaging modalities - such as light fields, point clouds and holograms - and this part defines the methodology, procedures and test suites used to verify that implementations (particularly decoders) conform to the JPEG Pleno specifications.

Key topics and technical requirements

• Scope and purpose: Defines conformance testing methods to promote interoperability across JPEG Pleno encoders and decoders. Conformance testing checks that implementations implement the required features of a given profile and level.
• Profiles and levels: Two baseline profiles are defined:
  • BBBP (baseline block-based profile) - aligned with 4D transform mode (4DTM) tools.
  • BVBP (baseline view-based profile) - aligned with 4D prediction mode (4DPM) tools. Levels constrain resolution and memory complexity; higher levels are supersets of lower ones.
• Objective metrics: Uses objective fidelity metrics such as maximum absolute error (Emax) and PSNR (computed via MSE) to evaluate decoded output. Example: for BBBP a decoder must decode all test codestreams (TCS) for a level with Emax ≤ 2 when comparing reconstructed views to references.
• Test artifacts and procedures:
  • Test codestreams (TCS) and executable test suites (ETS) support abstract test cases.
  • Annex A: normative decoder conformance test procedures.
  • Annex B: informative decoder test suites and tolerances.
  • Annex C: normative file-format syntax testing (uses a parser whose source is available via the ISO standards repository).
• Reference implementations: The standard references a reference decoder (ISO/IEC 21794‑4) and prescribes how reference outputs are obtained for comparisons.
• Scope limits: Conformance testing excludes robustness, performance, and acceptance testing.

Practical applications - who uses this standard

• Codec developers and implementers of JPEG Pleno encoders/decoders seeking formal conformance claims.
• Interoperability test labs and certification bodies running conformance suites.
• Device manufacturers (camera systems, AR/VR devices) integrating plenoptic imaging formats.
• Software vendors and multimedia platforms validating decoder compatibility.
• Researchers developing light-field, point-cloud or holographic compression tools who need reproducible conformance test methodology.

Related standards

• ISO/IEC 21794-1 - JPEG Pleno framework (file format, system concepts).
• ISO/IEC 21794-2:2021 - Light field coding tools and codestream definitions.
• ISO/IEC 21794-4 - Reference decoder (used for generating reference views in tests).

Keywords: ISO/IEC 21794-3, JPEG Pleno conformance testing, plenoptic image coding, light field, point cloud, hologram, decoder conformance, TCS, PSNR, MSE, BBBP, BVBP.

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 21794-3
First edition
2021-12
Information technology — Plenoptic
image coding system (JPEG Pleno) —
Part 3:
Conformance testing
Reference number
© ISO/IEC 2021
© ISO/IEC 2021
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
ii
© ISO/IEC 2021 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 3
5 General description . 3
5.1 Overview . 3
5.2 Profiles and levels . 3
5.3 Objective metrics . . 4
5.4 Test procedures to test decoders for conformance to ISO/IEC 21794-2 . 4
5.5 File format syntax testing . 4
Annex A (normative) Decoder conformance testing procedure . 5
Annex B (informative) Decoder conformance tests . 9
Annex C (normative) Codestream conformance .13
Bibliography .16
iii
© ISO/IEC 2021 – All rights reserved

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work.
The procedures used to develop this document and those intended for its further maintenance
are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria
needed for the different types of document should be noted. This document was drafted in
accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or
www.iec.ch/members_experts/refdocs).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents) or the IEC
list of patent declarations received (see https://patents.iec.ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see
www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 21794 series can be found on the ISO and IEC websites.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national-committees.
iv
© ISO/IEC 2021 – All rights reserved

Introduction
This document is part of a series of standards for a system known as JPEG Pleno. The ISO/IEC 21794
series aims to provide a standard framework for representing new imaging modalities. It facilitates
the capture, representation, exchange and visualization of plenoptic imaging modalities. A plenoptic
image modality can be a light field, point cloud or hologram, which are sampled representations of
the plenoptic function in the form of, respectively, a vector function that represents the radiance of a
discretized set of light rays, a collection of points with position and attribute information, or a complex
wavefront. The plenoptic function describes the radiance in time and in space obtained by positioning a
pinhole camera at every viewpoint in 3D spatial coordinates, every viewing angle and every wavelength,
resulting in a 7D function.
JPEG Pleno is designed primarily to facilitate the capture, representation, exchange and visualization of
point cloud, light field and holographic imaging modalities. It specifies tools for coding these modalities
while providing advanced functionality at the system level, such as support for data and metadata
manipulation, editing, random access and interaction, protection of privacy and ownership rights, as
well as other security mechanisms.
This document provides the framework, concepts and methodology for testing codestreams and
implementations, and the criteria to be achieved to claim conformance to the ISO/IEC 21794 series.
The objective of this document is to promote interoperability between JPEG Pleno decoders, and
to test these systems for conformance to one or multiple specifications that are part of the JPEG
Pleno. Conformance testing is the testing of a candidate implementation for the existence of specific
characteristics required by a standard. It involves testing the capabilities of an implementation against
the conformance requirements in the relevant standard.
The purpose of this document is to define a common test methodology, to provide a framework for
specific test cases and to define the procedures to be followed during conformance testing.
Any organization contemplating the use of the test methods defined in this document should carefully
consider the constraints on their applicability. Conformance testing does not include robustness testing,
acceptance testing or performance testing, all of which are outside the scope of this document.
v
© ISO/IEC 2021 – All rights reserved

INTERNATIONAL STANDARD ISO/IEC 21794-3:2021(E)
Information technology — Plenoptic image coding system
(JPEG Pleno) —
Part 3:
Conformance testing
1 Scope
This document provides the conformance testing of the ISO/IEC 21794 series, also known as JPEG Pleno.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 21794-1, Information technology — Plenoptic image coding system (JPEG Pleno) — Part 1:
Framework
ISO/IEC 21794-2:2021, Information technology — Plenoptic image coding system (JPEG Pleno) — Part 2:
Light field coding
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 21794-1 and
ISO/IEC 21794-2 and the following apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1
baseline block-based profile
BBBP
4D transform mode coding tools
3.2
baseline view-based profile
BVBP
4D prediction mode coding tools
3.3
box
structured collection of data describing a portion of the file format defined by a length and unique box
type
3.4
conformance
fulfilment of the specified requirements for a given profile
Note 1 to entry: The specified requirements are those defined in this document.
© ISO/IEC 2021 – All rights reserved

3.5
conformance test procedure
steps for assessing conformance (3.4)
3.6
executable test suite
ETS
set of executable test cases that support the abstract test cases
3.7
file format
codestream and its additional support information encoded for storage in a computer file
3.8
header
part of the codestream that contains only markers and marker segments
3.9
image
set of all components
3.10
implementation
realization of a specification
3.11
mean squared error
MSE
average squared difference between the decoded values and the pristine value
3.12
parser
program for syntax analysis
3.13
precision
number of binary digits allocated to a given sample
3.14
prediction mode codestream
ISO/IEC 21794-2 codestream obtained using the 4DPM coding tools
3.15
peak signal-to-noise ratio
PSNR
fidelity measurement between the original and decompressed signal
3.16
test codestream
TCS
available codestream designed to test specific tools
3.17
testing
process of evaluating conformance
3.18
transform mode codestream
ISO/IEC 21794-2 codestream obtained using the 4DTM coding tools
© ISO/IEC 2021 – All rights reserved

4 Abbreviated terms
4DPM four-dimensional prediction mode
4DTM four-dimensional transform mode
ASCII American Standard Code for Information Interchange
5 General description
5.1 Overview
JPEG Pleno encoders will possibly employ only a fraction of the features specified by ISO/IEC 21794-2.
Likewise, some decoders might not implement all the features specified by ISO/IEC 21794-2. It is
impossible to provide test cases for all possible configurations of tools that an encoder or decoder can
implement. This document provides abstract test procedures for JPEG Pleno encoders and decoders. A
developer may designate the features that have been implemented and determine a set of test cases that
applies to those features. This document specifies explicit decoder test procedures that aim to ensure
the greatest level of interoperability between various implementations of encoders and decoders. These
test procedures are used to derive tests that are run for a particular profile. Passing the explicit tests
for a given profile allows a decoder to be labelled as "conforming to a given profile".
Even with the explicit decoder tests, it is expected that some decoders will not decode all of the
information that was originally incorporated into the codestream by an encoder. Since ISO/IEC 21794-2
defines many auxiliary boxes, it is desirable to allow decoders to ignore information that is not of
interest to their target application.
This document describes conformance for JPEG Pleno decoders in terms of a system of assurance.
These assurances serve to discourage encoders from producing codestreams that will be exceedingly
difficult or impossible for a decoder to process, to encourage decoders to provide quality images from
any reasonable codestream, and to encourage the use of the flexibility and scalability of JPEG Pleno
codestreams.
5.2 Profiles and levels
Profiles define a subset of coding techniques, from the ISO/IEC 21794 series, that meet the needs of a
given application. Levels provide information about resolution and memory constraints in conforming
decoder implementations. Decoders implement the capabilities for all bitstreams encoded for a
particular profile. Encoders achieve quality guarantees for particular decoders by encoding bitstreams
which meet a particular profile and level definition.
If a JPEG Pleno encoder produces a codestream with certain properties, then a decoder of a certain
profile and level will be able to produce an image with some defined level of quality. The tests in this
document are designed to require a conforming decoder to be capable of decoding all codestreams.
Two profiles are labelled as baseline block-based profile (BBBP) and baseline view-based profile (BVBP).
These two profiles describe bitstream constraints for an encoder implementing ISO/IEC 21794-1 and
ISO/IEC 21794-2. BBBP corresponds to the 4DTM coding tools. Specifically, this profile considers the
tools described in Annexes A and B but excludes the tools detailed in ISO/IEC 21794-2:2021, Annexes C,
D and E. BVBP corresponds to the 4DPM tools and requires the implementation of all annexes but
Annex B. These profiles do not define a hierarchy, and hence are not subsets of each other. In other
words, they are independent of each other and no other simple relation holds between the other profiles.
Levels define the complexity of the decoding tools and serve as guidance for encoders to produce
codestreams that are easily decodable by decoders conforming to a given profile and level. A lower
level is a subset of a higher level. Hence, any implementation capable of decoding a higher-level test
codestream (TCS) shall be capable of passing the conformance tests for a lower-level codestream of the
same profile.
© ISO/IEC 2021 – All rights reserved

5.3 Objective metrics
Maximum absolute error and peak signal-to-noise ratio (PSNR) are the objective metrics used for
computing the distortions obtained from the conformance test procedure.
The maximum absolute error, E , between the original component, I, and the reconstructed
max
component, I’, both with size MN× samples, is computed as follows:
′
EI= max,()()ij −Ii(), j
max
00≤ The maximum absolute error of the whole light field is the maximum value obtained for all sub-aperture
images.
The PSNR between the original component, I, and the reconstructed component, I’, (both n-bit) is
computed as follows:
n 2
 
()21−
Sl=×10 og
 
PSNR 10
 
E
MSE
 
where E is the mean squared error (MSE) between the two M×N images, I and I’:
MSE
M−1N−1
′
E = ()Ii(),,jI− ()ij
MSE
∑∑
MN×
i=0 j=0
Once the PSNR of the Y-channel (S ), the PSNR of the Cb-channel (S ) and the PSNR of the
PSNR,Y PSNR,Cb
Cr-channel (S ) are computed, the PSNR of all three channels (S ) should be computed as
PSNR,Cr PSNR,YCbCr
follows:
6×+SS +S
PSNR,,YCPSNR bCPSNR, r
S =
PSNR,YCbCr
The PSNRs for the whole light field ( S ) are computed by averaging the PSNRs for all sub-
PSNR,LF
aperture images.
5.4 Test procedures to test decoders for conformance to ISO/IEC 21794-2
The test procedure specified in Annex A shall be used for testing whether a decoder under test
conforms to a particular profile and level from ISO/IEC 21794-2. Annex B specifies the test suites and
the references and tolerances allowed for each TCS.
5.5 File format syntax testing
The procedures defined in Annex C shall be used for testing JPEG Pleno files for conformity to the
file format specified in ISO/IEC 21794-1 and ISO/IEC 21794-2. They depend on a codestream syntax
parsing tool whose source code is available as a machine-readable file at https:// standards .iso .org/ iso
-iec/ 21794/ -3/ ed -1/ en.
© ISO/IEC 2021 – All rights reserved

Annex A
(normative)
Decoder conformance testing procedure
A.1 General
This annex defines procedures to follow for determining whether a decoder conforms to a particular
profile within a particular part.
A.2 Decoder test procedure for baseline block-based profile
The procedure defined in this clause will determine whether a decoder conforms to a given level of the
BBBP. A decoder under test is said to conform to a level of BBBP if it is able to decode all TCSs for that
level and the maximum absolute error E ≤2 when comparing the reconstructed views with their
max
reference view counterparts. Figure A.1 shows the test procedure.
Figure A.1 — Decoder conformance test procedure for a level of the baseline block-based profile
© ISO/IEC 2021 – All rights reserved

A.3 Decoder setting for baseline block-based profile
A.3.1 Reference software decoder settings
1)
The reference decoder software for the BBBP, specified in ISO/IEC 21794-4 along with build
instructions, is used to obtain the reference views. For a given TCS, the reference decoder software
shall be named as follows:
~/jpeg-pleno-refsw/bin$ jpl-decoder-bin --input bbbp-{###}.jpl \
--output ./bbbp-{###}/
The decoded reference views will be available after the decoding process is completed in the folder ./
bbbp-{###}/. in a specific file format defined in A.3.2.
A.3.2 Reference components file format
The organization or name convention of view files for the block-based profile reference decoder is
specified in ISO/IEC 21794-4:—, B.2. This decoder is agnostic to any colour space specification and
will decode only raw sample values. The reference decoded views are in PGX file format, specified
in ISO/IEC 15444-4. Such a file format comports only one component per file, thus light fields with
n-colour components will require the generation of n of such files per view. In order to compare decoded
views from the DUT with these reference decoded views, it is possible that several conversions will
be necessary. These conversions may be done as post-processing steps outside of the decoder solely
for determining conformance. There is no requirement for a conforming decoder to perform these
processes as part of its normal operation. These conversions shall not introduce a quality change (either
loss or gain) except as required by the specific conversions described in the following clauses.
A.4 Decoder test procedure for baseline view-based profile
A decoder under test is said to conform to a level of the BVBP if it is able to decode all TCSs for that level
and the PSNR is greater than the minimum permitted value for the given TCS. Figure A.2 shows the test
procedure.
1) Under preparation. Stage at the time of publicati
...