ISO/IEC 21794-1:2020
(Main)Information technology — Plenoptic image coding system (JPEG Pleno) — Part 1: Framework
Information technology — Plenoptic image coding system (JPEG Pleno) — Part 1: Framework
This document specifies the plenoptic image coding system framework architecture and its instantiation via a generic file format for storage of plenoptic modalities as well as associated metadata descriptors.
Technologies de l'information — Système de codage d'images plénoptiques (JPEG Pleno) — Partie 1: Cadre
General Information
Buy Standard
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 21794-1
First edition
2020-08
Information technology — Plenoptic
image coding system (JPEG Pleno) —
Part 1:
Framework
Technologies de l'information — Système de codage d'images
plénoptiques (JPEG Pleno) —
Partie 1: Cadre
Reference number
ISO/IEC 21794-1:2020(E)
©
ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC 21794-1:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 21794-1:2020(E)
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 3
5 Conventions . 3
5.1 Conformance language . 3
5.2 Naming conventions for numerical values . 3
6 Framework definition . 4
7 File format architecture . 5
8 Organization of the document . 5
Annex A (normative) JPEG Pleno file format (JPL) . 6
Annex B (informative) JPEG Pleno reference grid system .17
Annex C (informative) Conceptual example .18
Bibliography .20
© ISO/IEC 2020 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 21794-1:2020(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see https:// patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 21794 series can be found on the ISO website.
iv © ISO/IEC 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 21794-1:2020(E)
Introduction
This document is part of a series of standards for a system known as JPEG Pleno. This document defines
the JPEG Pleno framework. It facilitates the capture, representation, exchange and visualization of
plenoptic imaging modalities. A plenoptic image modality can be a light field, point cloud or hologram,
which are sampled representations of the plenoptic function in the form of, respectively, a vector
function that represents the radiance of a discretized set of light rays, a collection of points with
position and attribute information, or a complex wavefront. The plenoptic function describes the
radiance in time and in space obtained by positioning a pinhole camera at every viewpoint in 3D spatial
coordinates, every viewing angle and every wavelength, resulting in a 7D function.
JPEG Pleno specifies tools for coding these modalities while providing advanced functionality at system
level, such as support for data and metadata manipulation, editing, random access and interaction,
protection of privacy and ownership rights.
© ISO/IEC 2020 – All rights reserved v
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 21794-1:2020(E)
Information technology — Plenoptic image coding system
(JPEG Pleno) —
Part 1:
Framework
1 Scope
This document specifies the plenoptic image coding system framework architecture and its instantiation
via a generic file format for storage of plenoptic modalities as well as associated metadata descriptors.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 646, Information technology — ISO 7-bit coded character set for information interchange
ISO/IEC 15444-1:2019, Information technology — JPEG 2000 image coding system — Part 1: Core
coding system
ISO/IEC 15444-2:2004, Information technology — JPEG 2000 image coding system — Part 2: Extensions
1)
ISO/IEC 21794-2:—, Information technology — Plenoptic image coding system (JPEG Pleno) — Part 2:
Light field coding
3 Terms and definitions
For the purposes of this document the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
big-endian
byte ordering for which the most significant byte and least significant byte are sequentially ordered
from lower memory address to higher memory address, respectively
3.2
box
structured collection of data describing the image or the image decoding process
3.3
box content
data wrapped within the box structure
1) Under preparation. Stage at time of publication: ISO/IEC DIS 21794-2:2019.
© ISO/IEC 2020 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC 21794-1:2020(E)
3.4
box type
kind of information stored within the box
3.5
coder
embodiment of a coding process
3.6
codestream
coded data representation that includes all necessary data to allow a (full or approximate)
reconstruction of the sample values of a digital image
3.7
coding process
encoding process, decoding process, or both
3.8
complex wavefront
wavefront represented with a complex representation, which can be, for example, real-imaginary or
amplitude-phase
3.9
component
two-dimensional array of samples having the same designation in the output or display device, e. g.,
red, green or blue
3.10
decoder
embodiment of a decoding process
3.11
decoding process
process that takes as its input a codestream and outputs a continuous-tone image
3.12
encoder
embodiment of an encoding process
3.13
encoding process
process that takes as its input a continuous-tone image and outputs a codestream
3.14
hologram
sampled representation of the plenoptic function in the form of a complex wavefront
3.15
holographic display
three-dimensional display that renders a complex optical wavefront
3.16
light field
sampled representation of the plenoptic function in the form of a vector function that represents the
radiance of a discretized set of light rays
3.17
light field data
recorded light field
2 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 21794-1:2020(E)
3.18
plenoptic function
radiance in time and in space obtained by positioning a pinhole camera at every viewpoint in 3D spatial
coordinates, every viewing angle and every wavelength, resulting in a 7D function
3.19
plenoptic data
sampled representation of the plenoptic function
Note 1 to entry: Example plenoptic functions are light field, point cloud and holographic representation.
3.20
plenoptic object
plenoptic element
plenoptic data representing a part of the scene described by the plenoptic function
3.21
point cloud
sampled representation of the plenoptic function in the form of collection of points with position and
attribute information
3.22
superbox
box that carries other boxes as payload data
3.23
wavefront
locus of spatial points that share the same phase of the light wave
4 Abbreviated terms
3D three dimensional
CGH computer-generated holography
IPR intellectual property rights
JPEG Joint Photographic Experts Group
JPL Jpeg PLeno file format
UUID Universally Unique IDentifier
XML eXtensible Markup Language
5 Conventions
5.1 Conformance language
The keyword "reserved" indicates a provision that is not specified at this time, shall not be used, and
may be specified in the future. The keyword "forbidden" indicates "reserved" and in addition indicates
that the provision will never be specified in the future.
5.2 Naming conventions for numerical values
Integer numbers are expressed as bit patterns, hexadecimal values or decimal numbers. Bit patterns
and hexadecimal values have both a numerical value and an associated particular length in bits.
© ISO/IEC 2020 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC 21794-1:2020(E)
Hexadecimal notation, indicated by prefixing the hexadecimal number by "0x", is used instead of binary
notation to denote a bit pattern having a length that is an integer multiple of 4. For example, 0x41
represents an eight-bit pattern having only its second most significant bit and its least significant bit
equal to 1. An octal notation is indicated by prefixing the octal value “\”, this notation is used instead
of binary notation to denote a bit pattern having a length that is an integer multiple of 3. Numerical
values that are specified under a "Code" heading in tables that are referred to as "code tables" are bit
pattern values (specified as a string of digits equal to 0 or 1 in which the left-most bit is considered the
most-significant bit). Other numerical values not prefixed by "0x" are decimal values. When used in
expressions, a hexadecimal value is interpreted as having a value equal to the value of the corresponding
bit pattern evaluated as a binary representation of an unsigned integer (i.e., as the value of the number
formed by prefixing the bit pattern with a sign bit equal to 0 and interpreting the result as a two's
complement representation of an integer value). For example, the hexadecimal value 0xF is equivalent
to the 4-bit pattern '1111' and is interpreted in expressions as being equal to the decimal number 15.
6 Framework definition
The plenoptic image coding system (JPEG Pleno) specifies a framework that facilitates the capture,
representation and exchange of point cloud, light field and holographic imaging modalities. These
[1]
imaging modalities are light representations of the plenoptic function , regardless of which modality
was used to capture or create parts of the entire content:
— Point clouds are a set of data points representing the plenoptic function in a given, often 3D,
coordinate system with associated attributes;
— Light fields are a sampled representation of the plenoptic function by capturing it by either an
array of cameras (resulting in wide baseline light field data) or by a single light field camera that
uses microlenses to sample individual rays of light that contribute to the final image (resulting in
narrow baseline light field data);
— Holograms are a sampled interferometric representation of the plenoptic function that is based on
a wave-based light propagation model.
This framework supports conversations between different modalities throughout an end-to-end
processing chain. This document defines file format and codestream syntax for plenoptic modalities
while providing advanced functionalities at system level. It also supports data and metadata
manipulation, editing, random access and interaction, protection of privacy and ownership.
Figure 1 — JPEG Pleno framework
4 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 21794-1:2020(E)
The ISO/IEC 21794 series consists of the following Parts (Figure 1).
— This document outlines the overall JPEG Pleno framework and specifies the generic JPEG Pleno
file format.
— ISO/IEC 21794-2 specifies the file format and codestream syntax for coded light fields. Moreover, it
specifies output of JPEG Pleno light field encoders and provides documentation on the encoding and
decoding procedures.
2)
— ISO/IEC 21794-3 specifies conformance testing protocols.
2)
— ISO/IEC 21794-4 specifies reference software.
7 File format architecture
This document shares common definitions for the structure of files (a sequence of objects, called boxes
here, and atoms in other similar file formats), and a common definition of the general structure of an
object (the size and type).
The specification of the file format in Annex A is built upon the structure of ISO/IEC 15444-1 and
ISO/IEC 15444-2 file formats.
All these specifications require that readers ignore objects that are unrecognizable to them.
This document shall take precedence over those on which it is based, in any case where there are
differences or conflicts; however, no such conflicts are known to exist.
For better readability and understanding the syntax description for the different file formats is done in
the same way as in the base formats.
8 Organization of the document
Annex A specifies the JPEG Pleno box-based file format, which includes definitions of boxes to carry
light field, point cloud and holographic data. Every JPEG Pleno file shall conform to the specified file
format syntax.
Annex B specifies the JPEG Pleno reference grid system, a coordinate system that allows for positioning
plenoptic modalities in a global reference grid system, while using a local coordinate system per
plenoptic element to facilitate efficient local addressing.
Annex C provides an example of how plenoptic objects can be signalled using the JPEG Pleno framework.
2) Under preparation.
© ISO/IEC 2020 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC 21794-1:2020(E)
Annex A
(normative)
JPEG Pleno file format (JPL)
A.1 General
This annex defines a still image file format that applications may choose to wrap a codestream based
on a JPEG Pleno encoded image. This document is based on the same syntax as the box-based file format
for JPEG 2000 in ISO/IEC 15444-1:2019, Annex I or ISO/IEC 15444-2:2004, Annex M.
This annex:
— specifies a binary container (file) for both light field and metadata;
— specifies a mechanism to indicate light field properties, such as the tone scale or colour space of
the image;
— specifies a mechanism by which readers may recognize the existence of intellectual property rights
information in the file;
— specifies a mechanism by which metadata (including vendor-specific information) can be included
in files specified by this document.
A.2 Specification of the JPL file format
A.2.1 General
The JPL file format provides a foundation for storing application specific data (metadata) in association
with a JPEG Pleno codestream, such as information which is required to display the light field. As many
applications require a similar set of information to be associated with the coded light field data, it is
useful to define the format of that set of data along with the definition of the coding technology and
codestream syntax.
Conceptually, the JPL file format encapsulates the JPEG Pleno codestream along with other core pieces
of information about that codestream. The building-block of the JPL file format is called a "box". All
information contained within the JPL file is encapsulated in boxes. This document defines several types
of boxes; the definition of each specific box type defines the kinds of information that may be found
within a box of that type. Some boxes will be defined to contain other boxes.
A.2.2 File identification
JPL files can be identified using several mechanisms. When stored in traditional computer file systems,
JPL files should be given the file extension ".jpl" (readers should allow mixed case for the alphabetic
characters).
A.2.3 File organization
A JPL file represents a collection of boxes. Some of those boxes are independent, and some of those
boxes contain other boxes, denoted here as "superboxes". The binary structure of a file is a contiguous
sequence of boxes. The start of the first box shall be the first byte of the file, and the last byte of the last
box shall be the last byte of the file.
The binary structure of a box is identical to ISO/IEC 15444-1 and defined in Annex A.3.
6 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 21794-1:2020(E)
The conceptual structure of a JPL file is shown in Figure A.1. Boxes with dashed borders are optional in
conforming JPL files. However, an optional box may define mandatory boxes within that optional box.
In that case, if the optional box exists, those mandatory boxes within the optional box shall exist. If the
optional box does not exist, then the mandatory boxes within those boxes shall also not exist.
Figure A.1 specifies only the containment relationship between the boxes in the file.
The order of the boxes is not completely arbitrary and shall conform to the following rules:
— The JPEG Pleno Signature box shall be the first box in a JPL file and the File Type box (defined in
ISO/IEC 15444-1) shall immediately follow the JPEG Pleno Signature box.
— The JPEG Pleno Thumbnail box shall be signalled before the JPEG Pleno Light Field, JPEG Pleno Point
Cloud, and JPEG Pleno Hologram superboxes.
— A JPL file can contain an optional XML box signalling catalog information (see A.5.8). An XML box
containing catalogue information should be signalled after the File Type box and before the first
superbox containing plenoptic data.
— The JPEG Pleno Light Field, JPEG Pleno Point Cloud, and JPEG Pleno Hologram superboxes signalling
plenoptic data, shall be signalled as one monolithic block with no preferred ordering, and no other
boxes shall be signalled in between.
The file in Figure A.1 is a strict sequence of boxes. Other boxes may be found between the boxes defined
in this document, except the JPEG Pleno Light Field, JPEG Pleno Point Cloud, and JPEG Pleno Hologram
superboxes. However, all information contained within a JPL file shall be in the box format; byte-
streams not in the box format shall not be found in the file.
A JPL file may also contain an IPR box and one or more XML, UUID and UUID Info boxes. If present these
boxes shall be as defined in ISO/IEC 15444-1.
© ISO/IEC 2020 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/IEC 21794-1:2020(E)
Figure A.1 — Conceptual structure of a JPL file
A.2.4 Metadata
One important aspect of the JPL file format is the ability to add metadata to a JPL file. Some of the boxes
provide a set of tools by which applications can add vendor-specific information to the JPL file format,
like the XML box. These boxes are optional in conforming files and may be ignored by conforming
readers.
A.2.5 Conformance with the file format
All conforming files shall contain all boxes required by this document, and those boxes shall be as
defined in this document. Also, all conforming readers shall correctly interpret all required boxes
defined in this document and thus shall correctly interpret all conforming files.
Because all information is encapsulated in boxes, and all boxes have types, the format provides a
simple mechanism for a reader to extract relevant information, while ignoring any box that contains
8 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 13 ----------------------
ISO/IEC 21794-1:2020(E)
information that is not understood by that particular reader. In this way, new boxes can be created,
either through this document or other International Standards or private implementation. Also, any
new box added to a JPL file shall not change the visual appearance of the image.
A.3 Concept of boxes
A.3.1 Key to graphical descriptions
Each box is described in terms of its function, usage and length. The function describes the information
contained in the box. The usage describes the logical location and frequency of this box in the file.
These descriptions are followed by a figure that shows the order and relationship of the parameters
in the box. Figure A.2 provides an example. A rectangle is used to indicate the parameters in the box.
The width of the rectangle is proportional to the number of bytes in the parameter. A shaded rectangle
(diagonal stripes) indicates that the parameter is of varying size. Two parameters with superscripts
and a grey area between them indicate a run of several of these parameters. A sequence of two groups
of multiple parameters with superscripts separated by a grey area indicates a run of that group of
parameters (one set of each parameter in the group, followed by the next set of each parameter in the
group). Optional parameters or boxes will be shown with a dashed rectangle.
Figure A.2 — Example of a box description figure
The figure is followed by a list that describes the meaning of each parameter in the box. If parameters
are repeated, the length and nature of the run of parameters is defined. As an example, in Figure A.2,
0 N–1
parameters C, D, E and F are 8-, 16-, 32-bit and variable length respectively. The notation G and G
i i i
implies that there are N different parameters, G , in a row. The parameters pairs H and J , where i goes
0 0 1 1
from 0 to M-1, specify that the box will contain H , followed by J , followed by H and J , continuing to
M–1 M–1
H and J (M instances of each parameter in total). Finally, the field L is optional and may not be
found in this box.
After the list is a table that either describes the allowed parameter values or provides references to
other tables that describe these values.
Some boxes may carry other boxes as payload data. Such boxes are denoted as superboxes. The payload
size of a superbox is given by the sum of the box lengths of all the boxes it contains.
In addition, in a figure describing the contents of a superbox, an ellipsis (…) will be used to indicate that
the contents of the file between two boxes are not specifically defined. Any box (or sequence of boxes),
unless otherwise specified by the definition of that box, may be found in place of the ellipsis.
For example, the superbox shown in Figure A.3 must contain an AA box and a BB box, and the BB box
must follow the AA box. However, there may be other boxes found between boxes AA and BB. Dealing
with unknown boxes is discussed in A.6.
© ISO/IEC 2020 – All rights reserved 9
---------------------- Page: 14 ----------------------
ISO/IEC 21794-1:2020(E)
Figure A.3 — Example of the superbox description figures
A.3.2 Box definition
Physically, each object in the file is encapsulated within a binary structure called a box. That binary
structure is shown in Figure A.4 and Table A.1.
Figure A.4 — Organization of a box
LBox Box length. This field specifies the length of the box in multiples of bytes, stored as a
4-byte big-endian unsigned integer. This value includes all of the fields of the box, includ-
ing the length and type. If the value of this field is 1, then the XLBox field shall exist and
the value of that field shall be the actual length of the box. If the value of this field is 0, then
the length of the box was not known when the LBox field was written. In this case, this box
contains all bytes up to the end of the file. If a box of length 0 is contained within another
box (its superbox), then the length of that superbox shall also be 0. This means that this
box is the last box in the file. The values 2-7 are reserved for future use by ISO/IEC.
TBox Box type. This field specifies the type of information found in the DBox field. The value of
this field is encoded as a 4-byte big-endian unsigned integer. However, boxes are gener-
ally referred to by an ISO/IEC 646 character string translation of the integer value. For
all box types defined within this document, box types will be indicated as both character
string and as 4-byte hexadecimal integers. Also, a space character is shown in the charac-
ter string translation of the box type as “\040”. All values of TBox not defined within this
document are reserved for future use by ISO/IEC.
XLBox Box extended length. This field specifies the actual length of the box if the value of the
LBox field is 1. This field is stored as an 8-byte big-endian unsigned integer. The value
includes all of the fields of the box, including the LBox, TBox and XLBox fields.
DBox Box contents. This field contains the actual information contained within this box. The
format of the box contents depends on the box type and will be defined individually for
each type. While variable length fields can be held with DBox, the length of DBox shall be
an integer multiple of octets. Bit-padding shall be provided to ensure this.
Table A.1 — Binary structure of a box
Field name Size (bits) Value
0, 1, or
LBox 32
32
8 to (2 – 1)
TBox 32 Variable
64
64 16 to (2 – 1); if LBox = 1
XLBox
0 Not applicable; if LBox ≠ 1
DBox Variable Variable
For example, consider the illustration in Figure A.5 of a sequence of boxes, including one box that
contains other boxes:
10 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 15 ----------------------
ISO/IEC 21794-1:2020(E)
Figure A.5 — Illustration of box lengths
As shown in Figure A.5, the length of each box includes any boxes contained within that box. For
example, the length of Box 1 includes the length of Boxes 2 and 3, in addition to the LBox and TBox fields
for Box 1 itself. In this case, if the type of Box 1 was
...
DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 21794-1
ISO/IEC JTC 1/SC 29 Secretariat: JISC
Voting begins on: Voting terminates on:
2019-10-28 2020-01-20
Information technology — JPEG Pleno Plenoptic image
coding system —
Part 1:
Framework
ICS: 35.040.30
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 21794-1:2019(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
©
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2019
---------------------- Page: 1 ----------------------
ISO/IEC DIS 21794-1:2019(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2019 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC DIS 21794-1:2019(E)
Contents Page
1 Scope .1
2 Normative references .1
3 Terms and definitions .1
4 Abbreviations .4
5 Conventions .4
5.1 Conformance language .4
5.2 Naming conventions for numerical values .5
6 Framework definition .5
7 Conformance .6
8 Organization of the document .6
Annex A JPEG Pleno file format – JPL .7
A.1 General .7
A.2 Specification of the JPL file format .7
A.2.1 General .7
A.2.2 File identification .7
A.2.3 File organization .8
A.2.4 Metadata .9
A.2.5 Conformance with the file format .9
A.3 Concept of boxes . 10
A.3.1 Key to graphical descriptions . 10
A.3.2 Box definition . 11
A.4 Defined boxes . 12
A.5 Defined boxes . 13
A.5.1 JPEG Pleno Signature box . 13
A.5.2 File Type box . 14
A.5.3 JPEG Pleno Thumbnail box (superbox) . 15
A.5.4 Image Header box . 15
A.5.5 JPEG Pleno Light Field box (superbox) . 16
A.5.6 JPEG Pleno Point Cloud box (superbox) . 16
A.5.7 JPEG Pleno Hologram box (superbox). 16
A.5.8 XML box . 16
A.6 Dealing with unknown boxes . 18
Annex B JPEG Pleno Reference Grid System . 19
Annex C Conceptual example . 20
© ISO/IEC 2019 – All rights reserved
iii
---------------------- Page: 3 ----------------------
ISO/IEC DIS 21794-1:2019(E)
Figures Page
Figure 1 — JPEG Pleno Framework . 6
Figure A.1 — Conceptual structure of a JPL file . 9
Figure A.2 — Example of a box description figure . 10
Figure A.3 — Example of the superbox description figures . 11
Figure A.4 — Organization of a box. 11
Figure A.5 — Illustration of box lengths . 12
Figure A.6 — Organization of the contents of a File Type box. 14
Figure A.7 — Organization of the contents of a JPEG Pleno Thumbnail box . 15
Figure B.1 — The global and local reference grid . 19
Figure C.1 — Conceptual example of a JPEG Pleno workflow . 20
© ISO/IEC 2019 – All rights reserved
iv
---------------------- Page: 4 ----------------------
ISO/IEC DIS 21794-1:2019(E)
Tables
Table A.1 — Binary structure of a box . 11
Table A.2 — Defined boxes . 13
Table A.3 — Allowed Brand values . 14
Table A.4 — Format of the contents of the File Type box . 15
Table A.5 — Definition of JPEG Pleno Schema Descriptor Elements . 16
Table A.6 — XSD schema describing the JPL file content . 17
Table A.7 — Example XLM schema of JPL file containing a light field and point cloud element . 17
© ISO/IEC 2019 – All rights reserved
v
---------------------- Page: 5 ----------------------
ISO/IEC DIS 21794-1:2019(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical activity.
ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of document should be noted. This document was drafted in accordance with the editorial
rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights. Details
of any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the World
Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following
URL: www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 21794 series can be found on the ISO website.
© ISO/IEC 2019 – All rights reserved
vi
---------------------- Page: 6 ----------------------
ISO/IEC DIS 21794-1:2019(E)
Introduction
This document is part of a series of standards belonging to the JPEG Pleno framework. This standard
framework facilitates the capture, representation, exchange and visualization of point cloud, light field,
and holographic imaging modalities. It specifies tools for coding these modalities while providing
advanced functionality at system level such as support for data and metadata manipulation, editing,
random access and interaction, protection of privacy and ownership rights.
This document specifies the JPEG Pleno Framework architecture and its instantiation via a generic file
format for storage of plenoptic modalities as well as associated metadata descriptors.
© ISO/IEC 2019 – All rights reserved
vii
---------------------- Page: 7 ----------------------
DRAFT INTERNATIONAL STANDARD ISO/IEC DIS 21794-1:2019(E)
Information technology — JPEG Pleno Plenoptic image coding
system — Part 1: Framework
1 Scope
This document specifies the JPEG Pleno Framework architecture and its instantiation via a generic file
format for storage of plenoptic modalities as well as associated metadata descriptors.
2 Normative references
The following International Standards contain provisions which, through reference in this text, constitute
provisions of this document. At the time of publication, the editions indicated were valid.
All Standards are subject to revision, and parties to agreements based on this document are encouraged
to investigate the possibility of applying the most recent edition of the Standards listed below. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies. Members of IEC and ISO maintain registers of currently
valid International Standards.
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document.
ISO/IEC 15444-1, Information technology — JPEG 2000 image coding system: Core coding system
ISO/IEC 15444-2, Information technology — JPEG 2000 image coding system: Extensions
IEEE 754, IEEE Standard for Floating-Point Arithmetic
ISO/IEC 646, Information technology -- ISO 7-bit coded character set for information interchange
3 Terms and definitions
For the purposes of this document the terms, definitions, and abbreviated terms given in ISO/IEC 21794-
1, and the following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
⎯ ISO Online browsing platform: available at https://www.iso.org/obp
⎯ IEC Electropedia: available at http://www.electropedia.org/
© ISO/IEC 2019 – All rights reserved
1
---------------------- Page: 8 ----------------------
ISO/IEC DIS 21794-1:2019(E)
3.1
big-endian
byte ordering for which the most significant byte and least significant byte are sequentially ordered from
lower memory address to higher memory address, respectively
3.2
bit
binary choice encoded as either 0 or 1
3.3
box
structured collection of data describing the image or the image decoding process
3.4
box contents
refers to the data wrapped within the box structure
3.5
box type
specifies the kind of information that shall be stored with the box
3.6
byte
group of 8 bits (octet)
coder
embodiment of a coding process
3.7
codestream
coded data representation that includes all necessary data to allow a (full or approximate)
reconstruction of the sample values of a digital image
3.8
coding
encoding or decoding
3.9
coding process
general term for referring to an encoding process, a decoding process, or both
3.10
complex wavefront
wavefront represented with a complex representation, which can be for example real-imaginary or
amplitude-phase
3.11
component
two-dimensional array of samples having the same designation in the output or display device, e. g. red,
green or blue
© ISO/IEC 2019 – All rights reserved
2
---------------------- Page: 9 ----------------------
ISO/IEC DIS 21794-1:2019(E)
3.12
decoder
embodiment of a decoding process
3.13
decoding process
process that takes as its input coded image data and outputs a continuous-tone image
3.14
encoder
embodiment of an encoding process
3.15
encoding process
process that takes as its input a continuous-tone image and outputs coded image data
3.16
hologram
sampled representation of the plenoptic function in the form of a complex wavefront
3.17
holographic display
three-dimensional display that renders a complex optical wavefront
3.18
JPL
still image file format with JPEG Pleno coded images
3.19
light field
sampled representation of the plenoptic function in the form of a vector function that represents the
radiance of a discretized set of light rays
3.20
light field data
recorded light field
3.21
metadata
type of data that provides additional information about the encoded data
3.22
plenoptic function
radiance in time and in space obtained by positioning a pinhole camera at every viewpoint in 3D spatial
coordinates, every viewing angle and every wavelength, resulting in a 7D representation
3.23
plenoptic data
sampled representation of the plenoptic function (e.g. light field, point cloud, holographic representation)
© ISO/IEC 2019 – All rights reserved
3
---------------------- Page: 10 ----------------------
ISO/IEC DIS 21794-1:2019(E)
3.24
plenoptic object or element
light field, point cloud or hologram
3.25
point cloud
sampled representation of the plenoptic function in the form of collection of points with position and
attribute information
3.26
superbox
box that carries other boxes as payload data
3.27
wavefront
locus of spatial points that share the same phase of the light wave
4 Abbreviations
For the purposes of this document, the following abbreviations apply.
3D Three Dimensional
CGH Computer-Generated Holography
CIE Commission Internationale de l´Eclairage
IPR Intellectual Property Rights
JPEG Joint Photographic Experts Group
JPL Jpeg PLeno file format
UUID Universally Unique IDentifiers
XML eXtensible Markup Language
5 Conventions
5.1 Conformance language
This document consists of normative and informative text.
Normative text is that text which expresses mandatory requirements. The word "shall" is used to express
mandatory requirements strictly to be followed in order to conform to this document and from which no
deviation is permitted. A conforming implementation is one that fulfils all mandatory requirements.
Informative text is text that is potentially helpful to the user, but not indispensable and can be removed,
changed or added editorially without affecting interoperability. All text in this document is normative,
with the following exceptions: the Introduction, any parts of the text that are explicitly labelled as
"informative", and statements appearing with the preamble "NOTE" and behaviour described using the
© ISO/IEC 2019 – All rights reserved
4
---------------------- Page: 11 ----------------------
ISO/IEC DIS 21794-1:2019(E)
word "should". The word "should" is used to describe behaviour that is encouraged but is not required
for conformance to this document.
The keywords "may" and "need not" indicate a course of action that is permissible in a conforming
implementation.
The keyword "reserved" indicates a provision that is not specified at this time, shall not be used, and may
be specified in the future. The keyword "forbidden" indicates "reserved" and in addition indicates that
the provision will never be specified in the future.
5.2 Naming conventions for numerical values
Integer numbers are expressed as bit patterns, hexadecimal values, or decimal numbers. Bit patterns and
hexadecimal values have both a numerical value and an associated particular length in bits.
Hexadecimal notation, indicated by prefixing the hexadecimal number by "0x", may be used instead of
binary notation to denote a bit pattern having a length that is an integer multiple of 4. For example, 0x41
represents an eight-bit pattern having only its second most significant bit and its least significant bit equal
to 1. An octal notation is indicated by prefixing the octal value “\”, this notation may be used instead of
binary notation to denote a bit pattern having a length that is an integer multiple of 3. Numerical values
that are specified under a "Code" heading in tables that are referred to as "code tables" are bit pattern
values (specified as a string of digits equal to 0 or 1 in which the left-most bit is considered the most-
significant bit). Other numerical values not prefixed by "0x" are decimal values. When used in
expressions, a hexadecimal value is interpreted as having a value equal to the value of the corresponding
bit pattern evaluated as a binary representation of an unsigned integer (i.e., as the value of the number
formed by prefixing the bit pattern with a sign bit equal to 0 and interpreting the result as a two's
complement representation of an integer value). For example, the hexadecimal value 0xF is equivalent to
the 4-bit pattern '1111' and is interpreted in expressions as being equal to the decimal number 15.
6 Framework definition
JPEG Pleno specifies a framework that facilitates the capture, representation and exchange of point cloud,
light field, and holographic imaging modalities. These imaging modalities are light representations of the
plenoptic function [1], regardless of which modality was used to capture or create parts of the entire
content:
- Point clouds are a set of data points representing the plenoptic function in a given, often 3D,
coordinate system with associated attributes;
- Light fields are a sampled representation of the plenoptic function by capturing it by either an
array of cameras (resulting in wide baseline light field data) or by a single light field camera that
uses microlenses to sample individual rays of light that contribute to the final image (resulting in
narrow baseline light field data);
- Holograms are a sampled interferometric representation of the plenoptic function that is based
on a wave-based light propagation model.
This framework supports conversations between different modalities throughout an end-to-end
processing chain. The JPEG Pleno specification defines file format and codestream syntax for plenoptic
modalities while providing advanced functionalities at system level. It also supports data and metadata
manipulation, editing, random access and interaction, protection of privacy and ownership.
© ISO/IEC 2019 – All rights reserved
5
---------------------- Page: 12 ----------------------
ISO/IEC DIS 21794-1:2019(E)
Figure 1 — JPEG Pleno Framework
The JPEG Pleno specification is composed out of several parts (Figure 1).
- ISO/IEC 21794-1 “JPEG Pleno: Framework”, this document, outlines the overall JPEG Pleno
framework. This part specifies the generic JPEG Pleno file format as well;
- ISO/IEC 21794-2 “JPEG Pleno: Light Field Coding” specifies the file format and codestream
syntax for coded light fields. Moreover, it specifies normative output of JPEG Pleno light field
encoders and provides informative documentation on the encoding and decoding procedures;
7 Conformance
This document shares common definitions for the structure of files (a sequence of objects, called boxes
here, and atoms in other similar file formats), and a common definition of the general structure of an
object (the size and type).
The specification of the JPEG Pleno file format in Annex A is built upon the structure of the JP2
ISO/IEC 15444-1 and JPX ISO/IEC 15444-2 file formats.
All these specifications require that readers ignore objects that are unrecognizable to them.
This document takes precedence over those on which it is based, in any case where there are differences
or conflicts; however, no such conflicts are known to exist.
For better readability and understanding the syntax description for the different file formats is done in
the same way as in the base formats.
8 Organization of the document
Annex A specifies the JPEG Pleno box-based file format, which includes definitions of boxes to carry light
field, point cloud and holographic data. Every JPEG Pleno file shall comply to the specified file format
syntax.
Annex B specifies the normative JPEG Pleno reference grid system, a coordinate system that allows for
positioning plenoptic modalities in a global reference grid system, while using a local coordinate system
per plenoptic element to facilitate efficient local addressing.
Annex C provides an informative example of how plenoptic objects can be signalled utilizing the JPEG
Pleno framework.
© ISO/IEC 2019 – All rights reserved
6
---------------------- Page: 13 ----------------------
ISO/IEC DIS 21794-1:2019(E)
Annex A
(normative)
JPEG Pleno file format – JPL
A.1 General
This annex defines a still image file format that applications may choose to wrap a codestream based on
a JPEG Pleno encoded image. This document is based on the same syntax as the box-based file format for
JPEG 2000 in ISO/IEC 15444-1 Annex I or ISO/IEC 15444-2 Annex M.
This annex:
⎯ specifies a binary container (file) for both light field and metadata;
⎯ specifies a mechanism to indicate light field properties, such as the tone scale or colour space of the
image;
⎯ specifies a mechanism by which readers may recognize the existence of intellectual property rights
information in the file;
⎯ specifies a mechanism by which metadata (including vendor-specific information) can be included
in files specified by this document.
A.2 Specification of the JPL file format
A.2.1 General
The JPL file format provides a foundation for storing application specific data (metadata) in association
with a JPEG Pleno codestream, such as information which is required to display the light field. As many
applications require a similar set of information to be associated with the coded light field data, it is useful
to define the format of that set of data along with the definition of the coding technology and codestream
syntax.
Conceptually, the JPL file format encapsulates the JPEG Pleno codestream along with other core pieces of
information about that codestream. The building-block of the JPL file format is called a "box". All
information contained within the JPL file is encapsulated in boxes. This document defines several types
of boxes; the definition of each specific box type defines the kinds of information that may be found within
a box of that type. Some boxes will be defined to contain other boxes.
A.2.2 File identification
JPL files can be identified using several mechanisms. When stored in traditional computer file systems,
JPL files should be given the file extension ".jpl" (readers should allow mixed case for the alphabetic
characters).
© ISO/IEC 2019 – All rights reserved
7
---------------------- Page: 14 ----------------------
ISO/IEC DIS 21794-1:2019(E)
A.2.3 File organization
A JPL file represents a collection of boxes. Some of those boxes are independent, and some of those boxes
contain other boxes, denoted here as "superboxes". The binary structure of a file is a contiguous sequence
of boxes. The start of the first box shall be the first byte of the file, and the last byte of the last box shall be
the last byte of the file.
The binary structure of a box is identical to ISO/IEC 15444-1 and defined in Annex A.3 of this standard.
The conceptual structure of a JPL file is as shown in Figure A.1. Boxes with dashed borders are optional
in conforming JPL files. However, an optional box may define mandatory boxes within that optional box.
In that case, if the optional box exists, those mandatory boxes within the optional box shall exist. If the
optional box does not exist, then the mandatory boxes within those boxes shall also not exist.
Figure A.1 specifies only the containment relationship between the boxes in the file.
The order of the boxes is not completely arbitrary and has to comply with the following rules:
• The JPEG Pleno Signature box shall be the first box in a JPL file and the File Type box (defined in
ISO/IEC 15444-1) shall immediately follow the JPEG Pleno Signature box.
• The JPEG Pleno Thumbnail box shall be signalled before the JPEG Pleno Light Field, JPEG Pleno
Point Cloud, and JPEG Pleno Hologram superboxes.
• A JPL file can contain an optional XML box signalling catalog information (see A.5.8). An XML box
containing catalog information should be signalled after the File Type box and before the first
superbox containing plenoptic data.
• The JPEG Pleno Light Field, JPEG Pleno Point Cloud, and JPEG Pleno Hologram superboxes
signalling plenoptic data, shall be signalled as one monolithic block with no preferred ordering,
and no other boxes shall be signalled in between.
The file shown in Figure A.1 is a strict sequence of boxes. Other boxes may be found between the boxes
defined in this document, except the JPEG Pleno Light Field, JPEG Pleno Point Cloud, and JPEG Pleno
Hologram superboxes. However, all information contained within a JPL file shall be in the box format;
byte-streams not in the box
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.