ISO/IEC 21794-2:2021
(Main)Information technology — Plenoptic image coding system (JPEG Pleno) — Part 2: Light field coding
Information technology — Plenoptic image coding system (JPEG Pleno) — Part 2: Light field coding
This document specifies a coded codestream format for storage of light field modalities as well as associated metadata descriptors that are light field modality specific. This document also provides information on the encoding tools.
Technologies de l'information — Système de codage d'images plénoptiques (JPEG Pleno) — Partie 2: Codages des champs de lumière
General Information
Relations
Buy Standard
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 21794-2
First edition
2021-04
Information technology — Plenoptic
image coding system (JPEG Pleno) —
Part 2:
Light field coding
Technologies de l'information — Système de codage d'images
plénoptiques (JPEG Pleno) —
Partie 2: Codages des champs de lumière
Reference number
©
ISO/IEC 2021
© ISO/IEC 2021
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2021 – All rights reserved
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Symbols and abbreviated terms . 3
4.1 Symbols . 3
4.2 Abbreviated terms . 7
5 Conventions . 8
5.1 Naming conventions for numerical values . 8
5.2 Operators . 8
5.2.1 Arithmetic operators . 8
5.2.2 Logical operators . 9
5.2.3 Relational operators . 9
5.2.4 Precedence order of operators . 9
5.2.5 Mathematical functions .10
6 General .10
6.1 Functional overview on the decoding process .10
6.2 Encoder requirements .11
6.3 Decoder requirements.11
7 Organization of the document .11
Annex A (normative) JPEG Pleno Light Field superbox .12
Annex B (normative) 4D transform mode .29
Annex C (normative) JPEG Pleno light field reference view decoding .73
Annex D (normative) JPEG Pleno light field normalized disparity view decoding .81
Annex E (normative) JPEG Pleno Light Field Intermediate View superbox .89
Bibiliography .117
© ISO/IEC 2021 – All rights reserved iii
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives or www .iec .ch/ members
_experts/ refdocs).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see patents.iec.ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html. In the IEC, see www .iec .ch/ understanding -standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 21794 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html and www .iec .ch/ national
-committees.
iv © ISO/IEC 2021 – All rights reserved
Introduction
This document is part of a series of standards for a system known as JPEG Pleno. This document defines
the JPEG Pleno framework. It facilitates the capture, representation, exchange and visualization of
plenoptic imaging modalities. A plenoptic image modality can be a light field, point cloud or hologram,
which are sampled representations of the plenoptic function in the form of, respectively, a vector
function that represents the radiance of a discretized set of light rays, a collection of points with
position and attribute information, or a complex wavefront. The plenoptic function describes the
radiance in time and in space obtained by positioning a pinhole camera at every viewpoint in 3D spatial
coordinates, every viewing angle and every wavelength, resulting in a 7D function.
JPEG Pleno specifies tools for coding these modalities while providing advanced functionality at system
level, such as support for data and metadata manipulation, editing, random access and interaction,
protection of privacy and ownership rights.
© ISO/IEC 2021 – All rights reserved v
INTERNATIONAL STANDARD ISO/IEC 21794-2:2021(E)
Information technology — Plenoptic image coding system
(JPEG Pleno) —
Part 2:
Light field coding
1 Scope
This document specifies a coded codestream format for storage of light field modalities as well as
associated metadata descriptors that are light field modality specific. This document also provides
information on the encoding tools.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ITU-T Rec. T.800 | ISO/IEC 15444-1, Information technology — JPEG 2000 image coding system — Part 1:
Core coding system
ITU-T Rec. T.801 | ISO/IEC 15444-2, Information technology — JPEG 2000 image coding system — Part 2:
Extensions
ISO/IEC 21794-1:2020, Information technology — Plenoptic image coding system (JPEG Pleno) — Part 1:
Framework
ISO/IEC 60559, Information technology — Microprocessor Systems — Floating-Point arithmetic
3 Terms and definitions
For the purposes of this document the terms and definitions given in ISO/IEC 21794-1 and the
following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
arithmetic coder
entropy coder that converts variable length strings to variable length codes (encoding) and vice versa
(decoding)
3.2
bit-plane
two-dimensional array of bits
3.3
4D bit-plane
four-dimensional array of bits
© ISO/IEC 2021 – All rights reserved 1
3.4
coefficient
numerical value that is the result of a transformation or linear regression
3.5
compression
reduction in the number of bits used to represent source image data
3.6
depth
distance of a point in 3D space to the camera plane
3.7
disparity view
image that for each pixel of the subaperture view contains the apparent pixel shift between two
subaperture views along either horizontal or vertical axis
3.8
hexadeca-tree
division of a 4D region into 16 (sixteen) 4D subregions
3.9
pixel
collection of sample values in the spatial image domain having all the same sample coordinates
EXAMPLE A pixel may consist of three samples describing its red, green and blue value.
3.10
plenoptic function
amount of radiance in time and in space by positioning a pinhole camera at every viewpoint in 3D
spatial coordinates, every viewing angle and every wavelength, resulting in a 7D representation
3.11
reference view
subaperture view that is used as one of the references to generate the intermediate views
3.12
subaperture view
subaperture image
image taken of the 3D scene by a pinhole camera positioned at a particular viewpoint and viewing angle
3.13
texture
pixel attributes
EXAMPLE Colour information, opacity, etc.
3.14
transform
transformation
mathematical mapping from one signal space to another
2 © ISO/IEC 2021 – All rights reserved
4 Symbols and abbreviated terms
4.1 Symbols
Codestream_Body() coded image data in the codestream without Codestream_Header()
Codestream_Header() codestream header preceding the image data in the codestream
DEC
Dt,,sv,u
()
decoded normalized disparity value at view ts, for pixel location vu,
() ()
Dt,,sv,u
()
normalized disparity value at view ts, for pixel location vu,
() ()
DPEC
k pointer to contiguous codestream for normalized disparity view k
scaling parameter to translate quantized normalized disparity maps to pos-
D
shift
itive range
DCODEC disparity view codec type
f focal length
FPW
p fixed-weight merging parameter for view p
Ht,s
()
view hierarchy value for view ts,
()
HCCt,s
()
horizontal camera centre coordinate for view ts,
()
Ht,s
()
D
binary value defining the availability of a normalized disparity view ts,
()
J
0 Lagrangian encoding cost
J
1 Lagrangian encoding cost of spatial partitioning
J
2 Lagrangian encoding cost of view partitioning
KR
pc, sparse filter regressor mask of texture component c for view p
LightField() JPEG Pleno light field codestream
pc,
quantized least-squares merging weight of texture component c for view p ,
LSW
j
jN=…12,, , LS
p
MIDV absolute value of the minimum value over all quantized normalized disparity views
© ISO/IEC 2021 – All rights reserved 3
----------
...
INTERNATIONAL ISO/IEC
STANDARD 21794-2
First edition
2021-04
Information technology — Plenoptic
image coding system (JPEG Pleno) —
Part 2:
Light field coding
Technologies de l'information — Système de codage d'images
plénoptiques (JPEG Pleno) —
Partie 2: Codages des champs de lumière
Reference number
©
ISO/IEC 2021
© ISO/IEC 2021
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2021 – All rights reserved
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Symbols and abbreviated terms . 3
4.1 Symbols . 3
4.2 Abbreviated terms . 7
5 Conventions . 8
5.1 Naming conventions for numerical values . 8
5.2 Operators . 8
5.2.1 Arithmetic operators . 8
5.2.2 Logical operators . 9
5.2.3 Relational operators . 9
5.2.4 Precedence order of operators . 9
5.2.5 Mathematical functions .10
6 General .10
6.1 Functional overview on the decoding process .10
6.2 Encoder requirements .11
6.3 Decoder requirements.11
7 Organization of the document .11
Annex A (normative) JPEG Pleno Light Field superbox .12
Annex B (normative) 4D transform mode .29
Annex C (normative) JPEG Pleno light field reference view decoding .73
Annex D (normative) JPEG Pleno light field normalized disparity view decoding .81
Annex E (normative) JPEG Pleno Light Field Intermediate View superbox .89
Bibiliography .117
© ISO/IEC 2021 – All rights reserved iii
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives or www .iec .ch/ members
_experts/ refdocs).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see patents.iec.ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/
iso/ foreword .html. In the IEC, see www .iec .ch/ understanding -standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO/IEC 21794 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html and www .iec .ch/ national
-committees.
iv © ISO/IEC 2021 – All rights reserved
Introduction
This document is part of a series of standards for a system known as JPEG Pleno. This document defines
the JPEG Pleno framework. It facilitates the capture, representation, exchange and visualization of
plenoptic imaging modalities. A plenoptic image modality can be a light field, point cloud or hologram,
which are sampled representations of the plenoptic function in the form of, respectively, a vector
function that represents the radiance of a discretized set of light rays, a collection of points with
position and attribute information, or a complex wavefront. The plenoptic function describes the
radiance in time and in space obtained by positioning a pinhole camera at every viewpoint in 3D spatial
coordinates, every viewing angle and every wavelength, resulting in a 7D function.
JPEG Pleno specifies tools for coding these modalities while providing advanced functionality at system
level, such as support for data and metadata manipulation, editing, random access and interaction,
protection of privacy and ownership rights.
© ISO/IEC 2021 – All rights reserved v
INTERNATIONAL STANDARD ISO/IEC 21794-2:2021(E)
Information technology — Plenoptic image coding system
(JPEG Pleno) —
Part 2:
Light field coding
1 Scope
This document specifies a coded codestream format for storage of light field modalities as well as
associated metadata descriptors that are light field modality specific. This document also provides
information on the encoding tools.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ITU-T Rec. T.800 | ISO/IEC 15444-1, Information technology — JPEG 2000 image coding system — Part 1:
Core coding system
ITU-T Rec. T.801 | ISO/IEC 15444-2, Information technology — JPEG 2000 image coding system — Part 2:
Extensions
ISO/IEC 21794-1:2020, Information technology — Plenoptic image coding system (JPEG Pleno) — Part 1:
Framework
ISO/IEC 60559, Information technology — Microprocessor Systems — Floating-Point arithmetic
3 Terms and definitions
For the purposes of this document the terms and definitions given in ISO/IEC 21794-1 and the
following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
arithmetic coder
entropy coder that converts variable length strings to variable length codes (encoding) and vice versa
(decoding)
3.2
bit-plane
two-dimensional array of bits
3.3
4D bit-plane
four-dimensional array of bits
© ISO/IEC 2021 – All rights reserved 1
3.4
coefficient
numerical value that is the result of a transformation or linear regression
3.5
compression
reduction in the number of bits used to represent source image data
3.6
depth
distance of a point in 3D space to the camera plane
3.7
disparity view
image that for each pixel of the subaperture view contains the apparent pixel shift between two
subaperture views along either horizontal or vertical axis
3.8
hexadeca-tree
division of a 4D region into 16 (sixteen) 4D subregions
3.9
pixel
collection of sample values in the spatial image domain having all the same sample coordinates
EXAMPLE A pixel may consist of three samples describing its red, green and blue value.
3.10
plenoptic function
amount of radiance in time and in space by positioning a pinhole camera at every viewpoint in 3D
spatial coordinates, every viewing angle and every wavelength, resulting in a 7D representation
3.11
reference view
subaperture view that is used as one of the references to generate the intermediate views
3.12
subaperture view
subaperture image
image taken of the 3D scene by a pinhole camera positioned at a particular viewpoint and viewing angle
3.13
texture
pixel attributes
EXAMPLE Colour information, opacity, etc.
3.14
transform
transformation
mathematical mapping from one signal space to another
2 © ISO/IEC 2021 – All rights reserved
4 Symbols and abbreviated terms
4.1 Symbols
Codestream_Body() coded image data in the codestream without Codestream_Header()
Codestream_Header() codestream header preceding the image data in the codestream
DEC
Dt,,sv,u
()
decoded normalized disparity value at view ts, for pixel location vu,
() ()
Dt,,sv,u
()
normalized disparity value at view ts, for pixel location vu,
() ()
DPEC
k pointer to contiguous codestream for normalized disparity view k
scaling parameter to translate quantized normalized disparity maps to pos-
D
shift
itive range
DCODEC disparity view codec type
f focal length
FPW
p fixed-weight merging parameter for view p
Ht,s
()
view hierarchy value for view ts,
()
HCCt,s
()
horizontal camera centre coordinate for view ts,
()
Ht,s
()
D
binary value defining the availability of a normalized disparity view ts,
()
J
0 Lagrangian encoding cost
J
1 Lagrangian encoding cost of spatial partitioning
J
2 Lagrangian encoding cost of view partitioning
KR
pc, sparse filter regressor mask of texture component c for view p
LightField() JPEG Pleno light field codestream
pc,
quantized least-squares merging weight of texture component c for view p ,
LSW
j
jN=…12,, , LS
p
MIDV absolute value of the minimum value over all quantized normalized disparity views
© ISO/IEC 2021 – All rights reserved 3
----------
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.