Information technology — JPEG XS low-latency lightweight image coding system — Part 2: Profiles and buffer models

This document defines a number of subsets of the syntax specified in ISO/IEC 21122-1 as profiles. It also defines lower bounds on the throughput in the decoded domain via levels and the encoded domain via sublevels that a conforming decoder implementation shall support. Furthermore, it defines a buffer model to ensure interoperability between implementations in the presence of a latency constraint.

Technologies de l'information — Système de codage d'images léger à faible latence JPEG XS — Partie 2: Profils et modèles tampons

General Information

Status
Published
Publication Date
28-Mar-2022
Current Stage
9092 - International Standard to be revised
Completion Date
16-Jun-2022
Ref Project

RELATIONS

Buy Standard

Standard
ISO/IEC 21122-2:2022 - Information technology — JPEG XS low-latency lightweight image coding system — Part 2: Profiles and buffer models Released:3/29/2022
English language
49 pages
sale 15% off
Preview
sale 15% off
Preview
Draft
ISO/IEC FDIS 21122-2 - Information technology -- JPEG XS low-latency lightweight image coding system
English language
49 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

INTERNATIONAL ISO/IEC
STANDARD 21122-2
Second edition
2022-03
Information technology — JPEG XS
low-latency lightweight image coding
system —
Part 2:
Profiles and buffer models
Technologies de l'information — Système de codage d'images léger à
faible latence JPEG XS —
Partie 2: Profils et modèles tampons
Reference number
ISO/IEC 21122-2:2022(E)
© ISO/IEC 2022
---------------------- Page: 1 ----------------------
ISO/IEC 21122-2:2022(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2022

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on

the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below

or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 21122-2:2022(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction .................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ..................................................................................................................................................................................... 1

3 Terms, definitions, symbols and abbreviated terms .................................................................................................... 1

3.1 Terms and definitions ...................................................................................................................................................................... 1

3.2 Abbreviated terms .............................................................................................................................................................................. 4

3.3 Symbols ......................................................................................................................................................................................................... 4

4 Conventions ............................................................................................................................................................................................................... 6

4.1 Conformance language .................................................................................................................................................................... 6

4.2 Operators ..................................................................................................................................................................................................... 6

4.2.1 Arithmetic operators ...................................................................................................................................................... 6

4.2.2 Logical operators ............................................................................................................................................................... 6

4.2.3 Relational operators ........................................................................................................................................................ 6

4.2.4 Precedence order of operators .............................................................................................................................. 6

4.2.5 Mathematical functions ............................................................................................................................................... 7

5 Buffer model ............................................................................................................................................................................................................. 7

5.1 General system block diagram ................................................................................................................................................. 7

5.2 Influencing variables on the required buffer sizes ................................................................................................ 8

5.3 Role of the buffer model ................................................................................................................................................................. 9

6 Interpretation of Bayer data ................................................................................................................................................................... 9

Annex A (normative) Profiles, levels and sublevels .........................................................................................................................11

Annex B (normative) Packet-based JPEG XS decoder model ...................................................................................................25

Annex C (normative) Packet-based constant bit rate buffer model ................................................................................31

Annex D (informative) Encoder model, latency bounds and codestream conformance

properties for the packet-based constant bit rate buffer model ...................................................................37

Annex E (informative) JPEG XS latency analysis ..................................................................................................................................42

Bibliography .............................................................................................................................................................................................................................49

iii
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/IEC 21122-2:2022(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work.

The procedures used to develop this document and those intended for its further maintenance

are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria

needed for the different types of document should be noted. This document was drafted in

accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or

www.iec.ch/members_experts/refdocs).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents) or the IEC

list of patent declarations received (see patents.iec.ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to

the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see

www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.

This second edition cancels and replaces the first edition (ISO/IEC 21122-2:2019), which has been

technically revised.
The main changes are as follows:

— addition of new profiles to compress colour filter array images (CFA images), to allow mathematically

lossless image compression, and to compress 4:2:0 colour sampled images.

A list of all parts in the ISO/IEC 21122 series can be found on the ISO and IEC websites.

Any feedback or questions on this document should be directed to the user’s national standards

body. A complete listing of these bodies can be found at www.iso.org/members.html and

www.iec.ch/national-committees.
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 21122-2:2022(E)
Introduction

This document is part of a series of standards for a low-latency lightweight image coding system,

denoted as JPEG XS.

While ISO/IEC 21122-1 specifies a full set of compression coding tools needed to satisfy all of the

requirements of JPEG XS, a targeted application can often work with a simpler and reduced set

of coding tools, and with or without tighter constraints, to meet its targeted goals. For this reason,

profiles, levels, and sublevels are defined in this document. These three concepts facilitate partial and

reduced complexity implementations of ISO/IEC 21122-1 for such specific application use cases, while

also safeguarding interoperability.

This document specifies a limited number of profiles to represent interoperability subsets of the

codestream syntax specified in ISO/IEC 21122-1 with each profile serving specific application use

cases. In other word, profiles select a subset of the available coding tools. In addition, levels and

sublevels provide limits to the maximum throughput in respectively the encoded (codestream) and the

decoded (spatial/pixel) domains. In this way, profiles, levels and sublevels allow designing cost-efficient

implementations that serve the needs of the desired applications.

In addition to being light-weight, another major requirement of JPEG XS is to allow low end-to-end

latency, limited to a fraction of the frame size. To ensure this low-latency property, this document

also specifies a buffer model, consisting of a decoder model and a transmission channel model. The

models show the interaction of a hypothetical reference decoder, including its smoothing buffer with a

constant bitrate channel feeding this buffer. The size of the decoder smoothing buffer is computed from

the profile, level, and sublevel. Codestreams are formed such that the buffer of a decoder, operating

according to this buffer model, never overflows or underflows. In effect, the buffer model provides

encoders with the necessary information to generate codestreams that can be decoded by an arbitrary

decoder implementation, ensuring system interoperability.

In addition to the size of the decoder smoothing buffer, end-to-end latency also depends on the latency

inherent to each processing step of the encoding-decoding chain whose methods are described in

ISO/IEC 21122-1. To help implementers estimate the latency of their device, this document gives extra

information on the minimum latency that can be achieved by the different methods described in

ISO/IEC 21122-1.
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 21122-2:2022(E)
Information technology — JPEG XS low-latency lightweight
image coding system —
Part 2:
Profiles and buffer models
1 Scope

This document defines a number of subsets of the syntax specified in ISO/IEC 21122-1 as profiles. It

also defines lower bounds on the throughput in the decoded domain via levels and the encoded domain

via sublevels that a conforming decoder implementation shall support. Furthermore, it defines a buffer

model to ensure interoperability between implementations in the presence of a latency constraint.

2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 21122-1, JPEG XS low-latency lightweight image coding system — Part 1: Core coding system

3 Terms, definitions, symbols and abbreviated terms
3.1 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 21122-1 and the following

apply.

ISO and IEC maintain terminology databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1.1
blanking codestream fragment
placeholder codestream fragment representing blanking periods
3.1.2
buffer model

combination of a decoder model and a channel model whose behaviour can be defined by a set of

parameters
3.1.3
buffer model instance

specific configuration of a buffer model specified by the assignment of well-defined values to the buffer

model parameters
3.1.4
channel model

model describing the temporal behaviour of the transmission channel connecting an encoder and a

decoder
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC 21122-2:2022(E)
3.1.5
coded codestream fragment

continuous sequence of bits in the codestream containing exactly one packet body and a well-defined

number of packet headers, markers and marker segments
3.1.6
codestream fragment
either coded codestream fragment, or blanking codestream fragment
3.1.7
cycle
single clock period of an encoder or decoder clocked implementation
3.1.8
decoder model
combination of a decoder unit and a decoder smoothing buffer
3.1.9
decoder smoothing buffer

memory buffer that is used to level out changes in the number of bits read by a decoder unit per time

unit
3.1.10
decoder unit

module reading a variable number of bits per time unit to generate decoded output pixels with a fixed

rate
3.1.11
decomposition level

set of wavelet coefficients resulting from a particular level of recursive application of a wavelet

transform
3.1.12
encoder model
combination of an encoder unit and an encoder smoothing buffer
3.1.13
encoder smoothing buffer

memory buffer that is used to level out changes in the number of bits generated by an encoder unit per

time unit
3.1.14
encoder unit

module transforming a sequence of input pixels with constant rate into a conforming codestream,

producing a bit sequence with variable number of bits generated per time unit
3.1.15
fill level
number of bits stored in the encoder or decoder smoothing buffer
3.1.16
horizontal blanking period

timespan expressed in units of the grid point sampling rate between the last pixel of an image line ― not

being the last line of an image ― and the first pixel of the next image line
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 21122-2:2022(E)
3.1.17
level

defined set of constraints on the amount of decoded samples to be processed by an encoder or decoder,

both in the spatial and time dimensions

Note 1 to entry: The same set of levels is defined for all profiles. Individual implementations may, within the

specified constraints, support a different level for each supported profile.
3.1.18
nominal bits per pixel value

mean number of bits allocated per encoded pixel which is used to derive the sublevel constraints by

assuming an image with well-defined dimensions and frame rate derived from the level

3.1.19
profile

specified subset of the codestream syntax together with admissible parameter values

3.1.20
sampling grid point

position on the sample grid, specified by integer horizontal and vertical offset relative to the origin of

the sample grid
3.1.21
smoothing buffer unit

level and sublevel dependent number of bits by which the smoothing buffer size of the decoder model is

specified
3.1.22
start of transmission
SoT

time at which the transmission channel starts transmission relative to the start of encoding of the first

codestream fragment of a codestream
3.1.23
sublevel

defined set of constraints on the amount of codestream bits to be processed by an encoder or decoder,

per unit of time, per column, and per image

Note 1 to entry: The same set of sublevels is defined for all profiles. Individual implementations may, within the

specified constraints, support a different sublevel for each supported profile.
3.1.24
transmission channel
facility transferring bits from a source entity to a target entity
3.1.25
transmission channel capacity

maximum number of bits per time unit that a transmission channel can transfer from a source entity to

a target entity
3.1.26
vertical blanking period

timespan in units of the grid point sampling rate between the last line of an image ― including the

horizontal blanking periods ― and the first line of the next image
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC 21122-2:2022(E)
3.2 Abbreviated terms
bpp bits per pixel
CFA colour filter array
DWT discrete wavelet transform
IDWT inverse discrete wavelet transform
RCT reversible colour transform
IRCT inverse reversible colour transform
3.3 Symbols
C(i) codestream i

D number of clock cycles between the first bit written into the decoding smoothing buffer and

c2d

the decoding start of the first codestream fragment of a stream of codestream fragments

F (C(i)) first codestream fragment of codestream C(i)
first
F (C(i)) last codestream fragment of codestream C(i)
last
H height of the image in sampling grid points
H maximum picture height in sampling grid points
max
L maximum number of sampling grid points per image
max
l (t) fill level of the encoding smoothing buffer in bits at the end of cycle t
enc
l (t) fill level of the decoding smoothing buffer in bits at the end of cycle t
dec
l capacity in bits of the encoding smoothing buffer
enc,max
l capacity in bits of the decoding smoothing buffer
dec,max

Ĩ (t) number of bits that can be read from the decoding smoothing buffer in cycle t

dec
l (t) sum of encoder and decoder smoothing buffer fill level in bits at cycle t
sum
all integer numbers being strictly larger than zero
 all integer numbers being greater than or equal to zero
N size of the horizontal blanking line in sampling grid point clock periods
b,x
N size of the vertical blanking period in sampling grid lines
b,y
N nominal number of bits allocated per pixel for compression
bpp
N number of components in an image
N ( f ) number of coefficient groups within codestream fragment f

N number of coefficient groups associated to a codestream fragment representing a hori-

cg,hz
zontal blanking period
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 21122-2:2022(E)

N number of coefficient groups associated to a codestream fragment representing a vertical

cg,vt
blanking period
N (i) number of codestream fragments within a codestream i
N number of coefficients in a code group
N number of horizontal decomposition levels
L,x
N number of vertical decomposition levels
L,y
N number of precincts per sampling grid line
p,x
N number of precincts per sampling grid column
p,y
N number of decoder smoothing buffer units for a given profile
sbu
ℚ set of rational numbers

r (t) number of bits read and removed from the decoder smoothing buffer in clock cycle t

dec

R transmission channel capacity, expressed in bits per cycle (having a duration of T)

trans

R (l , l ) maximum admissible encoded throughput in bits per second for a given level

t,max m s
R maximum grid point sample rate (in samples per second) at decoder output
s,max
S ( f ) number of bits forming the codestream fragment f
bits
S targeted maximum number of bytes of an encoded codestream
c,max
S (l , l ) size of the smoothing buffer unit in bytes for level l and sublevel l
sbu m s m s
S (p) smoothing buffer offset in bits for a profile p
sbo

S (l , l ) maximum size of an encoded codestream in bytes of level l and sublevel l

sl,max m s m s
s [i] sampling factor of component i in horizontal direction
s [i] sampling factor of component i in vertical direction

T clock period defining the frequency by which code groups are processed by an encoder

enc

T clock period defining the frequency by which code groups are processed by a decoder

dec

t ( f ) timestamp in cycles at which the codestream fragment f is written to the encoder smooth-

enc,write
ing buffer

t ( f ) timestamp in cycles at which decoder starts decoding codestream fragment f

dec,start

t ( f ) timestamp in cycles at which codestream fragment f is removed from the decoder smooth-

dec,read
ing buffer
Tbmd buffer model type
W [i] width of component i in samples
W maximum column width in sampling grid points for a given profile
c,max
w (t) number of bits written into the decoder smoothing buffer in clock cycle t
dec
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC 21122-2:2022(E)
W width of the image in sampling grid points
W maximum picture width in sampling grid points
max
set of all integer numbers
4 Conventions
4.1 Conformance language

The keyword "reserved" indicates a provision that is not specified at this time, shall not be used, and

may be specified in the future. The keyword "forbidden" indicates "reserved" and in addition indicates

that the provision will never be specified in the future.
4.2 Operators

NOTE Many of the operators used in document are similar to those used in the C programming language.

4.2.1 Arithmetic operators
+ addition
− subtraction (as a binary operator) or negation (as a unary prefix operator)
× multiplication
/ division without truncation or rounding
4.2.2 Logical operators
|| logical OR
&& logical AND
! logical NOT
4.2.3 Relational operators
> greater than
≥ greater than or equal to
< less than
≤ less than or equal to
== equal to
!= not equal to
4.2.4 Precedence order of operators

Operators are listed in descending order of precedence. If several operators appear in the same line,

they have equal precedence. When several operators of equal precedence appear at the same level in an

expression, evaluation proceeds according to the associativity of the operator either from right to left

or from left to right.
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 21122-2:2022(E)
Operators Type of operation Associativity
() expression left to right
[] indexing of arrays left to right
– unary negation
×, / multiplication, division left to right
+, − addition and subtraction left to right
< , >, ≤, ≥ relational left to right
& bitwise AND left to right
| bitwise OR left to right
4.2.5 Mathematical functions
  ceil of x: returns the smallest integer that is greater than or equal to x
 
floor of x: returns the largest integer that is less than or equal to x
 
|x| absolute value of x, |x| equals –x for x < 0, otherwise x
sign(x) sign of x, 0 if x is 0, +1 if x is positive, −1 if x is negative
10t ≥
ξ t
() step function ξ t =
0otherwise
max (x ) maximum of a sequence of numbers [x ] enumerated by the index i
i i i
5 Buffer model
5.1 General system block diagram

The JPEG XS coding system addresses applications where coded images are transferred from a source

to a target, as shown in Figure 1. To this end, the encoder is compressing a continuous stream of input

pixels into a sequence of bits. These bits are forwarded by means of a transmission channel to the

decoder that decompresses the bits to produce a continuous stream of output pixels.

© ISO/IEC 2022 – All rights reserved
---------------------- Page: 12 ----------------------
ISO/IEC 21122-2:2022(E)
Key
1 encoder clock 6 decoder smoothing buffer
2 pixel data 7 decoder unit
3 encoder unit 8 decoder clock
4 encoder smoothing buffer 9 variable bit rate
5 transmission channel
Figure 1 — General system block diagram

The time instances at which the encoder processes each pixel are determined by an encoding clock.

Similarly, the time instances at which the decoder produces each output pixel are determined by a

decoding clock. Both clocks are generated by the system.

NOTE In implementations, these clocks can be the same or differ in both frequency and phase. The presented

model is independent of whether clocks are synchronized or not.

In accordance with ISO/IEC 21122-1, the pixels of an image are translated into coefficient groups

represented as code groups in the codestream. The number of bits necessary to code these code groups

may vary from group to group. As a consequence, the encoder writes encoded bits at a variable rate into

the encoder smoothing buffer. Similarly, the decoder reads the codestream at a variable rate from the

decoder smoothing buffer.

In case the maximum bit rate of the transmission channel is below the peak bit rate generated by the

encoder, an encoder smoothing buffer is necessary to decouple generation of bits by the encoder from

transmission of bits over the transmission channel. Similarly, a decoder smoothing buffer needs to be

provided that decouples the arrival of bits at the rate afforded by the transmission channel and the

consumption of bits by the decoder per clock.

Correct operation requires that the decoder buffer never overflows. This is because the decoder is

not able to pause the arrival of bits from the transmission channel. Moreover, a buffer underflow in

the decoder buffer needs to be avoided. This is because the decoder is required to output pixels in

accordance with the timing of its output interface. Hence it needs to be ensured that the bits to be read

from the decoding buffer to produce the next pixel in accordance with the decoding clock are available

in this decoding buffer.
5.2 Influencing variables on the required buffer sizes

Avoiding any buffer overflow or underflow, as discussed in subclause 5.1, requires sizing the decoder

smoothing buffer properly. Moreover, the time at which decoding starts is delayed relative to the

starting time of encoding and the start of transmission needs to be carefully set. Those values are

influenced by many system parameters, for example:
— The maximum transmission channel bit rate.

— The granularity at which the encoder writes the encoded data and the decoder reads the encoded

data.
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 13 ----------------------
ISO/IEC 21122-2:2022(E)
— The rate control strategy applied by the encoder.

These dependencies cause that encoders and decoders are only interoperable in well-defined conditions.

Defining these conditions is the purpose of the buffer model defined in Annex B and Annex C.

5.3 Role of the buffer model

The core coding system defined in ISO/IEC 21122-1 can be implemented on a large variety of platforms

using many different implementation strategies. Thus, interoperability cannot be achieved by precisely

specifying the temporal behaviour of a conforming decoding implementation. Instead, the buffer model

defines a simplified decoder model. Interoperability is then achieved by mandating that a conforming

decoder shall decode all bit streams being decodable by the simplified decoder model. Similarly, a

conforming encoder shall not create bit streams that cannot be decoded by the simplified decoder

model.

To this end, Annex B defines a generic JPEG XS decoder model that precisely defines the temporal

behaviour of the decoder model assuming a processing granularity of codestream packets. While such

a model already defines some fundamental properties of the decodable codestreams, it is still not

sufficient to ensure interoperability. The reason is that otherwise codestreams could be constructed

that would only be decodable by the decoder model if the transmission channel could transport bits

arbitrarily fast. In practice, this is obviously not the case. Consequently, interoperability also requires

defining a channel model over which an encoder sends the codestreams to the decoder.

Annex C defines such a channel model assuming a transmission channel with a fixed upper bit

rate that is related to the target compression ratio. Together with the decoder model of Annex B, it

defines the packet-based constant bit rate buffer model. It describes the conditions for a low latency

interoperability between any conforming encoder and any decoder. These conditions are expressed by

buffer model parameters that are specified by the profiles and levels defined in Annex A. The properties

of such conforming implementations are exemplified in Annex D. Since these properties are direct

consequences of Annex B and Annex C, Annex D is informative only.
6 Interpretation of Bayer data

ISO/IEC 21122-1 defines coding tools and signalling for compression of Bayer-type CFA image data.

According to this specificat
...

FINAL
INTERNATIONAL ISO/IEC
DRAFT
STANDARD FDIS
21122-2
ISO/IEC JTC 1/SC 29
Information technology — JPEG XS
Secretariat: JISC
low-latency lightweight image coding
Voting begins on:
2021-10-13 system —
Voting terminates on:
Part 2:
2021-12-08
Profiles and buffer models
RECIPIENTS OF THIS DRAFT ARE INVITED TO
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
THEY ARE AWARE AND TO PROVIDE SUPPOR TING
DOCUMENTATION.
IN ADDITION TO THEIR EVALUATION AS
Reference number
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO-
ISO/IEC FDIS 21122-2:2021(E)
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN-
DARDS TO WHICH REFERENCE MAY BE MADE IN
NATIONAL REGULATIONS. © ISO/IEC 2021
---------------------- Page: 1 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2021

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on

the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below

or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction .................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ..................................................................................................................................................................................... 1

3 Terms, definitions, symbols and abbreviated terms .................................................................................................... 1

3.1 Terms and definitions ...................................................................................................................................................................... 1

3.2 Abbreviated terms .............................................................................................................................................................................. 4

3.3 Symbols ......................................................................................................................................................................................................... 4

4 Conventions ............................................................................................................................................................................................................... 6

4.1 Conformance language .................................................................................................................................................................... 6

4.2 Operators ..................................................................................................................................................................................................... 6

4.2.1 Arithmetic operators ...................................................................................................................................................... 6

4.2.2 Logical operators ............................................................................................................................................................... 6

4.2.3 Relational operators ........................................................................................................................................................ 6

4.2.4 Precedence order of operators .............................................................................................................................. 6

4.2.5 Mathematical functions ............................................................................................................................................... 7

5 Buffer model ............................................................................................................................................................................................................. 7

5.1 General system block diagram ................................................................................................................................................. 7

5.2 Influencing variables on the required buffer sizes ................................................................................................ 8

5.3 Role of the buffer model ................................................................................................................................................................. 9

6 Interpretation of Bayer data ................................................................................................................................................................... 9

Annex A (normative) Profiles, levels and sublevels .........................................................................................................................11

Annex B (normative) Packet-based JPEG XS decoder model ...................................................................................................25

Annex C (normative) Packet-based constant bit rate buffer model ................................................................................31

Annex D (informative) Encoder model, latency bounds and codestream conformance

properties for the packet-based constant bit rate buffer model ...................................................................37

Annex E (informative) JPEG XS latency analysis ..................................................................................................................................42

Bibliography .............................................................................................................................................................................................................................49

iii
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work.

The procedures used to develop this document and those intended for its further maintenance

are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria

needed for the different types of document should be noted. This document was drafted in

accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or

www.iec.ch/members_experts/refdocs).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents) or the IEC

list of patent declarations received (see patents.iec.ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to

the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see

www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.

This second edition cancels and replaces the first edition (ISO/IEC 21122-2:2019), which has been

technically revised.
The main changes are as follows:

— addition of new profiles to compress colour filter array images (CFA images), to allow mathematically

lossless image compression, and to compress 4:2:0 colour sampled images.

A list of all parts in the ISO/IEC 21122 series can be found on the ISO and IEC websites.

Any feedback or questions on this document should be directed to the user’s national standards

body. A complete listing of these bodies can be found at www.iso.org/members.html and

www.iec.ch/national-committees.
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
Introduction

This document is part of a series of standards for a low-latency lightweight image coding system,

denoted as JPEG XS.

While ISO/IEC 21122-1 specifies a full set of compression coding tools needed to satisfy all of the

requirements of JPEG XS, a targeted application can often work with a simpler and reduced set

of coding tools, and with or without tighter constraints, to meet its targeted goals. For this reason,

profiles, levels, and sublevels are defined in this document. These three concepts facilitate partial and

reduced complexity implementations of ISO/IEC 21122-1 for such specific application use cases, while

also safeguarding interoperability.

This document specifies a limited number of profiles to represent interoperability subsets of the

codestream syntax specified in ISO/IEC 21122-1 with each profile serving specific application use

cases. In other word, profiles select a subset of the available coding tools. In addition, levels and

sublevels provide limits to the maximum throughput in respectively the encoded (codestream) and the

decoded (spatial/pixel) domains. In this way, profiles, levels and sublevels allow designing cost-efficient

implementations that serve the needs of the desired applications.

In addition to being light-weight, another major requirement of JPEG XS is to allow low end-to-end

latency, limited to a fraction of the frame size. To ensure this low-latency property, this document

also specifies a buffer model, consisting of a decoder model and a transmission channel model. The

models show the interaction of a hypothetical reference decoder, including its smoothing buffer with a

constant bitrate channel feeding this buffer. The size of the decoder smoothing buffer is computed from

the profile, level, and sublevel. Codestreams are formed such that the buffer of a decoder, operating

according to this buffer model, never overflows or underflows. In effect, the buffer model provides

encoders with the necessary information to generate codestreams that can be decoded by an arbitrary

decoder implementation, ensuring system interoperability.

In addition to the size of the decoder smoothing buffer, end-to-end latency also depends on the latency

inherent to each processing step of the encoding-decoding chain whose methods are described in ISO/

IEC 21122-1. To help implementers estimate the latency of their device, this document gives extra

information on the minimum latency that can be achieved by the different methods described in ISO/

IEC 21122-1.
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 5 ----------------------
FINAL DRAFT INTERNATIONAL STANDARD ISO/IEC FDIS 21122-2:2021(E)
Information technology — JPEG XS low-latency lightweight
image coding system —
Part 2:
Profiles and buffer models
1 Scope

This document defines a number of subsets of the syntax specified in ISO/IEC 21122-1 as profiles. It

also defines lower bounds on the throughput in the decoded domain via levels and the encoded domain

via sublevels that a conforming decoder implementation shall support. Furthermore, it defines a buffer

model to ensure interoperability between implementations in the presence of a latency constraint.

2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 21122-1, JPEG XS low-latency lightweight image coding system — Part 1: Core coding system

3 Terms, definitions, symbols and abbreviated terms
3.1 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 21122-1 and the following

apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1.1
blanking codestream fragment
placeholder codestream fragment representing blanking periods
3.1.2
buffer model

combination of a decoder model and a channel model whose behaviour can be defined by a set of

parameters
3.1.3
buffer model instance

specific configuration of a buffer model specified by the assignment of well-defined values to the buffer

model parameters
3.1.4
channel model

model describing the temporal behaviour of the transmission channel connecting an encoder and a

decoder
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
3.1.5
coded codestream fragment

continuous sequence of bits in the codestream containing exactly one packet body and a well-defined

number of packet headers, markers and marker segments
3.1.6
codestream fragment
either coded codestream fragment, or blanking codestream fragment
3.1.7
cycle
single clock period of an encoder or decoder clocked implementation
3.1.8
decoder model
combination of a decoder unit and a decoder smoothing buffer
3.1.9
decoder smoothing buffer

memory buffer that is used to level out changes in the number of bits read by a decoder unit per time

unit
3.1.10
decoder unit

module reading a variable number of bits per time unit to generate decoded output pixels with a fixed

rate
3.1.11
decomposition level

set of wavelet coefficients resulting from a particular level of recursive application of a wavelet

transform
3.1.12
encoder model
combination of an encoder unit and an encoder smoothing buffer
3.1.13
encoder smoothing buffer

memory buffer that is used to level out changes in the number of bits generated by an encoder unit per

time unit
3.1.14
encoder unit

module transforming a sequence of input pixels with constant rate into a conforming codestream,

producing a bit sequence with variable number of bits generated per time unit
3.1.15
fill level
number of bits stored in the encoder or decoder smoothing buffer
3.1.16
horizontal blanking period

timespan expressed in units of the grid point sampling rate between the last pixel of an image line ― not

being the last line of an image ― and the first pixel of the next image line
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
3.1.17
level

defined set of constraints on the amount of decoded samples to be processed by an encoder or decoder,

both in the spatial and time dimensions

Note 1 to entry: The same set of levels is defined for all profiles. Individual implementations may, within the

specified constraints, support a different level for each supported profile.
3.1.18
nominal bits per pixel value

mean number of bits allocated per encoded pixel which is used to derive the sublevel constraints by

assuming an image with well-defined dimensions and frame rate derived from the level

3.1.19
profile

specified subset of the codestream syntax together with admissible parameter values

3.1.20
sampling grid point

position on the sample grid, specified by integer horizontal and vertical offset relative to the origin of

the sample grid
3.1.21
smoothing buffer unit

level and sublevel dependent number of bits by which the smoothing buffer size of the decoder model is

specified
3.1.22
start of transmission
SoT

time at which the transmission channel starts transmission relative to the start of encoding of the first

codestream fragment of a codestream
3.1.23
sublevel

defined set of constraints on the amount of codestream bits to be processed by an encoder or decoder,

per unit of time, per column, and per image

Note 1 to entry: The same set of sublevels is defined for all profiles. Individual implementations may, within the

specified constraints, support a different sublevel for each supported profile.
3.1.24
transmission channel
facility transferring bits from a source entity to a target entity
3.1.25
transmission channel capacity

maximum number of bits per time unit that a transmission channel can transfer from a source entity to

a target entity
3.1.26
vertical blanking period

timespan in units of the grid point sampling rate between the last line of an image ― including the

horizontal blanking periods ― and the first line of the next image
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
3.2 Abbreviated terms
bpp bits per pixel
CFA colour filter array
DWT discrete wavelet transform
IDWT inverse discrete wavelet transform
RCT reversible colour transform
IRCT inverse reversible colour transform
3.3 Symbols
C(i) codestream i

D number of clock cycles between the first bit written into the decoding smoothing buffer and

c2d

the decoding start of the first codestream fragment of a stream of codestream fragments

F (C(i)) first codestream fragment of codestream C(i)
first
F (C(i)) last codestream fragment of codestream C(i)
last
H height of the image in sampling grid points
H maximum picture height in sampling grid points
max
L maximum number of sampling grid points per image
max
l (t) fill level of the encoding smoothing buffer in bits at the end of cycle t
enc
l (t) fill level of the decoding smoothing buffer in bits at the end of cycle t
dec
l capacity in bits of the encoding smoothing buffer
enc,max
l capacity in bits of the decoding smoothing buffer
dec,max

Ĩ (t) number of bits that can be read from the decoding smoothing buffer in cycle t

dec
l (t) sum of encoder and decoder smoothing buffer fill level in bits at cycle t
sum
all integer numbers being strictly larger than zero
 all integer numbers being greater than or equal to zero
N size of the horizontal blanking line in sampling grid point clock periods
b,x
N size of the vertical blanking period in sampling grid lines
b,y
N nominal number of bits allocated per pixel for compression
bpp
N number of components in an image
N (f) number of coefficient groups within codestream fragment f

N number of coefficient groups associated to a codestream fragment representing a hori-

cg,hz
zontal blanking period
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC FDIS 21122-2:2021(E)

N number of coefficient groups associated to a codestream fragment representing a vertical

cg,vt
blanking period
N (i) number of codestream fragments within a codestream i
N number of coefficients in a code group
N number of vertical decomposition levels
L,y
N number of precincts per sampling grid line
p,x
N number of precincts per sampling grid column
p,y
N number of decoder smoothing buffer units for a given profile
sbu
ℚ set of rational numbers

r (t) number of bits read and removed from the decoder smoothing buffer in clock cycle t

dec

R transmission channel capacity, expressed in bits per cycle (having a duration of T)

trans

R (l , l ) maximum admissible encoded throughput in bits per second for a given level

t,max m s
R maximum grid point sample rate (in samples per second) at decoder output
s,max
S ( f ) number of bits forming the codestream fragment f
bits
S targeted maximum number of bytes of an encoded codestream
c,max
S (l , l ) size of the smoothing buffer unit in bytes for level l and sublevel l
sbu m s m s
S (p) smoothing buffer offset in bits for a profile p
sbo

S (l , l ) maximum size of an encoded codestream in bytes of level l and sublevel l

sl,max m s m s
s [i] sampling factor of component i in horizontal direction
s [i] sampling factor of component i in vertical direction

T clock period defining the frequency by which code groups are processed by an encoder

enc

T clock period defining the frequency by which code groups are processed by a decoder

dec

t (f) timestamp in cycles at which the codestream fragment f is written to the encoder smooth-

enc,write
ing buffer
t (f) timestamp in cycles at which decoder starts decoding codestream fragment f
dec,start

t (f) timestamp in cycles at which codestream fragment f is removed from the decoder smooth-

dec,read
ing buffer
Tbmd buffer model type
W [i] width of component i in samples
W maximum column width in sampling grid points for a given profile
c,max
w (t) number of bits written into the decoder smoothing buffer in clock cycle t
dec
W width of the image in sampling grid points
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
W maximum picture width in sampling grid points
max
set of all integer numbers
4 Conventions
4.1 Conformance language

The keyword "reserved" indicates a provision that is not specified at this time, shall not be used, and

may be specified in the future. The keyword "forbidden" indicates "reserved" and in addition indicates

that the provision will never be specified in the future.
4.2 Operators

NOTE Many of the operators used in document are similar to those used in the C programming language.

4.2.1 Arithmetic operators
+ addition
− subtraction (as a binary operator) or negation (as a unary prefix operator)
× multiplication
/ division without truncation or rounding
4.2.2 Logical operators
|| logical OR
&& logical AND
! logical NOT
4.2.3 Relational operators
> greater than
≥ greater than or equal to
< less than
≤ less than or equal to
== equal to
!= not equal to
4.2.4 Precedence order of operators

Operators are listed in descending order of precedence. If several operators appear in the same line,

they have equal precedence. When several operators of equal precedence appear at the same level in an

expression, evaluation proceeds according to the associativity of the operator either from right to left

or from left to right.
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
Operators Type of operation Associativity
() expression left to right
[] indexing of arrays left to right
– unary negation
×, / multiplication, division left to right
+, − addition and subtraction left to right
< , >, ≤, ≥ relational left to right
& bitwise AND left to right
| bitwise OR left to right
4.2.5 Mathematical functions
  ceil of x: returns the smallest integer that is greater than or equal to x
 
floor of x: returns the largest integer that is less than or equal to x
 
|x| absolute value of x, |x| equals –x for x < 0, otherwise x
sign(x) sign of x, 0 if x is 0, +1 if x is positive, −1 if x is negative
10t ≥
ξ t
() step function ξ t =
0otherwise
max (x ) maximum of a sequence of numbers [x ] enumerated by the index i
i i i
5 Buffer model
5.1 General system block diagram

The JPEG XS coding system addresses applications where coded images are transferred from a source

to a target, as shown in Figure 1. To this end, the encoder is compressing a continuous stream of input

pixels into a sequence of bits. These bits are forwarded by means of a transmission channel to the

decoder that decompresses the bits to produce a continuous stream of output pixels.

© ISO/IEC 2021 – All rights reserved
---------------------- Page: 12 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
Key
1 encoder clock 6 decoder smoothing buffer
2 pixel data 7 decoder unit
3 encoder unit 8 decoder clock
4 encoder smoothing buffer 9 variable bit rate
5 transmission channel
Figure 1 — General system block diagram

The time instances at which the encoder processes each pixel are determined by an encoding clock.

Similarly, the time instances at which the decoder produces each output pixel are determined by a

decoding clock. Both clocks are generated by the system.

NOTE In implementations, these clocks can be the same or differ in both frequency and phase. The presented

model is independent of whether clocks are synchronized or not.

In accordance with ISO/IEC 21122-1, the pixels of an image are translated into coefficient groups

represented as code groups in the codestream. The number of bits necessary to code these code groups

may vary from group to group. As a consequence, the encoder writes encoded bits at a variable rate into

the encoder smoothing buffer. Similarly, the decoder reads the codestream at a variable rate from the

decoder smoothing buffer.

In case the maximum bit rate of the transmission channel is below the peak bit rate generated by the

encoder, an encoder smoothing buffer is necessary to decouple generation of bits by the encoder from

transmission of bits over the transmission channel. Similarly, a decoder smoothing buffer needs to be

provided that decouples the arrival of bits at the rate afforded by the transmission channel and the

consumption of bits by the decoder per clock.

Correct operation requires that the decoder buffer never overflows. This is because the decoder is

not able to pause the arrival of bits from the transmission channel. Moreover, a buffer underflow in

the decoder buffer needs to be avoided. This is because the decoder is required to output pixels in

accordance with the timing of its output interface. Hence it needs to be ensured that the bits to be read

from the decoding buffer to produce the next pixel in accordance with the decoding clock are available

in this decoding buffer.
5.2 Influencing variables on the required buffer sizes

Avoiding any buffer overflow or underflow, as discussed in subclause 5.1, requires sizing the decoder

smoothing buffer properly. Moreover, the time at which decoding starts is delayed relative to the

starting time of encoding and the start of transmission needs to be carefully set. Those values are

influenced by many system parameters, for example:
— The maximum transmission channel bit rate.

— The granularity at which the encoder writes the encoded data and the decoder reads the encoded

data.
© ISO/IEC 2021 – All rights reserved
---------------------- Page: 13 ----------------------
ISO/IEC FDIS 21122-2:2021(E)
— The rate control strategy applied by the encoder.

These dependencies cause that encoders and decoders are only interoperable in well-defined conditions.

Defining these conditions is the purpose of the buffer model defined in Annex B and Annex C.

5.3 Role of the buffer model

The core coding system defined in ISO/IEC 21122-1 can be implemented on a large variety of platforms

using many different implementation strategies. Thus, interoperability cannot be achieved by precisely

specifying the temporal behaviour of a conforming decoding implementation. Instead, the buffer model

defines a simplified decoder model. Interoperability is then achieved by mandating that a conforming

decoder shall decode all bit streams being decodable by the simplified decoder model. Similarly, a

conforming encoder shall not create bit streams that cannot be decoded by the simplified decoder

model.

To this end, Annex B defines a generic JPEG XS decoder model that precisely defines the temporal

behaviour of the decoder model assuming a processing granularity of codestream packets. While such

a model already defines some fundamental properties of the decodable codestreams, it is still not

sufficient to ensure interoperability. The reason is that otherwise codestreams could be constructed

that would only be decodable by the decoder model if the transmission channel could transport bits

arbitrarily fast. In practice, this is obviously not the case. Consequently, interoperability also requires

defining a channel model over which an encoder sends the codestreams to the decoder.

Annex C defines such a channel model assuming a transmission channel with a fixed upper bit

rate that is related to the target compression ratio. Together with the decoder model of Annex B, it

defines the packet-based constant bit rate buffer model. It describes the conditions for a low latency

interoperability between any conforming encoder and
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.