Information technology -- General video coding

Technologies de l'information -- Codage vidéo général

General Information

Status
Published
Publication Date
05-Sep-2022
Current Stage
4060 - Close of voting
Start Date
31-Dec-2021
Completion Date
30-Dec-2021
Ref Project

Buy Standard

Standard
ISO/IEC 23094-3:2022 - Information technology -- General video coding Released:6. 09. 2022
English language
27 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

INTERNATIONAL ISO/IEC
STANDARD 23094-3
First edition
2022-09
Information technology — General
video coding —
Part 3:
Conformance and reference software
for low complexity enhancement
video coding
Technologies de l'information — Codage vidéo général —
Partie 3: Conformité et logiciel de référence pour le codage vidéo
d'amélioration de faible complexité
Reference number
ISO/IEC 23094-3:2022(E)
© ISO/IEC 2022
---------------------- Page: 1 ----------------------
ISO/IEC 23094-3:2022(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2022

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on

the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below

or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 23094-3:2022(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ..................................................................................................................................................................................... 1

3 Terms and definitions .................................................................................................................................................................................... 1

4 Abbreviated terms ............................................................................................................................................................................................. 2

5 Conventions ............................................................................................................................................................................................................... 2

6 Conformance testing for ISO/IEC 23094-2 ................................................................................................................................ 3

6.1 General ........................................................................................................................................................................................................... 3

6.2 Bitstream conformance .................................................................................................................................................................. 3

6.3 Decoder conformance ...................................................................................................................................................................... 3

6.4 Procedure to test bitstreams ..................................................................................................................................................... 3

6.5 Procedure to test decoder conformance ......................................................................................................................... 4

6.5.1 Conformance bitstreams ............................................................................................................................................ 4

6.5.2 Contents of the bitstream file ................................................................................................................................. 4

6.5.3 Operation modes for decoder conformance testing ........................................................................... 4

6.5.4 Requirements on output of the decoding process and timing ................................................... 4

6.5.5 Recommendations............................................................................................................................................................. 5

6.5.6 Static tests for output order conformance .................................................................................................. 5

6.5.7 Dynamic tests for output timing conformance ....................................................................................... 5

6.5.8 Decoder conformance test of a particular profile and level ........................................................ 6

6.6 Specification of the test bitstreams ..................................................................................................................................... 6

6.6.1 General ........................................................................................................................................................................................ 6

6.6.2 Test bitstreams .................................................................................................................................................................... 7

6.7 Normative test suites for ISO/IEC 23094-2................................................................................................................ 18

7 Reference software description .......................................................................................................................................................21

7.1 General ........................................................................................................................................................................................................ 21

7.2 LTM repository ....................................................................................................................................................................................22

7.3 Encoder usage ...................................................................................................................................................................................... 22

7.4 Decoder usage ......................................................................................................................................................................................23

Annex A (informative) Information on accuracy tolerance of decoding implementations....................25

Annex B (informative) Demultiplexing of test bitstreams ........................................................................................................26

Bibliography .............................................................................................................................................................................................................................27

iii
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/IEC 23094-3:2022(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work.

The procedures used to develop this document and those intended for its further maintenance

are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria

needed for the different types of document should be noted. This document was drafted in

accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or

www.iec.ch/members_experts/refdocs).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents) or the IEC

list of patent declarations received (see https://patents.iec.ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to

the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see

www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.

A list of all parts in the ISO/IEC 23094 series can be found on the ISO and IEC websites.

Any feedback or questions on this document should be directed to the user’s national standards

body. A complete listing of these bodies can be found at www.iso.org/members.html and

www.iec.ch/national-committees.
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 4 ----------------------
INTERNATIONAL STANDARD ISO/IEC 23094-3:2022(E)
Information technology — General video coding —
Part 3:
Conformance and reference software for low complexity
enhancement video coding
1 Scope

This document specifies a set of tests and procedures designed to verify whether bitstreams and

decoders meet normative requirements specified in ISO/IEC 23094-2.

An encoder can claim conformance to ISO/IEC 23094-2 if the bitstreams that it generates are conforming

bitstreams. Characteristics of coded bitstreams and decoders are defined in ISO/IEC 23094-2. Decoder

characteristics define the properties and capabilities of the applied decoding process. The capabilities

of a decoder specify which bitstreams the decoder can decode and reconstruct. A bitstream can be

decoded by a decoder if the characteristics of the bitstream are within the specified decoder capabilities.

2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 23094-2:2021, Information technology – General video coding — Part 2: Low complexity

enhancement video coding
ISO/IEC 14882, Programming languages — C++
3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 23094-2 and the following

apply.

ISO and IEC maintain terminology databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1
bitstream

sequence of bits in the form of a NAL unit stream or a raw bitstream, that forms the representation of

coded pictures and associated data forming one or more coded video sequences
3.2
decoder

embodiment of a process that operates on a bitstream (3.1) and may conform to the decoding process

requirements specified for conformance

Note 1 to entry: The decoder does not include the display process, which is outside the scope of this document.

© ISO/IEC 2022 – All rights reserved
---------------------- Page: 5 ----------------------
ISO/IEC 23094-3:2022(E)
3.3
encoder
embodiment of a process that produces a bitstream (3.1)

Note 1 to entry: The process is not specified in this document (except in regard to identification of the reference

software encoder).
3.4
reference software decoder

software which may decode a bitstream (3.1) encoded according to the syntax structure and which is

accompanying this document
3.5
reference software encoder
encoding software which is accompanying this document
3.6
test base bitstream

bitstream (3.1) that is conformant to Rec. ITU-T H.264 | ISO/IEC 14496-10 or Rec. ITU-T H.265 |

ISO/IEC 23008-2
3.7
test bitstream

bitstream (3.1) that is the combination of a test base bitstream (3.6) and a test enhancement bitstream

(3.8)
3.8
test enhancement bitstream
bitstream (3.1) that is conformant to ISO/IEC 23094-2
4 Abbreviated terms

For the purposes of this document, the abbreviated terms given in ISO/IEC 23094-2 and the following

apply.
API Application programming interface
AVC Advanced video coding
EVC Essential video coding
HEVC High efficiency video coding
LTM LCEVC test model
VVC Versatile video coding
5 Conventions

For the purposes of this document, relevant conventions are specified in of ISO/IEC 23094-2:2021,

Clause 5.
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC 23094-3:2022(E)
6 Conformance testing for ISO/IEC 23094-2
6.1 General

The following subclauses specify normative tests for verifying conformance of video bitstreams as well

as decoders. Those normative tests make use of test data (bitstream test suites) provided as an electronic

attachment to this document and the reference software decoder specified in ISO/IEC 23094-2.

For the purpose of testing according to 6.5, test bitstreams are provided in the bitstream test suite. In

the following subclauses, the normative tests use the test bitstreams for verifying the conformance of

the decoding process of the test enhancement bitstreams.

A test base bitstream is conformant in accordance with the respective International Standard it

implements. The conformance testing of the test base bitstream is not in the scope of this document.

This document includes an electronic attachment containing the conformance bitstreams identified

within the text and the reference software. The electronic attachment is available at the following

address: https:// standards .iso .org/ iso -iec/ 23094/ -3/ ed -1/ en/ .
6.2 Bitstream conformance

Bitstream conformance for ISO/IEC 23094-2 is specified by ISO/IEC 23094-2:2021, C.4.

6.3 Decoder conformance

Decoder conformance for ISO/IEC 23094-2 is specified by ISO/IEC 23094-2:2021, C.5.

6.4 Procedure to test bitstreams

A bitstream that claims conformance with ISO/IEC 23094-2 shall pass the following normative test.

The bitstream shall be decoded by processing it with the reference software decoder. When processed by

the reference software decoder, the bitstream shall not cause any error or non-conformance messages

to be reported by the reference software decoder. This test should not be applied to bitstreams that

are known to contain errors introduced by transmission, as such errors are highly likely to result in

bitstreams that lack conformance to ISO/IEC 23094-2.

Successfully passing the reference software decoder test provides only a strong presumption that the

bitstream under test is conforming to the video layer, i.e., that it does indeed meet all the requirements

for the video layer (except Annexes C, D, and E) specified in ISO/IEC 23094-2 that are tested by the

reference software decoder.

Additional tests may be necessary to more thoroughly check that the bitstream properly meets all

the requirements specified in ISO/IEC 23094-2 including the hypothetical reference decoder (HRD)

conformance (based on Annexes C, D, and E). These complementary tests may be performed using other

video bitstream verifiers that perform more complete tests than those implemented by the reference

software decoder.

ISO/IEC 23094-2 contains several informative recommendations that are not an integral part of that

International Standard. When testing a bitstream for conformance, it may also be useful to test whether

or not the bitstream follows those recommendations.

To check correctness of a bitstream, it is necessary to parse the entire bitstream and to extract all the

syntax elements and other values derived from those syntactic elements and used by the decoding

process specified in ISO/IEC 23094-2.

A verifier may not necessarily perform all stages of the decoding process specified in ISO/IEC 23094-2

in order to verify bitstream correctness. Many tests can be performed on syntax elements in a state

prior to their use in some processing stages.
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 23094-3:2022(E)

Annex A of this document provides information on the accuracy tolerance of decoding implementations

if a different precision than the fixed-point precision specified in ISO/IEC 23094-2 is used.

6.5 Procedure to test decoder conformance
6.5.1 Conformance bitstreams

A bitstream has values of profile_idc and level_idc corresponding to a set of specified constraints on

a bitstream for which a decoder conforming to a specified profile and level is required in Annex A of

ISO/IEC 23094-2:2021 to properly perform the decoding process.
6.5.2 Contents of the bitstream file

The conformance bitstreams are included in this document as an electronic attachment. The following

information might be included in a single zipped file for each such bitstream.
— *.bit – bitstream
— *.txt – description
— *.yuv.md5 – MD5 check sum of the complete decoded yuv file
— *.md5 – MD5 check sum of the bitstream file
— *.opl – output picture log

— *.cfg – config file used to generate bitstream with LTM reference software encoder

— *_userdata.bin – Byte file with the content of the user data present within the bitstream

In case a single zipped file contains multiple bitstreams, each bitstream shall be tested separately. The

above listed files are available for each such bitstream within the zipped file.
6.5.3 Operation modes for decoder conformance testing
For decoder conformance testing, there are two operation modes:

1) Operation mode A: The reference software decoder may be used to separate the test enhancement

bitstream from the test bitstream and to generate the decoded base pictures, as specified in

Annex B. A decoder shall use as inputs the decoded base pictures and the test enhancement

bitstream. In this operation mode, the decoder shall be able to decode the test enhancement

bitstream in accordance with ISO/IEC 23094-2:2021, C.1.

2) Operation mode B: A decoder shall use as an input the test bitstream as provided. In this

operation mode, the decoder shall be able to decode the test bitstream in accordance with

ISO/IEC 23094-2:2021, C.1.

A decoder under test may choose either of the two operation modes to verify decoder conformance.

6.5.4 Requirements on output of the decoding process and timing
Two classes of decoder conformance are specified:
— output order conformance; and
— output timing conformance.

The output of the decoding process is specified in ISO/IEC 23094-2:2021, Clause 8 and Annex C.

For output order conformance, it is a requirement that all of the decoded pictures specified for output

in ISO/IEC 23094-2:2021, Annex C shall be output by a conforming decoder in the specified order and

© ISO/IEC 2022 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC 23094-3:2022(E)

that the values of the decoded samples in all of the pictures that are output shall be the values specified

in ISO/IEC 23094-2:2021, Clause 8.

For output timing conformance, it is a requirement that a conforming decoder shall also output the

decoded samples at the rates and times specified in ISO/IEC 23094-2:2021, Annex C.

The display process, which ordinarily follows the output of the decoding process, is outside the scope of

this document.
6.5.5 Recommendations
This clause does not form an integral part of this document.

In addition to the requirements, it is desirable that conforming decoders implement various informative

recommendations specified in ISO/IEC 23094-2 that are not an integral part of that International

Standard. This clause discusses some of these recommendations.

It is recommended that a conforming decoder be able to resume the decoding process as soon as possible

after the loss or corruption of part of a bitstream. In most cases it is possible to resume decoding at the

next IDR NALu. It is recommended that a conforming decoder be able to perform concealment for the

video packets for which all the coded data has not been received.
6.5.6 Static tests for output order conformance

Static tests of a video decoder require testing of the decoded samples. This clause will explain how this

test can be accomplished when the decoded samples at the output of the decoding process are available.

It may not be possible to perform this type of test with a production decoder (due to the lack of an

appropriate accessible interface in the design at which to perform the test). In that case this test should

be performed by the manufacturer during the design and development phase. Static tests are used for

testing the decoding process. The test will check that the values of the samples decoded by the decoder

under test shall be identical to the values of the samples decoded by the reference decoder. When a hash

of the values of the samples of the decoded pictures is attached to the bitstream file, a corresponding

hash operation performed on the values of the samples of the decoded pictures produced by the decoder

under test shall produce the same results.
6.5.7 Dynamic tests for output timing conformance

Dynamic tests are applied to check that all the decoded samples are output and that the timing of

the output of the decoder's decoded samples conforms to the specification of ISO/IEC 23094-2:2021,

Clause 8 and Annex C and to verify that the HRD models (as specified by the CPB and DPB specification

in ISO/IEC 23094-2:2021, Annex C) are not violated when the bits of the bitstream are delivered at the

proper rate.

The dynamic test is often easier to perform on a complete decoding system, which may include a

systems decoder, a video decoder and a display process. It may be possible to record the output of

the display process and to check that display order and timing of decoded pictures are correct at the

output of the display process. However, since the display process is not within the normative scope of

ISO/IEC 23094-2, there may be cases where the output of the display process differs in timing or value

even though the video decoder is conforming. In this case, the output of the video decoder itself (before

the display process) would need to be captured in order to perform the dynamic tests on the video

decoder. In particular the output order and timing of the decoded pictures shall be correct.

If buffering period and picture timing SEI messages are included in the base test bitstream, HRD

conformance shall be verified using the values of initial_cpb_removal_delay, initial_cpb_removal_delay_

offset, cpb_removal_delay and dpb_removal_delay that are included in the base test bitstream.

If buffering period and picture timing SEI messages are not included in the base test bitstream, the

following inferences shall be made to generate the missing parameters:
— fixed_pic_rate_flag shall be inferred to be equal to 1.
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 23094-3:2022(E)
— low_delay_hrd_flag shall be inferred to be equal to 0.
— cbr_flag shall be inferred to be equal to 0.

— The frame rate of the bitstream shall be inferred to be equal to the frame rate value specified in

Table 1, where the bitstream is listed. If this is missing, then a frame rate of either 25 or 30 000 ÷

1 001 can be inferred.

— time_scale shall be set equal to 90 000 and the value of num_units_in_tick shall be computed based

on frame rate.

— The bit rate of the base test bitstream shall be inferred to be equal to the maximum value for the

level specified in the specification that the base test bitstream implements.

— CPB and DPB sizes shall be inferred to be equal to the maximum value for the level as stated in the

specification of the base.
With the above inferences, the HRD shall be operated as follows.

— The CPB is filled starting at time t = 0, until it is full, before removal of the first access unit. This

means that the initial_cpb_removal_delay shall be inferred to be equal to the total CPB buffer size

divided by the bit rate divided by 90 000 (rounded downwards) and initial_cpb_removal_delay_

offset shall be inferred to be equal to zero.

— The first access unit is removed at time t = initial_cpb_removal_delay ÷ 90 000 and subsequent

access units are removed at intervals based on the frame distance, i.e., 2 * (90 000 ÷ num_units_in_

tick) or the field distance, i.e., (90 000 / num_units_in_tick), depending on whether the access unit

is coded as a frame picture or field picture.

— Using these inferences, the CPB will not overflow or underflow and the DPB will not overflow.

6.5.8 Decoder conformance test of a particular profile and level

In order for a decoder of a particular profile and level to claim output order conformance to

ISO/IEC 23094-2 as specified by this document, the decoder shall successfully pass the static test

specified in 6.5.6 with all the bitstreams of the normative test suite specified for testing decoders of

this particular profile and level combination.

In order for a decoder of a particular profile and level to claim output timing conformance to

ISO/IEC 23094-2 as specified by this document, the decoder shall successfully pass both the static test

specified in 6.5.6 and the dynamic test specified in 6.5.7 with all the bitstreams of the normative test

suite specified for testing decoders of this particular profile and level. Table 1 specifies the normative

test suites for each profile and level combination. The test suite for a particular profile and level

combination is the list of bitstreams that are marked with an 'X' in the column corresponding to that

profile and level combination.

'X' indicates that the bitstream is designed to test both the dynamic and static conformance of the

decoder.
The bitstream column specifies the bitstream used for each test.

A decoder that conforms to the profiles at a specific level shall be capable of decoding the specified

bitstreams in Table 1.
6.6 Specification of the test bitstreams
6.6.1 General

Some characteristics of each bitstream listed in Table 1 are specified in this clause.

© ISO/IEC 2022 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC 23094-3:2022(E)
6.6.2 Test bitstreams
6.6.2.1 Test bitstream MP_SET-1
Specification: Streams with sets of coding tools in Main profile.
Functional stage: Test the decoding process of Main profile.

Purpose: Check that the decoder can properly decode bitstreams in which the full set of coding tools in

Main profile is enabled.
6.6.2.2 Test bitstream TT-1

Specification: Streams with 2x2 directional decomposition transform used in Main profile.

Functional stage: Test the decoding process of the 2x2 directional decomposition transform in Main

profile.

Purpose: Check that the decoder can properly decode bitstreams in which the 2x2 directional

decomposition transform is used in Main profile.
6.6.2.3 Test bitstream TT-2

Specification: Streams with 4x4 directional decomposition transform changed to 2x2 directional

decomposition transform changed to 4x4 directional decomposition transform used in Main profile.

Functional stage: Test the decoding process with changes of the kernel size of the directional

decomposition transform within the bitstream in Main profile.

Purpose: Check that the decoder can properly decode bitstreams in which the kernel size of the

directional decomposition transform is changed within the bitstream in Main profile.

6.6.2.4 Test bitstream UPS-1

Specification: Streams with the nearest sample upsampler kernel used in Main profile.

Functional stage: Test the decoding process of the nearest sample upsampler kernel in Main profile.

Purpose: Check that the decoder can properly decode bitstreams in which the nearest sample upsampler

kernel is used in Main profile.
6.6.2.5 Test bitstream UPS-2
Specification: Streams with the bilinear upsampler kernel used in Main profile.

Functional stage: Test the decoding process of the bilinear upsampler kernel in Main profile.

Purpose: Check that the decoder can properly decode bitstreams in which the bilinear upsampler kernel

is used in Main profile.
6.6.2.6 Test bitstream UPS-3
Specification: Streams with the cubic upsampler kernel used in Main profile.

Functional stage: Test the decoding process of the cubic upsampler kernel in Main profile.

Purpose: Check that the decoder can properly decode bitstreams in which the cubic upsampler kernel is

used in Main profile.
© ISO/IEC 2022 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 23094-3:2022(E)
6.6.2.7 Test bitstream UPS-4

Specification: Streams with the modified cubic upsampler kernel used in Main profile.

Functional stage: Test the decoding process of the modified cubic upsampler kernel in Main profile.

Purpose: Check that the decoder can properly decode bitstreams in which the modified cubic upsampler

kernel is used in Main profile.
6.6.2.8 Test bitstream UPS-5
Specificat
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.