ISO/IEC FDIS 23092-1

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 23092-1
ISO/IEC JTC 1/SC 29 Secretariat: JISC
Voting begins on: Voting terminates on:
2020-01-13 2020-04-06
Information technology — Genomic information
representation —
Part 1:
Transport and storage of genomic information
Technologie de l'information — Représentation des informations génomiques —
Partie 1: Transport et stockage des informations génomiques
ICS: 35.040.99
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 23092-1:2020(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC DIS 23092-1:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

Foreword ........................................................................................................................................................................................................................................iv

Introduction ..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Mathematical operators ............................................................................................................................................................................... 4

4.1 Arithmetic operators ......................................................................................................................................................................... 4

4.2 Logical operators .................................................................................................................................................................................. 4

4.3 Relational operators ........................................................................................................................................................................... 4

4.4 Bitwise operators.................................................................................................................................................................................. 5

4.5 Assignment ................................................................................................................................................................................................. 5

4.6 Unary operators ..................................................................................................................................................................................... 5

5 Structure of coded genomic data ......................................................................................................................................................... 5

5.1 Genomic records .................................................................................................................................................................................... 5

5.2 Data classes ................................................................................................................................................................................................ 6

5.3 Access units ................................................................................................................................................................................................ 6

5.4 Datasets ......................................................................................................................................................................................................... 7

5.5 Selective access ....................................................................................................................................................................................... 7

6 Data format ................................................................................................................................................................................................................ 7

6.1 Format structure ................................................................................................................................................................................... 7

6.1.1 General...................................................................................................................................................................................... 7

6.1.2 Box order ................................................................................................................................................................................ 9

6.2 Syntax and semantics .....................................................................................................................................................................10

6.2.1 Method of specifying syntax in tabular form ........................................................................................10

6.2.2 Bit ordering .......................................................................................................................................................................11

6.2.3 Specification of syntax functions ....................................................................................................................11

6.3 Syntax for representation ...........................................................................................................................................................11

6.4 Output data unit ..................................................................................................................................................................................12

6.5 Data structures common to file format and transport format ....................................................................13

6.5.1 File header ..........................................................................................................................................................................13

6.5.2 Dataset group...................................................................................................................................................................13

6.5.3 Dataset ...................................................................................................................................................................................22

6.5.4 Access unit ..........................................................................................................................................................................29

6.5.5 Block ........................................................................................................................................................................................34

6.6 Data structures specific to file format ..............................................................................................................................34

6.6.1 General...................................................................................................................................................................................34

6.6.2 Indexing ................................................................................................................................................................................35

6.6.3 Descriptor stream ........................................................................................................................................................39

6.6.4 Offset .......................................................................................................................................................................................41

6.7 Data structures specific to transport format ..............................................................................................................41

6.7.1 General...................................................................................................................................................................................41

6.7.2 Data streams .....................................................................................................................................................................41

6.7.3 Dataset mapping table list ....................................................................................................................................42

6.7.4 Dataset mapping table .............................................................................................................................................43

6.7.5 Packet .....................................................................................................................................................................................44

6.8 Reference procedure to convert transport format to file format .............................................................45

Annex A (informative) IETF – RFC 3986 specification summary ..........................................................................................48

Annex B (informative) Selective access strategies ..............................................................................................................................49

Annex C (informative) Depacketization process ...................................................................................................................................52

Bibliography .............................................................................................................................................................................................................................54

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that

are members of ISO or IEC participate in the development of International Standards through

technical committees established by the respective organization to deal with particular fields of

technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other

international organizations, governmental and non-governmental, in liaison with ISO and IEC, also

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for

the different types of document should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

Any trade name used in this document is information given for the convenience of users and does not

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.

Any feedback or questions on this document should be directed to the user’s national standards body. A

The advent of high-throughput sequencing (HTS) technologies has the potential to boost the adoption

of genomic information in everyday practice, ranging from biological research to personalized genomic

medicine in clinics. As a consequence, the volume of generated data has increased dramatically during

the last few years, and an even more pronounced growth is expected in the near future.

At the moment, genomic information is mostly exchanged through a variety of data formats, such as

FASTA/FASTQ for unaligned sequencing reads and SAM/BAM/CRAM for aligned reads. With respect to

such formats, the ISO/IEC 23092 series provides a new solution for the representation and compression

— Specifying an abstract representation of the sequencing data rather than a specific format with its

— Being designed at a time point when technologies and use cases are more mature. This permits the

addressing of one limitation of the textual SAM format, for which incremental ad-hoc addition of

features followed along the years, resulting in an overall redundant and suboptimal format which

— Normatively separating free-field user-defined information with no clear semantics from the

normative genomic data representation. This allows a fully interoperable and automatic exchange

— Allowing multiplexing of relevant metadata information with the data since data and metadata are

— Following a strict and supervised development process which has proven successful in the last

30 years in the domain of digital media for the transport format, the file format, the compressed

This document provides the enabling technology that will allow the community to create an ecosystem

of novel, interoperable, solutions in the field of genomic information processing. In particular, it offers:

— Consistent, general and properly designed format definitions and data structures to store sequencing

and alignment information. A robust framework which can be used as a foundation to implement

— Speed and flexibility in the selective access to coded data, by means of newly-designed data

— Low latency in data transmission and consequent fast availability at remote locations, based on

— Built-in privacy and protection of sensitive information, thanks to a flexible framework which

— Reliability of the technology and interoperability among tools and systems, owing to the provision

of a normative procedure to assess conformance to this document on an exhaustive dataset.

— Support to the implementation of a complete ecosystem of compliant devices and applications,

through the availability of a normative reference implementation covering the totality of the

The fundamental structure of the ISO/IEC 23092 series data representation is the genomic record. The

genomic record is a data structure consisting of either a single sequence read, or a paired sequence

read, and its associated sequencing and alignment information; it may contain detailed mapping and

alignment data, a single or paired read identifier (read name) and quality values.

Without breaking traditional approaches, the genomic record introduced in the ISO/IEC 23092 series

provides a more compact, simpler and manageable data structure grouping all the information related

to a single DNA template, from simple sequencing data to sophisticated alignment information.

The genomic record, although it is an appropriate logic data structure for interaction and manipulation of

coded information, is not a suitable atomic data structure for compression. To achieve high compression

ratios, it is necessary to group genomic records into clusters and to transform the information of the

same type into sets of descriptors structured into homogeneous blocks. Furthermore, when dealing

with selective data access, the genomic record is a too small unit to allow effective and fast information

For these reasons, this document introduces the concept of access unit, which is the fundamental

The access unit is the smallest data structure that can be decoded by a decoder compliant with

ISO/IEC 23092-2. An access unit is composed of one block for each descriptor used to represent the

information of its genomic records; therefore, a block payload is the coded representation of all the data

In addition to clusters of genomic records compressed into access units, reads are further classified in

six data classes: five classes are defined according to the result of their alignment against one or more

reference sequences; the sixth class contains either reads that could not be mapped or raw sequencing

data. The classification of sequence reads into classes enables the development of powerful selective

data access. In fact, access units inherit a specific data characterization (e.g. perfect matches in Class

P, substitutions in Class M, indels in Class I, half-mapped reads in Class HM) from the genomic records

composing them, and thus constitute a data structure capable of providing powerful filtering capability

Access units are the fundamental, finest grain data structure in terms of content protection and in

terms of metadata association. In other words, each access unit can be protected individually and

independently. Figure 1 shows how access units, blocks and genomic records relate to each other in the

A dataset is a coded data structure containing headers and one or more access units. Typical datasets

could, for example, contain the complete sequencing of an individual, or a portion of it. Other datasets

could contain, for example, a reference genome or a subset of its chromosomes. Datasets are grouped in

This document defines the syntax and semantics of the data formats for both transport and storage of

genomic information. According to this document, the compressed sequencing data can be multiplexed

into a normative bitstream suitable for packetization for real-time transport over typical network

protocols. In storage use cases, coded data can be encapsulated into a file format with the possibility

to organize blocks per descriptor stream or per access units, to further optimize the selective access

performance to the type of data access required by the different application scenarios. This document

further provides a reference process to convert a normative transport stream into a normative file

The International Organization for Standardization (ISO) and International Electrotechnical

Commission (IEC) draw attention to the fact that it is claimed that compliance with this document may

ISO and IEC take no position concerning the evidence, validity and scope of this patent right. The

holder of this patent right has assured ISO and IEC that he/she is willing to negotiate licences under

reasonable and non-discriminatory terms and conditions with applicants throughout the world. In this

respect, the statement of the holder of this patent right is registered with ISO and IEC. Information may

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights other than those identified above. ISO and IEC shall not be held responsible for identifying

This document specifies data formats for both transport and storage of genomic information, including

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC/FDIS 23092-2, Information technology — Genomic information representation — Part 2: Coding

ISO/IEC/FDIS 23092-3, Information technology — Genomic information representation — Part 3:

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

logical data structure containing a coded representation of genomic information to facilitate bit stream

genomic range comprised between the access unit start position and the access unit end position,

position of the left-most mapped base among the first alignments of all genomic records contained in

position of the right-most mapped base among the first alignments of all genomic records contained in

genomic range comprised between the access unit start position and the right-most genomic record

genomic range comprised between the access unit start position and the access unit end position

information describing the similarity between a sequence (typically a sequencing read) and a reference

sequence of nucleotides that is common to most or all genomic records belonging to a cluster

position of the left-most mapped base among all alignments of all genomic records contained in the

position of the right-most mapped base among all alignments of all genomic records contained in the

integer number representing the zero-based position of a nucleotide within a reference sequence

genomic interval between a start nucleotide position and an end nucleotide position, inclusive

interval of positions on a reference sequence defined by a start position s and an end position e such

that s ≤ e; the start and the end positions of a genomic range are always included in the range

base of the aligned read that either matches the corresponding base on the reference sequence or can

be turned into the corresponding base on the reference sequence via a substitution

transmission unit transporting segments of any of the data structures defined in this document

Note 1 to entry: Genetic material meaning the sequences of the DNA molecules present in a typical cell of that

Note 1 to entry: Each reference sequence is indexed by a one-dimensional integer coordinate system whereby

each integer within range identifies a single nucleotide. Coordinate values can only be equal to or larger than

zero. The coordinate system in the context of this standard is zero-based (i.e. the first nucleotide has coordinate 0

and it is said to be at position 0) and linearly increasing within the string from left to right.

contiguous sequence of nucleotides, typically output of the sequencing process and sequenced from one

readout, by a specific technology more or less prone to errors, of a continuous part of a nucleic acid

Note 1 to entry: A template can be made of one or more segments, being called single-end sequencing read when

parameter either inferred from syntax fields or locally defined in a process description

NOTE The mathematical operators used in this document are similar to those used in the C programming

language. However, integer division with truncation and rounding are specifically defined. The bitwise operators

are defined assuming two's-complement representation of integers. Numbering and counting loops generally