ISO/IEC 23001-14:2019

Information technology - MPEG systems technologies - Part 14: Partial file format

This document specifies the partial file format, which is a generic format for describing file partially received over lossy communication channels. This format contains the correctly received data, missing block identification, and repair information such as location of the file or high-level original indexing information. This format can be used with any file formats, and provides additional helper tools for formats deriving from ISO/IEC 14496-12.

Technologies de l'information — Technologies des systèmes MPEG — Partie 14: Format de fichier partiel

General Information

• Status: Published
• Publication Date: 09-Jan-2019
• ICS: 35.040.40 - Coding of audio, video, multimedia and hypermedia information
• Technical Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Drafting Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Current Stage: 9060 - Close of review
• Completion Date: 02-Sep-2029

Relations

• Consolidated By: ISO 18562-1:2024 - Biocompatibility evaluation of breathing gas pathways in healthcare applications - Part 1: Evaluation and testing within a risk management process
• Effective Date: 06-Jun-2022
• Amended By: ISO/IEC 23001-14:2019/Amd 1:2021 - Information technology - MPEG systems technologies - Part 14: Partial file format - Amendment 1: Support for HTTP entities, enhanced file type and byte-range priorities
• Effective Date: 14-Aug-2021

Overview

ISO/IEC 23001-14:2019, "Information technology - MPEG systems technologies - Part 14: Partial file format," defines a generic, object‑oriented file format for describing files that have been partially received over lossy communication channels. The standard specifies how to encapsulate correctly received data, identify missing or corrupted blocks, and store repair metadata (for example, source locations or high‑level indexing information). It is designed to work with any file format and includes helper tools specifically for files derived from the ISO Base Media File Format (ISO/IEC 14496‑12).

Key topics and requirements

• Partial file structure: Files are composed of a header followed by one or more partial segments, all structured as ISO/IEC 14496‑12 style “boxes”.
• FileTypeBox branding: Partial files use compatible brands such as 'paff' (partial files) and 'pmff' (mixed partial files).
• Segment contents: Each partial segment contains received bytes, a byte‑map of valid ranges (relative to the segment), and optional repair information or source references.
• Box taxonomy: The standard defines mandatory and optional top‑level and segment‑level boxes (examples include PartialFileBox, PartialSegmentBox, PartialSegmentHeaderBox, PartialDataBox, PartialSegmentLocationBox, Source URL box, and helpers such as BoxFileIndexBox, BoxIndexBox, and DataIntegrityHashBox).
• Design considerations: The format supports continuous recording, resynchronization after byte losses, redistribution (local gateways), and out‑of‑band repair workflows.
• Metadata and integrity: It enables storage of reception metadata, repair pointers (e.g., source URL or offsets), and data integrity helpers for ISO/IEC 14496‑12 derived content.
• MIME registration: Annex A defines the associated MIME type for files conforming to the specification.

Practical applications and target users

• Content delivery and streaming engineers who need to record or exchange partially received media over unreliable networks.
• Software implementers building file repair tools, playback engines tolerant of missing bytes, or local gateway services that repackage and forward partially received files.
• Broadcasters and multicast distributors using lossy transports (e.g., FLUTE) who require documented loss traces and repair pointers.
• Archival and forensic use cases where documenting which bytes were received and which were lost is essential.
• Developers working with ISO Base Media File Format content who benefit from the standard’s helper tools (indexing and integrity).

Related standards

• ISO/IEC 14496‑12 (ISO Base Media File Format) - foundational box/object model used by this partial file format.
• Normative references include IETF RFC 5905 (NTP Version 4) for timing considerations.

Keywords: ISO/IEC 23001-14:2019, partial file format, MPEG systems, partial segments, FileTypeBox, paff, pmff, ISO/IEC 14496-12, file repair, lossy channels, data integrity.

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 23001-14
First edition
2019-01
Information technology — MPEG
systems technologies —
Part 14:
Partial file format
Technologies de l'information — Technologies des systèmes MPEG —
Partie 14: Format de fichier partiel
Reference number
©
ISO/IEC 2019
© ISO/IEC 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Fax: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2019 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms, definitions, symbols, abbreviated terms and conventions .1
3.1 Terms and definitions . 1
3.2 Abbreviated terms . 2
4 Partial file organization . 2
4.1 Object structure. 2
4.2 Design consideration . 2
4.2.1 Features . 2
4.2.2 Data layout . 3
4.2.3 Box order . 3
4.2.4 Mixed partial files . 4
4.2.5 Processing model . 5
5 Box structures . 5
5.1 Partial file general boxes . 5
5.1.1 Partial file box. 5
5.1.2 Partial file header box . 6
5.1.3 Partial segment box . 6
5.1.4 Partial segment header box . 7
5.1.5 Partial data box . 8
5.1.6 Partial segment location box . 9
5.1.7 Partial top level entry point box .10
5.1.8 Source URL box .11
5.2 Partial file ISO base media file helpers .12
5.2.1 General.12
5.2.2 Box file index box .12
5.2.3 Box index box .13
5.2.4 Data integrity box .14
5.2.5 Data integrity hash box .14
5.2.6 Front part box .16
Annex A (normative) Partial file MIME type registration .18
Bibliography .20
© ISO/IEC 2019 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that
are members of ISO or IEC participate in the development of International Standards through
technical committees established by the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other
international organizations, governmental and non-governmental, in liaison with ISO and IEC, also
take part in the work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for
the different types of document should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/patents) or the IEC
list of patent declarations received (see http: //patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso
.org/iso/foreword .html.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
A list of all parts in the ISO 23001 series can be found on the ISO website.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/members .html.
iv © ISO/IEC 2019 – All rights reserved

Introduction
The partial file format is designed to contain files partially received over a lossy link (with unreceived
or corrupted sections), for further processing such as playback or repair. The file structure is object-
oriented; a file can be decomposed into constituent objects very simply, and the structure of the objects
inferred directly from their type. Files conforming to this format may be exchanged without losing
information on losses, allowing redistribution (local gateways) or further repair processes.
The partial file format may be used to document reception of files, regardless of their bitstream format.
For generic cases, it provides ways for file readers to resynchronize their parsing in case of byte losses.
For cases where the documented file derives from ISO/IEC 14496-12, the partial file format provides
additional tools, such as an index of the source file structures and data integrity information.
Annex A of this document defines the associated MIME type for files conformant to this document.
© ISO/IEC 2019 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 23001-14:2019(E)
Information technology — MPEG systems technologies —
Part 14:
Partial file format
1 Scope
This document specifies the partial file format, which is a generic format for describing file partially
received over lossy communication channels. This format contains the correctly received data, missing
block identification, and repair information such as location of the file or high-level original indexing
information. This format can be used with any file formats, and provides additional helper tools for
formats deriving from ISO/IEC 14496-12.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 14496-12, Information technology — Coding of audio-visual objects — Part 12: ISO Base Media
file format
IETF RFC 5905, Network Time Protocol Version 4: Protocol and Algorithms Specification, Mills, D., et al,
June 2010
3 Terms, definitions, symbols, abbreviated terms and conventions
3.1 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
3.1.1
chunk
addressable set of bytes of a partial segment
3.1.2
indexed box
box in the source file (3.1.5) indexed by a BoxIndexBox in the partial file (3.1.3)
3.1.3
partial file
name of the files conforming to the file format described in this document
3.1.4
partial segment
portion of the partial file (3.1.3) containing a single PartialSegmentBox and identifying its
associated source data
© ISO/IEC 2019 – All rights reserved 1

3.1.5
source file
file described and encapsulated by the partial file (3.1.3)
3.2 Abbreviated terms
For the purposes of this document, the following abbreviated terms apply.
MD5 Message Digest 5
FDT File Delivery Table
FEC Forward Error Correction
FLUTE File Delivery over Unidirectional Transport
IANA Internet Assigned Numbers Authority
IETF Internet Engineering Task Force
RFC Request for Comments
SHA Secure Hash Algorithm
4 Partial file organization
4.1 Object structure
A partially received file, hereinafter referred to as partial file, is represented as a sequence of partial
segments, preceded by a header. The header and the partial segments are formed as a series of objects,
called boxes, as defined in ISO/IEC 14496-12; files under this document are object-structured files as
defined in ISO/IEC 14496-12.
All object-structured files conformant to Clause 4 shall contain a FileTypeBox as defined in ISO/IEC
14496-12. The FileTypeBox shall contain a compatible brand of type 'paff' for partial files, and of
type 'pmff' for mixed partial files.
In this document, top-level boxes (boxes not contained in other boxes) are indicated as being at ‘file’
level, with the notation “Container: File”.
The MIME type associated with a file conforming to this specification shall be formatted as defined in
Annex A.
4.2 Design consideration
4.2.1 Features
The partial file format is intended to serve as a storage and exchange format for other file formats
delivered over lossy channels. The format provides the following set of tools:
— Reception data, which provides means to store the received data and document transmission
information such as received or lost byte ranges and whether the corrupted/lost bytes are present
in the file.
— Repair information, such as location of the source file, possible byte offsets in that source, byte stream
position at which a parser can try processing a corrupted file; depending on the communication
channel, this information may be setup by the receiver or through out-of-band means.
2 © ISO/IEC 2019 – All rights reserved

— File format specific information, which depends on the type of file stored as a partial file; this
document only defines additional tools for files based on ISO/IEC 14496-12.
4.2.2 Data layout
The partial file format is designed to allow continuous recording and storing of received data in a
flexible way. A partial file is composed of a header providing information valid for the entire recording
followed by any number of segments; all of these segments are described by partial segment structures
and are hence called partial segments, although some can be correctly received. The header does not
contain any data from the source file and contains only information that can usually be inferred from
the parameters of the transmission, and hence does not require editing of the header values upon
finalizing the recording. Each partial segment contains a variable number of received bytes from a given
source, a map of valid byte ranges for these received bytes and optionally some repair information.
Byte maps give offsets relative to the first byte of data in this partial segment, rather than from the
first byte of data in the source file, in order to process partial segments without any knowledge of their
position in the partial file. Each partial segment may describe correctly received data, lost data or a mix
of correctly received and loss data.
4.2.3 Box order
An overall view of the normal encapsulation structure is provided in Table 1. In the event of a conflict
between this table and the prose, the prose prevails. The order of boxes within their container is not
necessarily indicated in the table.
The table shows those boxes that may occur at the top-level in the left-most column; indentation is
used to show possible containment. Not all boxes need to be used in all files; the mandatory boxes are
marked with an asterisk (*). See the description of the individual boxes for a discussion of assumptions
if the optional boxes are not present.
Objects using an extended type may be placed in a wide variety of containers, not just the top level.
The following rules shall be followed for the order of boxes in a partial file:
— The FileTypeBox shall occur first in the file.
— The PartialFileBox, if present, shall occur immediately after the FileTypeBox.
Table 1 — Box types, structure and cross-reference
Box types, structure and cross-reference
ftyp * file type and compatibility
pfil
5.1.1 container for metadata global to file
pfhd * 5.1.2 data about the entire partial file
surl
5.1.8 source URL and mime for the complete source file
ptle 5.1.7 entry points information for the complete source file
fidx
5.2.2 box file index for the complete source file
bidx 5.2.3 box index for a box of the source file
dint
5.2.4 data integrity container for the complete source file
dihd * 5.2.5 data integrity hash
frpa
5.2.6 front part (initial bytes) of an indexed box
dref 5.1.1 data reference box as defined in ISO/IEC 14496-12
pdat
5.1.2 partial segment data
pseg * 5.1.4 partial segment
Key
* Boxes that are mandatory within their container, which itself can be mandatory or optional.
© ISO/IEC 2019 – All rights reserved 3

Table 1 (continued)
Box types, structure and cross-reference
5.1.5 partial segment header containing information related to the partial
pshd *
segment
ploc
* 5.1.6 partial segment location
surl 5.1.8 source URL and mime of the source file for the partial segment
ptle
5.1.7 entry point information of the source file for the partial segment
fidx 5.2.2 box file index of the source file for the partial segment
bidx
5.2.3 box index for a box of the source file for the partial segment
dint 5.2.4 data integrity container of the source file for the partial segment
dihd
* 5.2.5 data integrity hash
frpa 5.2.6 front part (initial bytes) of an indexed box
Key
* Boxes that are mandatory within their container, which itself can be mandatory or optional.
4.2.4 Mixed partial files
When the source file uses a format derived from ISO/IEC 14496-12, it can be desirable to encapsulate
only the corrupted top-level boxes of the source file in a partial segment, while leaving the correctly
received top-level boxes untouched. Figure 1 shows the case where certain movie segments have been
received complete, other parts are only partial and there are yet other parts for which it is known that
no information has been received. The incomplete parts map to one or several partial segments.
Figure 1 — Mixed partial file
This design is referred to as mixed partial file. In this configuration, the FileTypeBox shall contain
a compatible brand of type 'pmff'. The FileTypeBox of the source file, whether or not correctly
received, shall not be modified; it shall be encapsulated in one partial segment and stored in a
PartialDataBox, in order to ensure unicity of the FileTypeBox at the container level. Mixed partial
file writers may insert, in the FileTypeBox of the partial file, brands from the source ISOBMFF file
if the requirements of these brands are still met in the mixed partial file. This design allows backward
compatibility with existing ISOBMFF file reader, as they would skip unrecognized boxes but still play
the rest of the presentation.
The mixed partial file format is designed to store partially received fragments of a given ISOBMFF file,
without any inspection or modification of the internal box structure of the source file. This format cannot
be used to indicate, through in-file structures such as SampleTableBox or TrackFragmentBox,
which items or samples are missing or corrupted. Such operation requires reprocessing of the source
file and is possible using dedicated structures defined in ISOBMFF.
NOTE 1 In case an ISOBMFF segment made of multiple movie fragments is corrupted and the initial
SegmentTypeBox is lost, a mixed partial file reader might have to insert a SegmentTypeBox at the appropriate
place for correct processing by higher application layers.
NOTE 2 When ISOBMFF segments are stored in a mixed partial file, track timing can be lost if correctly
received ISOBMFF segments do not contain TrackFragmentBaseMediaDecodeTimeBox.
4 © ISO/IEC 2019 – All rights reserved

4.2.5 Processing model
The processing model of partial files or mixed partial files is illustrated in Figure 2. In partial file mode,
there is a “partial file handler” that is used, which in turn delegates to the original file handler. In mixed
partial mode, the original file handler can be extended to handle partial files with their own original
MIME type.
Figure 2 — Processing model of partial files or mixed partial files
The source file shall be identical to the ordered concatenation of the data of each partial segment for
that given source if no losses occurred or if all corrupted blocks were successfully repaired. A mixed
partial file handler may typically optimize this process by not reconstructing the complete source file.
5 Box structures
5.1 Partial file general boxes
5.1.1 Partial file box
5.1.1.1 Definition
Box Type: 'pfil'
Container: File
Mandatory: No
Quantity: Zero or One
This box contains metadata information for the partial file. This metadata can be overridden with each
partial segment.
A PartialFileBox may contain zero or one DataReferenceBox as defined in ISO/IEC 14496-
12. Data references shall only be used to indicate an external location of (part of) the stored data;
consequently, entries of the DataReferenceBox shall not use the flag value 0x000001.
NOTE Usage of data references allows building a partial file describing the recorded result of a transmission
session stored in another format than the partial file format; for example, a partial file can be computed to
describe the current reception state of a torrent session.
When enclosed, the BoxFileIndexBox provides a summary of the box hierarchy of the complete
source file. It may be used to describe only type and size of a file-level box, or the complete or partial
© ISO/IEC 2019 – All rights reserved 5

hierarchy of child boxes of one or several file-level boxes. Each top-level box of the source file will
typically have zero or one associated BoxIndexBoxes in the BoxFileIndexBox.
If the set of boxes in the BoxFileIndexBox is also complete, the corresponding bit in file_flags
should also be set.
If one BoxFileIndexBox is present in the PartialFileBox, it indexes (part of) the complete file
and no other BoxFileIndexBox shall be present in any subsequent PartialSegmentBox.
5.1.1.2 Syntax
aligned(8) class PartialFileBox extends Box(’pfil’) {
Box other_boxes[]
}
5.1.2 Partial file header box
5.1.2.1 Definition
Box Type: 'pfhd'
Container: PartialFileBox
Mandatory: Yes, if PartialFileBox is present
Quantity: Exactly one
This box contains general information about the partial file data and structures.
The following flag is defined for the PartialFileHeaderBox:
— complete_index: flag value is 0x000001; when set, indicates that the BoxFileIndexBox
contained in this box is complete (the contents are always required to be dense and start at the
beginning, so this means that the box documents all bytes to the end of the original box-structured
file). This flag shall not be set for non-box-structured original files.
5.1.2.2 Syntax
aligned(8) class PartialFileHeaderBox extends FullBox(’pfhd’, 0,
flags) {
}
5.1.3 Partial segment box
5.1.3.1 Definition
Box Type: 'pseg'
Container: File
Mandatory: Yes
Quantity: One or more
The PartialSegmentBox is a container for metadata describing one or more blocks of received data
from a given source, missing or corrupted bytes, source or indexing information. There are usually
several instances of PartialSegmentBox in a partial file.
6 © ISO/IEC 2019 – All rights reserved

A PartialSegmentBox only contains boxes describing the source data; it does not contain
any source data. Each PartialSegmentBox shall contain one PartialSegmentHeaderBox
containing configuration of the partial segment; the associated source data may be located after or
before the PartialSegmentBox, as described by the PartialSegmentLocationBox, and may be
encapsulated in one or more PartialDataBox.
The resulting partial segment (i.e. the PartialSegmentBox and its associated data) shall conform to
the object structure defined in subclause 4.1.
Since reconstruction of the source file is based on information from partial segment, storing a segment
received without errors still requires the presence of a partial segment box describing the received data.
When enclosed, a BoxFileIndexBox provides a summary of the box hierarchy of the source file data
in the current partial segment. It may be used to describe only type and size of a file-level box, or the
complete or partial hierarchy of child boxes of one or several file-level boxes. Each top-level box of the
source file will typically have zero or one associated BoxIndexBoxes in the BoxFileIndexBox.
If the set of boxes in the BoxFileIndexBox box is also complete, the corresponding bit in segment_
flags should also be set.
If one BoxFileIndexBox is present in the PartialFileBox, it indexes (part of) the complete file
and no other BoxFileIndexBox shall be present in any subsequent PartialSegmentBox.
5.1.3.2 Syntax
aligned(8) class PartialSegmentBox extends Box(’pseg’) {
Box other_boxes[]
}
5.1.4 Partial segment header box
5.1.4.1 Definition
Box Type: 'pshd'
Container: PartialSegmentBox
Mandatory: Yes
Quantity: One per PartialSegmentBox
The PartialSegmentHeaderBox gives general information for the partial segment. The
PartialSegmentHeaderBox shall be placed first in the PartialSegmentBox.
The following flags are defined for the PartialSegmentHeaderBox:
— last_segment: flag value is 0x000001. Presence of this flag indicates that this partial segment
is the last one from the associated source; for a given file location as indicated by SourceURLBox,
there should not be more than one partial segment with last_segment flag set in the
PartialSegmentHeaderBox, and that partial segment should be the last one in the file. Readers
may stop processing the file data from this source once this fl
...