ISO/IEC 14496-16:2009
(Main)Information technology — Coding of audio-visual objects — Part 16: Animation Framework eXtension (AFX)
Information technology — Coding of audio-visual objects — Part 16: Animation Framework eXtension (AFX)
ISO/IEC 14496-16:2009 specifies MPEG-4 Animation Framework eXtension (AFX) model for representing and encoding 3D graphics assets to be used standalone or integrated in interactive multimedia presentations (the latter when combined with other parts of MPEG-4). Within this model, MPEG-4 is extended with higher-level synthetic objects for geometry, texture, and animation as well as dedicated compressed representations. AFX also specifies a backchannel for progressive streaming of view-dependent information.
Technologies de l'information — Codage des objets audiovisuels — Partie 16: Extension du cadre d'animation (AFX)
General Information
Relations
Buy Standard
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 14496-16
Third edition
2009-12-15
Information technology — Coding of
audio-visual objects —
Part 16:
Animation Framework eXtension (AFX)
Technologies de l'information — Codage des objets audiovisuels —
Partie 16: Extension du cadre d'animation (AFX)
Reference number
ISO/IEC 14496-16:2009(E)
©
ISO/IEC 2009
---------------------- Page: 1 ----------------------
ISO/IEC 14496-16:2009(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2009
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2009 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 14496-16:2009(E)
Contents Page
Foreword .vi
1 Scope.1
2 Normative references.1
3 Symbols and abbreviated terms .1
3.1 Introduction.2
3.2 The AFX place within computer animation framework .3
3.3 Geometry tools .5
3.3.1 Non-Uniform Rational B-Spline (NURBS) .5
3.3.2 Subdivision surfaces .8
3.3.3 MeshGrid representation.28
3.3.4 MorphSpace .34
3.3.5 MultiResolution FootPrint-Based Representation .35
3.3.6 Solid representation.39
3.4 Texture tools .49
3.4.1 Depth Image-Based Representation.49
3.4.2 Depth Image-based Representation Version 2.53
3.4.3 Multitexturing.57
3.5 Animation tools .62
3.5.1 Deformation tools.62
3.5.2 Generic skeleton, muscle and skin-based model definition and animation .64
3.6 Rendering tools .74
3.6.1 Shadows.74
4 AFX bitstream specification - 3D Graphics compression tools .76
4.1 Introduction.76
4.2 Geometry tools .77
4.2.1 3DMC Extension .77
4.2.2 Wavelet Subdivision Surfaces .134
4.2.3 MeshGrid stream .137
4.2.4 MultiResolution FootPrint-Based Representation .174
4.3 Texture tools .182
4.3.1 Depth Image-Based Representation.182
4.3.2 PointTexture stream.187
4.4 Animation tools .197
4.4.1 Bone-based animation.197
4.4.2 Frame-based Animated Mesh Compression (FAMC) stream.211
4.5 Generic tools.234
4.5.1 Multiplexing of 3D Compression Streams: the MPEG-4 3D Graphics stream (.m3d) syntax .234
4.5.2 AFX Generic Backchannel.238
5 AFX object codes .249
6 3D Graphics Profiles .250
6.1 Introduction.250
6.2 "Graphics" Dimension .251
6.2.1 MPEG-4 X3D Interactive Graphics Profiles and Levels.251
6.2.2 MPEG-4 "Basic AFX" Graphics Profiles and Levels.254
6.3 "Scene Graph" Dimension.256
6.3.1 MPEG-4 X3D Interactive Scene Graph Profile and Levels .256
6.3.2 PEG-4 "Basic AFX" Scene Graph Profile and Levels .259
6.4 "3D Compression" Dimension .261
6.4.1 "Core 3D Compression" Profile and Levels .262
© ISO/IEC 2009 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 14496-16:2009(E)
6.4.2 3D Multiresolution Compression Profile and Levels .265
7 XMT representation for AFX tools.269
7.1 AFX nodes .269
7.2 AFX encoding hints .269
7.2.1 WaveletSubdivision encoding hints .269
7.2.2 MeshGrid encoding hints.269
7.2.3 OctreeImage encoding hints .270
7.2.4 PointTexture encoding hints .270
7.2.5 BBA encoding hints.270
7.3 AFX encoding parameters .270
7.3.1 WaveletSubdivisionEncodingParameters.270
7.3.2 MeshGridEncodingParameters .271
7.3.3 PointTextureEncodingParameters.272
7.3.4 BBAEncodingParameters .272
7.4 AFX decoder specific info.273
7.4.1 WaveletSubdivision decoder specific info.273
7.4.2 MeshGrid decoder specific info .273
7.5 XMT for Bone-based Animation .274
7.5.1 BBA .274
7.5.2 BBA_header .274
7.5.3 BBA_encoding .275
7.5.4 BBA_body.275
7.5.5 BBA_frame .275
7.5.6 BBA_frameMask .275
7.5.7 BoneMask.276
7.5.8 MuscleMask.277
7.5.9 MorphMask.277
7.5.10 BBA_frameValues.278
Annex A (normative) Wavelet Mesh Decoding Process.279
A.1 Overview.279
A.2 Base mesh .279
A.3 Definitions and notations.279
A.4 Bitplanes extraction .280
A.5 Zero-tree decoder .281
A.6 Synthesis filters and mesh reconstruction.281
A.7 Basis change.282
Annex B (normative) MeshGrid Representation .283
B.1 The hierarchical multi-resolution MeshGrid .283
B.1.1 Building MeshGrid multi-resolution levels .283
B.1.2 Relation between the resolution level of the mesh and the number of reference-surfaces.284
B.1.3 Relation between the resolution level of the grid and the number of reference-surfaces.284
B.1.4 Region Of Interest (ROI): Computation of the ROIs size and position.285
B.2 Scalability Modes.287
B.2.1 Scalability types.287
B.2.2 The mesh resolution scalability .288
B.2.3 The scalability in shape precision .288
B.2.4 The vertex position scalability .289
B.3 Animation Possibilities .290
B.3.1 Introduction.290
B.3.2 Vertex-based animation .290
B.3.3 Rippling effects, changing the offsets.291
B.3.4 Animation of the reference-grid points .291
Annex C (informative) MeshGrid representation .293
C.1 Representing Quadrilateral meshes in the MeshGrid format .293
C.2 IndexedFaceSet models represented by the MeshGrid format. .294
C.3 Computation of the number of ROIs at the highest resolution level given an optimal ROI
size .294
iv © ISO/IEC 2009 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 14496-16:2009(E)
Annex D (informative) Solid representation .296
D.1 Overview.296
D.2 Solid Primitives.296
D.3 Arithmetics of Forms .297
D.3.1 Introduction.297
D.3.2 General Arithmetic Operators on Densities.298
D.3.3 Ternary Logic: The Kleene Operators .300
D.3.4 Densities Filtering .302
Annex E (informative) Face and Body animation: XMT compliant animation and encoding
parameter file format.303
E.1 XSD for FAP file format.303
E.2 XSD for BAP file format .304
E.3 XSD for EPF .305
Annex F (normative) Local refinements for MultiResolution FootPrint-Based Representation.309
Annex G (informative) Partition Encoding for FAMC .311
Annex H (informative) Animation Weights Encoding for FAMC .312
Annex I (normative) Layered decomposition for FAMC .313
I.1 Obtaining the layered decomposition.313
I.2 Deriving layers ld[l], l>0 using decoded data and connectivity
(layeredDecompositionIsEncoded equals 1).314
I.3 Deriving layers ld[l], l>0 using only connectivity (default option,
layeredDecompositionIsEncoded equals 0).314
I.4 Deriving layer ld[0] .317
I.5 Mesh simplification .319
Annex J (normative) Reconstruction of values from decoded prediction errors with LD technique
for FAMC.321
Annex K (normative) CABAC definitions, basic functions, and binarizations (as used for FAMC).324
K.1 CABAC definitions .324
K.1.1 Definition of the term CabacContext .324
K.1.2 Definition of the term DecodingEnvironment .324
K.2 CABAC basic functions .324
K.2.1 Definition of the function cabac.biari_init_context( ctx, preCtxState ) .324
K.2.2 Definition of the function cabac.arideco_start_decoding( dec_env ) .324
K.2.3 Definition of the function cabac.biari_decode_symbol( dec_env, ctx ).325
K.2.4 Definition of the function cabac.biari_decode_symbol_eq_prob( dec_env ) .327
K.2.5 Definition of the function cabac.biari_decode_final( dec_env ).328
K.3 CABAC binarization functionss .328
K.3.1 Definition of the function cabac.exp_golomb_decode_eq_prob( dec_env, k ) .328
K.3.2 Definition of the function cabac.unary_exp_golomb_decode( dec_env, ctx,
exp_start ).328
Annex L (normative) Node coding tables.330
L.1 Node Coding tables.330
L.2 Node Definition Type tables .330
Annex M (informative) Patent statements .331
Bibliography.332
© ISO/IEC 2009 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC 14496-16:2009(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 14496-16 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
This third edition cancels and replaces the second edition (ISO/IEC 14496-16:2006), of which it constitutes a
minor revision. It also incorporates ISO/IEC 14496-16:2006/Amd.1:2007, ISO/IEC 14496-16:2006/
Amd.1:2007/Cor.1:2008, ISO/IEC 14496-16:2006/Amd.1:2007/Cor.2:2008, ISO/IEC 14496-16:2006/
Amd.2:2009, ISO/IEC 14496-16:2006/Amd.3:2008.
ISO/IEC 14496 consists of the following parts, under the general title Information technology — Coding of
audio-visual objects:
⎯ Part 1: Systems
⎯ Part 2: Visual
⎯ Part 3: Audio
⎯ Part 4: Conformance testing
⎯ Part 5: Reference software
⎯ Part 6: Delivery Multimedia Integration Framework (DMIF)
⎯ Part 7: Optimized reference software for coding of audio-visual objects [Technical Report]
⎯ Part 8: Carriage of ISO/IEC 14496 contents over IP networks
⎯ Part 9: Reference hardware description
⎯ Part 10: Advanced Video Coding
⎯ Part 11: Scene description and application engine
⎯ Part 12: ISO base media file format
vi © ISO/IEC 2009 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC 14496-16:2009(E)
⎯ Part 13: Intellectual Property Management and Protection (IPMP) extensions
⎯ Part 14: MP4 file format
⎯ Part 15: Advanced Video Coding (AVC) file format
⎯ Part 16: Animation Framework eXtension (AFX)
⎯ Part 17: Streaming text format
⎯ Part 18: Font compression and streaming
⎯ Part 19: Synthesized texture stream
⎯ Part 20: Lightweight Application Scene Representation (LASeR) and Simple Aggregation Format (SAF)
⎯ Part 21: MPEG-J Graphics Framework eXtensions (GFX)
⎯ Part 22: Open Font Format
⎯ Part 23: Symbolic Music Representation
⎯ Part 24: Audio and systems interaction [Technical Report]
⎯ Part 25: 3D Graphics Compression Model
⎯ Part 27: 3D Graphics conformance
© ISO/IEC 2009 – All rights reserved vii
---------------------- Page: 7 ----------------------
INTERNATIONAL STANDARD ISO/IEC 14496-16:2009(E)
Information technology — Coding of audio-visual objects —
Part 16:
Animation Framework eXtension (AFX)
1 Scope
This Part of ISO/IEC 14496 specifies MPEG-4 Animation Framework eXtension (AFX) model for representing
and encoding 3D graphics assets to be used standalone or integrated in interactive multimedia presentations
(the latter when combined with other parts of MPEG-4). Within this model, MPEG-4 is extended with higher-
level synthetic objects for geometry, texture, and animation as well as dedicated compressed representations.
AFX also specifies a backchannel for progressive streaming of view-dependent information.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO/IEC 14496-1:2004, Information technology — Coding of audio-visual objects — Part 1: Systems
ISO/IEC 14496-2:2004, Information technology — Coding of audio-visual objects — Part 2: Visual
ISO/IEC 14496-11:2005, Information technology — Coding of audio-visual objects — Part 11: Scene
description and application engine
ISO/IEC 14496-25:2008, Information technology — Coding of audio-visual objects — Part 25: 3D Graphics
Compression Model
3 Symbols and abbreviated terms
List of symbols and abbreviated terms.
AFX Animation Framework eXtension
BIFS BInary Format for Scene
DIBR Depth-Image Based Representation
ES Elementary Stream
IBR Image-Based Rendering
NDT Node Data Type
© ISO/IEC 2009 – All rights reserved 1
---------------------- Page: 8 ----------------------
ISO/IEC 14496-16:2009(E)
OD Object Descriptor
VRML Virtual Reality Modelling Language
Animation Framework eXtension – 3D Graphics Primitives
3.1 Introduction
MPEG-4, ISO/IEC 14496, is a multimedia standard that enables composition of multiple audio-visual objects
on a terminal. Audio and visual objects can come from natural sources (e.g. a microphone, a camera) or from
synthetic ones (i.e. made by a computer); each source is called a media or a stream. On their terminals, users
can display, play, and interact with MPEG-4 audio-visual contents, which can be downloaded previously or
streamed from remote servers. Moreover, each object may be protected to ensure a user has the right
credentials before downloading and displaying it.
Unlike natural audio and video objects, computer graphics objects are purely synthetic. Mixing computer
graphics objects with traditional audio and video enables augmented reality applications, i.e. applications
mixing natural and synthetic objects. Examples of such contents range from DVD menus, and TV's Electronic
Programming Guides to medical and training applications, games, and so on.
Like other computer
...
INTERNATIONAL ISO/IEC
STANDARD 14496-16
Third edition
2009-12-15
Information technology — Coding of
audio-visual objects —
Part 16:
Animation Framework eXtension (AFX)
Technologies de l'information — Codage des objets audiovisuels —
Partie 16: Extension du cadre d'animation (AFX)
Reference number
ISO/IEC 14496-16:2009(E)
©
ISO/IEC 2009
---------------------- Page: 1 ----------------------
ISO/IEC 14496-16:2009(E)
PDF disclaimer
PDF files may contain embedded typefaces. In accordance with Adobe's licensing policy, such files may be printed or viewed but shall
not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading a PDF file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create the PDF file(s) constituting this document can be found in the General Info relative to
the file(s); the PDF-creation parameters were optimized for printing. Every care has been taken to ensure that the files are suitable for
use by ISO member bodies. In the unlikely event that a problem relating to them is found, please inform the Central Secretariat at the
address given below.
This CD-ROMcontains:
1) the publication ISO/IEC 14496-16:2009 in portable document format (PDF), which can be viewed
using Adobe® Acrobat® Reader;
2) electronic attachments for AFX.
Adobe and Acrobat are trademarks of Adobe Systems Incorporated.
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2009
All rights reserved. Unless required for installation or otherwise specified, no part of this CD-ROM may be reproduced, stored in a retrieval
system or transmitted in any form or by any means without prior permission from ISO. Requests for permission to reproduce this product
should be addressed to
ISO copyright office • Case postale 56 • CH-1211 Geneva
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.