ISO/IEC TR 14496-9:2009

Information technology - Coding of audio-visual objects - Part 9: Reference hardware description

ISO/IEC TR 14496-9:2009 specifies descriptions of the main video coding tools in hardware description language (HDL) form. Such alternative descriptions to the ones that are reported in ISO/IEC 14496-2, ISO/IEC 14496-5 and ISO/IEC TR 14496-7 correspond to the need of providing the public with conformant standard descriptions that are closer to the starting point of the development of codec implementations than textual descriptions or pure software descriptions. ISO/IEC TR 14496-9:2009 contains conformant descriptions of video tools that have been validated within the recommendation ISO/IEC TR 14496-7.

Technologies de l'information — Codage des objets audiovisuels — Partie 9: Description de matériel de référence

General Information

• Status: Published
• Publication Date: 19-Jan-2009
• ICS: 35.040 - Information coding
• ICS: 35.040.40 - Coding of audio, video, multimedia and hypermedia information
• Technical Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Drafting Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Current Stage: 9093 - International Standard confirmed
• Start Date: 12-Oct-2019
• Completion Date: 30-Oct-2025

Relations

• Revises: ISO/IEC TR 14496-9:2008 - Information technology - Coding of audio-visual objects - Part 9: Reference hardware description
• Effective Date: 10-Jan-2009

Overview

ISO/IEC TR 14496-9:2009 - Information technology - Coding of audio‑visual objects - Part 9: Reference hardware description - provides hardware‑oriented descriptions of the principal video coding tools used in the MPEG‑4 family. The Technical Report supplies hardware description language (HDL) representations as conformant, public reference material. These HDL descriptions are intended as alternatives to the textual and software descriptions in related MPEG‑4 parts and are validated within ISO/IEC TR 14496-7.

Key topics

• HDL reference implementations: Conformant descriptions of main video coding tools expressed in HDL to aid hardware development and verification.
• Conformance and validation: Reference designs validated within the ISO/IEC framework to ensure interoperability with MPEG‑4 video coding specifications.
• Bridging software and hardware: Provides descriptions closer to the starting point for ASIC/FPGA codec implementations than textual standards or software reference code.
• Electronic attachments: Includes supporting electronic files relevant to reference hardware for coding audio‑visual objects.

Practical applications

ISO/IEC TR 14496-9:2009 is practical for teams and projects that need an authoritative hardware starting point for MPEG‑4 video coding:

• Rapid prototype development of video codec IP on FPGA platforms using HDL.
• ASIC and SoC design for real‑time video compression/decompression where hardware efficiency and conformity are required.
• Conformance testing and verification labs that need a standard HDL baseline for interoperability tests.
• Academic and R&D projects studying hardware implementations of video coding algorithms.
• System integrators implementing hardware-accelerated video pipelines in consumer electronics, broadcast, and professional video equipment.

Who uses this standard

• Hardware designers (FPGA/ASIC) and IP developers
• Semiconductor and SoC vendors building video codec blocks
• Verification and QA engineers performing conformance and interoperability testing
• Standards developers and researchers comparing hardware implementations to MPEG‑4 textual specifications

Related standards

• ISO/IEC 14496-2 - MPEG‑4 Video (textual tool descriptions)
• ISO/IEC 14496-5 - Reference software for MPEG‑4 systems
• ISO/IEC TR 14496-7 - Validation and conformance guidance referenced by Part 9

ISO/IEC TR 14496-9:2009 is a practical, implementation‑focused resource for anyone needing standardized HDL descriptions of MPEG‑4 video coding tools, supporting faster, more reliable hardware codec development and interoperability.

Standards Content (Sample)

TECHNICAL ISO/IEC
REPORT TR
14496-9
Third edition
2009-02-01
Information technology — Coding of
audio-visual objects —
Part 9:
Reference hardware description
Technologies de l'information — Codage des objets audiovisuels —
Partie 9: Description de matériel de référence

Reference number
©
ISO/IEC 2009
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.

©  ISO/IEC 2009
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail [e-mail]
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2009 – All rights reserved

Contents Page
Foreword. v
Introduction . vii
1 Scope .1
2 Copyright disclaimer for HDL software modules .1
3 Abbreviated terms .2
4 HDL software availability .2
5 HDL coding format and standards.2
5.1 HDL standards and libraries.2
5.2 Conditions and tools for the synthesis of HDL modules .3
5.3 Conformance with the reference software.3
6 Integrated Framework supporting the “Virtual Socket” between HDL modules described
in Part 9 and the MPEG Reference Software (Implementation 1). .3
6.1 Introduction.3
6.2 Addressing .4
6.3 Memory Map.4
6.4 Hardware Accelerator Interface.6
6.5 User Hardware Accelerator Sockets.12
7 Integrated Framework supporting the “Virtual Socket” between HDL modules described
in Part 9 and the MPEG Reference Software (Implementation 2). .14
7.1 Introduction.14
7.2 Development Example of a Typical Module : Calc_Sum_Product Module.14
7.3 Second Example of a Typical Module : fifo_transfer module .18
7.4 Integrating the Multi-Modules within the Framework .23
7.5 Calc_Sum_Product Module Controller (memory data transfer) .30
7.6 Simulation of the whole system.44
7.7 Debug Menu .46
8 Integrated Framework supporting the “Virtual Socket” between HDL modules described
in Part 9 and the MPEG Reference Software (Implementation 3). .47
8.1 An Integrated Virtual Socket Hardware-Accelerated Co-design Platform for MPEG-4 .47
8.2 Reference for Virtual Socket API Function Calls.72
8.3 Tutorial on the Integrated Virtual Socket Hardware-Accelerated Co-design Platform for
MPEG-4 Part 9 Implementation 3 .95
8.4 An Integration of the MPEG-4 Part 10/AVC DCT/Q Hardware Module into the Virtual
Socket Co-design Platform.140
8.5 Migrating Virtual Socket Hardware-Accelerated Co-design Platform From WildCard-II to
WildCard-4 .153
9 Integrated Framework supporting the “Virtual Socket” between HDL modules described
in Part 9 and the MPEG Reference Software: Implementation 4 - Virtual Memory
Extension.167
9.1 Introduction.167
9.2 Overview of the “Virtual Socket Platform” implementation 4.167
9.3 Development information .171
9.4 Technical details.172
9.5 How to build the platform .196
9.6 Simulation of the platform .203
9.7 Synthesis of the platform.209
9.8 Building the platform system software .212
9.9 How to use the platform.214
© ISO/IEC 2009 – All rights reserved iii

9.10 Understanding VHDL code. 216
9.11 Appendix. 220
9.12 Glossary. 222
10 HDL MODULES. 225
10.1 INVERSE QUANTIZER HARDWARE IP BLOCK FOR MPEG-4 PART 2 . 225
10.2 2-D IDCT HARDWARE IP BLOCK FOR MPEG-4 PART 2. 232
10.3 VLD+IQ+IDCT for MPEG-4 Part 2. 242
10.4 A SYSTEM C MODEL FOR 2X2 HADAMARD TRANSFORM AND QUANTIZATION FOR
MPEG–4 PART 10. 247
10.5 A VHDL HARDWARE BLOCK FOR 2X2 HADAMARD TRANSFORM AND QUANTIZATION
WITH APPLICATION TO MPEG–4 PART 10 AVC . 256
10.6 A SYSTEMC MODEL FOR 4X4 HADAMARD TRANSFORM AND QUANTIZATION FOR
MPEG-4 PART 10. 262
10.7 A VHDL HARDWARE IP BLOCK FOR 4X4 HADAMARD TRANSFORM AND QUANTIZATION
FOR MPEG-4 PART 10 AVC . 269
10.8 A HARDWARE BLOCK FOR THE MPEG-4 PART 10 4X4 DCT-LIKE TRANSFORMATION
AND QUANTIZATION . 275
10.9 A SYSTEMC MODEL FOR THE MPEG-4 PART 10 4X4 DCT-LIKE TRANSFORMATION AND
QUANTIZATION . 281
10.10 A 8X8 INTEGER APPROXIMATION DCT TRANSFORMATION AND QUANTIZATION
SYSTEMC IP BLOCK FOR MPEG-4 PART 10 AVC. 289
10.11 INTEGER APPROXIMATION OF 8X8 DCT TRANSFORMATION AND QUANTIZATION, A
HARDWARE IP BLOCK FOR MPEG-4 PART 10 AVC. 299
10.12 A VHDL CONTEXT-BASED ADAPTIVE VARIABLE LENGTH CODING (CAVLC) IP BLOCK
FOR MPEG-4 PART 10 AVC . 306
10.13 A VERILOG HARDWARE IP BLOCK FOR SA-DCT FOR MPEG-4. 311
10.14 A VERILOG HARDWARE IP BLOCK FOR SA-IDCT FOR MPEG-4. 322
10.15 A VERILOG HARDWARE IP BLOCK FOR 2D-DCT (8X8). 335
10.16 SHAPE CODING BINARY MOTION ESTIMATION HARDWARE ACCELERATION MODULE. 344
10.17 A SIMD ARCHITECTURE FOR FULL SEARCH BLOCK MATCHING ALGORITHM. 358
10.18 HARDWARE MODULE FOR MOTION ESTIMATION (4xPE) . 367
10.19 A IP BLOCK FOR H.264/AVC QUARTER PEL FULL SEARCH VARIABLE BLOCK MOTION
ESTIMATION . 381
10.20 AN IP BLOCK FOR VARIABLE BLOCK SIZE MOTION ESTIMATION IN H.264/MPEG-4 AVC . 389
10.21 An IP Block for MPEG-4 Part 10 AVC Deblocking Filter . 400
10.22 A HW BLOCK FOR MPEG-4 PART 10 AVC CONTEXT ADAPTIVE VARIABLE LENGTH
CODING (CAVLC) . 407
10.23 HARDWARE IMPLEMENTATION OF FULL SEARCH MPEG-4 PART 10 AVC MOTION
ESTIMATION . 421
Annex A (Informative) Specification of directory structure for reference SW, HDL and
documentation files of MPEG-4 Part 9 Reference HW Description . 433
A.1 Introduction . 433
A.2 Directory Structure of TR SW Modules. 433
CVS Module Name. 434
Integration Framework Version . 435
Reference Software Version and Modifications. 435
Annex B (Informative) Tutorial on Part 9 CVS Client Installation & Operation . 436
Annex C (Informative) Additional utility software . 446
Annex D (Informative) Providers of reference hardware code . 447
Bibliography . 448

iv © ISO/IEC 2009 – All rights reserved

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
In exceptional circumstances, the joint technical committee may propose the publication of a Technical Report
of one of the following types:
⎯ type 1, when the required support cannot be obtained for the publication of an International Standard,
despite repeated efforts;
⎯ type 2, when the subject is still under technical development or where for any other reason there is the
future but not immediate possibility of an agreement on an International Standard;
⎯ type 3, when the joint technical committee has collected data of a different kind from that which is
normally published as an International Standard (“state of the art”, for example).
Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether
they can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to
be reviewed until the data they provide are considered to be no longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC TR 14496-9, which is a Technical Report of type 3, was prepared by Joint Technical Committee
ISO/IEC JTC 1, Information technology, Subcommittee SC 29, Coding of audio, picture, multimedia and
hypermedia information.
This third edition cancels and replaces the second edition (ISO/IEC TR 14496-9:2008) which has been
technically revised.
ISO/IEC 14496 consists of the following parts, under the general title Information technology — Coding of
audio-visual objects:
⎯ Part 1: Systems
⎯ Part 2: Visual
⎯ Part 3: Audio
⎯ Part 4: Conformance testing
⎯ Part 5: Reference software
© ISO/IEC 2009 – All rights reserved v

⎯ Part 6: Delivery Multimedia Integration Framework (DMIF)
⎯ Part 7: Optimized reference software for coding of audio-visual objects [Technical Report]
⎯ Part 8: Carriage of ISO/IEC 14496 contents over IP networks
⎯ Part 9: Reference hardware description [Technical Report]
⎯ Part 10: Advanced Video Coding
⎯ Part 11: Scene description and application engine
⎯ Part 12: ISO base media file format
⎯ Part 13: Intellectual Property Management and Protection (IPMP) extensions
⎯ Part 14: MP4 file format
⎯ Part 15: Advanced Video Coding (AVC) file format
⎯ Part 16: Animation Framework eXtension (AFX)
⎯ Part 17: Streaming text format
⎯ Part 18: Font compression and streaming
⎯ Part 19: Synthesized texture streaming
⎯ Part 20: Lightweight Application Scene Representation (LASeR) and Simple Aggregation Format (SAF)
⎯ Part 21: MPEG-J Graphical Framework eXtension (GFX)
⎯ Part 22: Open Font Format
⎯ Part 23: Symbolic Music Representation
⎯ Part 24: Audio and systems interaction
⎯ Part 25: 3D Graphics Compression Model

vi © ISO/IEC 2009 – All rights reserved

Introduction
The main goal of this Technical Report is to facilitate a more widespread use of the MPEG-4 standard.
Design methodologies of the EDA industry have evolved from schematics to Hardware Description Languages
(HDLs) to address the needs of the vast number of gates available on a single device. The increased number
of gates allowed more elaborate algorithms to be deployed but also required a shift in design paradigm to
handle the complexity created. Through HDLs, more complicated systems could be designed faster through
the enabling technology of synthesis of the HDL code towards different silicon technologies where trade offs
could be explored. Now the EDA industry again faces challenges where HDLs may not provide the level of
abstraction needed for system designers to evaluate system level parameters and complexity issues. There
have been a number of tool investigations under way to address this problem. Profiling tools aid in exposing
bottlenecks in an abstract way so that early design decisions can be made. C to gates tools allow a C based
simulation environment while also enabling direct synthesis to gates for hardware acceleration.
In conclusion, it is the aim of this Technical Report to enable more widespread use of the MPEG-4 standard
through reference hardware descriptions and close integration with MPEG-4 Part 7 Optimized Reference
Software. Additionally, it is aimed that exposure to such a platform will enable a more systematic way to
investigate the complexity of new codecs and open up the algorithm search space with an order of magnitude
more compute cycles.
© ISO/IEC 2009 – All rights reserved vii

TECHNICAL REPORT ISO/IEC TR 14496-9:2009(E)

Information technology — Coding of audio-visual objects —
Part 9:
Reference hardware description
1 Scope
This part of ISO/IEC 14496 specifies descriptions of the main video coding tools in hardware description
language (HDL) form. Such alternative descriptions to the ones that are reported in ISO/IEC 14496-2,
ISO/IEC 14496-5 and ISO/IEC TR 14496-7 correspond to the need of providing the public with conformant
standard descriptions that are closer to the starting point of the development of codec implementations than
textual descriptions or pure software descriptions. This part of ISO/IEC 14496 contains conformant
descriptions of video tools that have been validated within the recommendation ISO/IEC TR 14496-7.
2 Copyright disclaimer for HDL software modules
Each HDL module shall be accompanied by the following copyright disclaimer, to be included in each HDL
module and all derivative modules:
/*********************************************************************
This software module was originally developed by
, , ,
(date: ,)
and edited by: , ,
This HDL module is an implementation of a part of one or more MPEG-4
tools(ISO/IEC 14496).
ISO/IEC gives users of the MPEG-4 free license to this HDL module or
modifications thereof for use in hardware or software products claiming
conformance to the MPEG-4 Standard.
Those intending to use this HDL module in hardware or software products are
advised that its use may infringe existing patents.
The original developer of this HDL module and his/her company, the subsequent
editors and their companies, and ISO/IEC have no liability for use of this HDL
module or modifications thereof in an implementation.
Copyright is not released for non MPEG-4 Video conforming products.
retains full right to use the code for his/her own purpose, assign
or donate the code to a third party and to inhibit third parties from using the
code for non MPEG standard conforming products.
© ISO/IEC 2009 – All rights reserved 1

This copyright notice must be included in all copies or derivative works.
Copyright (c) .
Module Name: .vhd
Abstract:
Revision History:
**********************************************************************/
3 Abbreviated terms
For the purposes of this document, the following abbreviated terms apply:
AV Audio-Visual
DCT Discrete Cosine Transform
IDCT Inverse Discrete Cosine Transform
HDL Hardware Description language
ISO International Organization for Standardization
MPEG Moving Picture Experts Group
Verilog A Hardware Description Language
VHDL VHSIC high speed Hardware Description Language
SAD Sum of Absolute Differences
MAC Multiply ACcumulate
MAD Minimum Absolute Difference
SIMD Single Instruction Multiple Data
DA Distributive Arithmetic
EDA Electronic Design and Automation
IEEE Institute of Electrical and Electronic Engineers
IMEC Interuniversity Micro Electronic Center
EPFL École Polytechnique Fédérale de Lausanne

4 HDL software availability
The HDL and System C software modules described in this part of ISO/IEC 14496 are available within the zip
file containing this Technical Report. Each module contains a separate directory structure for the source code
with a readme.txt file explaining the top level and all files to be included for simulation and synthesis.
5 HDL coding format and standards
5.1 HDL standards and libraries
As the IEEE has several HDL coding standards that are commonly used in hardware reference code (i.e.
VHDL1076-1987, VHDL 1164-1993, Verilog 1364-2000, Verilog 1364-1995), the modules constituting this part
2 © ISO/IEC 2009 – All rights reserved

of ISO/IEC 14496 are made of the latest IEEE standard possible at the time of coding for all reference HDL
code. As the IEEE has provided libraries to assist in the use of HDL, only IEEE standard libraries are needed
to use the HDL code.
Custom libraries which are specific to the vendor's (Silicon) base library elements are used only if they are
freely available for synthesis and simulation and are provided in an accompanying module version of the
submitted HDL code using the standard libraries mentioned above.
5.2 Conditions and tools for the synthesis of HDL modules
As there are many choices commercially for HDL synthesis and HDL simulation software tools, specific
synthesis or simulation libraries that are used for reference HDL code are properly documented. The same
code that is used to synthesize towards an implementation is also used to perform HDL behavioral simulation
of the MPEG-4 tool. The code is properly documented with respect to the synthesis and simulation tool (and
version) that has been used to perform the work. HDL module codes with multiple synthesis and simulation
tools are also possible. In the event a source code modification must be made to support an additional
synthesis or simulation tool, an additional source code is provided with proper documentation.
5.3 Conformance with the reference software
HDL reference code provides sufficient test bench code and documentation on how it is conformant with
respect to the reference software. To the extent possible, bit and cycle true models are provided which can be
used directly in the reference software code for verification. In the case that the reference HDL code is derived
from other languages such as: C, C++, System C, Java, it is recommended that that this code and information
on the methodology used to generate HDL should be provided to improve verification of conformance of the
HDL code.
6 Integrated Framework supporting the “Virtual Socket” between HDL modules
described in Part 9 and the MPEG Reference Software (Implementation 1).
6.1 Introduction
The aim of this clause is to document the framework developed by Xilinx Research Labs for the integration of
HW modules with the MPEG-4 reference software. The purpose of this virtual socket framework is to create
an abstraction between the specific physical layer and specific software driver library to facilitate a reusable
hardware/software co-design environment. By acting as an intermediary between specific physical layer bus
protocols, the hardware accelerator designer can focus on the acceleration algorithm rather than lower level
interface protocols.
The framework of the Virtual Socket allows for 31 addressable hardware accelerators to be present in a single
device (see Figure 1). Each specific hardware accelerator will be assigned a bit of the 32-bit hardware
identification register and these bit locations shall be assigned to particular MPEG development teams (see
Figure 2 for an example containing two accelerators at slots 1 and 6). If an accelerator socket is not present
then its bit in the identification register will be de-asserted. Unassigned sockets will also be de-asserted
indicating no accelerator is present. In the event that hardware accelerator designers wish to put further
identification of their socket they may do so by allocating further identification registers within their socket’s
assigned register space.
© ISO/IEC 2009 – All rights reserved 3

Figure 1 — Block Diagram of Virtual Socket Platform.

Figure 2 — Example 32-Bit Hardware Identification Register.

6.2 Addressing
The virtual socket provides four strobes that indicate what region of the memory space, register or memory,
has been accessed as well as the type of operation, write or read. Although a 16-bit is provided to each
socket, the least significant nine bits are only necessary to address within the 512 word assigned memory
region. The Virtual Socket API uses macros that assist the software designer in transferring data to and from
memory locations.
6.3 Memory Map
Table 1 — Memory Mapping for Register File Allocation.
Register Read-Only Register Write-Only
Socket
# Begin End Begin End
Master 0000 01FF 4000 41FF
1 0200 03FF 4200 43FF
2 0400 05FF 4400 45FF
3 0600 07FF 4600 47FF
4 0800 09FF 4800 49FF
5 0A00 0BFF 4A00 4BFF
6 0C00 0DFF 4C00 4DFF
7 0E00 0FFF 4E00 4FFF
4 © ISO/IEC 2009 – All rights reserved

8 1000 11FF 5000 51FF
9 1200 13FF 5200 53FF
10 1400 15FF 5400 55FF
11 1600 17FF 5600 57FF
12 1800 19FF 5800 59FF
13 1A00 1BFF 5A00 5BFF
14 1C00 1DFF 5C00 5DFF
15 1E00 1FFF 5E00 5FFF
16 2000 21FF 6000 61FF
17 2200 23FF 6200 63FF
18 2400 25FF 6400 65FF
19 2600 27FF 6600 67FF
20 2800 29FF 6800 69FF
21 2A00 2BFF 6A00 6BFF
22 2C00 2DFF 6C00 6DFF
23 2E00 2FFF 6E00 6FFF
24 3000 31FF 7000 71FF
25 3200 33FF 7200 73FF
26 3400 35FF 7400 75FF
27 3600 37FF 7600 77FF
28 3800 39FF 7800 79FF
29 3A00 3BFF 7A00 7BFF
30 3C00 3DFF 7C00 7DFF
31 3E00 3FFF 7E00 7FFF
Table 2 — Memory Mapping for the Block RAM Allocation.
Memory Read-Only Memory Write-Only
Socket
# Begin End Begin End
Master 8000 81FF C000 C1FF
1 8200 83FF C200 C3FF
2 8400 85FF C400 C5FF
3 8600 87FF C600 C7FF
4 8800 89FF C800 C9FF
5 8A00 8BFF CA00 CBFF
6 8C00 8DFF CC00 CDFF
7 8E00 8FFF CE00 CFFF
8 9000 91FF D000 D1FF
9 9200 93FF D200 D3FF
10 9400 95FF D400 D5FF
11 9600 97FF D600 D7FF
12 9800 99FF D800 D9FF
13 9A00 9BFF DA00 DBFF
14 9C00 9DFF DC00 DDFF
15 9E00 9FFF DE00 DFFF
16 A000 A1FF E000 E1FF
17 A200 A3FF E200 E3FF
18 A400 A5FF E400 E5FF
19 A600 A7FF E600 E7FF
20 A800 A9FF E800 E9FF
21 AA00 ABFF EA00 EBFF
© ISO/IEC 2009 – All rights reserved 5

22 AC00 ADFF EC00 EDFF
23 AE00 AFFF EE00 EFFF
24 B000 B1FF F000 F1FF
25 B200 B3FF F200 F3FF
26 B400 B5FF F400 F5FF
27 B600 B7FF F600 F7FF
28 B800 B9FF F800 F9FF
29 BA00 BBFF FA00 FBFF
30 BC00 BDFF FC00 FDFF
31 BE00 BFFF FE00 FFFF
Table 1 and Table 2 show the memory mapping for the 31 hardware sockets in the virtual socket platform.
Note in Figure 1 that the memory is allocated into four distinct sections: 1) read-only register file; 2) write-only
register file; 3) read-only block RAM; and 4) write-only block RAM. The allocation size for each type of
memory for every HW socket is 512 bytes.

6.4 Hardware Accelerator Interface
Figure 3 shows a typical block diagram of a hardware accelerator socket. Note that input and output block
RAMs are provided for input and output data while important flags are mapped to the register file sections,
such as start and finish flags.

Figure 3 — Block Diagram of Typical Hardware Accelerator.

When a hardware socket is selected for a particular transaction, one of its strobes will be asserted. It is up to
the user’s particular socket designs whether register or memory regions will be treated differently, however in
most cases their behaviour may be identical. The necessary signals to interface to the virtual socket with
respect to the hardware accelerator socket are shown in Table 3 below.
6 © ISO/IEC 2009 – All rights reserved

Table 3 — Hardware Accelerator Socket Interface.
Signal Length Direction* Polarity Description
Globals <2>
Clk 1 Input R hardware accelerator socket clock
global_reset 1 Input H global reset

Strobes <4>
strobe_reg_read 1 Input H read-only register space selected
strobe_reg_write 1 Input H write-only register space selected
strobe_ram_read 1 Input H read-only memory space selected
strobe_ram_write 1 Input H write-only memory space selected

Write Signals <50>
write_addr 16 Input  write address
data_in 32 Input  data to write into socket
write_valid 1 Input H data_in is valid
socket available to take more write
write_rdy 1 Output H data
Read Signals <49>
read_addr 16 Input  read address
data_out 32 Output  data for read operation
strobe_out 1 Output H data_out has requested data

External Memory
Manager <92>
ZBT_ReadEmpty 1 Input H read fifo is empty
ZBT_WriteFull 1 Input H write fifo is full
ZBT_ack_job 1 Input H job to memory manager accepted
ZBT_wf_grant 1 Input H write fifo access granted
ZBT_rf_grant 1 Input H read fifo access granted
ZBT_ReadData 32 Input  data read from external memory
ZBT_issue_job 1 Output H issue job to memory manager
ZBT_rwb 1 Output H job is read = '1' or write = '0'
ZBT_popfifo 1 Output H retrieve word of data from read fifo
ZBT_pushfifo 1 Output H place data onto write fifo
address to access data to in
ZBT_addr 19 Output  external memory
ZBT_dpush 32 Output  data to send to external memory

The user may optionally connect their device to the external memory manager that allows access to either the
ZBT SRAM or DDR DRAM (see subclause 6.4.2). The block move and external block move example VHDL
modules demonstrate the basic interface to the virtual socket (see subclause 6.5). Hardware socket
designers are strongly encouraged to read this section and use them as building blocks for their own sockets.
6.4.1 Transferring Data To/From a Socket
When a socket detects a write access to it, it should check to see if the write_valid signal is asserted. This
signal will indicate that the data present on Data_In is valid and ready to be processed by the socket.
Whenever the user is capable of taking data from the virtual socket interface it should drive its write_rdy signal
high. This will bring new data to it from the interface. Register or Memory writes to a socket may be multiple
words. The write_rdy signal provides a flow control mechanism back to the virtual socket interface. Below are
two waveforms demonstrating example writes to register and memory space.
© ISO/IEC 2009 – All rights reserved 7

Figure 4 — Timing Diagram of Register Write Operation.

Figure 5 — Timing Diagram of Memory Write Operation.

To perform a read transaction the user should observe a register or memory read strobe asserted. The user
has up to 16 clocks to respond to the read transaction by providing the data requested at the read address on
data_out and asserting the strobe_out signal. Read operations only return a single word. Samples waveforms
are provided below in Figure 6 and Figure 7:

8 © ISO/IEC 2009 – All rights reserved

Figure 6 — Timing Diagram of Register Read Operation.

Figure 7 — Timing Diagram of Memory Read Operation.

The hardware socket should implement a simple state machine as shown below in Figure 8.
© ISO/IEC 2009 – All rights reserved 9

Figure 8 — State Machine of Hardware Accelerator.

6.4.2 External Memory Interface
The external memory manager allows for three hardware accelerator sockets to access the two external
memories contained on the WildCard-II, ZBT SRAM and DDR DRAM. The manager arbitrates access to the
memory using a round-robin decision method. Sockets requesting access to static or dynamic RAM must first
issue a job request to the manager and the manager will respond with an acknowledgement. The socket
should then wait until the manager returns either a write or read grant depending on the requested operation.
Once a read or write grant is asserted, the socket should either withdraw or deposit (read or write
respectively) from the manager. The accesses are for single words only so if a socket requires multiple words,
it must request multiple jobs with the manager. An example state machine is provided in Figure 9
demonstrating how the user’s socket should interface with the memory manager.
10 © ISO/IEC 2009 – All rights reserved

Figure 9 — State Machine of External Memory Interface.

The addressing to the external memories is shown in Figure 10. Note that the user writes a start address
value into register location 1 of the master socket. The least-significant nine bits of the memory write value
sent to the master socket is then added to the start address register to obtain the final address sent to the
external memory.
© ISO/IEC 2009 – All rights reserved 11

Figure 10 — Addressing Technique Used to Access External Memories.

6.5 User Hardware Accelerator Sockets
Developers of their own hardware accelerator socket should follow the examples listed below to instantiate
and activate their own sockets. The VHDL source code for these two examples is provided with the platform
and is already pre-connected to the platform. Users are encouraged to copy these examples to use as a
template for their own accelerator design. The virtual socket interface is comprised of two major modules that
cannot be altered by the user – the master socket and memory manager. These modules must be present to
allow software to access external memory as well as provide status information back to control software.
6.5.1 Block Move
The block move example connects to the virtual socket interface connections and performs a very simple task
- copying the contents of one internal RAM to another. Specifically the block move example copies a region of
the Write-Only memory to the Read-Only memory. The example watches for a write to its Start register and
begins a loop reading from one memory and writing to the other. Once the block move module has finished
this task, it sets its Valid register high. Figure 11 and Figure 12 show the interface and internal block diagram
of the block move example, respectively. Note that the interface matches the basic interface listed in Table 3.

Figure 11 — Interface of Block Move Example.

12 © ISO/IEC 2009 – All rights reserved

Figure 12 — Block Diagram of Block Move Example.

6.5.2 External Memory Block Move
The external block move example VHDL module performs a copy from one region of ZBT SRAM memory to
another. Figure 13 shows the interface of this block. It demonstrates the additional connectivity to the
external memory manager. This socket provides additional registers to configure the source and destination
addresses of the copy and the length of the transfer. A write to the Start register begins the copy and the
status of the transfer is provided in the Status register. The number of words moved is present in the high bits
of the Status register and the current state of the socket in the lower three bits. When the transfer has
completed, the number of words should match the requested length to copy and the state should be all zeroes
indicating it has finished.
Figure 13 — Interface of External Block Move Example.
© ISO/IEC 2009 – All rights reserved 13

7 Integrated Framework supporting the “Virtual Socket” between HDL modules
described in Part 9 and the MPEG Reference Software (Implementation 2).
7.1 Introduction
The aim of this clause is to document the framework developed at the University of Calgary for the integration
of HW modules with the MPEG-4 reference software.
The clause is formatted in the form of a tutorial that takes the reader step by step through a typical HDL
module development case. After the introduction, two examples are described. The first one permit to transfer
from host data into FPGA internal memory block, then to do a simple process and then the data are
transferred back to host. The second example is similar, but instead of data processing the data are stored
into a internal fifo (designed with FPGA memory blocks) and straight away read back by the host. For both
examples, a testbench is descripted to test the two IPs and to help user to understand how host’s command
words could be interpreted and the processing performed. The following shows the integration of these two
examples into the platform and how host controls these IPs. Then it is shown how to use API function and
how to use the debug platform.
A typical data follow for an IP accelerator is: The data is transferred from the host computer to FPGA memory
block (BRAM) or SRAM memory, (using either direct transfer or DMA protocol) the processing is done, then
the results are stored respectively into BRAM or SRAM or (DRAM which is not presented in this document)
depending on the source of the original data. The computer host initiates and controls (through software calls)
the IP with an IP controller (designed by the user). Different types of controller are detailed into the next
sections. The system includes two parts (layers): A software part in C/C++ and a Part in HDL that simulate the
IP block. Both share the operation to perform the functionality required. In all examples of this document, it
was chosen that the S/W layer receives and responds to the all interrupts (act as Interrupt management). Also
the S/W controls the start and acknowledges the operation of the H/W module.
7.2 Development Example of a Typical Module : Calc_Sum_Product Module
In this example, The data are transferred from the host computer to FPGA memory block (BRAM) memory
with no DMA, nor SRAM the processing is done, then the results are stored respectively into BRAM. The host
initiates and controls the IP with an IP controller. The controller is into the VHDL file calc_sum_product.vhd.
entity calc_sum_product is
generic(no_of_inputs: positive:=64);
port(
reset:  in std_logic;
clock:  in std_logic;
--------------------------------------------
module_ready:  out std_logic;
input_available: in std_logic;
datain:  in std_logic_vector(7 downto 0);
--------------------------------------------
output_available: out std_logic;
dataout:  out std_logic_vector(15 downto 0);
---------------------------async transfers--
async_in: in std_logic;
async_out: out std_logic
);
end calc_sum_product;
Figure 14 — Interface of a Typical Module.

The specifications of this calculator:
1. Takes in 64 words-8 bits each, one word at a time (asynchronous transfer), each word is stored
into FPGA memory block.
2. It calculates the sum and the product of each two consecutive words. The sum and the product
are separately stored in 16-bit words.
14 © ISO/IEC 2009 – All rights reserved

3. When the calculation for all the input is done, it begins to output the 64 results, one word at a time,
then waits for new input.
4. The interface signals are:
a. Reset. (Input)
b. Clock. (Input)
c. module_ready. (Output) (the module is ready for receiving new input stream)
d. Input_ready (Input) (signals start of the input stream)
e. Output_ready. (Output) (signals start of the output stream)
f. Data_in. (Input)
g. Data_out. (Output)
h. Async_in (Input), Async_out (Output) asynchronous data transfer handshake signals.
They are used in inverse sense when data source and data destination switch roles.
Assumptions:
a. Reset is active high.
b. System is positive edge triggered.
c. One clock delay between data and handshake signals.

The second step is to have this module tested by preferably a testbench file similar to the following one shown
in Figure 15.
------------------------------------------------------------
-- example testbench for the example calculator module
-- Tamer Mohamed and Wael Badawy ([e-mail])
-- University of Calgary
------------------------------------------------------------

library ieee;
use ieee.std_logic_1164.all;
use ieee.std_logic_arith.all;
entity tb_calc_
...

TECHNICAL ISO/IEC
REPORT TR
14496-9
Third edition
2009-02-01
Information technology — Coding of
audio-visual objects —
Part 9:
Reference hardware description
Technologies de l'information — Codage des objets audiovisuels —
Partie 9: Description de matériel de référence

Reference number
©
ISO/IEC 2009
PDF disclaimer
PDF files may contain embedded typefaces. In accordance with Adobe's licensing policy, such files may be printed or viewed but shall
not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading a PDF file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create the PDF file(s) constituting this document can be found in the General Info relative to
the file(s); the PDF-creation parameters were optimized for printing. Every care has been taken to ensure that the files are suitable for
use by ISO member bodies. In the unlikely event that a problem relating to them is found, please inform the Central Secretariat at the
address given below.
This CD-ROM contains:
1) the publication ISO/IEC TR 14496-9:2009 in portable document format (PDF), which can be viewed
using Adobe® Acrobat® Reader;
2) electronic attachments relevant to reference hardware for coding of audio-visual objects.
Adobe and Acrobat are trademarks of Adobe Systems Incorporated.

This third edition cancels and replaces the second edition (ISO/IEC TR 14496-9:2008), which has been
technically revised.
©  ISO/IEC 2009
All rights reserved. Unless required for installation or otherwise specified, no part of this CD-ROM may be reproduced, stored in a retrieval
system or transmitted in any form or by any means without prior permission from ISO. Requests for permission to reproduce this
...