ISO/IEC 23004-4:2007
(Main)Information technology — Multimedia Middleware — Part 4: Resource and quality management
Information technology — Multimedia Middleware — Part 4: Resource and quality management
ISO/IEC 23004-4:2007 specifies the interfaces of the support application programming interface and the realization technology used for resource management in MPEG Multimedia Middleware (M3W). Resource management is an optional framework for M3W platforms. ISO/IEC 23004-4:2007 specifies entities and interfaces for resource budget creation, assignment and removal, entity and interfaces for assessing the feasibility and selecting resource configurations (resource configuration = set of assigned budgets), interfaces implemented by quality-aware entities (Quality-aware entities can provide multiple quality levels and know the resource needed to provide each quality level.), entity and interfaces for coordination of the "budget--quality level" negotiation (includes interfaces for registration and setting priorities).
Technologies de l'information — Intergiciel multimédia — Partie 4: Management des ressources et de la qualité
General Information
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 23004-4
First edition
2007-09-15
Information technology — Multimedia
Middleware —
Part 4:
Resource and quality management
Technologies de l'information — Intergiciel multimédia —
Partie 4: Management des ressources et de la qualité
Reference number
ISO/IEC 23004-4:2007(E)
©
ISO/IEC 2007
---------------------- Page: 1 ----------------------
ISO/IEC 23004-4:2007(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2007
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2007 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 23004-4:2007(E)
Contents Page
Foreword .iv
Introduction.v
1 Scope.1
2 Organization of this document .1
3 Normative references.2
4 Terms and definitions .2
5 Overview of interface suites.2
5.1 Introduction.2
5.2 Resource management.2
5.3 Quality management .3
6 Resource management interface suite .6
6.1 Overview.6
6.2 Types and constants.6
6.3 Logical component.20
6.4 Roles.20
6.5 Interfaces.21
7 Overview of realization .29
8 Resource management.30
8.1 Overview.30
8.2 Responsibilities of roles in the Resource Management Framework .30
8.3 Realization of a resource chief .31
8.4 Composition of quality information.34
Annex A (informative) Dynamic view of the Resource and Power Management Framework.36
Annex B (informative) CPU chief details .38
Annex C (informative) Approach to power management.40
Annex D (informative) Composition of quality information example .43
Bibliography.47
© ISO/IEC 2007 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 23004-4:2007(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 23004-4 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information.
ISO/IEC 23004 consists of the following parts, under the general title Information technology — Multimedia
Middleware:
⎯ Part 1: Architecture
⎯ Part 2: Multimedia application programming interface
⎯ Part 3: Component model
⎯ Part 4: Resource and quality management
⎯ Part 5: Component download
⎯ Part 6: Fault management
⎯ Part 7: System integrity management
iv © ISO/IEC 2007 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 23004-4:2007(E)
Introduction
MPEG, ISO/IEC JTC 1/SC 29/WG 11, has produced many important standards (MPEG-1, MPEG-2, MPEG-4,
MPEG-7, and MPEG-21). MPEG feels that it is important to standardize an application programming interface
(API) for Multimedia Middleware (M3W) that complies with the requirements found in the annex to the
Multimedia Middleware (M3W) Requirements Document Version 2.0 (ISO/IEC JTC 1/SC 29/WG 11 N 6981).
The objectives of Multimedia middleware (M3W) are to allow applications to execute multimedia functions with
a minimum knowledge of the middleware and to allow applications to trigger updates to the middleware to
extend the middleware API. The first goal can be achieved by standardizing the API that the middleware offers.
The second goal is much more challenging, as it requires mechanisms to manage the middleware API and to
ensure that this functions according to application needs. The second goal can support the first, by reducing
the needed standard APIs to those that provide middleware management. Consequently, applications can use
these standard management APIs to generate the multimedia system they require.
The aim of M3W is to define a component-based middleware layer in high-volume embedded appliances that
enables robust and reliable operation. These types of product are heavily resource constrained, with a high
pressure on silicon cost and power consumption. In order to be able to compete with dedicated hardware
solutions, the available resources will have to be used very cost-effectively, while enabling robustness and
meeting stringent timing requirements imposed by high-quality elements such as digital audio and video
processing.
High-volume embedded appliances are considered business-critical devices. Their failure can have important
economic implications for the producing company. Hence, they have stringent and demanding quality
requirements. Unfortunately, in the software industry misbehaving products are commonplace. However, this
is not the case with consumer electronics, home appliances and mobile devices. Users are accustomed to
robust and well-behaved devices.
In consumer electronics, there are demanding quality requirements and a need to use the available hardware
resources cost-effectively. The term “Application” is used to refer to the software entity that (indirectly)
provides certain functionality to an end-user; the Application may include Service Instances that are bound to
it. In order to ensure that an Application provides the correct service, it needs to be assigned the hardware
resources it requires. This can be achieved by assigning it the “worst case” resources required. However, with
many applications, the worst-case resource usage is much less than the mean case. If this worst-case
approach is followed, hardware resources will be wasted. On the other hand, problems may arise if the
running applications require more resources than can be made available. If the platform follows a typical fair
policy for the assignment, the behaviour of the system will be unpredictable, especially for applications that
must provide results according to some specified time constraints.
The goal of Resource Management is to assign budgets or resource reserves to Applications. These budgets
are guaranteed, so that they are available in any situation. This approach helps to enhance system
robustness, because an Application is unable to affect the resource reserves of others.
In order to benefit from Resource Management, Applications participating in Resource Management should be
resource-aware (RA). This means that they are aware of the resources they need during their execution and
should adapt their behaviour to the resources which are made available. This ensures that applications can
function correctly without exhausting all of the system resources that have been allocated to them.
Quality-Aware (QA) Applications are RA applications that are aware of the quality of service that they deliver.
Typically, they are capable of providing different quality levels. They are characterized by the quality they
provide and the resources needed for this purpose. Usually the higher the quality level, the higher the
resource needs. This type of Application is able to dynamically change the provided quality level, depending
on the assigned budgets.
© ISO/IEC 2007 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC 23004-4:2007(E)
Often, QA and RA applications are real-time applications. The assigned budgets allow them to provide a
suitable output within some time interval. A hard real-time application can be viewed as QA with only two
quality levels: maximum quality or nothing. The Resource Management framework can also deal with Non
Resource-Aware Applications (NRA) in the sense that they are assigned a fixed budget all together.
The relation between the Resource Management framework and QA applications is based on a contract
model. The system provides resources and the Applications commit to generate the required results (outputs)
with a specific and stable quality. Budget assignment must be obtainable, which means that it should not be
possible to assign more resources than those actually available. Contracts are negotiated with the
Applications with the goal of maximizing overall system quality, as perceived by the user. In this process, the
“importance” of the Applications is a primary parameter for this operation.
Power is considered to be one of the most important resources to be managed in the next generation of
consumer electronics. Its importance is clear for mobile devices, where the goal is to maximize battery life. In
stationary devices, it is also relevant for environmental conditions, fan-less operation and to increase the
lifetime of the silicon devices. In addition, power management is related to heat control, so that the
temperature of different parts of the device can be maintained within a certain threshold.
The basic goals in M3W with respect to power management are as follows.
⎯ Reduce power consumption.
⎯ Increase battery life, for mobile devices.
⎯ Ensure a system-wide limit for heating: If it is detected that the temperature of the device it is too high, it
may be required to reduce it.
⎯ Take advantage of power-aware hardware and M3W components.
⎯ Ensure that the system and relevant Applications are always executed, in spite of the power saving policy
selected.
Power management is very much related to resource management. A number of techniques for reducing
power consumption are based on moving hardware components to less power-consuming modes of operation,
which implies reducing/modifying the available resources (CPU capacity, memory size, bandwidth, etc.). This
is the case, for example, with CPU voltage and frequency scaling, which can reduce power consumption and
consequentially, CPU computational power. For this reason, it is desirable to integrate power management
with quality and resource management.
The Resource Management Framework is in charge of achieving the functionality that has been sketched in
this introductory clause. Its functions and architecture are defined in this part of ISO/IEC 23004.
vi © ISO/IEC 2007 – All rights reserved
---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 23004-4:2007(E)
Information technology — Multimedia Middleware —
Part 4:
Resource and quality management
1 Scope
This part of ISO/IEC 23004 defines the Resource and Quality Management framework of the MPEG
Multimedia Middleware (M3W) technology.
2 Organization of this document
This part of ISO/IEC 23004 has the following high level structure:
⎯ Clause 1 defines the scope of this part of ISO/IEC 23004.
⎯ Clause 3 gives an overview of documents that are indispensable for the application of this part of
ISO/IEC 23004.
⎯ Clause 4 gives the terms and definitions used in this part of ISO/IEC 23004.
⎯ Clause 5 provides an overview of and introduction to the Resource and Power Management Framework.
⎯ Clause 6 contains a detailed specification of the interfaces of the Resource and Power Management
Framework that are part of the M3W API.
⎯ Clause 7 gives an overview of the realization of the Resource and Power Management Framework.
This part of ISO/IEC 23004 has the following annexes:
Annex A, Dynamic view of the Resource and Power Management Framework, describes the interactions
between the different entities of the Resource and Power Management Framework.
Annex B, CPU chief details: The Resource and Power Management Framework distinguishes four different
roles that are together responsible for Resource and Power Management:
1) resource chief;
2) resource manager;
3) quality chief;
4) quality manager.
The CPU chief is discussed in this annex. The CPU chief is a specialization of a resource chief. This is
the chief responsible for monitoring and enforcing the CPU budgets.
© ISO/IEC 2007 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC 23004-4:2007(E)
Annex C, Approach to power management: Power is a resource as well. This annex discusses managing this
specific resource.
Annex D, Composition of quality information example: Often we need to manage the Resource Consumption
and delivered Quality of a composition of entities instead of a single atomic entity. This annex discusses how
to compose the Quality and Resource information in such cases.
3 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO/IEC 23004-1, Information technology — Multimedia Middleware — Part 1: Architecture
4 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO/IEC 23004-1 apply.
5 Overview of interface suites
5.1 Introduction
This is an informative clause that gives an overview of the API specification of the resource management
framework.
5.2 Resource management
The optional M3W Resource Management Framework provides timely, guaranteed and protected access to
system resources. These functions allow Applications to specify their resource demands, leaving the
Resource Management Framework to satisfy those demands using multimedia platform specific resource
management schemes. In this situation, two models (or views) can be defined:
⎯ Resource specification models: Applications and Services must specify the budget or resource share they
need. For this purpose, the runtime provides a certain model and an associated interface. If it is feasible,
a given resource share can be committed to these entities.
⎯ Resource management models: The runtime must provide a means for accounting, enforcing and
monitoring resource usage. In this way, it is possible for the runtime to guarantee the committed budgets
and to provide useful information to applications so that they can adapt their behaviour to their current
resource shares / budgets.
These two models are somewhat independent. For a given specification model, a number of management
models can be implemented to satisfy it, and vice versa.
The resource specification model is based on assigning resource shares or budgets to the Applications. As an
example, the specification model for handling memory and CPU is now described:
⎯ Memory: The corresponding entities should make a claim for a certain amount of memory. The manager
should respond whether the grant can be approved or not. Ideally, these entities should request all the
memory they will need for their entire execution lifetime. If they dynamically claim memory, it could be that
in the middle of its execution there is no additional free memory, and the program is then unable to
continue with its execution. On the other hand, it may be difficult for an application to know in advance the
exact memory needed, and it may ask for much more than it really needs. In any case, memory feasibility
tests are very simple and can be made frequently, with little overhead.
2 © ISO/IEC 2007 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC 23004-4:2007(E)
⎯ CPU: The specifying model is based on requesting a certain CPU share (percentage) for a given period
of time. This budget is refilled periodically. CPU budgets can either be allocated to a thread or to a group
of threads (also named a cluster). For example, a thread may be allowed to use the CPU for 1ms every
10ms (budget is 1 ms and the refill period is 10ms). In the context of this specification, a thread is the unit
of concurrency.
Resource shares are assigned by the Resource Management framework and shall be enforced during system
operation. For this purpose, it is necessary to account for resource usage and take some action if a budget
user tries to use more resources than budgeted. If an Application requests more resources, it should be
checked whether this is possible, given the current system situation. If there are available resources, the
request may be satisfied. Otherwise, it should be denied or a new system assignment should be made.
QA Applications and Services have static information on their quality levels and their associated resource
needs. In the case of the CPU, the required budget is an estimation that is e.g. calculated by measurements
taken when running the program with a meaningful set of input streams and data. In some domains, e.g.
multimedia, the real resource needs depend on the type of data, so it is difficult to provide accurate
estimations. Applications may dynamically adapt and update their required budget estimations for use in
future negotiations. The same considerations can be made with respect to memory, although typically in this
case the required memory is subject to less variation, and estimations can be made more accurately.
An Application or Service may provide the required quality level with different sets of resources. For example,
it can use a lot of CPU and little memory or only a little CPU but with plenty of memory to deliver the same
quality level. In the context of the Resource Management Framework the set of resources that an Application
requires for a certain quality level is called a configuration. A quality level can have several configurations
associated. The Resource Management framework is in charge of deciding which configuration will be
selected, depending on the needs of the applications and the available resources in the system.
In a M3W system, Resource Aware and Non-Resource Aware applications can co-exist. The approach for
dealing with this situation is to divide the system resources into two partitions:
⎯ RA: This partition is composed by all Resource Aware Applications and Services. The QM has assigned
them a certain budget, which shall be guaranteed, for the execution (until a re-negotiation).
⎯ NRA: This partition includes the rest of resources and it is used for the execution of the Non-Resource
Aware Applications and Services.
One possible way of implementing this division is by assigning priorities to RA threads (those that belong to
RA Applications or RA Services) that are higher than those assigned to NRA elements. When the budget of a
RA element is exhausted, its priority could be reduced. When the budget is replenished, the priority is raised.
A real-time operating system, with appropriate scheduling policies, is required for the execution of the RA
threads. The scheduling of NRA threads can be done using a classical quantum scheduler.
Power management is an important aspect of resource management, as the available system resources
depends on the power settings of the corresponding hardware components. If the power settings are changed,
(for example to reduce consumption, then the negotiated resource shares cannot be kept, and the contract
may be broken. For this reason, the approach is to consider power as another resource, although with special
characteristics, and it is managed together with the rest of the system resources. More details on power
management can be found in Annex C.
5.3 Quality management
The purpose of quality management is to drive the interaction with the QA Applications, so that the quality of
the overall system is maximised. A system may be composed of a set of QA Applications. Where each
application can provide a set of quality levels. A quality level is characterized by a description of the provided
quality and by the required resources for this purpose. The problem is to decide the quality level for each
Application, and as a consequence the resource assignment, so that system quality is maximized and the
resource assignment is feasible.
© ISO/IEC 2007 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/IEC 23004-4:2007(E)
The criteria for making this selection vary from system to system. The information that has to be considered
includes:
⎯ Importance of Applications: There are applications that are fundamental in a particular device and should
always provide some minimal quality, such as the handling of incoming calls in a mobile phone.
Information from the execution context is also useful for identifying the important Applications. In a system
with multiple windows, it is usually the case that the Application running in the window with the user focus
is the most important.
⎯ User settings: The goal is to maximize user satisfaction. Hence, it is important to consider user settings.
⎯ Domain knowledge: This information is used in the decision process, e.g. to decide which Application is
most important and has to receive more resources.
The quality management functions do not finish when a contract is negotiated. It is necessary to monitor the
system to handle situations where the system is overloaded or has free resources. The resources required by
Applications are usually estimations. In multimedia applications, the required resources strongly depend on
the input data. In particular, the computation time may depend on whether the scene is static or with a lot of
action and on the compression algorithm used. Hence, it can be that an Application requires more resources
or has more than needed. In this situation, there should be mechanisms for its correction. As a consequence,
the resources needed for the quality level may change and the contracts must be (at least partially)
renegotiated.
The operations that quality management provides can be described as follows:
⎯ Reservation is the most basic mechanism for providing QoS guarantees. It consists of assigning shares
of resources (budgets) to Applications and guaranteeing them based on runtime monitoring and
enforcement of their usage. Most of the functions related to reservation are performed as part of resource
management.
⎯ Negotiation can be defined as the operation for creating a new contract (or modifying the current one)
between the system software and the different Applications. This long-term contract determines a set of
minimum requirements for both sides. The system guarantees a minimum budget, and the Application
agrees on executing the functionality required by the user providing, at least, a minimum utility. The
process is based on having a dialogue between both domains to check whether new settings are feasible
in the system. Differentiating negotiation (feasibility test) from the actual settings into two different phases
is useful when we want to check the feasibility of several mutually exclusive configurations, and then
select one of them. Negotiation can be executed at Application (or Service) start-up, or dynamically when
it is detected that the current contract is no longer valid. This renegotiation may be in response to either
external or internal events. External events are those signalled externally to the system, while internal
events are those detected by runtime monitoring.
⎯ Setting is the operation of making the system configuration resulting from a negotiation operational. As
part of this procedure it is necessary to communicate to the Applications their executing quality level and
to set the resource budgets, by using the facilities provided by the resource manager. This configuration
change has to be done following a certain procedure that depends on how abrupt the changes can be
made. Some Applications require a smooth change, so it is necessary to follow a protocol that ensures
this requirement. For example, Applications that reduce their quality level may have to change before
others raise theirs, in order to ensure that budgets are guaranteed during the setting operation.
⎯ Monitoring is the process in charge of obtaining information about the execution of Applications and their
threads, in order to know how they behave. This information includes (i) resource usage made by
Applications and threads, (ii) progress monitoring (usually in the form of milestones reached or
percentage of processing finished), (iii) adaptation measurements (performance achieved by threads, and
data dropping made by Applications), and (iv) application domain metrics (utility provided as a function of
provided QoS parameters and QoS violations).
⎯ Adaptation is the operation of responding to transient overloads, detected at runtime, by performing some
short-term changes. The goal is to adapt the processing algorithm in order to have a near-constant
4 © ISO/IEC 2007 – All rights reserved
---------------------- Page: 10 ---
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.