Information Technology — Data centres — Application Platform Energy Effectiveness (APEE)

This document specifies application platform energy effectiveness (APEE) as a Key Performance Indicator (KPI) which quantifies the energy effectiveness of an application platform for an IT service in data centres. This KPI evaluates the energy consumption of an application platform prior to deployment. The purpose of this KPI is to measure the energy effectiveness of a set of target IT equipment, operating systems and middleware, to enable the selection of an energy effective IT stack. This document specifies a formula for calculating APEE and definitions of components of the formula. This document specifies a measurement method for assessing and reporting the energy effectiveness of an application platform. This document also specifies requirements for benchmarks to be used for APEE and requirements for reporting. The following topics are outside of the scope of this document: 1) KPIs intended to solely evaluate the energy effectiveness of target IT equipment hardware, 2) energy effectiveness of data centre facilities.

Technologies de l'information — Centres de traitement de données — Efficacité énergétique des plateformes d'applications (APEE)

General Information

Status
Published
Publication Date
14-Jun-2021
Current Stage
6060 - International Standard published
Start Date
15-Jun-2021
Completion Date
15-Jun-2021
Ref Project

Buy Standard

Standard
ISO/IEC 23544:2021 - Information Technology -- Data centres -- Application Platform Energy Effectiveness (APEE)
English language
23 pages
sale 15% off
Preview
sale 15% off
Preview
Draft
ISO/IEC FDIS 23544:Version 06-mar-2021 - Information Technology -- Data centres -- Application Platform Energy Effectiveness (APEE)
English language
23 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

INTERNATIONAL ISO/IEC
STANDARD 23544
First edition
2021-06
Information Technology — Data
centres — Application Platform
Energy Effectiveness (APEE)
Technologies de l'information — Centres de traitement de données —
Efficacité énergétique des plateformes d'applications (APEE)
Reference number
ISO/IEC 23544:2021(E)
ISO/IEC 2021
---------------------- Page: 1 ----------------------
ISO/IEC 23544:2021(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2021

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2021 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 23544:2021(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction ..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms, definitions and abbreviated terms ................................................................................................................................ 1

3.1 Terms and definitions ....................................................................................................................................................................... 1

3.2 Abbreviated terms ............................................................................................................................................................................... 3

4 Relevance of APEE ............................................................................................................................................................................................... 3

5 Determination of APEE .................................................................................................................................................................................. 4

6 Determination of application outcome and energy consumption of the application

platform for APEE ................................................................................................................................................................................................ 4

6.1 Prerequisites for the benchmark and application platform ............................................................................ 4

6.2 Determination of application outcome .............................................................................................................................. 5

6.3 Determination of energy consumption of the application platform......................................................... 5

6.3.1 Requirements for obtaining energy consumption of the application platform......... 5

6.3.2 Requirements for the measurement method for obtaining energy consumption .. 6

7 Reporting of APEE ............................................................................................................................................................................................... 6

7.1 Requirements ........................................................................................................................................................................................... 6

7.2 Recommendations ............................................................................................................................................................................... 7

Annex A (informative) Use cases of APEE ........................................................................................................................................................ 8

Annex B (informative) Example of APEE calculation ........................................................................................................................10

Annex C (informative) Example of an APEE report .............................................................................................................................11

Annex D (informative) Examples of acceptable benchmarks ..................................................................................................21

Bibliography .............................................................................................................................................................................................................................23

© ISO/IEC 2021 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 23544:2021(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that are

members of ISO or IEC participate in the development of International Standards through technical

committees established by the respective organization to deal with particular fields of technical

activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international

organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the

work.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for

the different types of document should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives or www .iec .ch/ members

_experts/ refdocs).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

list of patent declarations received (see patents.iec.ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/

iso/ foreword .html. In the IEC, see www .iec .ch/ understanding -standards.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 39, Sustainability, IT & Data Centres.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html and www .iec .ch/ national

-committees.
iv © ISO/IEC 2021 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 23544:2021(E)
Introduction

The growth of the Internet of Things (IoT) has resulted in the storage of increasingly large amounts

of data in data centres and the increased utilization of this data by technologies such as artificial

intelligence (AI). As a result, energy consumption of data centres is also increasing. In this recent

utilization of large amounts of data by technologies such as AI, various kinds of data are combined and

analysed, and processing requests for such combinations and analysis are also increasing. Middleware,

such as a database management system (DBMS), is also becoming more important for this data

utilization. The energy effectiveness of application platforms can be greatly improved through the

selection of middleware.

In addition to improving the energy effectiveness of target IT equipment itself, it is necessary to improve

application platform energy effectiveness by choosing an optimally energy-effective combination of

target IT equipment, operating systems and middleware.

Although there are several KPIs for the energy effectiveness of target IT equipment itself (for example,

ISO/IEC 30134-4, ISO/IEC 21836), there have previously been no KPIs for energy effectiveness to

calculate the energy effectiveness of a combination of target IT equipment, operating systems and

middleware. This document, therefore, introduces a KPI for the energy effectiveness of an entire

application platform. A typical use case of application platform energy effectiveness (APEE) is a

criterion for procuring an energy efficient application platform for an IT service.

This KPI provides assistance in selecting an optimal application platform for energy effectiveness.

This KPI does not apply to the energy effectiveness of an entire data centre. The colloquial term of

"efficiency" is commonly used in regional programmes and "effectiveness" can be referenced as

"efficiency" in those programmes.
© ISO/IEC 2021 – All rights reserved v
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 23544:2021(E)
Information Technology — Data centres — Application
Platform Energy Effectiveness (APEE)
1 Scope

This document specifies application platform energy effectiveness (APEE) as a Key Performance

Indicator (KPI) which quantifies the energy effectiveness of an application platform for an IT service in

data centres. This KPI evaluates the energy consumption of an application platform prior to deployment.

The purpose of this KPI is to measure the energy effectiveness of a set of target IT equipment, operating

systems and middleware, to enable the selection of an energy effective IT stack.

This document specifies a formula for calculating APEE and definitions of components of the formula.

This document specifies a measurement method for assessing and reporting the energy effectiveness of

an application platform.

This document also specifies requirements for benchmarks to be used for APEE and requirements for

reporting.
The following topics are outside of the scope of this document:

1) KPIs intended to solely evaluate the energy effectiveness of target IT equipment hardware,

2) energy effectiveness of data centre facilities.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 21836:2020, Information technology — Data centres — Server energy effectiveness metric

ISO/IEC 21878, Information technology — Security techniques — Security guidelines for design and

implementation of virtualized servers
3 Terms, definitions and abbreviated terms
3.1 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 21878 and the following

apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
© ISO/IEC 2021 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC 23544:2021(E)
3.1.1
application platform

platform composed of target IT equipment (3.1.4), operating systems, and middleware (3.1.3) with the

purpose of providing an IT service (3.1.5)

Note 1 to entry: An application platform is an environment for executing software applications (3.1.2) such as

ERP and CRM software. Therefore, a software application itself is not included as part of an application platform.

3.1.2
software application
computer program for implementing an IT service (3.1.5)

Note 1 to entry: A software application is usually developed independently for each IT service.

3.1.3
middleware
software layer between an operating system and the software applications (3.1.2)

Note 1 to entry: Examples of middleware include DBMS, web server software, application server software,

transaction monitor.

Note 2 to entry: An operating system is the software directly managing the hardware, e.g. conventional operating

system, hypervisor, etc.

[SOURCE: ISO/IEC/IEEE 24765:2017, 3.2459, modified – Notes 1 and 2 to entry have been added.]

3.1.4
target IT equipment

server, storage and network equipment that are components of an application platform (3.1.1)

3.1.5
IT service
service that provides an outcome to its users via information technology
3.1.6
application outcome
information provided by an IT service (3.1.5) to its users

Note 1 to entry: An example of an application outcome is a set of reports to support decision-making at retail

outlets (such as sales analysis reports, and stock status reports).
3.1.7
measurement period
duration in which the benchmark performs the intended data processing

Note 1 to entry: Typically, the measurement period(s) is/are the only portion of the benchmark execution which

is/are used to calculate the benchmark score.

Note 2 to entry: The intended data processing routine is a routine including the benchmark calculation.

Note 3 to entry: In the case of using TPC-H , the measurement target of TPC-H is a performance for executing

SQL queries, so a period of executing SQL queries is a measurement period. Other periods, such as a data loading,

are not a measurement period.
3.1.8
benchmark identifier

string which identifies a specific benchmark, which consists of the name and the major version (3.1.9),

when applicable

1) TPC-H, TPC-E, and TPCx-IoT are trademarks of the Transaction Processing Performance Council. This

information is given for the convenience of users of this document and does not constitute an endorsement by ISO/

IEC of the product named. Equivalent products may be used if they can be shown to lead to the same results.

2 © ISO/IEC 2021 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 23544:2021(E)
3.1.9
version

particular form or variation of a resource that differs from other instantiations of the resource in at

least one aspect or item of information

Note 1 to entry: The first digit of a version is called the major version, and any numbers right of that digit is called

a minor version.

Note 2 to entry: Example: in the version designated as “v1.2.3,” the Major Version identification is “1”, and the

Minor Version identification is “2.3”.
[SOURCE: ISO 24619:2011, 3.1.9, modified — Notes 1 and 2 to entry added.]
3.1.10
internal power supply

independent device that converts commercial AC power to DC power as required inside target IT

equipment (3.1.4)
3.2 Abbreviated terms
AC alternating current
AI artificial intelligence
APEE application platform energy effectiveness
CRM customer relationship management
DBMS database management system
DC direct current
EAP energy consumption of application platform during the measurement periods
ERP enterprise resource planning
FC fibre channel
IoT Internet of Things
NoAO number of application outcomes during an APEE measurement
OS operating system
RDBMS relational database management system
4 Relevance of APEE

Many kinds of IT services are provided by using data centres, and it is necessary to conserve the

amount of energy used by these IT services. A lot of IT services use various technologies, such as AI, IoT

and big data technologies. For each of these technologies, there are several combinations of target IT

equipment, OSs and middleware that can be used to provide IT services. These combinations of target

IT equipment, OSs and middleware are called an application platform.

Even if the results provided by an IT service are the same, if the application platform used to provide

the IT service is different, the energy consumption is also different. For example, when the same query

is executed on different DBMSs, because the processing method and processing time are different for

each DBMS, energy consumption also differs.
© ISO/IEC 2021 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC 23544:2021(E)

In order to choose the optimum application platform for energy effectiveness, a KPI for measuring the

energy effectiveness of an application platform is necessary. There are KPIs for measuring the energy

effectiveness of target IT equipment only, and ISO/IEC 21836 is one of KPIs for a server. APEE is a KPI

for measuring the energy effectiveness of an application platform. Use cases of APEE are described in

Annex A.

NOTE APEE is a KPI for measuring the energy effectiveness of an application platform for a particular IT

service, not the energy effectiveness of an entire data centre. Therefore, APEE is not suitable for use in regulations

that apply to an entire data centre.
5 Determination of APEE
APEE shall be determined and described by Formula (1):
c = a / b (1)
where

a is the number of application outcomes during an APEE measurement (NoAO) (see Clause 6);

b is the energy consumption (kWh) of the application platform during the measurement

periods (EAP) (see Clause 6);
c is the APEE.

The unit dimension of APEE is (application outcome(s)/kWh). The APEE value should have at least 2

significant digits.
Reporting of data for APEE is described in Clause 7.
An example of APEE calculation is given in Annex B.
6 Determination of application outcome and energy consumption of the
application platform for APEE
6.1 Prerequisites for the benchmark and application platform

The following prerequisites shall apply when measuring the APEE of an application platform that

provides an IT service:

a) The measurer determines one benchmark to be used for the calculation of APEE. Benchmarks shall

meet the following requirements. Examples of benchmarks that can be used for APEE measurement

are given in Annex D.
1) The benchmark is representative of the target IT service.
2) The start and the end of the execution of the benchmark are clear.
3) All iterations of the benchmark run produce comparable results.

4) An application outcome defined by the benchmark is identical regardless of how many times it

is executed on any application platform.

NOTE 1 In the case of TPC-H as an example of an application outcome, it defines a set of results of

business-oriented ad-hoc queries as an application outcome of the execution. A performance metric

such as throughput and processing time is not an application outcome.
4 © ISO/IEC 2021 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 23544:2021(E)

EXAMPLE 1 A benchmark that executes transactions while increasing the multiplicity until

the system cannot keep the specified response time in order to obtain maximum multiplicity is not

applicable because the processes of measurement target for these benchmarks have the potential to

change from run to run.

5) Start date and time, and end date and time of all the measurement period(s) are clear.

NOTE 2 Some benchmarks have multiple measurement periods in one benchmark execution.

EXAMPLE 2 (Example of clarification) A benchmark has warmup and cooldown phases which do not

effort a benchmark performance score. It records start date and time and end date and time of these

phases in a log file, so these phases are distinguishable from measurement period(s).

6) Measurement period start and stop time are measured in seconds or higher precision.

7) To ensure that the result is verifiable, the benchmark is made publicly available.

b) The measurer determines the target IT equipment, OSs and middleware that are to serve as

the elements of the application platform. The application platform shall meet the following

requirements:

1) The application platform contains all of the target IT equipment necessary to execute the

benchmark.
2) Every target IT equipment component can be measured for energy consumption.
6.2 Determination of application outcome

For the determination of application outcome, the following terms are introduced:

— One application outcome is defined as one valid execution of the benchmark.
The following requirements for measurement shall be met:

a) The benchmark is executed at least one time. If the benchmark is executed multiple times, the

number of benchmark executions is a positive integer.

NOTE The tester can interact with the server between benchmark iterations, typically to calculate the

total measured time for benchmarks which include unmeasured periods.

b) If the total time of the whole measurement period is less than one hour, the benchmark is repeatedly

executed until the total time of the whole measurement period exceeds one hour.

c) If the benchmark has a setting for official or valid or reportable results for publication or use in

comparisons, this mode shall be used for APEE comparisons.
6.3 Determination of energy consumption of the application platform
6.3.1 Requirements for obtaining energy consumption of the application platform

Energy consumption of the application platform shall be obtained by the sum of energy consumption

(kWh) of all the target IT equipment in the application platform during all benchmark measurement

period(s) of the APEE measurement. Non-benchmark software should be minimal during the

measurement period(s). Execution of non-benchmark software on the application platform shall not

invalidate energy results. Energy measurements during benchmark execution shall not be adjusted to

remove the effects of non-benchmark software.

NOTE The application platform is usually used solely by the benchmark during the measuring period for

accuracy.

The energy consumption of each item of target IT equipment in the application platform is obtained by

the measurement method. Details of this method are given in 6.3.2.
© ISO/IEC 2021 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC 23544:2021(E)
6.3.2 Requirements for the measurement method for obtaining energy consumption

The measurement method shall meet the following requirements. The measurer shall replace “SEEM”

or “SERTv2” with “APEE” and “SUT” with “application platform” in the text of the SEEM referenced

below and apply.
a) Environment
1) ISO/IEC 21836:2020, 7.2 applies for measurement environment.
2) ISO/IEC 21836:2020, 7.4 applies for power requirements.
b) Power measurement

1) One or more power analysers shall be used to measure the entire energy consumption of the

application platform.

2) Maximum sampling interval of the power analysers shall be at least one sample per second.

3) ISO/IEC 21836:2020, 7.3 and 9.2.2.2 apply for power analyser requirements.
4) Power analysers shall comply with ISO/IEC 21836:2020, 9.2.2.3 and 9.2.2.4.

5) The following requirements shall be preferentially followed over the above ISO/IEC 21836

requirements.

i) The energy consumption shall be measured by using one or more power analysers.

ii) All clocks of all power analysers and target IT equipment shall be synchronized such that

time stamps are aligned within 1 second.

NOTE This requirement is intended to support measurements of the energy consumption of

multiple target IT equipment. ISO/IEC 21836 targets a server, but this document targets multiple target

IT equipment.
c) Temperature measurement

1) The instantaneous temperature of an inlet of one typical element of the application platform

shall be measured by a temperature sensor both within 1 minute of the start time of the

benchmark execution and within 1 minute of the end time of the benchmark execution.

2) ISO/IEC 21836:2020, 9.2.3 applies for temperature sensor requirements.
d) Approved measuring instrument

1) ISO/IEC 21836:2020, Annex B, Table 4 and Table 5 apply for pre-approved power analysers and

temperature sensors.
7 Reporting of APEE
7.1 Requirements

When presenting an APEE, both the benchmark identifier and the APEE value shall be presented, and

the benchmark identifier shall be written in parentheses as follows:
APEE(benchmark identifier)

For all benchmarks which include version numbers, the corresponding benchmark identifier shall

include the benchmark's major version number.

EXAMPLE APEE is written as APEE(TPC-H v2) when TPC-H version 2.17.3 is used as the benchmark.

6 © ISO/IEC 2021 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 23544:2021(E)

When presenting APEE, APEE shall be presented along with a report or link to a report. The report shall

include at least the following information:
a) APEE value
b) NoAO value and EAP value
c) start date and time of measurement, and end date and time of measurement

d) start date and time of each measurement period, and end date and time of each measurement

period
e) target IT service
f) details of the selected application platform:
1) vendor names and model names of target IT equipment;

2) vendor names, model names, and version numbers of operating systems and middleware;

3) system configuration diagram of the target IT equipment, operating systems, and middleware;

NOTE 1 For an example of a configuration diagram, see Annex C.

4) all target IT equipment configuration and software settings that affect NoAO and EAP values

or are required to reproduce the results shall be documented, including load range, power

management settings, and virtualization resource allocation configuration.

NOTE 2 Examples of load range in DBMS are database size and transaction throughput.

g) information about the selected benchmark:
1) name and version;
2) reference to the specifications of the benchmark;

3) benchmark settings descriptions and values used for all benchmark settings that affect NoAO

and EAP values.

h) power supply information (e.g. power supply mode, power supplies 80 plus rating, AC/DC, voltage,

frequency, and phase)

i) instantaneous temperatures of the inlet at start and end of benchmark execution, and location of

the temperature sensor
j) information about the measurement method:
1) vendor names and model names of the selected power analysers;

2) physical locations of the power analyser(s) between the power source and the measured

systems, including servers, storage, and network equipment;

3) any deviations from the measurement procedure and any unusual features observed.

k) number, publication year, and title of this document, i.e. ISO/IEC 23544:2021, Information

Technology — Data centres — Application Platform Energy Effectiveness (APEE)
7.2 Recommendations
The report should include the following information:
a) purpose of measuring of the APEE value;
b) reason for selecting the benchmark.
© ISO/IEC 2021 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/IEC 23544:2021(E)
Annex A
(informative)
Use cases of APEE
A.1 Introduction
The life cycle of an IT service consists of the following phases:
a) planning,
b) procurement,
c) operation,
d) update,
e) disposal.

APEE can be effectively used in the procurement phase. This annex provides two use cases for the

procurement phase, from the perspective of the party procuring the IT service and from the perspective

of the party proposing the IT service. These parties fulfil the following roles in the use cases described

in this annex. The following description of characters is used only in this Annex A.

— IT service provider: entity that provides IT services to end users.

— System integrator: entity that builds an IT system that implements IT services by combining target

IT equipment and software.
A.2 Use cases
This annex provides the following two use cases.

Use case 1: Procurement of an IT service — A case where the IT service provider uses APEE as a

criterion for energy effectiveness when narrowing down candidates from among multiple system

integrator proposals.

a) The service provider chooses one benchmark, which is representative of the IT service in question,

and informs multiple system integrators of this benchmark, as shown in Annex D.

b) Each system integrator specifies the middleware and builds an application platform.

c) On their respective application platforms, each system integrator executes the benchmark selected

by the service provider.

d) The system integrator measures the energy consumption of the application platform, and calculates

the APEE value.
e) Each system integrator informs the service provider of the APEE value.

f) The service provider receives the APEE value from the multiple system integrators, and uses the

APEE value, as an expression of energy effectiveness, to narrow down candidates from among

multiple system integrator proposals.
8 © ISO/IEC 2021 – All rights reserved
---------------------- Page: 13 ----------------------
...

FINAL
INTERNATIONAL ISO/IEC
DRAFT
STANDARD FDIS
23544
ISO/IEC JTC 1/SC 39
Information Technology — Data
Secretariat: ANSI
centres — Application Platform
Voting begins on:
2021­03-10 Energy Effectiveness (APEE)
Voting terminates on:
2021­05­05
RECIPIENTS OF THIS DRAFT ARE INVITED TO
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
THEY ARE AWARE AND TO PROVIDE SUPPOR TING
DOCUMENTATION.
IN ADDITION TO THEIR EVALUATION AS
Reference number
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO­
ISO/IEC FDIS 23544:2021(E)
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN­
DARDS TO WHICH REFERENCE MAY BE MADE IN
NATIONAL REGULATIONS. ISO/IEC 2021
---------------------- Page: 1 ----------------------
ISO/IEC FDIS 23544:2021(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2021

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH­1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2021 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC FDIS 23544:2021(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction ..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms, definitions and abbreviated terms ................................................................................................................................ 1

3.1 Terms and definitions ....................................................................................................................................................................... 1

3.2 Abbreviated terms ............................................................................................................................................................................... 3

4 Relevance of APEE ............................................................................................................................................................................................... 3

5 Determination of APEE .................................................................................................................................................................................. 4

6 Determination of application outcome and energy consumption of the application

platform for APEE ................................................................................................................................................................................................ 4

6.1 Prerequisites for the benchmark and application platform ............................................................................ 4

6.2 Determination of application outcome .............................................................................................................................. 5

6.3 Determination of energy consumption of the application platform......................................................... 5

6.3.1 Requirements for obtaining energy consumption of the application platform......... 5

6.3.2 Requirements for the measurement method for obtaining energy consumption .. 5

7 Reporting of APEE ............................................................................................................................................................................................... 6

7.1 Requirements ........................................................................................................................................................................................... 6

7.2 Recommendations ............................................................................................................................................................................... 7

Annex A (informative) Use cases of APEE ........................................................................................................................................................ 8

Annex B (informative) Example of APEE calculation ........................................................................................................................10

Annex C (informative) Example of an APEE report .............................................................................................................................11

Annex D (informative) Examples of acceptable benchmarks ..................................................................................................21

Bibliography .............................................................................................................................................................................................................................23

© ISO/IEC 2021 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC FDIS 23544:2021(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that

are members of ISO or IEC participate in the development of International Standards through

technical committees established by the respective organization to deal with particular fields of

technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other

international organizations, governmental and non­governmental, in liaison with ISO and IEC, also

take part in the work.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for

the different types of document should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives or www .iec .ch/ members

_experts/ refdocs).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

list of patent declarations received (see patents.iec.ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www .iso .org/

iso/ foreword .html. In the IEC, see www .iec .ch/ understanding ­standards.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 39, Sustainability, IT & Data Centres.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html and www .iec .ch/ national

­committees.
iv © ISO/IEC 2021 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC FDIS 23544:2021(E)
Introduction

The growth of the Internet of Things (IoT) has resulted in the storage of increasingly large amounts

of data in data centres and the increased utilization of this data by technologies such as artificial

intelligence (AI). As a result, energy consumption of data centres is also increasing. In this recent

utilization of large amounts of data by technologies such as AI, various kinds of data are combined and

analysed, and processing requests for such combinations and analysis are also increasing. Middleware,

such as a database management system (DBMS), is also becoming more important for this data

utilization. The energy effectiveness of application platforms can be greatly improved through the

selection of middleware.

In addition to improving the energy effectiveness of target IT equipment itself, it is necessary to improve

application platform energy effectiveness by choosing an optimally energy-effective combination of

target IT equipment, operating systems and middleware.

Although there are several KPIs for the energy effectiveness of target IT equipment itself (for example,

ISO/IEC 30134-4, ISO/IEC 21836), there have previously been no KPIs for energy effectiveness to

calculate the energy effectiveness of a combination of target IT equipment, operating systems and

middleware. This document, therefore, introduces a KPI for the energy effectiveness of an entire

application platform. A typical use case of application platform energy effectiveness (APEE) is a

criterion for procuring an energy efficient application platform for an IT service.

This KPI provides assistance in selecting an optimal application platform for energy effectiveness.

This KPI does not apply to the energy effectiveness of an entire data centre. The colloquial term of

"efficiency" is commonly used in regional programmes and "effectiveness" can be referenced as

"efficiency" in those programmes.
© ISO/IEC 2021 – All rights reserved v
---------------------- Page: 5 ----------------------
FINAL DRAFT INTERNATIONAL STANDARD ISO/IEC FDIS 23544:2021(E)
Information Technology — Data centres — Application
Platform Energy Effectiveness (APEE)
1 Scope

This document specifies application platform energy effectiveness (APEE) as a Key Performance

Indicator (KPI) which quantifies the energy effectiveness of an application platform for an IT service in

data centres. This KPI evaluates the energy consumption of an application platform prior to deployment.

The purpose of this KPI is to measure the energy effectiveness of a set of target IT equipment, operating

systems and middleware, to enable the selection of an energy effective IT stack.

This document specifies a formula for calculating APEE and definitions of components of the formula.

This document specifies a measurement method for assessing and reporting the energy effectiveness of

an application platform.

This document also specifies requirements for benchmarks to be used for APEE and requirements for

reporting.
The following topics are outside of the scope of this document:

1) KPIs intended to solely evaluate the energy effectiveness of target IT equipment hardware,

2) energy effectiveness of data centre facilities.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 21836:2020, Information technology — Data centres — Server energy effectiveness metric

3 Terms, definitions and abbreviated terms
3.1 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 21878 and the

following apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1.1
application platform

platform composed of target IT equipment (3.1.4), operating system, and middleware (3.1.3) with the

purpose of providing an IT service (3.1.5)

Note 1 to entry: An application platform is an environment for executing software applications (3.1.2) such as

ERP and CRM software. Therefore, a software application itself is not included as part of an application platform.

© ISO/IEC 2021 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC FDIS 23544:2021(E)
3.1.2
software application
computer program for implementing an IT service (3.1.5)

Note 1 to entry: A software application is usually developed independently for each IT service.

3.1.3
middleware
software layer between an operating system and the software applications (3.1.2)

Note 1 to entry: Examples of middleware include DBMS, web server software, application server software,

transaction monitor.

Note 2 to entry: A base operating system is the software directly managing the hardware, e.g. conventional

operating system, hypervisor, etc.

[SOURCE: ISO/IEC/IEEE 24765:2017, 3.2459, modified – Notes 1 and 2 to entry have been added.]

3.1.4
target IT equipment

server, storage and network equipment that are components of an application platform (3.1.1)

3.1.5
IT service
service that provides an outcome to its users via information technology
3.1.6
application outcome
information provided by an IT service (3.1.5) to its users

Note 1 to entry: An example of an application outcome is a set of reports to support decision-making at retail

outlets (such as sales analysis reports, and stock status reports).
3.1.7
measurement period
duration in which the benchmark performs the intended data processing

Note 1 to entry: Typically, the measurement period(s) is/are the only portion of the benchmark execution which

is/are used to calculate the benchmark score.

Note 2 to entry: The intended data processing routine is a routine including the benchmark calculation.

Note 3 to entry: In the case of using TPC-H , the measurement target of TPC-H is a performance for executing

SQL queries, so a period of executing SQL queries is a measurement period. Other periods, such as a data loading,

are not a measurement period.
3.1.8
benchmark identifier

string which identifies a specific benchmark, which consists of the name and the major version (3.1.9),

when applicable
3.1.9
version

particular form or variation of a resource that differs from other instantiations of the resource in at

least one aspect or item of information

Note 1 to entry: The first digit of a version is called the major version, and any numbers right of that digit is called

a minor version.

1) TPC-H, TPC-E, and TPCx-IoT are trademarks of the Transaction Processing Performance Council. This

information is given for the convenience of users of this document and does not constitute an endorsement by ISO/

IEC of the product named. Equivalent products may be used if they can be shown to lead to the same results.

2 © ISO/IEC 2021 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC FDIS 23544:2021(E)

Note 2 to entry: Example: in the version designated as “v1.2.3,” the Major Version identification is “1”, and the

Minor Version identification is “2.3”.
[SOURCE: ISO 24619:2011, 3.1.9, modified — Notes 1 and 2 to entry added.]
3.1.10
internal power supply

independent device that converts commercial AC power to DC power as required inside target IT

equipment (3.1.4)
3.2 Abbreviated terms
AC alternating current
AI artificial intelligence
APEE application platform energy effectiveness
CRM customer relationship management
DBMS database management system
DC direct current
EAP energy consumption of application platform during the measurement periods
ERP enterprise resource planning
FC fibre channel
IoT Internet of Things
NoAO number of application outcomes during an APEE measurement
OS operating system
RDBMS relational database management system
4 Relevance of APEE

Many kinds of IT services are provided by using data centres, and it is necessary to conserve the

amount of energy used by these IT services. A lot of IT services use various technologies, such as AI, IoT

and big data technologies. For each of these technologies, there are several combinations of target IT

equipment, OSs and middleware that can be used to provide IT services. These combinations of target

IT equipment, OSs and middleware are called an application platform.

Even if the results provided by an IT service are the same, if the application platform used to provide

the IT service is different, the energy consumption is also different. For example, when the same query

is executed on different DBMSs, because the processing method and processing time are different for

each DBMS, energy consumption also differs.

In order to choose the optimum application platform for energy effectiveness, a KPI for measuring the

energy effectiveness of an application platform is necessary. There are KPIs for measuring the energy

effectiveness of target IT equipment only, and ISO/IEC 21836 is one of KPIs for a server. APEE is a KPI

for measuring the energy effectiveness of an application platform. Use cases of APEE are described in

Annex A.

NOTE APEE is a KPI for measuring the energy effectiveness of an application platform for a particular IT

service, not the energy effectiveness of an entire data centre. Therefore, APEE is not suitable for use in regulations

that apply to an entire data centre.
© ISO/IEC 2021 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC FDIS 23544:2021(E)
5 Determination of APEE
APEE shall be determined and described by Formula (1):
c = a / b (1)
where

a is the number of application outcomes during an APEE measurement (NoAO) (see Clause 6);

b is the energy consumption (kWh) of the application platform during the measurement

periods (EAP) (see Clause 6);
c is the APEE.

The unit dimension of APEE is (application outcome(s)/kWh). The APEE value should have at least 2

significant digits.
Reporting of data for APEE is described in Clause 7.
An example of APEE calculation is given in Annex B.
6 Determination of application outcome and energy consumption of the
application platform for APEE
6.1 Prerequisites for the benchmark and application platform

The following prerequisites shall apply when measuring the APEE of an application platform that

provides an IT service:

a) The measurer determines one benchmark to be used for the calculation of APEE. Benchmarks shall

meet the following requirements. Examples of benchmarks that can be used for APEE measurement

are given in Annex D.
1) The benchmark is representative of the target IT service.
2) The start and the end of the execution of the benchmark are clear.
3) All iterations of the benchmark run produce comparable results.

4) An application outcome defined by the benchmark is identical regardless of how many times it

is executed on any application platform.

NOTE 1 In the case of TPC-H as an example of an application outcome, it defines a set of results of

business-oriented ad-hoc queries as an application outcome of the execution. A performance metric

such as throughput and processing time is not an application outcome.

EXAMPLE 1 A benchmark that executes transactions while increasing the multiplicity until

the system cannot keep the specified response time in order to obtain maximum multiplicity is not

applicable because the processes of measurement target for these benchmarks have the potential to

change from run to run.

5) Start date and time, and end date and time of all the measurement period(s) are clear.

NOTE 2 Some benchmarks have multiple measurement periods in one benchmark execution.

EXAMPLE 2 (Example of clarification) A benchmark has warmup and cooldown phases which do not

effort a benchmark performance score. It records start date and time and end date and time of these

phases in a log file, so these phases are distinguishable from measurement period(s).

6) Measurement period start and stop time are measured in seconds or higher precision.

4 © ISO/IEC 2021 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC FDIS 23544:2021(E)

7) To ensure that the result is verifiable, the benchmark is made publicly available.

b) The measurer determines the target IT equipment, OS and middleware that are to serve as

the elements of the application platform. The application platform shall meet the following

requirements:

1) The application platform contains all of the target IT equipment necessary to execute the

benchmark.
2) Every target IT equipment component can be measured for energy consumption.
6.2 Determination of application outcome

For the determination of application outcome, the following terms are introduced:

— One application outcome is defined as one valid execution of the benchmark.
The following requirements for measurement shall be met:

a) The benchmark is executed at least one time. If the benchmark is executed multiple times, the

number of benchmark executions is a positive integer.

NOTE The tester can interact with the server between benchmark iterations, typically to calculate the

total measured time for benchmarks which include unmeasured periods.

b) If the total time of the whole measurement period is less than one hour, the benchmark is repeatedly

executed until the total time of the whole measurement period exceeds one hour.

c) If the benchmark has a setting for official or valid or reportable results for publication or use in

comparisons, this mode shall be used for APEE comparisons.
6.3 Determination of energy consumption of the application platform
6.3.1 Requirements for obtaining energy consumption of the application platform

Energy consumption of the application platform shall be obtained by the sum of energy consumption

(kWh) of all the target IT equipment in the application platform during all benchmark measurement

period(s) of the APEE measurement. Non-benchmark software should be minimal during the

measurement period(s). Execution of non-benchmark software on the application platform shall not

invalidate energy results. Energy measurements during benchmark execution shall not be adjusted to

remove the effects of non-benchmark software.

NOTE The application platform is usually used solely by the benchmark during the measuring period for

accuracy.

The energy consumption of each item of target IT equipment in the application platform is obtained by

the measurement method. Details of this method are given in 6.3.2.
6.3.2 Requirements for the measurement method for obtaining energy consumption

The measurement method shall meet the following requirements. The measurer shall replace “SEEM”

or “SERTv2” with “APEE” and “SUT” with “application platform” in the text of the SEEM referenced

below and apply.
a) Environment
1) ISO/IEC 21836:2020, 7.2 applies for measurement environment.
© ISO/IEC 2021 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC FDIS 23544:2021(E)
2) ISO/IEC 21836:2020, 7.4 applies for power requirements.
b) Power measurement

1) One or more power analysers shall be used to measure the entire energy consumption of the

application platform.

2) Maximum sampling interval of the power analysers shall be at least one sample per second.

3) ISO/IEC 21836:2020, 7.3 and 9.2.2.2 apply for power analyser requirements.
4) Power analysers shall comply with ISO/IEC 21836:2020, 9.2.2.3 and 9.2.2.4.

5) The following requirements shall be preferentially followed over the above ISO/IEC 21836

requirements.

i) The energy consumption shall be measured by using one or more power analysers.

ii) All clocks of all power analysers and target IT equipment shall be synchronized such that

time stamps are aligned within 1 second.

NOTE This requirement is intended to support measurements of the energy consumption of

multiple target IT equipment. ISO/IEC 21836 targets a server, but this document targets multiple target

IT equipment.
c) Temperature measurement

1) The instantaneous temperature of an inlet of one typical element of the application platform

shall be measured by a temperature sensor both within 1 minute of the start time of the

benchmark execution and within 1 minute of the end time of the benchmark execution.

2) ISO/IEC 21836:2020, 9.2.3 applies for temperature sensor requirements.
d) Approved measuring instrument

1) ISO/IEC 21836:2020, Annex B, Table 4 and Table 5 apply for pre-approved power analysers and

temperature sensors.
7 Reporting of APEE
7.1 Requirements

When presenting an APEE, both the benchmark identifier and the APEE value shall be presented, and

the benchmark identifier shall be written in parentheses as follows:
APEE(benchmark identifier)

For all benchmarks which include version numbers, the corresponding benchmark identifier shall

include the benchmark's major version number.

EXAMPLE APEE is written as APEE(TPC-H v2) when TPC-H version 2.17.3 is used as the benchmark.

When presenting APEE, APEE shall be presented along with a report or link to a report. The report shall

include at least the following information:
a) APEE value
b) NoAO value and EAP value
c) start date and time of measurement, and end date and time of measurement

d) start date and time of each measurement period, and end date and time of each measurement period

6 © ISO/IEC 2021 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC FDIS 23544:2021(E)
e) target IT service
f) details of the selected application platform:
1) vendor names and model names of target IT equipment;

2) vendor names, model names, and version numbers of operating systems and middleware;

3) system configuration diagram of the target IT equipment, operating systems, and middleware;

NOTE 1 For an example of a configuration diagram, see Annex C.

4) all target IT equipment configuration and software settings that affect NoAO and EAP values

or are required to reproduce the results shall be documented, including load range, power

management settings, and virtualization resource allocation configuration.

NOTE 2 Examples of load range in DBMS are database size and transaction throughput.

g) information about the selected benchmark:
1) name and version;
2) reference to the specifications of the benchmark;

3) benchmark settings descriptions and values used for all benchmark settings that affect NoAO

and EAP values.

h) power supply information (e.g. power supply mode, power supplies 80 plus rating, AC/DC, voltage,

frequency, and phase)

i) instantaneous temperatures of the inlet at start and end of benchmark execution, and location of

the temperature sensor
j) information about the measurement method:
1) vendor names and model names of the selected power analysers;

2) physical locations of the power analyser(s) between the power source and the measured

systems, including servers, storage, and network equipment;

3) any deviations from the measurement procedure and any unusual features observed.

k) number, publication year, and title of this document, i.e. ISO/IEC 23544:2021, Information

Technology — Data centres — Application Platform Energy Effectiveness (APEE)
7.2 Recommendations
The report should include the following information:
a) purpose of measuring of the APEE value;
b) reason for selecting the benchmark.
© ISO/IEC 2021 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/IEC FDIS 23544:2021(E)
Annex A
(informative)
Use cases of APEE
A.1 Introduction
The life cycle of an IT service consists of the following phases:
a) planning,
b) procurement,
c) operation,
d) update,
e) disposal.

APEE can be effectively used in the procurement phase. This annex provides two use cases for the

procurement phase, from the perspective of the party procuring the IT service and from the perspective

of the party proposing the IT service. These parties fulfil the following roles in the use cases described

in this annex. The following description of characters is used only in this Annex A.

— IT service provider: entity that provides IT services to end users.

— System integrator: entity that builds an IT system that implements IT services by combining target

IT equipment and software.
A.2 Use cases
This annex provides the following two use cases.

Use case 1: Procurement of an IT service — A case where the IT service provider uses APEE as a

criterion for energy effectiveness when narrowing down candidates from among multiple system

integrator proposals.

a) The service provider chooses one benchmark, which is representative of the IT service in question,

and informs multiple system integrators of this benchmark, as shown in Annex D.

b) Each system integrator specifies the middleware and builds an application platform.

c) On their respective application platforms, each system integrator executes the benchmark selected

by the service provider.

d) The system integrator measures the energy consumption of the application platform, and calculates

the APEE value.
e) Each system
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.