ISO/IEC DIS 23544

Information Technology -- Data Centres -- Application Platform Energy Effectiveness (APEE)

Titre manque

Standards Content (sample)

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 23544
ISO/IEC JTC 1/SC 39 Secretariat: ANSI
Voting begins on: Voting terminates on:
2020-08-19 2020-11-11
Information Technology — Data Centres — Application
Platform Energy Effectiveness (APEE)
ICS: 35.020
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 23544:2020(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC DIS 23544:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC DIS 23544:2020(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction ..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms, definitions and abbreviations ............................................................................................................................................ 1

3.1 Terms and definitions ....................................................................................................................................................................... 1

3.2 Abbreviations ........................................................................................................................................................................................... 3

4 Relevance of APEE ............................................................................................................................................................................................... 3

5 Determination of APEE .................................................................................................................................................................................. 4

6 Determination of application outcome and energy consumption of the application

platform for APEE ................................................................................................................................................................................................ 4

6.1 Prerequisites for benchmark and application platform ...................................................................................... 4

6.2 Determination of application outcome .............................................................................................................................. 5

6.3 Determination of energy consumption of the application platform......................................................... 5

6.3.1 Requirements to obtain energy consumption of the application platform ................... 5

6.3.2 Requirement of method to obtain energy consumption ............................................................... 5

7 Reporting of APEE ............................................................................................................................................................................................... 6

7.1 Requirements ........................................................................................................................................................................................... 6

7.2 Recommendations ............................................................................................................................................................................... 7

Annex A (informative) Use cases of APEE ........................................................................................................................................................ 8

Annex B (informative) Example of APEE calculation ........................................................................................................................... 9

Annex C (informative) Example of an APEE report .............................................................................................................................10

Annex D (informative) Examples of acceptable benchmarks ..................................................................................................20

Bibliography .............................................................................................................................................................................................................................21

© ISO/IEC 2020 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC DIS 23544:2020(E)
Foreword

ISO (the International Organization for Standardization) is a worldwide federation of national standards

bodies (ISO member bodies). The work of preparing International Standards is normally carried out

through ISO technical committees. Each member body interested in a subject for which a technical

committee has been established has the right to be represented on that committee. International

organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.

ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of

electrotechnical standardization.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the

different types of ISO documents should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of

any patent rights identified during the development of the document will be in the Introduction and/or

on the ISO list of patent declarations received (see www .iso .org/ patents).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation on the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following

URL: www .iso .org/ iso/ foreword .html.

This document was prepared by ISO/IEC JTC 1, Information technology, Subcommittee SC 39,

Sustainability for and by Information Technology.
iv © ISO/IEC 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC DIS 23544:2020(E)
Introduction

This document provides a Key Performance Indicator (KPI) for the energy effectiveness of application

platform. The purpose of this KPI is to improve the energy effectiveness of application platform through

the selection of an optimally energy-effective combination of IT equipment, operating system, and

middleware.

The growth of Internet of Things (IoT) has resulted in the storage of increasingly large amounts of data

in data centres, and the increased utilization of this data by technologies such as artificial intelligence

(AI). As a result, energy consumption of data centres is also increasing. In this recent utilization of

large amounts of data by technologies such as AI, various kinds of data are combined and analysed,

and processing requests for such combinations and analysis are also increasing. Middleware, such as

a database management system (DBMS), is also becoming more important for this data utilization.

The energy effectiveness of application platform can be greatly improved through the selection of

middleware.

In addition to improving the energy effectiveness of IT equipment itself, it is necessary to improve the

energy effectiveness of application platform by choosing an optimally energy-effective combination of

IT equipment, operating system, and middleware.

Although there are several KPIs (ISO/IEC 30134-4 IT Equipment Energy Efficiency for servers(ITEEsv),

ISO/IEC 21836 Server Energy Effectiveness Metric) for the energy effectiveness of IT equipment itself,

there was previously no KPI for energy effectiveness to calculate energy effectiveness of a combination

of IT equipment, operating system, and middleware. This document, therefore, introduces a KPI for the

energy effectiveness of an entire application platform. A typical use case of Application Platform Energy

Effectiveness (APEE) is a criterion to procure an energy efficient application platform for an IT service.

This KPI provides assistance in selecting an optimal application platform for energy effectiveness. This

KPI does not apply to the energy effectiveness of an entire data centre.
© ISO/IEC 2020 – All rights reserved v
---------------------- Page: 5 ----------------------
DRAFT INTERNATIONAL STANDARD ISO/IEC DIS 23544:2020(E)
Information Technology — Data Centres — Application
Platform Energy Effectiveness (APEE)
1 Scope

This document specifies APEE as a Key Performance Indicator (KPI) which quantifies the energy

effectiveness of an application platform for an IT service in data centres. This KPI evaluates energy

consumption of application platform prior to deployment.

This document specifies a formula for APEE, and definitions of components of the formula.

This document specifies a measurement method to assess and report the energy effectiveness of an

application platform.
This document specifies requirements for benchmarks to be used for APEE.
This document specifies requirements for reporting.
The following topics are outside of the scope of this project:
1) energy effectiveness KPIs for only IT equipment,
2) energy effectiveness of data centre facilities.
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 30134-1:2016, Information technology — Data centres — Key performance indicators — Part 1:

Overview and general requirements

ISO/IEC 21836, Information technology — Data centres — Server Energy Effectiveness Metric

3 Terms, definitions and abbreviations
3.1 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 21878 and the

following apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— IEC Electropedia: available at http:// www .electropedia .org/
— ISO Online browsing platform: available at https:// www .iso .org/ obp
3.1.1
application platform

platform composed of IT equipment, operating system, and middleware with the purpose of providing

an IT service

Note 1 to entry: An application platform is an environment for executing software application such as ERP and

CRM software. Therefore, software application itself is not included as part of an application platform.

© ISO/IEC 2020 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC DIS 23544:2020(E)
3.1.2
software application
software for implementing an IT service by using an application platform

Note 1 to entry: Software application is usually developed independently for each IT service.

3.1.3
middleware
software layer between a base operating system and the software applications

Note 1 to entry: Example of middleware: DBMS, web server software, application server software, transaction

monitor.

Note 2 to entry: Base operating system is the software directly managing hardware. e.g. conventional operating

system, hypervisor

[SOURCE: ISO/IEC/IEEE 24765:2017, 3.2459, modified – The note 1 and the note 2 has been added.]

3.1.4
IT equipment

server, storage, and network equipment that are components of an application platform

3.1.5
IT service
service that provides a valuable outcome to its users via information technology
3.1.6
application outcome
valuable outcome provided by an IT service to its users

Note 1 to entry: Example of application outcome: a set of reports to support decision-making at retail outlets

(such as sales analysis reports, and stock status reports)
3.1.7
measurement period

during which the benchmark is performing the data processing intended to be measured by the

benchmark

Note 1 to entry: Typically, the measurement period(s) are the only portion of the benchmark execution which are

used to calculate the benchmark score.

Note 2 to entry: In the case of using TPC-H, the measurement target of TPC-H is a performance for executing

SQL queries, so a period of executing SQL queries is a measurement period. Other period such as a data loading is

not a measurement period.
3.1.8
benchmark identifier

string to identify a benchmark, which consists of the name and the major version if applicable

3.1.9
version

particular form or variation of a resource that differs from other instantiations of the resource in at

least one aspect or item of information
[SOURCE: ISO 24619:2011, 3.1.9]

Note 1 to entry: The first digit of a version is called the major version, and any numbers right of that digit is called

a minor version.

1) TPC-H, TPC-E, and TPCx-IoT are trademarks of the Transaction Processing Performance Council. This

information is given for the convenience of users of this document and does not constitute an endorsement by ISO/

IEC of the product named. Equivalent products may be used if they can be shown to lead to the same results.

2 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC DIS 23544:2020(E)

Note 2 to entry: Example: in the version designated as “v1.2.3,” the Major Version identification is “1”, and the

Minor Version identification is “2.3 ”.

[SOURCE: ISO/IEC 21836:2020, 3.1.59. Under preparation. Stage at the time of publication: ISO/IEC DIS

21836:2020.]
3.1.10
internal power supply

device that converts commercial AC power to the DC power as required by IT equipment

3.2 Abbreviations
AC Alternating Current
AI Artificial Intelligence
APEE Application Platform Energy Effectiveness
CRM Customer Relationship Management
DBMS DataBase Management System
DC Direct Current
ERP Enterprise Resource Planning
FC Fibre Channel
IoT Internet of Things
OS Operation System
RDBMS Relational DataBase Management
4 Relevance of APEE

Many kinds of IT services are provided by using data centres, and it is necessary to conserve the amount

of energy used by these IT services. Lots of IT services use various technologies, such as AI, IoT, and big

data technologies. For each of these technologies, there are several combinations of IT equipment, OS,

and middleware that can be used to provide IT services. These combinations of IT equipment, OS, and

middleware are called application platform.

Even if the results provided by an IT service are the same, if the application platform used to provide

the IT service is different, the energy consumption is also different. For example, when the same query

is executed on different DBMSs, because the processing method and processing time are different for

each DBMS, energy consumption also differs.

In order to choose the optimum application platform for energy effectiveness, a KPI for measuring the

energy effectiveness of an application platform is necessary. There are KPIs for measuring the energy

effectiveness of IT equipment only, and ISO/IEC 21836 is one of KPIs for a server. APEE is a KPI for

measuring the energy effectiveness of an application platform. Use cases of APEE are described in

Annex A.

NOTE APEE is a KPI for measuring the energy effectiveness of an application platform for a particular IT

service, not the energy effectiveness of an entire data centre. Therefore, APEE is not suitable for use in regulations

that apply to an entire data centre.
© ISO/IEC 2020 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC DIS 23544:2020(E)
5 Determination of APEE
APEE shall be determined and described by Formula (1):
APEE = NoAO / EAP (1)
where

NoAO = the Number of Application Outcomes during an APEE measurement (see clause 6);

EAP = the Energy Consumption(kWh) of the Application platform during the measurement

periods (see clause 6).

The unit dimension of APEE is (application outcome(s) / kWh). APEE value should have at least 2

significant digits.
Reporting of data for APEE is described in clause 7.
An example of APEE calculation is given in Annex B.
6 Determination of application outcome and energy consumption
of the application platform for APEE
6.1 Prerequisites for benchmark and application platform

The following prerequisites shall apply when measuring the APEE of an application platform that

provides an IT service:

a) The measurer determines one benchmark to be used for the calculation of APEE. Benchmarks shall

meet the following requirements. Examples of benchmarks that can be used for APEE measurement

are given in Annex D.
1) The benchmark is representative of the target IT service.
2) The start and the end of the execution of the benchmark are clear.
3) All iterations of the benchmark run produce comparable results.

4) An application outcome defined by the benchmark is identical regardless of how many times it

is executed on any application platform.

NOTE 1 In the case of TPC-H as an example of an application outcome, it defines a set of results of

business-oriented ad-hoc queries as an application outcome of the execution. Performance metric such

as throughput and processing time is not an application outcome.

NOTE 2 For example, benchmark that executes transactions while increasing the multiplicity until

the system cannot keep the specified response time in order to obtain maximum multiplicity is not

applicable. Because the processes of measurement target for these benchmarks have the potential to

change from run to run.

5) Start date and time, and end date and time of all the measurement period(s) are clear.

NOTE 1 Some benchmarks have multiple measurement periods in one benchmark execution.

NOTE 2 Example of clarification: a benchmark records start date and time, and end date and time of

each measurement period in log file.

6) Measurement period start and stop time are measured in second or higher precision.

4 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC DIS 23544:2020(E)
7) To ensure that result is verifiable, the benchmark is publicly available.

b) The measurer determines the IT equipment, OS and middleware that are to serve as the elements

of the application platform. The application platform shall meet the following requirements:

1) The application platform contains all of the IT equipment necessary to execute the benchmark.

2) Each piece of IT equipment can be measured for energy consumption.
6.2 Determination of application outcome

For the determination of application outcome, the following terms are introduced:

a) One application outcome is defined as one execution of the benchmark.
The following requirements for measurement shall be met:

a) The benchmark is executed at least one time. If the benchmark is executed multiple times, the

number of benchmark executions is a positive integer.

NOTE The tester may interact with the server between benchmark iterations, typically to calculate

the total measured time for benchmarks which include unmeasured periods.

b) If the total time of all the measurement period is less than one hour, the benchmark is repeatedly

executed until the total time of all the measurement period exceeds one hour.
6.3 Determination of energy consumption of the application platform
6.3.1 Requirements to obtain energy consumption of the application platform

Energy consumption of the application platform shall be obtained by the sum of energy

consumption(kWh) of all the IT equipment in the application platform during all measurement period(s)

of the APEE measurement. Execution of non-benchmark software on the application platform shall not

invalidate energy results. Energy measurements during benchmark execution shall not be adjusted to

remove the effects of non-benchmark software.

NOTE The application platform is usually used solely by the benchmark during the measuring period for

accuracy.

Energy consumption of each IT equipment in the application platform is obtained by the measurement

method. Details of this method is given in 6.3.2.
6.3.2 Requirement of method to obtain energy consumption

The measurement method shall meet following requirements. The measurer shall replace “SEEM” with

“APEE” and “SUT” with “application platform” in the text of the SEEM referenced below and apply.

a) Environment

1) ISO/IEC 21836:2020, Information technology – Data centres – Server Energy Effectiveness

Metric , 7.2 is applied for measurement environment.

2) ISO/IEC 21836:2020, Information technology – Data centres – Server Energy Effectiveness

Metric , 7.4 is applied for power requirements.
b) Power measurement

1) One or more power analysers shall be used to measure the entire energy consumption of the

application platform.
2) Under preparation. Stage at the time of publication: ISO/IEC DIS 21836:2020.
© ISO/IEC 2020 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC DIS 23544:2020(E)

2) Maximum sampling interval of the power meter shall be at least one sample per second.

3) ISO/IEC 21836:2020, Information technology – Data centres – Server Energy Effectiveness

Metric , 7.3, 9.2.2.2, 9.2.2.3, and 9.2.2.4 are applied for power analyser requirements.

4) The following requirements are preferentially followed over the above ISO/IEC 21836

requirements.
i) The energy consumption is measured by using one or more power analysers.

ii) All clocks of all power analysers and IT equipment shall be synchronized such that time

stamps are aligned within 1 second.

NOTE This requirement intends to support measurements of the energy consumption of multiple

IT equipment. ISO/IEC 21836 targets a server, but this standard targets multiple IT equipment.

c) Temperature measurement

1) The inlet temperature of one typical element of the application platform needs to be measured

by a temperature sensor both within 1 minute of the start time of the benchmark execution

and within 1 minute of the end time of the benchmark execution.

2) ISO/IEC 21836:2020, Information technology – Data centres – Server Energy Effectiveness

Metric , 9.2.3 is applied for temperature sensor requirements.
d) Approved measuring instrument

1) ISO/IEC 21836:2020, Information technology – Data centres – Server Energy Effectiveness

Metric , Annex B, Table 4 and Table 5 are applied for pre-approved power analysers and

temperature sensors.
7 Reporting of APEE
7.1 Requirements

When presenting an APEE, both the benchmark identifier and the APEE value shall be presented, and

the benchmark identifier shall be written in parentheses as follows:
APEE(benchmark identifier)

For all benchmarks which include version numbers, the corresponding benchmark identifier shall

include the benchmark's major version number.

NOTE APEE is written as APEE(TPC-H v2) when TPC-H version 2.17.3 is used as the benchmark.

When presenting APEE, APEE shall be presented along with a report or link to a report. The report shall

include at least the following information:
a) APEE value
b) NoAO value and EAP value
c) start date and time of measurement, and end date and time of measurement

d) start date and time of each measurement period, and end date and time of each measurement period

e) target IT service
f) details of the selected application platform:
1) vendor names and model names of IT equipment
6 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC DIS 23544:2020(E)

2) vendor names, model names, and version numbers of operating system and middleware

3) system configuration diagram of the IT equipment, operating system, and middleware

NOTE Example of configuration diagram is in Annex C.

4) all IT equipment configuration and software settings that affect NoAO and EAP values shall

be documented including load range, power management settings, and virtualization resource

allocation configuration

NOTE Examples of load range in DBMS are database size and transaction throughput.

g) information about the selected benchmark:
1) name and version
2) reference to the specifications of the benchmark

3) benchmark settings descriptions and values used for all benchmark settings that affect NoAO

and EAP values
h) power supply information (e.g. AC/DC, voltage, frequency, and phase)

i) inlet temperatures and location of the selected application platform at start and end of benchmark

execution
j) information about the measurement method:
1) vendor names and model names of the selected power analysers
2) connection locations of the power analysers
7.2 Recommendations
The report should include the following information:
a) purpose of measuring of the APEE value
b) reason for selecting the benchmark
c) standard number, publication year, and title of this standard
© ISO/IEC 2020 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/IEC DIS 23544:2020(E)
Annex A
(informative)
Use cases of APEE
A.1 Introduction

The life cycle of an IT service consists of the following phases: planning, procurement, operation,

update, and disposal. APEE can be effectively used in the procurement phase. This annex provides two

use cases for the procurement phase, from the perspective of the party procuring the IT service and

from the perspective of the party proposing the IT service. These parties fulfil the following roles in the

use cases described in this annex. This description of characters is used only in this Annex A.

IT service provider: entity that provides IT services to end users.

System integrator: entity that builds an IT system that implements IT services by combining IT

equipment and software.
A.2 Use cases
This annex provides the following two use cases.

Use case 1: Procurement of an IT service - A case where the IT service provider uses APEE as a criterion

for energy effectiveness when narrowing down candidates from among multiple system integrator

proposals.

a) The service provider chooses one benchmark, that is representative of the IT service in question,

and informs multiple system integrators of this benchmark, as shown in Annex D.

b) Each system integrator specifies the middleware and builds an application platform.

c) On their respective application platforms, each system integrator executes the benchmark selected

by the service provider.

d) The system integrator measures the energy consumption of the application platform, and calculates

the APEE value.
e) Each system integrator informs the service provider of the APEE value.

f) The service provider receives the APEE value from the multiple system integrators, and uses the

APEE value, as an expression of energy effectiveness, to narrow down candidates from among

multiple system integrator proposals.

Use case 2: Promoting sales of an application platform - A case where the system integrator uses APEE to

appeal to customers regarding the energy effectiveness of the system integrator's application platform.

a) The system integrator clearly identifies the target IT service and the target customer of the

system integrator, and determines the benchmark based on the workload type of the IT service, as

described in Annex D.

b) The system integrator builds an application platform that can implement the IT service in an

energy-efficient manner.

c) The system integrator measures the APEE value for the application platform and the selected

benc
...