Information technology — Data centres — Server energy effectiveness metric

This document specifies a measurement method to assess and report the energy effectiveness of a computer server. This document does not set any pass/fail criteria for servers.

Technologies de l'information — Centres de données — Grandeurs de mesure de l'efficacité énergétique des serveurs

General Information

Status
Published
Publication Date
24-Aug-2020
Current Stage
6060 - International Standard published
Start Date
25-Aug-2020
Completion Date
25-Aug-2020
Ref Project

Buy Standard

Standard
ISO/IEC 21836:2020 - Information technology -- Data centres -- Server energy effectiveness metric
English language
70 pages
sale 15% off
Preview
sale 15% off
Preview
Draft
ISO/IEC FDIS 21836 - Information technology -- Data centres -- Server energy effectiveness metric
English language
71 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (sample)

INTERNATIONAL ISO/IEC
STANDARD 21836
First edition
2020-08
Information technology — Data centres
— Server energy effectiveness metric
Technologies de l'information — Centres de données — Grandeurs de
mesure de l'efficacité énergétique des serveurs
Reference number
ISO/IEC 21836:2020(E)
ISO/IEC 2020
---------------------- Page: 1 ----------------------
ISO/IEC 21836:2020(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2020

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2020 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 21836:2020(E)
Contents Page

Foreword ..........................................................................................................................................................................................................................................v

Introduction ................................................................................................................................................................................................................................vi

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms, definitions and abbreviated terms ................................................................................................................................ 1

3.1 Terms and definitions ....................................................................................................................................................................... 1

3.2 Abbreviated terms ............................................................................................................................................................................... 9

4 Applicability of Server Energy Effectiveness Metric (SEEM) ...............................................................................10

4.1 General ........................................................................................................................................................................................................10

4.2 Applicability of SERTv2 ................................................................................................................................................................10

4.3 Determination of applicability for an “implementer-specified” metric ..............................................11

5 Determination of Server Energy Effectiveness Metric (SEEM) ..........................................................................11

5.1 General ........................................................................................................................................................................................................11

5.2 Power supply requirements .....................................................................................................................................................12

5.3 SERTv2 active state energy effectiveness metric ...................................................................................................12

5.4 Determination of “implementer-specified” metrics ............................................................................................14

5.4.1 General...................................................................................................................................................................................14

5.4.2 “Implementer-specified” active state metrics .....................................................................................14

5.4.3 “Implementer-specified” idle state metric .............................................................................................15

6 SEEM implementation ..................................................................................................................................................................................16

6.1 General ........................................................................................................................................................................................................16

6.2 Implementation steps for SEEM ...........................................................................................................................................16

6.3 Implementation steps for SERTv2 ......................................................................................................................................16

6.4 Implementation steps for “implementer-specified” metrics .......................................................................16

6.5 Threshold selection ......... .................................................................................................................................................................17

7 Server testing ........................................................................................................................................................................................................18

7.1 Configuration .........................................................................................................................................................................................18

7.2 Environment ..........................................................................................................................................................................................18

7.3 Power analyser calibration requirement ......................................................................................................................18

7.4 Power requirements........................................................................................................................................................................18

7.5 SEEM idle state power test method ...................................................................................................................................19

7.6 Testing Variance ..................................................................................................................................................................................19

7.6.1 General...................................................................................................................................................................................19

7.6.2 Software Testing Variance Specification ..................................................................................................20

7.6.3 SEEM Hardware Testing Variance Specification ................................................................................20

7.7 Technical support ..............................................................................................................................................................................20

7.7.1 Technical support requirements .....................................................................................................................20

7.7.2 SPEC SERTv2 technical support ......................................................................................................................20

7.8 Server product family and special configuration testing .................................................................................21

7.8.1 General...................................................................................................................................................................................21

7.8.2 Single configuration testing .................................................................................................................................22

7.8.3 SERTv2 product family and special configurations ........................................................................22

7.8.4 Product family and special configurations for “implementer-specified” metrics 24

7.9 “Implementer-specified” metric creation testing ..................................................................................................24

7.9.1 General...................................................................................................................................................................................24

7.9.2 Functionality and reproducibility testing ...............................................................................................24

7.9.3 Real-world energy savings testing ................................................................................................................26

8 Reporting SEEM ..................................................................................................................................................................................................26

8.1 Server categories ................................................................................................................................................................................26

8.1.1 General...................................................................................................................................................................................26

8.1.2 SERTv2 server category definitions .............................................................................................................26

© ISO/IEC 2020 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 21836:2020(E)

8.1.3 “Implementer-specified” metric server categories .........................................................................29

8.2 Documentation ....................................................................................................................................................................................29

8.2.1 General...................................................................................................................................................................................29

8.2.2 Implementation documentation .....................................................................................................................29

8.2.3 SEEM end user documentation ........................................................................................................................30

9 Minor update acceptance testing .....................................................................................................................................................33

9.1 General ........................................................................................................................................................................................................33

9.2 SEEM minor update testing and requirements ........................................................................................................33

9.2.1 SEEM component versions .................. .................................................................................................................33

9.2.2 Power analysers ............................................................................................................................................................33

9.2.3 Temperature sensors ................................................................................................................................................35

9.3 SERTv2 minor update testing and requirements ...................................................................................................35

9.3.1 General...................................................................................................................................................................................35

9.3.2 SERTv2 Component versions .............................................................................................................................35

9.3.3 SERTv2 ...................................................................................................................................................................................36

9.3.4 PTDaemon ..........................................................................................................................................................................36

9.3.5 SERTv2 Client Configuration XML .................................................................................................................36

9.4 “Implementer-specified” metric minor update requirements ......... ...........................................................44

Annex A (normative) SERTv2 pre-approved CPU architectures, operating systems, and Java

Virtual Machines (JVMs) ............................................................................................................................................................................46

Annex B (normative) SEEM pre-approved power analysers and temperature sensors ...............................57

Annex C (normative) SERTv2 required documentation fields ................................................................................................59

Annex D (normative) Deployed power assessment ...........................................................................................................................61

Bibliography .............................................................................................................................................................................................................................69

iv © ISO/IEC 2020 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 21836:2020(E)
Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical

Commission) form the specialized system for worldwide standardization. National bodies that

are members of ISO or IEC participate in the development of International Standards through

technical committees established by the respective organization to deal with particular fields of

technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other

international organizations, governmental and non-governmental, in liaison with ISO and IEC, also

take part in the work.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for

the different types of document should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject

of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent

rights. Details of any patent rights identified during the development of the document will be in the

Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC

list of patent declarations received (see http:// patents .iec .ch).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/

iso/ foreword .html.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,

Subcommittee SC 39, Sustainability, IT & Data Centres.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/ members .html.
© ISO/IEC 2020 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/IEC 21836:2020(E)
Introduction

The global economy is now totally reliant on information and communication technologies (ICT) and the

associated generation, transmission, dissemination, computation and storage of digital data. While the

internet backbone carries the traffic, it is data centres which find themselves at the nodes and hubs of a

wide variety of both private enterprise and shared/collocation facilities. With the large and continually

increasing data capacity demands placed on data centres worldwide, efficient use of data centre energy

is an extremely important strategy for managing environmental, cost, electrical grid capacity and other

impacts.

The ISO/IEC 30134 series specifies data centre energy effectiveness key performance indicators (KPI)

to help data centre operators measure and improve specific aspects of data centre energy effectiveness.

ISO/IEC 30134-4 in particular defines a method to measure the peak capacity and utilization of

servers operating in a data centre using operator selected benchmarks. However, it does not provide

a method for comparing individual server energy effectiveness across data centres, and as stated

in ISO/IEC 30134-4, “should not be used to set regulations for a data centre or individual server”.

There is stakeholder demand for an international standard to measure the energy effectiveness of

servers before procurement and installation, particularly for use in worldwide server energy

effectiveness regulations and programmes.

This document provides a server energy effectiveness metric (SEEM) to measure and report the

energy effectiveness of specific server designs and configurations. This document will be useful to

stakeholders, including vendors, users and governments, from the design verification testing phase

all the way through conformance verification, procurement and operation. Organizations that wish to

establish conformance or reporting programmes will find that the test methods and scoring specified

in this document will save them significant time and effort in implementing such programmes.

Standardization across such programmes will allow vendors to comply to stakeholder requirements

more quickly and efficiently.

For applicable servers, this document builds upon the widely adopted Server Efficiency Rating

1) ® 2)

Tool (SERT™) suite developed by the Standard Performance Evaluation Corporation (SPEC )

benchmark consortium, as the energy effectiveness metric and test method. For servers where SERT

is not applicable, this document provides requirements for the creation of alternate server energy

effectiveness metrics, referred to as “implementer-specified” metrics.

1) SERT is a trademark of the Standard Performance Evaluation Corporation. This information is given for the

convenience of users of this document. References to SERT do not constitute an endorsement by ISO/IEC.

2) SPEC is a trademark of the Standard Performance Evaluation Corporation. This information is given for the

convenience of users of this document. References to SPEC do not constitute an endorsement by ISO/IEC.

vi © ISO/IEC 2020 – All rights reserved
---------------------- Page: 6 ----------------------
INTERNATIONAL STANDARD ISO/IEC 21836:2020(E)
Information technology — Data centres — Server energy
effectiveness metric
1 Scope

This document specifies a measurement method to assess and report the energy effectiveness of a

computer server. This document does not set any pass/fail criteria for servers.
2 Normative references
There are no normative references in this document.
3 Terms, definitions and abbreviated terms
For the purposes of this document, the following terms and definitions apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1 Terms and definitions
3.1.1
64-bit CPU

CPU (3.1.12) which has data path widths, memory (3.1.34) addressing, registers, and other architectural

features which are 64-bits wide
3.1.2
active state
operational state in which the server (3.1.49) is carrying out data processing

Note 1 to entry: an example is data retrieval from memory (3.1.34), cache, or storage while awaiting further input

over the network.
3.1.3
auxiliary processing accelerator

additional compute device installed in the computer server (3.1.49) that handles parallelized workloads

(3.1.60) in conjunction with the CPU (3.1.12)
3.1.4
blade chassis

enclosure that contains shared resources for the operation of blade servers (3.1.5), blade storage (3.1.6),

and other blade form-factor devices

Note 1 to entry: Shared resources provided by a chassis may include power supplies, data storage and hardware

for DC power distribution, thermal management, system management and network services.

© ISO/IEC 2020 – All rights reserved 1
---------------------- Page: 7 ----------------------
ISO/IEC 21836:2020(E)
3.1.5
blade server
server (3.1.49) that is designed for use in a blade chassis (3.1.4)

Note 1 to entry: A blade server is a high-density device that functions as an independent server (3.1.49) and

includes at least one processor (3.1.40) and system memory (3.1.54), but is dependent upon shared blade chassis

(3.1.4) resources (e.g. power supplies, cooling) for operation.
3.1.6
blade storage

storage device that is designed for use in a blade chassis (3.1.4) that is dependent upon shared blade

chassis (3.1.4) resources, like power supplies or cooling, for operation
3.1.7
buffered memory

circuitry between the server’s memory (3.1.34) and memory controller to either increase memory

capacity, increase bandwidth, and/or reduce the electrical load on the memory controller

3.1.8
coefficient of determination

statistic used to determine the strength of a fit between a mathematical model and a set of observed

data values
[SOURCE: ISO 15551-1:2015, 3.26, modified — Note 1 to entry has been removed.]
3.1.9
coefficient of variation
standard deviation divided by the mean
[SOURCE: ISO 3534-1:2006, 2.38, modified — Note 1 to entry has been removed.]
3.1.10
configuration

interrelated functional and physical characteristics of a product defined in product configuration

information

Note 1 to entry: This document employs the following configurations: low-end (3.1.33), high-end (3.1.25) and

typical.

Note 2 to entry: For server (3.1.49) products, a configuration is one of many possible combinations of components

including CPU (3.1.12), storage devices, memory (3.1.34) size, and capacity and input/output devices for a single

server (3.1.49) product within a larger product family. There are a large number of possible configurations within

a product family.

[SOURCE: ISO 17599:2015, 3.15, modified — Notes 1 and Note 2 to entry have been added.]

3.1.11
core

component of a processor (3.1.40) which can independently receive instructions and takes actions or

performs calculations in response
3.1.12
CPU

central processing element with functions for interpreting and executing instructions

Note 1 to entry: In this document, cache memory (3.1.34) is included with the CPU.

Note 2 to entry: This document uses the terms CPU and processor (3.1.40) interchangeably.

[SOURCE: ISO/IEC 14576:1999, 2.1.9, modified — Note 1 and Note 2 to entry have been added.]

2 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/IEC 21836:2020(E)
3.1.13
CPU architecture

CPU (3.1.12) design with significant similarities to other CPU architectures within the same CPU

architecture class (3.1.14)

Note 1 to entry: CPU architectures are used to create CPU models (3.1.15) which are often released in a similar

timeframe.

Note 2 to entry: Examples of different CPU architectures in the same CPU architecture class (3.1.14) are

® ®

Intel Haswell, Intel Broadwell, and Intel Skylake, or separately, AMD Bulldozer, AMD Piledriver, and AMD

Steamroller .
3.1.14
CPU architecture class

group of one or more CPU architectures (3.1.13) which share the same instruction set architecture and

in which newer architecture designs are derived from previous architecture designs

Note 1 to entry: Within a CPU architecture class, the initial CPU architecture (3.1.13) is, for the most part, a new

design, and subsequent CPU architectures (3.1.13) are derived from preceding CPU architectures (3.1.13).

® ®3

Note 2 to entry: Examples of different CPU architecture classes are ARM v8-A and AMD EPYC .

Note 3 to entry: In certain cases, software programmes need to be recompiled for use with different CPU

architecture classes.
3.1.15
CPU model
specific CPU (3.1.12) that is sold in the marketplace

Note 1 to entry: All CPU (3.1.12) of the same model share the same technical characteristics, such as core (3.1.11)

frequencies and core counts, and can be used interchangeably.

Note 2 to entry: Examples of different CPU models are AMD EYPC 7601, AMD EYPC 7251 and Intel Xeon

Platinum 8180 .
3.1.16
CPU nominal frequency

CPU core clock frequency, which is the main frequency used in naming, marketing and selling the CPU

(3.1.12)
3.1.17
data averaging interval

for a power analyser (3.1.38), the time period over which all samples captured by the high-speed

sampling electronics of the analyser are averaged to provide a set of measured data

3.1.18
double data rate
DDR

computer bus characteristic of transferring data on the rising and falling edges of the clock signal,

resulting in twice the data bandwidth at a specific clock frequency, versus a single data rate bus

3.1.19
end user

person or persons who will ultimately be using the system for its intended purpose

Note 1 to entry: For the purposes of this document, the end user refers to a SEEM end user, which is the entity

applying for certification of a server (3.1.49) model to a SEEM conformant regulation or programme. For example,

if server manufacturer A was submitting a server model to ENERGY STAR for certification, server manufacturer

A would be the end user.

3) AMD and EPYC are trademarks of Advanced Micro Devices, Intel is a trademark of the Intel Corporation and ARM

is a trademark of Arm Limited. This information is given for the convenience of users of this document. References

to AMD, EPYC, Intel and ARM do not constitute an endorsement by ISO/IEC.
© ISO/IEC 2020 – All rights reserved 3
---------------------- Page: 9 ----------------------
ISO/IEC 21836:2020(E)

[SOURCE: ISO/IEC 19770-5:2015, 3.13, modified — Note 1 to entry removed, new Note 1 to entry added.]

3.1.20
energy effectiveness

measure of the amount of data processing performed for a given amount of energy consumed

Note 1 to entry: For the purposes of this document, energy effectiveness is equivalent to the term energy

efficiency as used in server (3.1.49) compliance regulations and programmes.
3.1.21
expansion auxiliary processing accelerator
expansion APA

auxiliary processing accelerator that is an add-in card installed in a general-purpose add-in

expansion slot.

Note 1 to entry: An expansion APA add-in card may include one or more APAs and/or separate, dedicated

removable switches.
EXAMPLE A GPGPU installed in a PCI-e slot.
3.1.22
fully fault tolerant server

computer server (3.1.49) that is designed with complete hardware redundancy, in which every

computing component is replicated between two nodes running identical and concurrent workloads

(3.1.60)

Note 1 to entry: A fully fault tolerant server uses two systems to simultaneously and repetitively run a single

workload for continuous availability in a mission critical application.

Note 2 to entry: An example of a fault tolerant server ; if one node fails or needs repair, the second node can run

the workload alone to avoid downtime.
3.1.23
hardware threads

in a CPU core (3.1.11), the number of fully independent instruction streams which can be executed

through SMT
3.1.24
high-performance computing system
HPC system
HPC server

computing system which is designed, marketed, sold, and optimized to execute highly parallel

applications for high performance, deep learning, or artificial intelligence applications

Note 1 to entry: HPC systems consist of multiple clustered servers (3.1.49, primarily for increased computational

capability, high speed inter-processing interconnects, large and high bandwidth memory (3.1.34) capability and

often accelerators such as GPGPUs or FPGAs.

Note 2 to entry: HPC systems may be purposely built or assembled from more commonly available computer

servers.
3.1.25
high-end configuration

server (3.1.49) equipped with a specific selection of high-performance components, which is required

to be tested as part of measuring a server product family (3.1.51)
4 © ISO/IEC 2020 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC 21836:2020(E)
3.1.26
idle state

operational mode in which the OS and other software have completed loading, the server (3.1.49) is

capable
...

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 21836
ISO/IEC JTC 1/SC 39 Secretariat: ANSI
Voting begins on: Voting terminates on:
2019-03-07 2019-05-30
Information Technology — Data Centres — Server Energy
Effectiveness Metric

Technologies de l'information — Centres de données — Grandeurs de mesure de l'efficacité énergétique des

serveurs
ICS: 35.020
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 21836:2019(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2019
---------------------- Page: 1 ----------------------
ISO/IEC DIS 21836:2019(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2019

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2019 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC DIS 21836:2019(E)
40 CONTENTS

42 Foreword ................................................................................................................................ 6

43 Introduction ............................................................................................................................. 7

44 1 Scope ............................................................................................................................... 8

45 2 Normative references ....................................................................................................... 8

46 3 Definitions and abbreviations ............................................................................................ 8

47 3.1 Definitions ............................................................................................................... 8

48 3.2 Abbreviations ........................................................................................................ 17

49 4 Applicability of Server Energy Effectiveness Metric (SEEM) ............................................ 19

50 4.1 General ................................................................................................................. 19

51 4.2 Applicability of SERTv2 ......................................................................................... 19

52 4.3 Determination of applicability for an “implementer-specified” metric ....................... 19

53 5 Determination of Server Energy Effectiveness Metric (SEEM) ......................................... 20

54 5.1 General ................................................................................................................. 20

55 5.2 Power supply requirements ................................................................................... 20

56 5.3 SERTv2 active energy effectiveness metric ........................................................... 20

57 5.4 Determination of “implementer-specified” metrics .................................................. 22

58 General ..................................................................................................... 22

59 “Implementer-specified” active metrics ....................................................... 22

60 “Implementer-specified” idle metric ............................................................ 23

61 6 SEEM implementation .................................................................................................... 24

62 6.1 General ................................................................................................................. 24

63 6.2 Implementation steps for SEEM ............................................................................. 24

64 6.3 Implementation steps for SERTv2 .......................................................................... 24

65 6.4 Implementation steps for “implementer-specified” metrics ...................................... 25

66 6.5 Threshold selection ............................................................................................... 25

67 7 Server Testing ................................................................................................................ 26

68 7.1 Configuration ......................................................................................................... 26

69 7.2 Environment .......................................................................................................... 26

70 7.3 Power analyser requirements ................................................................................ 26

71 7.4 Power requirements .............................................................................................. 26

72 7.5 SEEM idle power test method ................................................................................ 26

73 7.6 Technical support .................................................................................................. 27

74 Technical support requirements ................................................................. 27

75 SPEC SERTv2 technical support ............................................................... 27

76 7.7 Server product family and special configuration testing ......................................... 28

77 General ..................................................................................................... 28

78 Single configuration testing ........................................................................ 29

79 SERTv2 product family and special configurations ..................................... 29

80 Product family and special configurations for “implementer-specified”

81 metrics ...................................................................................................... 31

82 7.8 “Implementer-specified” metric creation testing...................................................... 31

83 General ..................................................................................................... 31

84 Functionality and reproducibility testing ..................................................... 31

iii
© ISO/IEC 2019 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/IEC DIS 21836:2019(E)

85 Real-world energy savings testing ............................................................. 32

86 8 Reporting SEEM ............................................................................................................. 33

87 8.1 Server categories .................................................................................................. 33

88 General ..................................................................................................... 33

89 SERTv2 server category definitions ........................................................... 33

90 “Implementer-specified” metric server categories ....................................... 36

91 8.2 Documentation ...................................................................................................... 36

92 General ..................................................................................................... 36

93 Implementation documentation .................................................................. 36

94 End user documentation ............................................................................ 37

95 9 Minor update acceptance testing .................................................................................... 40

96 9.1 General ................................................................................................................. 40

97 9.2 SEEM minor update testing and requirements ....................................................... 40

98 SEEM component versions ........................................................................ 40

99 Power analysers ........................................................................................ 40

100 Temperature sensors ................................................................................. 42

101 9.3 SERTv2 minor update testing and requirements .................................................... 42

102 Component versions and allowed updates ................................................. 42

103 Requirements for new software and hardware compliance ......................... 42

104 9.4 “Implementer-specified” metric minor update requirements .................................... 51

105 (normative) SERTv2 pre-approved CPU architectures, operating systems, and

106 Java Virtual Machines..................................................................................................... 53

107 (normative) SEEM pre-approved power analysers and temperature sensors ........... 64

108 (normative) SERTv2 required documentation fields ................................................. 66

109 (normative) Deployed power assessment ................................................................ 68

110 D.1 General .................................................................................................................... 68

111 D.2 Deployed power calculations .................................................................................... 68

112 D.3 Deployed power visualization ................................................................................... 69

113 D.3.1 Charting deployed power ................................................................................. 69

114 D.3.2 Creating a deployed power ranking table ......................................................... 71

115 D.4 Deployed power analysis ...................................................................................... 73

116 D.4.1 General ........................................................................................................... 73

117 D.4.2 Interpreting deployed power chart results ........................................................ 73

118 D.4.3 Interpreting deployed power ranking tables ..................................................... 74

119 Bibliography .......................................................................................................................... 76

120
121 List of tables
122

123 Table 1 - SERTv2 server categories by form factor and function (Option 1) ........................... 33

124 Table 2 - SERTv2 server categories by CPU architecture class (Option 2) ............................ 34

125 Table A.1 - Pre-approved CPU Architectures, Operating Systems, and Java Virtual

126 Machines .............................................................................................................................. 53

127 Table B.1 – Pre-approved power analysers ........................................................................... 64

128 Table B.2 – Pre-approved temperature sensors .................................................................... 65

129 Table D.1 - Example Ranking Analysis Table ........................................................................ 73

iv © ISO/IEC 2019 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC DIS 21836:2019(E)
130
131 List of figures
132

133 Figure 1 - Example SERTv2 HTML Result Output ................................................................. 39

134 Figure 2 - Example SERTv2 Text Result Output .................................................................... 39

135 Figure D.1 - Example Ideal Deployed Power Chart ................................................................ 71

136 Figure D.2 - Example Deployed Idle Power Chart.................................................................. 71

137
© ISO/IEC 2019 – All rights reserved
---------------------- Page: 5 ----------------------
ISO/IEC DIS 21836:2019(E)
138 Foreword

139 ISO (the International Organization for Standardization) is a worldwide federation of

140 national standards bodies (ISO member bodies). The work of preparing International

141 Standards is normally carried out through ISO technical committees. Each member body

142 interested in a subject for which a technical committee has been established has the right to

143 be represented on that committee. International organizations, governmental and non-

144 governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with

145 the International Electrotechnical Commission (IEC) on all matters of electrotechnical

146 standardization.

147 The procedures used to develop this document and those intended for its further

148 maintenance are described in the ISO/IEC Directives, Part 1. In particular the different

149 approval criteria needed for the different types of ISO documents should be noted. This

150 document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part

151 2 (see www.iso.org/directives).

152 Attention is drawn to the possibility that some of the elements of this document may be the

153 subject of patent rights. ISO shall not be held responsible for identifying any or all such

154 patent rights. Details of any patent rights identified during the development of the

155 document will be in the Introduction and/or on the ISO list of patent declarations received

156 (see https://www.iso.org/iso-standards-and-patents.html).

157 Any trade name used in this document is information given for the convenience of users and

158 does not constitute an endorsement.

159 For an explanation on the meaning of ISO specific terms and expressions related to

160 conformity assessment, as well as information about ISO’s adherence to the WTO principles

161 in the Technical Barriers to Trade (TBT) see, https://www.iso.org/foreword-
162 supplementary-information.html.

163 The committee responsible for this document is ISO/IEC JTC 1, Information technology, SC

164 39, Sustainability for and by Information Technology.
165
vi © ISO/IEC 2019 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC DIS 21836:2019(E)
166 Introduction

167 The global economy is now totally reliant on information and communication technologies

168 (ICT) and the associated generation, transmission, dissemination, computation, and storage

169 of digital data. While the Internet backbone carries the traffic, it is data centres which find

170 themselves at the nodes and hubs of a wide variety of both private enterprise and

171 shared/collocation facilities. With the large and continually increasing data capacity

172 demands placed on data centres worldwide, efficient use of data centre energy is an

173 extremely important strategy to manage environmental, cost, electrical grid capacity and

174 other impacts.

175 The ISO/IEC 30134 series of International Standards specify data centre energy

176 effectiveness KPIs to help data centre operators measure and improve specific aspects of

177 data centre energy effectiveness. ISO/IEC 30134-4, in particular, defines a method to

178 measure the peak capacity and utilization of servers operating in a data centre using

179 operator selected benchmarks. It, however, does not provide a method of comparing

180 individual server energy effectiveness across data centres and as stated in ISO/IEC 30134-

181 4, “should not be used to set regulation for a data centre or individual server”. There is

182 stakeholder demand for an international standard to measure the energy effectiveness of

183 servers before procurement and installation, particularly for use in worldwide server

184 energy effectiveness regulations and programs.

185 This standard provides a server energy effectiveness metric (SEEM) to measure and report

186 the energy effectiveness of specific server designs and configurations. This document will

187 be useful to stakeholders, including vendors, users and governments, from the design

188 verification testing phase all the way through compliance verification, procurement and

189 operation. Organizations that wish to establish compliance or reporting programs will find

190 that the test methods and scoring specified in ISO/IEC 21836 will save them significant time

191 and effort in implementing such programs. Standardization across such programs will allow

192 vendors to comply to stakeholder requirements more quickly and efficiently.

193 For applicable servers, ISO/IEC 21836 builds upon the widely-adopted Server Efficiency

194 Rating Tool™ (SERT), developed by the Standard Performance Evaluation Corporation

195 (SPEC ), as the energy effectiveness metric and test method. For servers where SERT is not

196 applicable, this standard provides requirements for the creation of alternate server energy

197 effectiveness metrics, referred to as “implementer-specified” metrics.
198
199
vii
© ISO/IEC 2019 – All rights reserved
---------------------- Page: 7 ----------------------
DRAFT INTERNATIONAL STANDARD ISO/IEC DIS 21836:2019(E)
200 Information technology – Data centres – Server Energy
201 Effectiveness Metric
202
203 1 Scope

204 This document specifies a measurement method to assess and report the energy effectiveness of

205 a computer server. This standard does not set any pass/fail criteria for servers.

206 2 Normative references

207 The following documents are referred to in the text in such a way that some or all of their content

208 constitutes requirements of this document. For dated references, only the edition cited applies.

209 For undated references, the latest edition of the referenced document (including any

210 amendments) applies.
211 SPEC Server Efficiency Rating Tool (SERT) version 2.x.x
212 SPEC PTDaemon™ version 1.8.1 and later
213 SPEC SERT Run and Reporting Rules version 2.0.1 (20170929)

214 SPEC Client Configuration XML (https://www.spec.org/sert2/SERT-JVM_Options-2.0.html)

215 3 Definitions and abbreviations
216 3.1 Definitions

217 For the purposes of this document, the following terms and definitions apply.

218 ISO and IEC maintain terminological databases for use in standardization at the following

219 addresses:
220 • IEC Electropedia: http://www.electropedia.org/
221 • ISO Online browsing platform: http://www.iso.org/obp
222
223 64-bit server

224 server which includes a CPU which has data path widths, memory addressing, registers, and other

225 architectural features which are 64-bits wide
226
227 active state

228 operational state in which the server is carrying out data processing in response to external

229 requests

230 Note 1 to entry: Example: active processing, data seeking/retrieval from memory, cache, internal/external

231 storage while awaiting further input over the network
© ISO/IEC 2019 – All rights reserved 1
---------------------- Page: 8 ----------------------
ISO/IEC DIS 21836:2019(E)
232
233 active state power
234 average server power, in watts, when server is performing data processing
235
236 auxiliary processing accelerator

237 additional compute device installed in the computer server that handles parallelized workloads

238 in conjunction with the CPU
239
240 blade chassis

241 enclosure that contains shared resources for the operation of blade servers, blade storage, and

242 other blade form-factor devices

243 Note 1 to entry: Shared resources provided by a chassis may include power supplies, data storage and

244 hardware for DC power distribution, thermal management, system management, and network services

245
246 blade server
247 server that is designed for use in a blade chassis

248 Note 1 to entry: A blade server is a high-density device that functions as an independent server and includes

249 at least one processor and system memory, but is dependent upon shared blade chassis resources (e.g.

250 power supplies, cooling) for operation
251
252 blade storage

253 storage device that is designed for use in a blade chassis that is dependent upon shared blade

254 chassis resources, like power supplies or cooling, for operation.
255
256 buffered memory

257 circuitry between the server’s memory and memory controller to either increase memory

258 capacity, increase bandwidth, and/or reduce the electrical load on the memory controller

259
260 coefficient of determination

261 statistic used to determine the strength of a fit between a mathematical model and a set of

262 observed data values
263 [SOURCE: ISO 15551-1:2015, 3.26, Note 1 has been removed]
264
265 coefficient of variation
266 standard deviation divided by the mean
267 [SOURCE: ISO 3534-1:2006]
© ISO/IEC 2019 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC DIS 21836:2019(E)
268
269 configuration

270 interrelated functional and physical characteristics of a product specified in product configuration

271 information

272 Note 1 to entry: This standard employs the following configurations: low-end, high-end and typical

273 Note 2 to entry: For server products, a configuration is one of many possible combinations of components

274 including CPU, storage devices, memory size and capacity and input/output devices for a single server

275 product within a larger product family. There are a large number of possible configurations within a

276 product family.
277 [SOURCE: ISO 17599:2015, 3.15 – modified, Note 1 and Note 2 have been added]
278
279 core

280 component of a processor which can independently receive instructions and takes actions or

281 performs calculations in response
282
283 CPU

284 a central processing element with functions for interpreting and executing instructions.

285 Note 1 to entry: In this standard, cache memory is included with the CPU

286 Note 2 to entry: This standard uses the terms CPU and processor interchangeably

287 [SOURCE: ISO 14576:1999, 2.1.9 – modified, Definition is shortened, Note 1 and Note 2 have been added]

288
289 CPU architecture

290 CPU design with significant similarities to other CPU architectures within the same CPU

291 architecture class

292 Note 1 to entry: CPU architectures are used to create CPU models which are often released in a similar

293 timeframe

294 Note 2 to entry: Examples of CPU architectures in the same CPU architecture class are Intel Haswell, Intel

295 Broadwell, and Intel Skylake, or separately, AMD Bulldozer, AMD Piledriver, and AMD Steamroller

296
297 CPU architecture class

298 group of one or more CPU architectures which share the same instruction set architecture and in

299 which newer architectures designs are derived from previous architecture designs

300 Note 1 to entry: Within a CPU architecture class, the initial CPU architecture is, for the most part, a new

301 design, and subsequent CPU architectures are derived from preceding CPU architectures

® ®

302 Note 2 to entry: Examples of different CPU architecture classes are ARMv8-A and AMD EPYC

303 Note 3 to entry: In certain cases, software programs need to be recompiled for use with different CPU

304 architecture classes
305
306 CPU model
307 specific CPU that is sold in the marketplace
© ISO/IEC 2019 – All rights reserved 3
---------------------- Page: 10 ----------------------
ISO/IEC DIS 21836:2019(E)

308 Note 1 to entry: All CPUs of the same model share the same technical characteristics, such as core

309 frequencies and core counts, and can be used interchangeably

310 Note 2 to entry: Examples of different CPU models are AMD EYPC 7601, AMD EYPC 7251 and Intel Xeon

311 Platinum 8180
312
313 CPU nominal frequency

314 the CPU core clock frequency which is the main frequency used in naming, marketing, and selling

315 the CPU
316
317 data averaging internal

318 for a power analyser, the time period over which all samples captured by the high-speed sampling

319 electronics of the analyser are averaged to provide a set of measured data
320
321 double data rate

322 transfers data on the rising and falling edges of the memory clock signal, resulting in twice the

323 memory bandwidth capacity at a specific clock frequency
324
325 end user

326 person or persons who will ultimately be using the system for its intended purpose

327 Note 1 to entry: for the purposes of this standard, the end user is the entity applying for certification of a

328 server model to a SEEM compliant regulation or program. For example, if server manufacturer A was

329 submitting a server model to ENERGY STAR for certification, server manufacturer A would be the end user.

330 [SOURCE: ISO/IEC/IEEE 24765:2010, 3.990, Note 1 has been added]
331
332 energy effectiveness

333 measure of the amount of data processing performed for a given amount of energy consumed

334 Note 1 to entry: for the purposes of this standard energy effectiveness is equivalent to the term energy

335 efficiency as used in server compliance regulations and programs such as ENERGY STAR®.

336
337 expansion auxiliary processing accelerator

338 auxiliary processing accelerator that is an add-in card installed in a general-purpose add-in

339 expansion slot.

340 Note 1 to entry: An expansion APA add-in card may include one or more APAs and/or separate, dedicated

341 removable switches
342 Note 2 to entry: example is a GPGPU installed in a PCI-e slot
343
344 fair use

345 uses of content that are considered valid defences to copyright infringement, such as for criticism

346 or educational purposes
© ISO/IEC 2019 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC DIS 21836:2019(E)

347 Note 1 to entry: For the purposes of this standard, fair use refers to the guidelines the developer of a metric

348 has specified related to how results can be used

349 [SOURCE: ISO 19153:2014, 4.14 – modified, original Note 1 remove, Note 1 added]

350
351 fully fault tolerant server

352 computer server that is designed with complete hardware redundancy, in which every computing

353 component is replicated between two nodes running identical and concurrent workloads

354 Note 1 to entry: A fully fault tolerant server uses two systems to simultaneously and repetitively run a

355 single workload for continuous availability in a mission critical application

356 Note 2 to entry: An example of a fault tolerant server; if one node fails or needs repair, the second node

357 can run the workload alone to avoid downtime
358
359 hardware threads

360 in a CPU core, the number of fully independent instruction streams which can be executed through

361 SMT
362
363 high performance computing (HPC) system

364 computing system which is designed, marketed, sold, and optimized to execute highly parallel

365 applications for high performance, deep learning, or artificial intelligence applications

366 Note 1 to entry: HPC systems consist of multiple clustered servers, primarily for increased computational

367 capability, high speed inter-processing interconnects, large and high bandwidth memory capability and

368 often accelerators such as GPGPUs or FPGAs

369 Note 2 to entry: HPC systems may be purposely built or assembled from more commonly available computer

370 servers
371
372 high-end configuration

373 server equipped with a specific selection of high performance components, which is required to

374 be tested as part of measuring a server product family
375
376 idle state

377 operational mode in which the OS and other software have completed loading, the server is

378 capable of completing workload transactions, but no active workload transactions are requested

379 or pending by the system

380 Note 1 to entry: In the idle state, the server is operational, but not performing any useful data

381 processing

382 Note 2 to entry: For systems where Advanced Configuration and Power Interface (ACPI) has

383 been implemented, idle state is the ACPI G0 global state and S0 sleep state.
384
385 idle state power
386 average server power, in watts, when in idle state
© ISO/IEC 2019 – All rights reserved 5
---------------------- Page: 12 ----------------------
ISO/IEC DIS 21836:2019(E)

387 Note 1 to entry: SERT provides a standard way to measure the idle power of a server, which is included

388 with the result output, and is in addition to power measurement while the server is actively performing data

389 processing
390
391 implementer
392 entity that transforms specified designs into their physical realization

393 Note 1 to entry: For the purposes of this standard, implementer is the entity which creates a selection or

394 procurement program based on SEEM.
395 [SOURCE: IEC 62279:2015, 3.1.15, Note 1 added]
396
397 integrated auxiliary processing accelerator
398 auxiliary proces
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.