SIST EN 303 470 V1.1.1:2019
(Main)Environmental Engineering (EE) - Energy Efficiency measurement methodology and metrics for servers
Environmental Engineering (EE) - Energy Efficiency measurement methodology and metrics for servers
The present document specifies a metric using the Server Efficiency Rating Tool (SERT™), test conditions and product family configuration for the assessment of energy efficiency of computer servers using reliable, accurate and reproducible measurement methods. The metric applies to general purpose computer servers with up to four processor sockets and with their own dedicated power supply.
NOTE 1: The term "socket" also applies to design in which processors are installed without sockets (e.g. soldered products).
The metric applies to a computer server model and to a computer server product family, including type and count of CPU, memory, storage, power supplies, cooling (e.g. fans) and any other add-on hardware expected to be present when deployed.
The present document defines:
• an energy efficiency metric to support procurement or market entry requirements;
• requirements for equipment to perform the measurements and analysis;
• requirements for the measurement process;
• requirements for the management of the metric calculation;
• operation or run rules to configure, execute, and monitor the testing;
• documentation and reporting requirements;
• a validation process for the metric using the Deployed Power Assessment.
The present document is not applicable to:
• fully fault tolerant servers;
• High Performance Computing (HPC) systems;
• hyper-converged servers;
• large scale servers;
• servers with integrated APA(s);
• networking equipment including network servers;
• server appliances;
• storage device including blade storage and storage servers.
NOTE 2: Products whose feature set and intended operation are not addressed by active mode testing parameters are excluded from this evaluation method. The above list shows products for which SERT™ efficiency evaluations are not appropriate.
The present document does not address home servers and small servers that fall under the scope of mandate M/545 [i.8].
Okoljski inženiring (EE) - Metodologija merjenja energijske učinkovitosti in metrika za strežnike
Ta dokument določa metriko z uporabo orodja Server Efficiency Rating Tool (SERT™), preskusnih pogojev in konfiguracije skupine izdelkov za ocenjevanje energijske učinkovitosti računalniških strežnikov z uporabo zanesljivih, natančnih in ponovljivih merilnih metod. Metrika se uporablja za računalniške strežnike za splošne namene z največ štirimi priključki procesorja in lastnim namenskim napajanjem.
OPOMBA 1: Izraz »priključek« se uporablja tudi za zasnovo, pri kateri so procesorji nameščeni brez priključkov (npr. spajkani izdelki).
Metrika se uporablja za model in skupino izdelkov računalniškega strežnika, vključno z vrsto in številom CPE, pomnilnikov, shramb, napajalnikov, hladilnih enot (npr. ventilatorjev) in katere koli druge dodatne strojne opreme, za katero se pričakuje, da je prisotna ob razmestitvi.
Ta dokument določa:
• metriko energijske učinkovitosti za podporo zahtev za nabavo ali vstop na trg;
• zahteve za opremo v zvezi izvajanjem meritev in analizo;
• zahteve za postopek merjenja;
• zahteve za upravljanje izračuna metrike;
• pravila delovanja za konfiguriranje, izvajanje in spremljanje preskusov;
• zahteve za dokumentiranje in poročanje;
• proces potrjevanja za metriko z uporabo postopka »Deployed Power Assessment«.
Ta dokument se ne uporablja za:
• strežnike s popolno toleranco napak;
• sisteme visoko zmogljivega računalništva (HPC);
• hiperkonvergentne strežnike;
• velike strežnike;
• strežnike s pomožnimi procesorskimi pospeševalniki (APA);
• omrežno opremo, vključno z omrežnimi strežniki;
• strežniške naprave;
• naprave za shranjevanje, vključno z rezinskimi pomnilniki in strežniki za shranjevanje podatkov.
OPOMBA 2: Izdelki z naborom funkcij in predvidenim delovanjem, ki jih parametri za preskušanje aktivnega delovanja ne obravnavajo, so izvzeti iz te metode vrednotenja. Na zgornjem seznamu so navedeni izdelki, za katere vrednotenje učinkovitosti SERT™ ni primerno.
Ta dokument ne obravnava strežnikov za domačo uporabo in majhnih strežnikov, ki spadajo na področje uporabe mandata M/545 [i.8].
General Information
Standards Content (Sample)
Draft ETSI EN 303 470 V1.0.0 (2018-06)
EUROPEAN STANDARD
Environmental Engineering (EE);
Energy Efficiency measurement methodology
and metrics for servers
2 Draft ETSI EN 303 470 V1.0.0 (2018-06)
Reference
DEN/EE-EEPS24
Keywords
energy efficiency, ICT, server, sustainability
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.
© ETSI 2018.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are trademarks of ETSI registered for the benefit of its Members.
TM TM
3GPP and LTE are trademarks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
oneM2M logo is protected for the benefit of its Members. ®
GSM and the GSM logo are trademarks registered and owned by the GSM Association.
ETSI
3 Draft ETSI EN 303 470 V1.0.0 (2018-06)
Contents
Intellectual Property Rights . 6
Foreword . 6
Modal verbs terminology . 6
Introduction . 6
1 Scope . 8
2 References . 9
2.1 Normative references . 9
2.2 Informative references . 9
3 Definitions and abbreviations . 10
3.1 Definitions . 10
3.2 Abbreviations . 14
4 Server product categories and representative product family configurations . 15
4.1 General . 15
4.2 Applications and metric applicability . 15
4.3 Computer servers . 16
4.3.1 General requirements . 16
4.3.2 Form-factors . 16
4.4 Computer server product categories . 16
4.5 Computer server configuration . 17
4.5.1 General . 17
4.5.2 "High-end" performance configuration . 17
4.5.3 "Low-end" performance configuration . 17
5 Metrics . 18
5.1 Active state metric . 18
5.1.1 Worklets . 18
5.1.2 Formulae . 18
5.1.2.1 General . 18
5.1.2.2 Metric definition . 18
5.1.3 Weightings . 19
5.2 Idle state metric . 19
6 Test setup . 20
6.1 General . 20
6.2 Input power . 20
6.3 Environmental conditions . 20
6.3.1 Ambient Temperature . 20
6.3.2 Relative Humidity . 21
6.4 Power analyser . 21
6.5 Temperature sensor . 21
6.6 Active state test tool . 21
6.7 Controller system . 21
6.8 General SERT™ requirements . 22
7 Equipment Under Test (EUT) . 22
7.1 Configuration . 22
7.2 Test procedure . 23
8 Measurement . 24
8.1 Measurement for active state . 24
8.2 Sensitivity Analysis . 25
8.3 Measurement for power supply . 25
8.3.1 Measurement for internal power supply . 25
8.3.2 Measurement for test board power supply . 26
8.3.2.1 General . 26
ETSI
4 Draft ETSI EN 303 470 V1.0.0 (2018-06)
8.3.2.2 Test loads . 26
8.3.2.3 Test leads and wiring . 26
8.3.2.4 Warm up time . 26
8.3.2.5 Power measurements . 26
9 Measurement report . 26
Annex A (normative): Resilient server requirements . 28
A.1 Reliability, Availability and Serviceability (RAS) features . 28
A.2 Reliability, Availability and Serviceability (RAS) requirements . 28
Annex B (informative): Deployed Power Assessment . 30
B.1 Overview . 30
B.2 Determining the number of deployed servers . 30
B.2.1 General . 30
B.2.2 Establishing target performance . 31
B.2.3 Weighting factors . 31
Annex C (informative): Alternative calculation . 33
Annex D (informative): Bibliography . 34
History . 35
ETSI
5 Draft ETSI EN 303 470 V1.0.0 (2018-06)
List of tables
Table 1: Configuration of EUT .22
Table 2: Test configuration of EUT .23
Table 3: Measurement of active state .25
Table 4: Measurement of internal power supply .25
Table A.1: Resilient Server requirements.29
ETSI
6 Draft ETSI EN 303 470 V1.0.0 (2018-06)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
Foreword
This draft European Standard (EN) has been produced by ETSI Technical Committee Environmental Engineering (EE),
and is now submitted for the combined Public Enquiry and Vote phase of the ETSI standards EN Approval Procedure.
Proposed national transposition dates
Date of latest announcement of this EN (doa): 3 months after ETSI publication
Date of latest publication of new National Standard
or endorsement of this EN (dop/e): 6 months after doa
Date of withdrawal of any conflicting National Standard (dow): 6 months after doa
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
The present document specifies a metric for the assessment of energy efficiency of computer servers using reliable,
accurate and reproducible measurement methods, which take into account the recognized state of the art.
The present document formalizes the tools, conditions and calculations used to generate a single figure of merit of a
single computer server representing its relative efficiency and power impact. The metric is targeted for use as a tool in
the selection process of servers to be provisioned.
ETSI
7 Draft ETSI EN 303 470 V1.0.0 (2018-06)
For comparisons, evaluations should be conducted across similar server types or categories. The efficiency metric is
targeted for use in a pass/fail selection process by differentiating the ability of servers to be provisioned for general
purpose operations. The present document does not prescribe the levels or values for acceptance but prescribes standard
method of evaluation from which energy efficiency programs would use to establish such criteria.
As there are many operational deployments of servers resulting in a range of specialized equipment and configurations
for a single server product, a metric that evaluates provisioning impacts to general purpose operations may not be
applicable. ICT equipment and servers in particular, are generally customized and commissioned on site for
deployment. As with most IT equipment, new technologies are regularly introduced, which may require product level
customization or an industry wide tool upgrade to more appropriately represent the efficiency of the servers. The
present document categorizes servers to address applicability, configuration groupings to represent a family of servers
to address the broad range of custom configurations possible within each server product family, and tool revision
control to ensure comparability and consistency of the resulting metric value.
The present document is based upon the Server Efficiency Rating Tool™ (SERT™) of the Standard Performance
Evaluation Corporation (SPEC) and takes into account:
• the Ecodesign Technical Assistance Study on Standards for ErP Lot 9 Enterprise Servers and Enterprise Data
Storage;
• activity related to the analysis of output of Server Efficiency Rating Tool (SERT™) tool measurements and
deployed power by The Green Grid; ®
• ENERGY STAR for Computer Servers [i.2].
The present document defines energy efficiency metrics and measurement methodology for server equipment under
standardization Mandate M/462 of the European Commission [i.3].
ETSI
8 Draft ETSI EN 303 470 V1.0.0 (2018-06)
1 Scope
The present document specifies a metric using the Server Efficiency Rating Tool ((SERT™), test conditions and
product family configuration for the assessment of energy efficiency of computer servers using reliable, accurate and
reproducible measurement methods. The metric applies to general purpose computer servers with up to four processor
sockets, with their own dedicated power supply.
NOTE 1: The term "socket" also applies to design in which processors are installed without sockets (e.g. soldered
products).
The metric applies to a computer server product family, including type and count of CPU, memory, storage, power
supplies, cooling (e.g. fans) and any other add-on hardware expected to be present when deployed.
The present document defines:
• an energy efficiency metric to support procurement or market entry requirements;
• requirements for equipment to perform the measurements and analysis;
• requirements for the measurement process;
• requirements for the management of the metric calculation;
• operation or run rules to configure, execute, and monitor the testing;
• documentation and reporting requirements;
• a validation process for the metric using the Deployed Power Assessment.
The present document is not applicable to:
• fully fault tolerant servers;
• High Performance Computing (HPC) systems;
• hyper-converged servers;
• large scale servers;
• servers with non-removable APA(s);
• networking equipment including network servers;
• server appliances;
• storage products including blade storage and storage servers.
NOTE 2: Products whose feature set and intended operation are not addressed by active mode testing parameters
are excluded from this evaluation method. The above list shows products for which SERT™ efficiency
evaluations are not appropriate.
The present document does not address home servers and small servers under the Mandate M/545 [i.8].
ETSI
9 Draft ETSI EN 303 470 V1.0.0 (2018-06)
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference/.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
[1] CENELEC EN 62623:2013: "Desktop and notebook computers. Measurement of energy
consumption".
[2] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT)
version 2 Run and Reporting Rules".
NOTE: Available at https://www.spec.org/sert2/SERT-runrules.pdf.
[3] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT)
version 2 User Guide".
NOTE: Available at https://www.spec.org/sert2/SERT-userguide.pdf.
[4] IEEE 802.3™: "IEEE Standard for Ethernet".
NOTE: Available at https://standards.ieee.org/findstds/standard/802.3-2015.html.
[5] IEEE 802.3az™: "Energy Efficient Ethernet".
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] CENELEC EN 60297 series: "Mechanical structures for electrical and electronic equipment.
Dimensions of mechanical structures of the 482,6 mm (19 in) series". ®
[i.2] ENERGY STAR : "Product Specification for Computer Servers".
[i.3] Standardization Mandate M/462: "M/462 Standardisation mandate addressed to CEN, CENELEC
and ETSI in the field of ICT to enable efficient energy use in fixed and mobile information and
communication networks".
[i.4] ETSI EN 300 119 series: "Equipment Engineering (EE); European telecommunication standard for
equipment practice".
[i.5] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT)
version 2 Design Document".
NOTE: Available at https://www.spec.org/sert2/SERT-designdocument.pdf.
ETSI
10 Draft ETSI EN 303 470 V1.0.0 (2018-06)
[i.6] SERT Client Configurations (JVM Options).
NOTE: Available at https://www.spec.org/sert2/SERT-JVM_Options-2.0.html.
[i.7] SERT Result File Fields.
NOTE: Available at https://www.spec.org/sert2/SERT-resultfilefields.html.
[i.8] Mandate M/545 Commission Implementing Decision of 6.1.2016 on a standardisation request to
the European standardisation organisations as regards computers and computer servers, in support
of the implementation of Commission Regulation (EU) No 617/2013 of 26 June 2013,
implementing Directive 2009/125/EC of the European Parliament and of the Council with regard
to ecodesign requirements for computers and computer servers".
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the following terms and definitions apply:
a.c.-d.c. power supply unit: power supply unit that converts line-voltage alternating current (a.c.) input power into one
or more direct current (d.c.) power outputs for powering a computer server
active state: operational state of a computer server (as opposed to the idle state) in which the computer server is
carrying out work in response to prior or concurrent external requests (e.g. instruction over the network)
NOTE: The work includes, but is not restricted to, active processing and data seeking/retrieval from memory,
cache, or internal/external storage while awaiting further input over the network.
Auxiliary Processing Accelerator (APA): additional compute device installed in the computer server that handles
parallelized workloads
NOTE 1: This includes, but is not limited to, Graphical Processing Units (GPUs) or Field Programmable Gate
Array chips which can be installed in a server either on Graphics or Extension add-in cards installed in
general-purpose add-in expansion slots (e.g. GPGPUs, CPU accelerators, etc. installed in a PCI slot) or
direct attached to a server component such as the motherboard.
NOTE 2: There are two specific types of APAs used in servers:
a) Expansion APA: An APA that is on an add-in card installed in an add-in expansion slot (e.g.
GPGPUs, CPU accelerators, etc. installed in a PCI slot). An expansion APA add-in card may
include one or more APAs.
b) Integrated APA: An APA that is integrated into the motherboard or CPU package or an expansion
APA that has part of its subsystem, such as switches, included in the non-APA server configuration
that would be used to run the energy efficiency test (SERT suite).
blade chassis: enclosure that contains shared resources for the operation of blade servers, blade storage, and other blade
form-factor devices
NOTE: Shared resources provided by a chassis include, but are not restricted to, power supplies, data storage, and
hardware for d.c. power distribution, thermal management, system management, and network services.
blade server: computer server, designed for use in a blade chassis, that is a high-density device and functions as an
independent computer server and includes at least one processor and system memory, which is dependent upon shared
blade chassis resources (e.g. power supplies, cooling) for operation
NOTE: A processor or memory module that is intended to scale up a standalone server is not considered a blade
server.
blade storage: storage device that is designed for use in a blade chassis and that is dependent upon shared blade chassis
resources (e.g. power supplies, cooling) for operation
ETSI
11 Draft ETSI EN 303 470 V1.0.0 (2018-06)
blade system: blade chassis and one or more removable blade servers and/or other units (e.g. blade storage, blade
networking equipment) which provide a scalable means for combining multiple blade server or storage units in a single
enclosure
NOTE: A blade system is designed to allow service technicians to easily add or replace (hot-swap) blades in the
field.
buffered Double Data Rate (DDR) channel: channel or memory port connecting a memory controller to a defined
number of memory devices (e.g. dual in-line memory modules (DIMMs)) in a computer server
NOTE 1: A typical computer server may contain multiple memory controllers, which may in turn support one or
more buffered DDR channels.
NOTE 2: Each buffered DDR channel serves only a fraction of the total addressable memory space in a computer
server.
computer server: computer, sold through enterprise channels, that provides services and manages networked resources
for client devices
NOTE 1: Client devices include, but are not restricted to desktop computers, notebook computers, thin clients,
wireless devices, Personal Digital Assistants, IP telephones, other computer servers, or other network
devices.
NOTE 2: A computer server is primarily accessed via network connections, versus directly-connected user input
devices such as a keyboard or mouse.
controller system: computer or computer server that manages a benchmark evaluation process
data averaging interval: time period over which all samples captured by the high-speed sampling electronics of the
power analyser are averaged to provide the measurement set
d.c.-d.c. power supply unit: power supply unit that converts line-voltage direct current (d.c.) input power to one or
more d.c. outputs for powering a computer server
NOTE: For purposes of the present document, a d.c-d.c. converter that is internal to a computer server and is used
to convert a low voltage d.c. (e.g. 12 VDC) into other d.c. power outputs for use by computer server
components is not considered a d.c-d.c. power supply.
deployed power: average power level of the utilization applicable to the total number of servers provisioned to meet an
aggregate peak load
direct current server: computer server that is designed solely to operate on a direct current (d.c.) power source
double-wide blade server: blade server requiring twice the width of a standard blade server bay
efficiency: defined workload output divided by the resource input to the system
fully fault tolerant server: computer server that is designed with complete hardware redundancy, in which every
computing component is replicated between two nodes running identical and concurrent workloads (i.e. if one node
fails or needs repair, the second node can run the workload alone to avoid downtime) and that uses two systems to
simultaneously and repetitively run a single workload for continuous availability in a mission critical application
half-height blade server: blade server requiring one half the height of a standard blade server bay
hard disk drive: primary computer storage device which reads and writes to one or more rotating magnetic disk
platters
High Performance Computing (HPC) system: computing system which is designed (or assembled), optimized,
marketed and sold to execute highly parallel applications for higher performance computing applications
NOTE 1: HPC systems support applications including, but not restricted to, deep learning or artificial intelligence.
NOTE 2: HPC systems feature multiple clustered nodes to increase computational capability and often featuring
high speed inter-processing interconnects as well as large memory capability and bandwidth.
ETSI
12 Draft ETSI EN 303 470 V1.0.0 (2018-06)
NOTE 3: HPC systems may be comprised of multiple clusters of homogenous nodes, for which the clusters may be
heterogeneous.
hyper-converged server: highly integrated enterprise device which contains the same components as a computer
server in addition to the features of a network server and storage server
hypervisor: supervisory system level software that establishes and manages a virtualized environment which enables
multiple operating systems to run on a single physical system at the same time
ICT equipment: equipment providing data storage, processing and transport services
NOTE: A combination of Information Technology Equipment and Network Telecommunications Equipment.
idle state: operational state of a computer server in which the operating system and other software have completed
loading but is not performing any useful work
NOTE 1: The computer server is capable of completing workload transactions, but no active workload transactions
are requested or pending by the system.
NOTE 2: For systems where ACPI standards are applicable, idle state correlates only to ACPI System Level S0.
I/O device: device which provides data input and output capability between a computer server and other devices
NOTE: An I/O device may be integral to the computer server motherboard or may be connected to the
motherboard via expansion slots.
I/O port: physical circuitry within an I/O device where an independent I/O session can be established
NOTE: A port is not the same as a connector receptacle; it is possible that a single connector receptacle can
service multiple ports of the same interface.
large scale server: resilient/scalable server which ships as a pre-integrated/pre-tested system housed in one or more full
frames or racks and that includes a high connectivity input/output subsystem with a minimum of 32 dedicated
input/output slots
maximum power: peak sustained or root means square power consumption value while operating the worst case
functions
memory: server component external to the processor in which information is stored for immediate use by the processor
motherboard: main circuit board of the server typically accommodating the processor, memory, expansion slots and
enabling the attachment of additional circuit boards
multi-bay blade server: blade server requiring more than one bay for installation in a blade chassis
multi-output power supply unit: power supply unit designed to deliver the majority of its rated output power to more
than one primary direct current (d.c.) output for the purpose of powering a computer server
NOTE 1: Multi-output power supply units may offer one or more standby outputs that remain active whenever
connected to an input power source.
NOTE 2: The total rated power output from any additional power supply unit outputs, other than primary or
standby outputs, is greater than or equal to 20 W.
network client (testing): computer or computer server that generates workload traffic for transmission to an Equipment
Under Test connected via a network switch
network server: large network device which contains the same components as a computer server together with more
than 11 ports, has a total line rate throughput of greater than or equal to 12 Gb/s and is designed to dynamically
reconfigure ports and speed and to support a virtualized network environment, software defined networking
NOTE: Supporting features are described by the product's datasheet description and are either accompanied with
vendor specific utilities and/or commercially available software supporting these functions.
networking equipment: device whose primary function is to pass data among various network interfaces, providing
data connectivity among connected devices (e.g. routers and switches) via the routing of data packets
ETSI
13 Draft ETSI EN 303 470 V1.0.0 (2018-06)
normalized performance: relative performance values calibrated to a baseline common to the set of equipment being
evaluated
pedestal server: self-contained computer server that is designed with power supply units, cooling, input/output devices,
and other resources necessary for stand-alone operation within a frame similar to that of a tower client computer
power supply unit: self-contained device, physically separable from the motherboard of the computer server, that
converts a.c. or d.c. input power to one or more d.c. power outputs for powering the computer server via a removable or
hard-wired electrical connection
processor: central processing unit of the computer server comprising logic circuitry that responds to and processes the
basic instructions that drive the server
product category: second-order classification or sub-type within a product group or form-factor that is based on
product features and installed components and used in the present document to determine qualification and test
requirements
quarter-height blade server: blade server requiring one quarter the height of a standard server bay
rack-mounted server: computer server that is designed for deployment in a standard 19 inch ICT equipment rack as
defined by CENELEC EN 60297 [i.1] or ETSI EN 300 119 [i.4]
NOTE: For the purposes of the present document, a blade server is considered under a separate product category
and excluded from the rack-mounted product category.
reported maximum power: highest maximum power recorded on the eleven SERT™ worklet scores for the two tested
configurations
resilient server: computer server designed with extensive Reliability, Availability, Serviceability (RAS) and scalability
features integrated in the micro architecture of the system, central processor unit (CPU) and chipset
NOTE: The requirements are listed in annex A.
server processor utilization: ratio of processor computing activity to full-load processor computing activity at a
specified voltage and frequency, measured instantaneously or with a short term average of use over a set of active
and/or idle cycles
server product family: high-level description for a group of servers sharing one chassis and motherboard combination
that may contain multiple hardware and software configurations
single output power supply unit: power supply unit designed to deliver the majority of its rated output power to one
primary direct current (d.c.) output for the purpose of powering a computer server
NOTE 1: Single-output power supply units may offer one or more standby outputs that remain active whenever
connected to an input power source.
NOTE 2: The total rated power output from any additional power supply units outputs, other than primary and
standby outputs, is not greater than 20 W.
NOTE 3: Power supply units that offer multiple outputs at the same voltage as the primary output are considered
single-output power supply units unless those outputs are generated from separate converters or have
separate output rectification stages, or have independent current limits.
single-wide blade server: blade server requiring the width of a standard blade server bay
solid state drive: storage device that uses memory chips instead of rotating magnetic platters for data storage
storage product: fully-functional storage system that supplies data storage services to clients and devices attached
directly or through a network
NOTE 1: A storage product may be composed of integrated storage controllers, storage devices, embedded network
elements, software, and other devices.
NOTE 2: While storage products may contain one or more embedded processors, these processors do not execute
user-supplied software applications but may execute data-specific applications (e.g. data replication,
backup utilities, data compression, install agents).
ETSI
14 Draft ETSI EN 303 470 V1.0.0 (2018-06)
NOTE 3: Components and subsystems that are an integral part of the storage product architecture (e.g. to provide
internal communications between controllers and disks) are considered to be part of the storage product.
NOTE 4: Components that are normally associated with a storage environment (e.g. devices required for operation
of an external Storage Area Network) are not considered to be part of the storage product.
storage server: enterprise storage device which contains the same components as a computer server together with
rd
≥ 10 storage devices and software (vendor or 3 party) that supports storage system connectivity, capacity optimization
management, virtualized storage environment and software defined storage
NOTE: Supporting features are described by the product's datasheet description and is either accompanied with
vendor specific utilities and/or commercially available software supporting these functions.
Uninterruptible Power Supply (UPS): combination of convertors, switches, and energy storage devices (such as
batteries) constituting a power system for maintaining continuity of load power in case of input power failure
weighted geometric mean: geometric mean calculated using a predetermined factor for each the elements prior to
aggregation
worklet: synthetic software routine, using real application functions focused on a particular type of computing activity,
which stresses a particular characteristic of the system
NOTE: A floating point and integer performance stress code is an example of a CPU worklet.
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
a.c., AC Alternating Current
ACPI Advanced Configuration and Power Interface
APA Auxiliary Processing Accelerator
BIOS Basic Input/Output System
CENELEC European Committee for Electrotechnical Standardization
CPU Central Processor Unit
d.c., DC Direct Current
DDR Double Data Rate
DIMM Dual In-line Memory Module
EEE Energy Efficient Ethernet
EUT Equipment Under Test
FPGA Field Programmable Gate Array
GB GigaByte
GPGPU General-Purpose computing on Graphics Processing Units
HDD Hard Disk Drive
HPC Higher Performance Computing
I/O Input/Output
ICT Information and Communication Technology
ISO/IEC International Standards Organization/International Electrotechnical Committee
IT Information Technology
JVM Java™ Virtual Machine
LU Lower-Upper
OS Operating System
PCI Peripheral Component Interconnect
PDU Power Distribution Unit
PSU Power Supply Unit
RAS Reliability, Availability and Serviceability
rms root mean square
SERT™ Server Efficiency Rating Tool
SOR Successive Over-relaxation
SPEC Standard Performance Evaluation Corporation
SSD Solid State Drive
SSJ Server Side Java™
UPS Uninterruptible Power Supply
ETSI
15 Draft ETSI EN 303 470 V1.0.0 (2018-06)
VDC Volts (DC)
WCPU Weighting of CPU
WMemory Weighting of Memory
WStorage Weighting of Storage
4 Server product categories and representative
product family configurations
4.1 General
Servers are sold in different form factors, processor socket counts, resilience levels and configuration types and
different groups of servers will have distinct performance capability and power demands.
Servers are categorized as defined in clause 4.4 by specific form factor and configuration parameters to enable the
setting of appropriate idle power or active efficiency thresholds and assessment of like products with regards to
procurement or market entry requirements.
To compare or evaluate systems, the evaluation or metric used shall only be made against like products. Like products
are grouped into categories. Products of different categories shall not be compared using the metric of the present
document.
Even though servers are classified in categories by the type of system, each server is customized by its configuration to
best match the application for which they are being sold or purchased. As a result, a product is represented by a fixed
set of configurations which bound the family of configurations sold.
For an appropriate evaluation, the category shall be defined and the product family configurations itemized.
The product family configuration establishes a single group representing the efficiency of the product, as this then
covers the range of configurations of that product that would be sold. Since the products sold are custom-configured,
there should be a minimum of 2 configurations that would represent the family of configurations, bounded by the
high-end configuration (see clause 4.5.2) and the low-end configuration (see clause 4.5.3).
A typical configuration is optional and would be represented by a computer server with characteristics within the range
defined by the high-end and low end configurations.
NOTE: For some categories, such as 1-socket blade servers, the number of configurations is limited. For
categories which contain limited configurations, "typical" may be a slight variation of the low-end and
high-end configurations.
4.2 Applications and metric applicability
Computer servers are architect
...
Final draft ETSI EN 303 470 V1.1.0 (2019-01)
EUROPEAN STANDARD
Environmental Engineering (EE);
Energy Efficiency measurement methodology
and metrics for servers
2 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
Reference
DEN/EE-EEPS24
Keywords
energy efficiency, ICT, server, sustainability
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.
© ETSI 2019.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are trademarks of ETSI registered for the benefit of its Members.
TM TM
3GPP and LTE are trademarks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and
of the oneM2M Partners. ®
GSM and the GSM logo are trademarks registered and owned by the GSM Association.
ETSI
3 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
Contents
Intellectual Property Rights . 6
Foreword . 6
Modal verbs terminology . 6
Introduction . 6
1 Scope . 8
2 References . 9
2.1 Normative references . 9
2.2 Informative references . 9
3 Definition of terms, symbols and abbreviations . 10
3.1 Terms . 10
3.2 Symbols . 14
3.3 Abbreviations . 14
4 Server product categories and representative product family configurations . 15
4.1 General . 15
4.2 Applications and metric applicability . 15
4.3 Computer servers . 16
4.3.1 General requirements . 16
4.3.2 Form-factors . 16
4.4 Computer server product categories . 16
4.5 Server product family configuration. 17
4.5.1 General . 17
4.5.2 "High-end" performance configuration . 17
4.5.3 "Low-end" performance configuration . 17
5 Metrics . 18
5.1 Active state metric . 18
5.1.1 Worklets . 18
5.1.2 Formulae . 18
5.1.2.1 General . 18
5.1.2.2 Active State Metric definition . 18
5.1.3 Weightings . 19
5.2 Idle state metric . 19
6 Test setup . 20
6.1 General . 20
6.2 Input power . 21
6.3 Environmental conditions . 21
6.3.1 Ambient temperature . 21
6.3.2 Relative humidity . 21
6.4 Power analyser . 21
6.5 Temperature sensor . 22
6.6 Active state test tool . 22
6.7 Controller system . 22
6.8 General SERT™ requirements . 22
7 Equipment Under Test (EUT) . 23
7.1 Configuration . 23
7.2 Test procedure . 24
8 Measurement . 25
8.1 Measurement for active state . 25
8.2 Sensitivity analysis . 25
8.3 Measurement for power supply . 26
8.3.1 Measurement for internal power supply . 26
8.3.2 Measurement for test board power supply . 26
ETSI
4 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
8.3.2.1 General . 26
8.3.2.2 Test loads . 26
8.3.2.3 Test leads and wiring . 26
8.3.2.4 Warm up time . 26
8.3.2.5 Power measurements . 27
9 Measurement report . 27
Annex A (normative): Resilient server requirements . 28
A.1 Reliability, Availability and Serviceability (RAS) features . 28
A.2 Reliability, Availability and Serviceability (RAS) requirements . 28
Annex B (informative): Deployed Power Assessment . 30
B.1 Overview . 30
B.2 Determining the number of deployed servers . 30
B.2.1 General . 30
B.2.2 Establishing target performance . 31
B.2.3 Weighting factors . 31
Annex C (informative): Alternative calculation . 33
Annex D (informative): Bibliography . 34
History . 35
ETSI
5 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
List of tables
Table 1: Configuration of EUT .23
Table 2: Test configuration of EUT .24
Table 3: Measurement of active state .25
Table 4: Measurement of internal power supply .26
Table A.1: Resilient Server requirements.29
ETSI
6 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
Foreword
This final draft European Standard (EN) has been produced by ETSI Technical Committee Environmental Engineering
(EE), and is now submitted for the Vote phase of the ETSI standards EN Approval Procedure.
Proposed national transposition dates
Date of latest announcement of this EN (doa): 3 months after ETSI publication
Date of latest publication of new National Standard
or endorsement of this EN (dop/e): 6 months after doa
Date of withdrawal of any conflicting National Standard (dow): 6 months after doa
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
The present document specifies a metric for the assessment of energy efficiency of computer servers using reliable,
accurate and reproducible measurement methods, which take into account the recognized state of the art.
The present document formalizes the tools, conditions and calculations used to generate a single figure of merit of a
single computer server representing its relative efficiency and power impact. The metric is targeted for use as a tool in
the selection process of servers to be provisioned.
ETSI
7 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
For comparisons, evaluations should be conducted across similar server types or categories. The efficiency metric is
targeted for use in a pass/fail selection process by differentiating the ability of servers to be provisioned for general
purpose operations. The present document does not prescribe the levels or values for acceptance but prescribes a
standard method of evaluation that energy efficiency programs would use to establish such criteria.
As there are many operational deployments of servers resulting in a range of specialized equipment and configurations
for a single server product, a metric that evaluates provisioning impacts to general purpose operations may not be
applicable. ICT equipment and servers in particular, are generally customized and commissioned on site for
deployment. As with most IT equipment, new technologies are regularly introduced, which may require product level
customization or an industry wide tool upgrade to more appropriately represent the efficiency of the servers. The
present document categorizes servers to address applicability, configuration groupings to represent a family of servers
to address the broad range of custom configurations possible within each server product family, and tool revision
control to ensure comparability and consistency of the resulting metric value.
The present document is based upon the Server Efficiency Rating Tool™ (SERT™) of the Standard Performance
Evaluation Corporation (SPEC) and takes into account:
• the Eco-design Technical Assistance Study on Standards for ErP Lot 9 Enterprise Servers and Enterprise Data
Storage;
• activity related to the analysis of output of Server Efficiency Rating Tool (SERT™) measurements and
deployed power by The Green Grid; ®
• ENERGY STAR for Computer Servers [i.2].
The present document defines energy efficiency metrics and measurement methodology for server equipment under
standardization mandate M/462 of the European Commission [i.3].
ETSI
8 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
1 Scope
The present document specifies a metric using the Server Efficiency Rating Tool (SERT™), test conditions and product
family configuration for the assessment of energy efficiency of computer servers using reliable, accurate and
reproducible measurement methods. The metric applies to general purpose computer servers with up to four processor
sockets and with their own dedicated power supply.
NOTE 1: The term "socket" also applies to design in which processors are installed without sockets (e.g. soldered
products).
The metric applies to a computer server model and to a computer server product family, including type and count of
CPU, memory, storage, power supplies, cooling (e.g. fans) and any other add-on hardware expected to be present when
deployed.
The present document defines:
• an energy efficiency metric to support procurement or market entry requirements;
• requirements for equipment to perform the measurements and analysis;
• requirements for the measurement process;
• requirements for the management of the metric calculation;
• operation or run rules to configure, execute, and monitor the testing;
• documentation and reporting requirements;
• a validation process for the metric using the Deployed Power Assessment.
The present document is not applicable to:
• fully fault tolerant servers;
• High Performance Computing (HPC) systems;
• hyper-converged servers;
• large scale servers;
• servers with integrated APA(s);
• networking equipment including network servers;
• server appliances;
• storage device including blade storage and storage servers.
NOTE 2: Products whose feature set and intended operation are not addressed by active mode testing parameters
are excluded from this evaluation method. The above list shows products for which SERT™ efficiency
evaluations are not appropriate.
The present document does not address home servers and small servers that fall under the scope of mandate M/545 [i.8].
ETSI
9 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference/.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
[1] CENELEC EN 62623:2013: "Desktop and notebook computers. Measurement of energy
consumption".
[2] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT)
version 2 Run and Reporting Rules".
NOTE: Available at https://www.spec.org/sert2/SERT-runrules.pdf.
[3] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT)
version 2 User Guide".
NOTE: Available at https://www.spec.org/sert2/SERT-userguide.pdf.
TM
[4] IEEE 802.3 : "IEEE Standard for Ethernet".
NOTE: Available at https://standards.ieee.org/findstds/standard/802.3-2015.html.
TM
[5] IEEE 802.3az : "Energy Efficient Ethernet".
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] CENELEC EN 60297 series: "Mechanical structures for electrical and electronic equipment.
Dimensions of mechanical structures of the 482,6 mm (19 in) series". ®
[i.2] ENERGY STAR : "Product Specification for Computer Servers".
[i.3] Standardization mandate M/462: "M/462 Standardisation mandate addressed to CEN, CENELEC
and ETSI in the field of ICT to enable efficient energy use in fixed and mobile information and
communication networks".
[i.4] ETSI EN 300 119 series: "Equipment Engineering (EE); European telecommunication standard for
equipment practice".
[i.5] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT)
version 2 Design Document".
NOTE: Available at https://www.spec.org/sert2/SERT-designdocument.pdf.
ETSI
10 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
[i.6] SERT Client Configurations (JVM Options).
NOTE: Available at https://www.spec.org/sert2/SERT-JVM_Options-2.0.html.
[i.7] SERT Result File Fields.
NOTE: Available at https://www.spec.org/sert2/SERT-resultfilefields.html.
[i.8] Mandate M/545: "Commission Implementing Decision of 6.1.2016 on a standardisation request to
the European standardisation organisations as regards computers and computer servers, in support
of the implementation of Commission Regulation (EU) No 617/2013 of 26 June 2013,
implementing Directive 2009/125/EC of the European Parliament and of the Council with regard
to ecodesign requirements for computers and computer servers".
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the following terms apply:
a.c.-d.c. power supply unit: power supply unit that converts line-voltage alternating current (a.c.) input power into one
or more direct current (d.c.) power outputs for powering a computer server
active state: operational state of a computer server (as opposed to the idle state) in which the computer server is
carrying out work in response to prior or concurrent external requests (e.g. instruction over the network)
NOTE: The work includes, but is not restricted to, active processing and data seeking/retrieval from memory,
cache, or internal/external storage while awaiting further input over the network.
Auxiliary Processing Accelerator (APA): additional compute device installed in the computer server that handles
parallelized workloads
NOTE 1: This includes, but is not limited to, Graphical Processing Units (GPUs) or Field Programmable Gate
Array chips which can be installed in a server either on Graphics or Extension add-in cards installed in
general-purpose add-in expansion slots (e.g. GPGPUs, CPU accelerators, etc. installed in a PCI slot) or
directly attached to a server component such as the motherboard.
NOTE 2: There are two specific types of APAs used in servers:
a) Expansion APA: An APA that is on an add-in card installed in an add-in expansion slot (e.g.
GPGPUs, CPU accelerators, etc. installed in a PCI slot). An expansion APA add-in card may
include one or more APAs.
b) Integrated APA: An APA that is integrated into the motherboard or CPU package or an expansion
APA that has part of its subsystem, such as switches, included in the non-APA server configuration
TM
that would be used to run the energy efficiency test (SERT suite).
blade chassis: enclosure that contains shared resources for the operation of blade servers, blade storage, and other blade
form-factor devices
NOTE: Shared resources provided by a chassis include, but are not restricted to, power supplies, data storage, and
hardware for d.c. power distribution, thermal management, system management, and network services.
blade server: computer server, designed for use in a blade chassis, that is a high-density device and functions as an
independent computer server and includes at least one processor and system memory, which is dependent upon shared
blade chassis resources (e.g. power supplies, cooling) for operation
NOTE: A processor or memory module that is intended to scale up a standalone server is not considered a blade
server.
blade storage: storage device that is designed for use in a blade chassis and that is dependent upon shared blade chassis
resources (e.g. power supplies, cooling) for operation
ETSI
11 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
blade system: blade chassis and one or more removable blade servers and/or other units (e.g. blade storage, blade
networking equipment) which provide a scalable means for combining multiple blade server or storage units in a single
enclosure
NOTE: A blade system is designed to allow service technicians to easily add or replace (hot-swap) blades in the
field.
buffered Double Data Rate (DDR) channel: channel or memory port connecting a memory controller to a defined
number of memory devices (e.g. dual in-line memory modules (DIMMs)) in a computer server
NOTE 1: A typical computer server may contain multiple memory controllers, which may in turn support one or
more buffered DDR channels.
NOTE 2: Each buffered DDR channel serves only a fraction of the total addressable memory space in a computer
server.
computer server: computer, sold through enterprise channels, that provides services and manages networked resources
for client devices
NOTE 1: Client devices include, but are not restricted to desktop computers, notebook computers, thin clients,
wireless devices, Personal Digital Assistants, IP telephones, other computer servers, or other network
devices.
NOTE 2: A computer server is primarily accessed via network connections, versus directly-connected user input
devices such as a keyboard or mouse.
controller system: computer or computer server that manages a benchmark evaluation process
data averaging interval: time period over which all samples captured by the high-speed sampling electronics of the
power analyser are averaged to provide the measurement set
d.c.-d.c. power supply unit: power supply unit that converts line-voltage direct current (d.c.) input power to one or
more d.c. outputs for powering a computer server
NOTE: For purposes of the present document, a d.c-d.c. converter that is internal to a computer server and is used
to convert a low voltage d.c. (e.g. 12 VDC) into other d.c. power outputs for use by computer server
components is not considered a d.c-d.c. power supply unit.
deployed power: average power level of the utilization applicable to the total number of servers provisioned to meet an
aggregate peak load
direct current server: computer server that is designed solely to operate on a direct current (d.c.) power source
double-wide blade server: blade server requiring twice the width of a standard blade server bay
efficiency: defined workload output divided by the resource input to the system
fully fault tolerant server: computer server that is designed with complete hardware redundancy, in which every
computing component is replicated between two nodes running identical and concurrent workloads (i.e. if one node
fails or needs repair, the second node can run the workload alone to avoid downtime) and that uses two systems to
simultaneously and repetitively run a single workload for continuous availability in a mission critical application
half-height blade server: blade server requiring one half the height of a standard blade server bay
hard disk drive: primary computer storage device which reads and writes to one or more rotating magnetic disk
platters
High Performance Computing (HPC) system: computing system which is designed (or assembled), optimized,
marketed and sold to execute highly parallel applications for higher performance computing applications
NOTE 1: HPC systems support applications including, but not restricted to, deep learning or artificial intelligence.
NOTE 2: HPC systems feature multiple clustered nodes to increase computational capability and often featuring
high speed inter-processing interconnects as well as large memory capability and bandwidth.
ETSI
12 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
NOTE 3: HPC systems may be comprised of multiple clusters of homogenous nodes, for which the clusters may be
heterogeneous.
hyper-converged server: highly integrated enterprise device which contains the same components as a computer
server in addition to the features of a network server and storage server
hypervisor: supervisory system level software that establishes and manages a virtualized environment which enables
multiple operating systems to run on a single physical system at the same time
ICT equipment: equipment providing data storage, processing and transport services
NOTE: A combination of Information Technology Equipment and Network Telecommunications Equipment.
idle state: operational state of a computer server in which the operating system and other software have completed
loading but is not performing any useful work
NOTE 1: The computer server is capable of completing workload transactions, but no active workload transactions
are requested or pending by the system.
NOTE 2: For systems where ACPI standards are applicable, idle state correlates only to ACPI System Level S0.
I/O device: device which provides data input and output capability between a computer server and other devices
NOTE: An I/O device may be integral to the computer server motherboard or may be connected to the
motherboard via expansion slots.
I/O port: physical circuitry within an I/O device where an independent I/O session can be established
NOTE: A port is not the same as a connector receptacle; it is possible that a single connector receptacle can
service multiple ports of the same interface.
large scale server: resilient/scalable server which ships as a pre-integrated/pre-tested system housed in one or more full
frames or racks and that includes a high connectivity input/output subsystem with a minimum of 32 dedicated
input/output slots
maximum power: peak sustained or root means square power consumption value while operating the worst case
functions
memory: server component external to the processor in which information is stored for immediate use by the processor
motherboard: main circuit board of the server typically accommodating the processor, memory, expansion slots and
enabling the attachment of additional circuit boards
multi-bay blade server: blade server requiring more than one bay for installation in a blade chassis
multi-output power supply unit: power supply unit designed to deliver the majority of its rated output power to more
than one primary direct current (d.c.) output for the purpose of powering a computer server
NOTE 1: Multi-output power supply units may offer one or more standby outputs that remain active whenever
connected to an input power source.
NOTE 2: The total rated power output from any additional power supply unit outputs, other than primary or
standby outputs, is greater than or equal to 20 W.
network client (testing): computer or computer server that generates workload traffic for transmission to an Equipment
Under Test connected via a network switch
network server: large network device which contains the same components as a computer server together with more
than 11 ports, has a total line rate throughput of greater than or equal to 12 Gb/s and is designed to dynamically
reconfigure ports and speed and to support a virtualized network environment, software defined networking
NOTE: Supporting features are described by the product's datasheet description and are either accompanied with
vendor specific utilities and/or commercially available software supporting these functions.
networking equipment: device whose primary function is to pass data among various network interfaces, providing
data connectivity among connected devices via the routing of data packets (e.g. routers and switches)
ETSI
13 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
normalized performance: relative performance values calibrated to a baseline common to the set of equipment being
evaluated
pedestal server: self-contained computer server that is designed with power supply units, cooling, input/output devices,
and other resources necessary for stand-alone operation within a frame similar to that of a tower client computer
power supply unit: self-contained device, physically separable from the motherboard of the computer server, that
converts a.c. or d.c. input power to one or more d.c. power outputs for powering the computer server via a removable or
hard-wired electrical connection
processor: central processing unit of the computer server comprising logic circuitry that responds to and processes the
basic instructions that drive the server
product category: second-order classification or sub-type within a product group or form-factor that is based on
product features and installed components
NOTE: Used in the present document to determine qualification and test requirements.
quarter-height blade server: blade server requiring one quarter the height of a standard server bay
rack-mounted server: computer server that is designed for deployment in a standard 19 inch ICT equipment rack as
defined by CENELEC EN 60297 [i.1] or ETSI EN 300 119 [i.4]
NOTE: For the purposes of the present document, a blade server is considered under a separate product category
and excluded from the rack-mounted product category.
reported maximum power: highest maximum power recorded on the eleven SERT™ worklet scores for the two tested
configurations
resilient server: computer server designed with extensive Reliability, Availability, Serviceability (RAS) and scalability
features integrated in the micro architecture of the system, Central Processor Unit (CPU) and chipset
NOTE: The requirements are listed in annex A.
server appliance: server that is not intended to execute user-supplied software, delivers services through one or more
networks, is typically managed through a web or command line interface and is bundled with a pre-installed OS and
application software that is used to perform a dedicated function or set of tightly coupled functions
server processor utilization: ratio of processor computing activity to full-load processor computing activity at a
specified voltage and frequency, measured instantaneously or with a short term average of use over a set of active
and/or idle cycles
server product family: group of servers sharing one chassis and motherboard combination that may contain multiple
hardware and software configurations
single output power supply unit: power supply unit designed to deliver the majority of its rated output power to one
primary direct current (d.c.) output for the purpose of powering a computer server
NOTE 1: Single-output power supply units may offer one or more standby outputs that remain active whenever
connected to an input power source.
NOTE 2: The total rated power output from any additional power supply units outputs, other than primary and
standby outputs, is not greater than 20 W.
NOTE 3: Power supply units that offer multiple outputs at the same voltage as the primary output are considered
single-output power supply units unless those outputs are generated from separate converters or have
separate output rectification stages, or have independent current limits.
single-wide blade server: blade server requiring the width of a standard blade server bay
solid state drive: storage device that uses memory chips instead of rotating magnetic platters for data storage
ETSI
14 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
storage device: fully-functional storage system that supplies data storage services to clients and devices attached
directly or through a network
NOTE 1: A storage product may be composed of integrated storage controllers, storage devices, embedded network
elements, software, and other devices.
NOTE 2: While storage products may contain one or more embedded processors, these processors do not execute
user-supplied software applications but may execute data-specific applications (e.g. data replication,
backup utilities, data compression, install agents).
NOTE 3: Components and subsystems that are an integral part of the storage product architecture (e.g. to provide
internal communications between controllers and disks) are considered to be part of the storage product.
NOTE 4: Components that are normally associated with a storage environment (e.g. devices required for operation
of an external Storage Area Network) are not considered to be part of the storage product.
storage server: enterprise storage device which contains the same components as a computer server together with
rd
≥ 10 storage devices and software (vendor or 3 party) that supports storage system connectivity, capacity optimization
management, virtualized storage environment and software defined storage
NOTE: Supporting features are described by the product's datasheet description and are either accompanied with
vendor specific utilities and/or commercially available software supporting these functions.
Uninterruptible Power Supply (UPS): combination of convertors, switches, and energy storage devices (such as
batteries) constituting a power system for maintaining continuity of load power in case of input power failure
weighted geometric mean: geometric mean calculated using a predetermined factor for each of the elements prior to
aggregation
worklet: synthetic software routine, using real application functions focused on a particular type of computing activity,
which stresses a particular characteristic of the system
NOTE: A floating point and integer performance stress code is an example of a CPU worklet.
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
a.c., AC Alternating Current
ACPI Advanced Configuration and Power Interface
APA Auxiliary Processing Accelerator
BIOS Basic Input/Output System
CENELEC European Committee for Electrotechnical Standardization
CPU Central Processor Unit
d.c., DC Direct Current
DDR Double Data Rate
DIMM Dual In-line Memory Module
EEE Energy Efficient Ethernet
EUT Equipment Under Test
FPGA Field Programmable Gate Array
GB GigaByte
GPGPU General-Purpose computing on Graphics Processing Units
HDD Hard Disk Drive
HPC Higher Performance Computing
I/O Input/Output
ICT Information and Communication Technology
ID Identification
ISO/IEC International Organization for Standardization/International Electrotechnical Committee
ETSI
15 Final draft ETSI EN 303 470 V1.1.0 (2019-01)
IT Information Technology
JVM Java™ Virtual Machine
LU Lower-Upper
OS Operating System
PCI Peripheral Component Interconnect
PDU Power Distribution Unit
PSU Power Supply Unit
RAS Reliability, Availability and Serviceability
rms root mean square
SERT™ Server Efficiency Rating Tool
SOR Successive Over-Relaxation
SPEC Standard Performance Evaluation Corporation
SSD Solid State Drive
SSJ Server Side Java™
UPS Uninterruptible Power Supply
VDC Volts (DC)
W Weighting of CPU
CPU
W Weighting of Memory
Memory
W Weighting of Storage
Storage
4 Server product categories and representative
product family configurations
4.1 General
Servers are sold in different form factors, processor socket counts, resilience levels and configuration types and
different groups of servers will have distinct performance capability and power demands.
Servers are categorized as defined in clause 4.4 by specific form factor and configuration parameters to enable the
setting of appropriate idle power or active efficiency thresholds and assessment of like products with regards to
procurement or market entry requirements.
To compare or evaluate systems, the evaluation or metric used shall only be made against like products. Like products
are grouped into categories. Products of different categories shall not be compared using the metric of the present
document.
Even though servers are classified in categories by the type of system, each server is customized by its configuration to
best match the application for which they are being sold or purchased. As a result, a product is represented by a fixed
set of configurations.
For an appropriate evaluation, the category shall be defined and the product family configurations itemized.
The product family configuration establishes a single group representing the efficiency of the product, as this then
covers the range of configurations of that product that would be sold. Since the products sold are custom-configured,
there should be a minimum of 2 configurations that would represe
...
EUROPEAN STANDARD
Environmental Engineering (EE);
Energy Efficiency measurement methodology
and metrics for servers
2 ETSI EN 303 470 V1.1.1 (2019-03)
Reference
DEN/EE-EEPS24
Keywords
energy efficiency, ICT, server, sustainability
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
The present document can be downloaded from:
http://www.etsi.org/standards-search
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx
If you find errors in the present document, please send your comment to one of the following services:
https://portal.etsi.org/People/CommiteeSupportStaff.aspx
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.
© ETSI 2019.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are trademarks of ETSI registered for the benefit of its Members.
TM TM
3GPP and LTE are trademarks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and
of the oneM2M Partners. ®
GSM and the GSM logo are trademarks registered and owned by the GSM Association.
ETSI
3 ETSI EN 303 470 V1.1.1 (2019-03)
Contents
Intellectual Property Rights . 6
Foreword . 6
Modal verbs terminology . 6
Introduction . 6
1 Scope . 8
2 References . 9
2.1 Normative references . 9
2.2 Informative references . 9
3 Definition of terms, symbols and abbreviations . 10
3.1 Terms . 10
3.2 Symbols . 14
3.3 Abbreviations . 14
4 Server product categories and representative product family configurations . 15
4.1 General . 15
4.2 Applications and metric applicability . 15
4.3 Computer servers . 16
4.3.1 General requirements . 16
4.3.2 Form-factors . 16
4.4 Computer server product categories . 16
4.5 Server product family configuration. 17
4.5.1 General . 17
4.5.2 "High-end" performance configuration . 17
4.5.3 "Low-end" performance configuration . 17
5 Metrics . 18
5.1 Active state metric . 18
5.1.1 Worklets . 18
5.1.2 Formulae . 18
5.1.2.1 General . 18
5.1.2.2 Active State Metric definition . 18
5.1.3 Weightings . 19
5.2 Idle state metric . 19
6 Test setup . 20
6.1 General . 20
6.2 Input power . 21
6.3 Environmental conditions . 21
6.3.1 Ambient temperature . 21
6.3.2 Relative humidity . 21
6.4 Power analyser . 21
6.5 Temperature sensor . 22
6.6 Active state test tool . 22
6.7 Controller system . 22
6.8 General SERT™ requirements . 22
7 Equipment Under Test (EUT) . 23
7.1 Configuration . 23
7.2 Test procedure . 24
8 Measurement . 25
8.1 Measurement for active state . 25
8.2 Sensitivity analysis . 25
8.3 Measurement for power supply . 26
8.3.1 Measurement for internal power supply . 26
8.3.2 Measurement for test board power supply . 26
ETSI
4 ETSI EN 303 470 V1.1.1 (2019-03)
8.3.2.1 General . 26
8.3.2.2 Test loads . 26
8.3.2.3 Test leads and wiring . 26
8.3.2.4 Warm up time . 26
8.3.2.5 Power measurements . 27
9 Measurement report . 27
Annex A (normative): Resilient server requirements . 28
A.1 Reliability, Availability and Serviceability (RAS) features . 28
A.2 Reliability, Availability and Serviceability (RAS) requirements . 28
Annex B (informative): Deployed Power Assessment . 30
B.1 Overview . 30
B.2 Determining the number of deployed servers . 30
B.2.1 General . 30
B.2.2 Establishing target performance . 31
B.2.3 Weighting factors . 31
Annex C (informative): Alternative calculation . 33
Annex D (informative): Bibliography . 34
History . 35
ETSI
5 ETSI EN 303 470 V1.1.1 (2019-03)
List of tables
Table 1: Configuration of EUT .23
Table 2: Test configuration of EUT .24
Table 3: Measurement of active state .25
Table 4: Measurement of internal power supply .26
Table A.1: Resilient Server requirements.29
ETSI
6 ETSI EN 303 470 V1.1.1 (2019-03)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
Foreword
This European Standard (EN) has been produced by ETSI Technical Committee Environmental Engineering (EE).
National transposition dates
Date of adoption of this EN: 11 March 2019
Date of latest announcement of this EN (doa): 30 June 2019
Date of latest publication of new National Standard
or endorsement of this EN (dop/e): 31 December 2019
Date of withdrawal of any conflicting National Standard (dow): 31 December 2019
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
The present document specifies a metric for the assessment of energy efficiency of computer servers using reliable,
accurate and reproducible measurement methods, which take into account the recognized state of the art.
The present document formalizes the tools, conditions and calculations used to generate a single figure of merit of a
single computer server representing its relative efficiency and power impact. The metric is targeted for use as a tool in
the selection process of servers to be provisioned.
ETSI
7 ETSI EN 303 470 V1.1.1 (2019-03)
For comparisons, evaluations should be conducted across similar server types or categories. The efficiency metric is
targeted for use in a pass/fail selection process by differentiating the ability of servers to be provisioned for general
purpose operations. The present document does not prescribe the levels or values for acceptance but prescribes a
standard method of evaluation that energy efficiency programs would use to establish such criteria.
As there are many operational deployments of servers resulting in a range of specialized equipment and configurations
for a single server product, a metric that evaluates provisioning impacts to general purpose operations may not be
applicable. ICT equipment and servers in particular, are generally customized and commissioned on site for
deployment. As with most IT equipment, new technologies are regularly introduced, which may require product level
customization or an industry wide tool upgrade to more appropriately represent the efficiency of the servers. The
present document categorizes servers to address applicability, configuration groupings to represent a family of servers
to address the broad range of custom configurations possible within each server product family, and tool revision
control to ensure comparability and consistency of the resulting metric value.
The present document is based upon the Server Efficiency Rating Tool™ (SERT™) of the Standard Performance
Evaluation Corporation (SPEC) and takes into account:
• the Eco-design Technical Assistance Study on Standards for ErP Lot 9 Enterprise Servers and Enterprise Data
Storage;
• activity related to the analysis of output of Server Efficiency Rating Tool (SERT™) measurements and
deployed power by The Green Grid; ®
• ENERGY STAR for Computer Servers [i.2].
The present document defines energy efficiency metrics and measurement methodology for server equipment under
standardization mandate M/462 of the European Commission [i.3].
ETSI
8 ETSI EN 303 470 V1.1.1 (2019-03)
1 Scope
The present document specifies a metric using the Server Efficiency Rating Tool (SERT™), test conditions and product
family configuration for the assessment of energy efficiency of computer servers using reliable, accurate and
reproducible measurement methods. The metric applies to general purpose computer servers with up to four processor
sockets and with their own dedicated power supply.
NOTE 1: The term "socket" also applies to design in which processors are installed without sockets (e.g. soldered
products).
The metric applies to a computer server model and to a computer server product family, including type and count of
CPU, memory, storage, power supplies, cooling (e.g. fans) and any other add-on hardware expected to be present when
deployed.
The present document defines:
• an energy efficiency metric to support procurement or market entry requirements;
• requirements for equipment to perform the measurements and analysis;
• requirements for the measurement process;
• requirements for the management of the metric calculation;
• operation or run rules to configure, execute, and monitor the testing;
• documentation and reporting requirements;
• a validation process for the metric using the Deployed Power Assessment.
The present document is not applicable to:
• fully fault tolerant servers;
• High Performance Computing (HPC) systems;
• hyper-converged servers;
• large scale servers;
• servers with integrated APA(s);
• networking equipment including network servers;
• server appliances;
• storage device including blade storage and storage servers.
NOTE 2: Products whose feature set and intended operation are not addressed by active mode testing parameters
are excluded from this evaluation method. The above list shows products for which SERT™ efficiency
evaluations are not appropriate.
The present document does not address home servers and small servers that fall under the scope of mandate M/545 [i.8].
ETSI
9 ETSI EN 303 470 V1.1.1 (2019-03)
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference/.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
[1] CENELEC EN 62623:2013: "Desktop and notebook computers. Measurement of energy
consumption".
[2] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT)
version 2 Run and Reporting Rules".
NOTE: Available at https://www.spec.org/sert2/SERT-runrules.pdf.
[3] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT)
version 2 User Guide".
NOTE: Available at https://www.spec.org/sert2/SERT-userguide.pdf.
[4] IEEE 802.3™: "IEEE Standard for Ethernet".
NOTE: Available at https://standards.ieee.org/findstds/standard/802.3-2015.html.
[5] IEEE 802.3az™: "Energy Efficient Ethernet".
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] CENELEC EN 60297 series: "Mechanical structures for electrical and electronic equipment.
Dimensions of mechanical structures of the 482,6 mm (19 in) series". ®
[i.2] ENERGY STAR : "Product Specification for Computer Servers".
[i.3] Standardization mandate M/462: "M/462 Standardisation mandate addressed to CEN, CENELEC
and ETSI in the field of ICT to enable efficient energy use in fixed and mobile information and
communication networks".
[i.4] ETSI EN 300 119 series: "Equipment Engineering (EE); European telecommunication standard for
equipment practice".
[i.5] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT)
version 2 Design Document".
NOTE: Available at https://www.spec.org/sert2/SERT-designdocument.pdf.
ETSI
10 ETSI EN 303 470 V1.1.1 (2019-03)
[i.6] SERT Client Configurations (JVM Options).
NOTE: Available at https://www.spec.org/sert2/SERT-JVM_Options-2.0.html.
[i.7] SERT Result File Fields.
NOTE: Available at https://www.spec.org/sert2/SERT-resultfilefields.html.
[i.8] Mandate M/545: "Commission Implementing Decision of 6.1.2016 on a standardisation request to
the European standardisation organisations as regards computers and computer servers, in support
of the implementation of Commission Regulation (EU) No 617/2013 of 26 June 2013,
implementing Directive 2009/125/EC of the European Parliament and of the Council with regard
to ecodesign requirements for computers and computer servers".
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the following terms apply:
a.c.-d.c. power supply unit: power supply unit that converts line-voltage alternating current (a.c.) input power into one
or more direct current (d.c.) power outputs for powering a computer server
active state: operational state of a computer server (as opposed to the idle state) in which the computer server is
carrying out work in response to prior or concurrent external requests (e.g. instruction over the network)
NOTE: The work includes, but is not restricted to, active processing and data seeking/retrieval from memory,
cache, or internal/external storage while awaiting further input over the network.
Auxiliary Processing Accelerator (APA): additional compute device installed in the computer server that handles
parallelized workloads
NOTE 1: This includes, but is not limited to, Graphical Processing Units (GPUs) or Field Programmable Gate
Array chips which can be installed in a server either on Graphics or Extension add-in cards installed in
general-purpose add-in expansion slots (e.g. GPGPUs, CPU accelerators, etc. installed in a PCI slot) or
directly attached to a server component such as the motherboard.
NOTE 2: There are two specific types of APAs used in servers:
a) Expansion APA: An APA that is on an add-in card installed in an add-in expansion slot (e.g.
GPGPUs, CPU accelerators, etc. installed in a PCI slot). An expansion APA add-in card may
include one or more APAs.
b) Integrated APA: An APA that is integrated into the motherboard or CPU package or an expansion
APA that has part of its subsystem, such as switches, included in the non-APA server configuration
TM
that would be used to run the energy efficiency test (SERT suite).
blade chassis: enclosure that contains shared resources for the operation of blade servers, blade storage, and other blade
form-factor devices
NOTE: Shared resources provided by a chassis include, but are not restricted to, power supplies, data storage, and
hardware for d.c. power distribution, thermal management, system management, and network services.
blade server: computer server, designed for use in a blade chassis, that is a high-density device and functions as an
independent computer server and includes at least one processor and system memory, which is dependent upon shared
blade chassis resources (e.g. power supplies, cooling) for operation
NOTE: A processor or memory module that is intended to scale up a standalone server is not considered a blade
server.
blade storage: storage device that is designed for use in a blade chassis and that is dependent upon shared blade chassis
resources (e.g. power supplies, cooling) for operation
ETSI
11 ETSI EN 303 470 V1.1.1 (2019-03)
blade system: blade chassis and one or more removable blade servers and/or other units (e.g. blade storage, blade
networking equipment) which provide a scalable means for combining multiple blade server or storage units in a single
enclosure
NOTE: A blade system is designed to allow service technicians to easily add or replace (hot-swap) blades in the
field.
buffered Double Data Rate (DDR) channel: channel or memory port connecting a memory controller to a defined
number of memory devices (e.g. dual in-line memory modules (DIMMs)) in a computer server
NOTE 1: A typical computer server may contain multiple memory controllers, which may in turn support one or
more buffered DDR channels.
NOTE 2: Each buffered DDR channel serves only a fraction of the total addressable memory space in a computer
server.
computer server: computer, sold through enterprise channels, that provides services and manages networked resources
for client devices
NOTE 1: Client devices include, but are not restricted to desktop computers, notebook computers, thin clients,
wireless devices, Personal Digital Assistants, IP telephones, other computer servers, or other network
devices.
NOTE 2: A computer server is primarily accessed via network connections, versus directly-connected user input
devices such as a keyboard or mouse.
controller system: computer or computer server that manages a benchmark evaluation process
data averaging interval: time period over which all samples captured by the high-speed sampling electronics of the
power analyser are averaged to provide the measurement set
d.c.-d.c. power supply unit: power supply unit that converts line-voltage direct current (d.c.) input power to one or
more d.c. outputs for powering a computer server
NOTE: For purposes of the present document, a d.c-d.c. converter that is internal to a computer server and is used
to convert a low voltage d.c. (e.g. 12 VDC) into other d.c. power outputs for use by computer server
components is not considered a d.c-d.c. power supply unit.
deployed power: average power level of the utilization applicable to the total number of servers provisioned to meet an
aggregate peak load
direct current server: computer server that is designed solely to operate on a direct current (d.c.) power source
double-wide blade server: blade server requiring twice the width of a standard blade server bay
efficiency: defined workload output divided by the resource input to the system
fully fault tolerant server: computer server that is designed with complete hardware redundancy, in which every
computing component is replicated between two nodes running identical and concurrent workloads (i.e. if one node
fails or needs repair, the second node can run the workload alone to avoid downtime) and that uses two systems to
simultaneously and repetitively run a single workload for continuous availability in a mission critical application
half-height blade server: blade server requiring one half the height of a standard blade server bay
hard disk drive: primary computer storage device which reads and writes to one or more rotating magnetic disk
platters
High Performance Computing (HPC) system: computing system which is designed (or assembled), optimized,
marketed and sold to execute highly parallel applications for higher performance computing applications
NOTE 1: HPC systems support applications including, but not restricted to, deep learning or artificial intelligence.
NOTE 2: HPC systems feature multiple clustered nodes to increase computational capability and often featuring
high speed inter-processing interconnects as well as large memory capability and bandwidth.
ETSI
12 ETSI EN 303 470 V1.1.1 (2019-03)
NOTE 3: HPC systems may be comprised of multiple clusters of homogenous nodes, for which the clusters may be
heterogeneous.
hyper-converged server: highly integrated enterprise device which contains the same components as a computer
server in addition to the features of a network server and storage server
hypervisor: supervisory system level software that establishes and manages a virtualized environment which enables
multiple operating systems to run on a single physical system at the same time
ICT equipment: equipment providing data storage, processing and transport services
NOTE: A combination of Information Technology Equipment and Network Telecommunications Equipment.
idle state: operational state of a computer server in which the operating system and other software have completed
loading but is not performing any useful work
NOTE 1: The computer server is capable of completing workload transactions, but no active workload transactions
are requested or pending by the system.
NOTE 2: For systems where ACPI standards are applicable, idle state correlates only to ACPI System Level S0.
I/O device: device which provides data input and output capability between a computer server and other devices
NOTE: An I/O device may be integral to the computer server motherboard or may be connected to the
motherboard via expansion slots.
I/O port: physical circuitry within an I/O device where an independent I/O session can be established
NOTE: A port is not the same as a connector receptacle; it is possible that a single connector receptacle can
service multiple ports of the same interface.
large scale server: resilient/scalable server which ships as a pre-integrated/pre-tested system housed in one or more full
frames or racks and that includes a high connectivity input/output subsystem with a minimum of 32 dedicated
input/output slots
maximum power: peak sustained or root means square power consumption value while operating the worst case
functions
memory: server component external to the processor in which information is stored for immediate use by the processor
motherboard: main circuit board of the server typically accommodating the processor, memory, expansion slots and
enabling the attachment of additional circuit boards
multi-bay blade server: blade server requiring more than one bay for installation in a blade chassis
multi-output power supply unit: power supply unit designed to deliver the majority of its rated output power to more
than one primary direct current (d.c.) output for the purpose of powering a computer server
NOTE 1: Multi-output power supply units may offer one or more standby outputs that remain active whenever
connected to an input power source.
NOTE 2: The total rated power output from any additional power supply unit outputs, other than primary or
standby outputs, is greater than or equal to 20 W.
network client (testing): computer or computer server that generates workload traffic for transmission to an Equipment
Under Test connected via a network switch
network server: large network device which contains the same components as a computer server together with more
than 11 ports, has a total line rate throughput of greater than or equal to 12 Gb/s and is designed to dynamically
reconfigure ports and speed and to support a virtualized network environment, software defined networking
NOTE: Supporting features are described by the product's datasheet description and are either accompanied with
vendor specific utilities and/or commercially available software supporting these functions.
networking equipment: device whose primary function is to pass data among various network interfaces, providing
data connectivity among connected devices via the routing of data packets (e.g. routers and switches)
ETSI
13 ETSI EN 303 470 V1.1.1 (2019-03)
normalized performance: relative performance values calibrated to a baseline common to the set of equipment being
evaluated
pedestal server: self-contained computer server that is designed with power supply units, cooling, input/output devices,
and other resources necessary for stand-alone operation within a frame similar to that of a tower client computer
power supply unit: self-contained device, physically separable from the motherboard of the computer server, that
converts a.c. or d.c. input power to one or more d.c. power outputs for powering the computer server via a removable or
hard-wired electrical connection
processor: central processing unit of the computer server comprising logic circuitry that responds to and processes the
basic instructions that drive the server
product category: second-order classification or sub-type within a product group or form-factor that is based on
product features and installed components
NOTE: Used in the present document to determine qualification and test requirements.
quarter-height blade server: blade server requiring one quarter the height of a standard server bay
rack-mounted server: computer server that is designed for deployment in a standard 19 inch ICT equipment rack as
defined by CENELEC EN 60297 [i.1] or ETSI EN 300 119 [i.4]
NOTE: For the purposes of the present document, a blade server is considered under a separate product category
and excluded from the rack-mounted product category.
reported maximum power: highest maximum power recorded on the eleven SERT™ worklet scores for the two tested
configurations
resilient server: computer server designed with extensive Reliability, Availability, Serviceability (RAS) and scalability
features integrated in the micro architecture of the system, Central Processor Unit (CPU) and chipset
NOTE: The requirements are listed in annex A.
server appliance: server that is not intended to execute user-supplied software, delivers services through one or more
networks, is typically managed through a web or command line interface and is bundled with a pre-installed OS and
application software that is used to perform a dedicated function or set of tightly coupled functions
server processor utilization: ratio of processor computing activity to full-load processor computing activity at a
specified voltage and frequency, measured instantaneously or with a short term average of use over a set of active
and/or idle cycles
server product family: group of servers sharing one chassis and motherboard combination that may contain multiple
hardware and software configurations
single output power supply unit: power supply unit designed to deliver the majority of its rated output power to one
primary direct current (d.c.) output for the purpose of powering a computer server
NOTE 1: Single-output power supply units may offer one or more standby outputs that remain active whenever
connected to an input power source.
NOTE 2: The total rated power output from any additional power supply units outputs, other than primary and
standby outputs, is not greater than 20 W.
NOTE 3: Power supply units that offer multiple outputs at the same voltage as the primary output are considered
single-output power supply units unless those outputs are generated from separate converters or have
separate output rectification stages, or have independent current limits.
single-wide blade server: blade server requiring the width of a standard blade server bay
solid state drive: storage device that uses memory chips instead of rotating magnetic platters for data storage
ETSI
14 ETSI EN 303 470 V1.1.1 (2019-03)
storage device: fully-functional storage system that supplies data storage services to clients and devices attached
directly or through a network
NOTE 1: A storage product may be composed of integrated storage controllers, storage devices, embedded network
elements, software, and other devices.
NOTE 2: While storage products may contain one or more embedded processors, these processors do not execute
user-supplied software applications but may execute data-specific applications (e.g. data replication,
backup utilities, data compression, install agents).
NOTE 3: Components and subsystems that are an integral part of the storage product architecture (e.g. to provide
internal communications between controllers and disks) are considered to be part of the storage product.
NOTE 4: Components that are normally associated with a storage environment (e.g. devices required for operation
of an external Storage Area Network) are not considered to be part of the storage product.
storage server: enterprise storage device which contains the same components as a computer server together with
rd
≥ 10 storage devices and software (vendor or 3 party) that supports storage system connectivity, capacity optimization
management, virtualized storage environment and software defined storage
NOTE: Supporting features are described by the product's datasheet description and are either accompanied with
vendor specific utilities and/or commercially available software supporting these functions.
Uninterruptible Power Supply (UPS): combination of convertors, switches, and energy storage devices (such as
batteries) constituting a power system for maintaining continuity of load power in case of input power failure
weighted geometric mean: geometric mean calculated using a predetermined factor for each of the elements prior to
aggregation
worklet: synthetic software routine, using real application functions focused on a particular type of computing activity,
which stresses a particular characteristic of the system
NOTE: A floating point and integer performance stress code is an example of a CPU worklet.
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
a.c., AC Alternating Current
ACPI Advanced Configuration and Power Interface
APA Auxiliary Processing Accelerator
BIOS Basic Input/Output System
CENELEC European Committee for Electrotechnical Standardization
CPU Central Processor Unit
d.c., DC Direct Current
DDR Double Data Rate
DIMM Dual In-line Memory Module
EEE Energy Efficient Ethernet
EUT Equipment Under Test
FPGA Field Programmable Gate Array
GB GigaByte
GPGPU General-Purpose computing on Graphics Processing Units
HDD Hard Disk Drive
HPC Higher Performance Computing
I/O Input/Output
ICT Information and Communication Technology
ID Identification
ISO/IEC International Organization for Standardization/International Electrotechnical Committee
ETSI
15 ETSI EN 303 470 V1.1.1 (2019-03)
IT Information Technology
JVM Java™ Virtual Machine
LU Lower-Upper
OS Operating System
PCI Peripheral Component Interconnect
PDU Power Distribution Unit
PSU Power Supply Unit
RAS Reliability, Availability and Serviceability
rms root mean square
SERT™ Server Efficiency Rating Tool
SOR Successive Over-Relaxation
SPEC Standard Performance Evaluation Corporation
SSD Solid State Drive
SSJ Server Side Java™
UPS Uninterruptible Power Supply
VDC Volts (DC)
W Weighting of CPU
CPU
W Weighting of Memory
Memory
W Weighting of Storage
Storage
4 Server product categories and representative
product family configurations
4.1 General
Servers are sold in different form factors, processor socket counts, resilience levels and configuration types and
different groups of servers will have distinct performance capability and power demands.
Servers are categorized as defined in clause 4.4 by specific form factor and configuration parameters to enable the
setting of appropriate idle power or active efficiency thresholds and assessment of like products with regards to
procurement or market entry requirements.
To compare or evaluate systems, the evaluation or metric used shall only be made against like products. Like products
are grouped into categories. Products of different categories shall not be compared using the metric of the present
document.
Even though servers are classified in categories by the type of system, each server is customized by its configuration to
best match the application for which they are being sold or purchased. As a result, a product is represented by a fixed
set of configurations.
For an appropriate evaluation, the category shall be defined and the product family configurations itemized.
The product family configuration establishes a single group representing the efficiency of the product, as this then
covers the range of configurations of that product that would be sold. Since the products sold are custom-configured,
there should be a minimum of 2 configurations that would represent the family of configurations, bounded by the
high-end configuration (see clause 4.5.2) and the low-end configuration (see clause 4.5.3).
4.2 Applications and metric applicability
Computer servers are architected in such a way as to be configurable to different groups of applica
...
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.Environmental Engineering (EE) - Energy Efficiency measurement methodology and metrics for servers35.020Informacijska tehnika in tehnologija na splošnoInformation technology (IT) in general27.015VSORãQREnergy efficiency. Energy conservation in generalICS:Ta slovenski standard je istoveten z:ETSI EN 303 470 V1.1.1 (2019-03)SIST EN 303 470 V1.1.1:2019en01-maj-2019SIST EN 303 470 V1.1.1:2019SLOVENSKI
STANDARD
EUROPEAN STANDARD SIST EN 303 470 V1.1.1:2019
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 2
Reference DEN/EE-EEPS24 Keywords energy efficiency, ICT, server, sustainability ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00
Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C Association à but non lucratif enregistrée à la Sous-Préfecture de Grasse (06) N° 7803/88
Important notice The present document can be downloaded from: http://www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI deliverable is the one made publicly available in PDF format at www.etsi.org/deliver. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document, please send your comment to one of the following services: https://portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF version shall not be modified without the written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media.
© ETSI 2019. All rights reserved.
DECTTM, PLUGTESTSTM, UMTSTM and the ETSI logo are trademarks of ETSI registered for the benefit of its Members. 3GPPTM and LTETM are trademarks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of the oneM2M Partners. GSM® and the GSM logo are trademarks registered and owned by the GSM Association. SIST EN 303 470 V1.1.1:2019
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 3 Contents Intellectual Property Rights . 6 Foreword . 6 Modal verbs terminology . 6 Introduction . 6 1 Scope . 8 2 References . 9 2.1 Normative references . 9 2.2 Informative references . 9 3 Definition of terms, symbols and abbreviations . 10 3.1 Terms . 10 3.2 Symbols . 14 3.3 Abbreviations . 14 4 Server product categories and representative product family configurations . 15 4.1 General . 15 4.2 Applications and metric applicability . 15 4.3 Computer servers . 16 4.3.1 General requirements . 16 4.3.2 Form-factors . 16 4.4 Computer server product categories . 16 4.5 Server product family configuration. 17 4.5.1 General . 17 4.5.2 "High-end" performance configuration . 17 4.5.3 "Low-end" performance configuration . 17 5 Metrics . 18 5.1 Active state metric . 18 5.1.1 Worklets . 18 5.1.2 Formulae . 18 5.1.2.1 General . 18 5.1.2.2 Active State Metric definition . 18 5.1.3 Weightings . 19 5.2 Idle state metric . 19 6 Test setup . 20 6.1 General . 20 6.2 Input power . 21 6.3 Environmental conditions . 21 6.3.1 Ambient temperature . 21 6.3.2 Relative humidity . 21 6.4 Power analyser . 21 6.5 Temperature sensor . 22 6.6 Active state test tool . 22 6.7 Controller system . 22 6.8 General SERT™ requirements . 22 7 Equipment Under Test (EUT) . 23 7.1 Configuration . 23 7.2 Test procedure . 24 8 Measurement . 25 8.1 Measurement for active state . 25 8.2 Sensitivity analysis . 25 8.3 Measurement for power supply . 26 8.3.1 Measurement for internal power supply . 26 8.3.2 Measurement for test board power supply . 26 SIST EN 303 470 V1.1.1:2019
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 4 8.3.2.1 General . 26 8.3.2.2 Test loads . 26 8.3.2.3 Test leads and wiring . 26 8.3.2.4 Warm up time . 26 8.3.2.5 Power measurements . 27 9 Measurement report . 27 Annex A (normative): Resilient server requirements . 28 A.1 Reliability, Availability and Serviceability (RAS) features . 28 A.2 Reliability, Availability and Serviceability (RAS) requirements . 28 Annex B (informative): Deployed Power Assessment . 30 B.1 Overview . 30 B.2 Determining the number of deployed servers . 30 B.2.1 General . 30 B.2.2 Establishing target performance . 31 B.2.3 Weighting factors . 31 Annex C (informative): Alternative calculation . 33 Annex D (informative): Bibliography . 34 History . 35
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 5 List of tables Table 1: Configuration of EUT .23 Table 2: Test configuration of EUT .24 Table 3: Measurement of active state .25 Table 4: Measurement of internal power supply .26 Table A.1: Resilient Server requirements.29
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 6 Intellectual Property Rights Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (https://ipr.etsi.org/). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Trademarks The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners. ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks. Foreword This European Standard (EN) has been produced by ETSI Technical Committee Environmental Engineering (EE).
National transposition dates Date of adoption of this EN: 11 March 2019 Date of latest announcement of this EN (doa): 30 June 2019 Date of latest publication of new National Standard or endorsement of this EN (dop/e):
31 December 2019 Date of withdrawal of any conflicting National Standard (dow): 31 December 2019
Modal verbs terminology In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). "must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation. Introduction The present document specifies a metric for the assessment of energy efficiency of computer servers using reliable, accurate and reproducible measurement methods, which take into account the recognized state of the art. The present document formalizes the tools, conditions and calculations used to generate a single figure of merit of a single computer server representing its relative efficiency and power impact. The metric is targeted for use as a tool in the selection process of servers to be provisioned. SIST EN 303 470 V1.1.1:2019
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 7 For comparisons, evaluations should be conducted across similar server types or categories. The efficiency metric is targeted for use in a pass/fail selection process by differentiating the ability of servers to be provisioned for general purpose operations. The present document does not prescribe the levels or values for acceptance but prescribes a standard method of evaluation that energy efficiency programs would use to establish such criteria. As there are many operational deployments of servers resulting in a range of specialized equipment and configurations for a single server product, a metric that evaluates provisioning impacts to general purpose operations may not be applicable. ICT equipment and servers in particular, are generally customized and commissioned on site for deployment. As with most IT equipment, new technologies are regularly introduced, which may require product level customization or an industry wide tool upgrade to more appropriately represent the efficiency of the servers. The present document categorizes servers to address applicability, configuration groupings to represent a family of servers to address the broad range of custom configurations possible within each server product family, and tool revision control to ensure comparability and consistency of the resulting metric value.
The present document is based upon the Server Efficiency Rating Tool™ (SERT™) of the Standard Performance Evaluation Corporation (SPEC) and takes into account: • the Eco-design Technical Assistance Study on Standards for ErP Lot 9 Enterprise Servers and Enterprise Data Storage; • activity related to the analysis of output of Server Efficiency Rating Tool (SERT™) measurements and deployed power by The Green Grid; • ENERGY STAR® for Computer Servers [i.2]. The present document defines energy efficiency metrics and measurement methodology for server equipment under standardization mandate M/462 of the European Commission [i.3].
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 8 1 Scope The present document specifies a metric using the Server Efficiency Rating Tool (SERT™), test conditions and product family configuration for the assessment of energy efficiency of computer servers using reliable, accurate and reproducible measurement methods. The metric applies to general purpose computer servers with up to four processor sockets and with their own dedicated power supply. NOTE 1: The term "socket" also applies to design in which processors are installed without sockets (e.g. soldered products).
The metric applies to a computer server model and to a computer server product family, including type and count of CPU, memory, storage, power supplies, cooling (e.g. fans) and any other add-on hardware expected to be present when deployed.
The present document defines: • an energy efficiency metric to support procurement or market entry requirements; • requirements for equipment to perform the measurements and analysis; • requirements for the measurement process; • requirements for the management of the metric calculation; • operation or run rules to configure, execute, and monitor the testing; • documentation and reporting requirements; • a validation process for the metric using the Deployed Power Assessment. The present document is not applicable to: • fully fault tolerant servers; • High Performance Computing (HPC) systems; • hyper-converged servers; • large scale servers; • servers with integrated APA(s); • networking equipment including network servers; • server appliances; • storage device including blade storage and storage servers. NOTE 2: Products whose feature set and intended operation are not addressed by active mode testing parameters are excluded from this evaluation method. The above list shows products for which SERT™ efficiency evaluations are not appropriate.
The present document does not address home servers and small servers that fall under the scope of mandate M/545 [i.8].
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 9 2 References 2.1 Normative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. Referenced documents which are not found to be publicly available in the expected location might be found at https://docbox.etsi.org/Reference/. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are necessary for the application of the present document. [1] CENELEC EN 62623:2013: "Desktop and notebook computers. Measurement of energy consumption". [2] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT) version 2 Run and Reporting Rules". NOTE: Available at https://www.spec.org/sert2/SERT-runrules.pdf. [3] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT) version 2 User Guide". NOTE: Available at https://www.spec.org/sert2/SERT-userguide.pdf. [4] IEEE 802.3™: "IEEE Standard for Ethernet". NOTE: Available at https://standards.ieee.org/findstds/standard/802.3-2015.html. [5] IEEE 802.3az™: "Energy Efficient Ethernet". 2.2 Informative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. [i.1] CENELEC EN 60297 series: "Mechanical structures for electrical and electronic equipment. Dimensions of mechanical structures of the 482,6 mm (19 in) series". [i.2] ENERGY STAR®: "Product Specification for Computer Servers". [i.3] Standardization mandate M/462: "M/462 Standardisation mandate addressed to CEN, CENELEC and ETSI in the field of ICT to enable efficient energy use in fixed and mobile information and communication networks". [i.4] ETSI EN 300 119 series: "Equipment Engineering (EE); European telecommunication standard for equipment practice". [i.5] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT) version 2 Design Document". NOTE: Available at https://www.spec.org/sert2/SERT-designdocument.pdf. SIST EN 303 470 V1.1.1:2019
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 10 [i.6] SERT Client Configurations (JVM Options). NOTE: Available at https://www.spec.org/sert2/SERT-JVM_Options-2.0.html. [i.7] SERT Result File Fields. NOTE: Available at https://www.spec.org/sert2/SERT-resultfilefields.html. [i.8] Mandate M/545: "Commission Implementing Decision of 6.1.2016 on a standardisation request to the European standardisation organisations as regards computers and computer servers, in support of the implementation of Commission Regulation (EU) No 617/2013 of 26 June 2013, implementing Directive 2009/125/EC of the European Parliament and of the Council with regard to ecodesign requirements for computers and computer servers". 3 Definition of terms, symbols and abbreviations 3.1 Terms For the purposes of the present document, the following terms apply: a.c.-d.c. power supply unit: power supply unit that converts line-voltage alternating current (a.c.) input power into one or more direct current (d.c.) power outputs for powering a computer server active state: operational state of a computer server (as opposed to the idle state) in which the computer server is carrying out work in response to prior or concurrent external requests (e.g. instruction over the network) NOTE: The work includes, but is not restricted to, active processing and data seeking/retrieval from memory, cache, or internal/external storage while awaiting further input over the network. Auxiliary Processing Accelerator (APA): additional compute device installed in the computer server that handles parallelized workloads NOTE 1: This includes, but is not limited to, Graphical Processing Units (GPUs) or Field Programmable Gate Array chips which can be installed in a server either on Graphics or Extension add-in cards installed in general-purpose add-in expansion slots (e.g. GPGPUs, CPU accelerators, etc. installed in a PCI slot) or directly attached to a server component such as the motherboard.
NOTE 2: There are two specific types of APAs used in servers: a) Expansion APA: An APA that is on an add-in card installed in an add-in expansion slot (e.g. GPGPUs, CPU accelerators, etc. installed in a PCI slot). An expansion APA add-in card may include one or more APAs. b) Integrated APA: An APA that is integrated into the motherboard or CPU package or an expansion APA that has part of its subsystem, such as switches, included in the non-APA server configuration that would be used to run the energy efficiency test (SERTTM suite). blade chassis: enclosure that contains shared resources for the operation of blade servers, blade storage, and other blade form-factor devices NOTE: Shared resources provided by a chassis include, but are not restricted to, power supplies, data storage, and hardware for d.c. power distribution, thermal management, system management, and network services. blade server: computer server, designed for use in a blade chassis, that is a high-density device and functions as an independent computer server and includes at least one processor and system memory, which is dependent upon shared blade chassis resources (e.g. power supplies, cooling) for operation NOTE: A processor or memory module that is intended to scale up a standalone server is not considered a blade server. blade storage: storage device that is designed for use in a blade chassis and that is dependent upon shared blade chassis resources (e.g. power supplies, cooling) for operation SIST EN 303 470 V1.1.1:2019
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 11 blade system: blade chassis and one or more removable blade servers and/or other units (e.g. blade storage, blade networking equipment) which provide a scalable means for combining multiple blade server or storage units in a single enclosure NOTE: A blade system is designed to allow service technicians to easily add or replace (hot-swap) blades in the field. buffered Double Data Rate (DDR) channel: channel or memory port connecting a memory controller to a defined number of memory devices (e.g. dual in-line memory modules (DIMMs)) in a computer server NOTE 1: A typical computer server may contain multiple memory controllers, which may in turn support one or more buffered DDR channels. NOTE 2:
Each buffered DDR channel serves only a fraction of the total addressable memory space in a computer server. computer server: computer, sold through enterprise channels, that provides services and manages networked resources for client devices
NOTE 1: Client devices include, but are not restricted to desktop computers, notebook computers, thin clients, wireless devices, Personal Digital Assistants, IP telephones, other computer servers, or other network devices. NOTE 2: A computer server is primarily accessed via network connections, versus directly-connected user input devices such as a keyboard or mouse. controller system: computer or computer server that manages a benchmark evaluation process data averaging interval: time period over which all samples captured by the high-speed sampling electronics of the power analyser are averaged to provide the measurement set d.c.-d.c. power supply unit: power supply unit that converts line-voltage direct current (d.c.) input power to one or more d.c. outputs for powering a computer server
NOTE: For purposes of the present document, a d.c-d.c. converter that is internal to a computer server and is used to convert a low voltage d.c. (e.g. 12 VDC) into other d.c. power outputs for use by computer server components is not considered a d.c-d.c. power supply unit. deployed power: average power level of the utilization applicable to the total number of servers provisioned to meet an aggregate peak load direct current server: computer server that is designed solely to operate on a direct current (d.c.) power source double-wide blade server: blade server requiring twice the width of a standard blade server bay efficiency: defined workload output divided by the resource input to the system fully fault tolerant server: computer server that is designed with complete hardware redundancy, in which every computing component is replicated between two nodes running identical and concurrent workloads (i.e. if one node fails or needs repair, the second node can run the workload alone to avoid downtime) and that uses two systems to simultaneously and repetitively run a single workload for continuous availability in a mission critical application half-height blade server: blade server requiring one half the height of a standard blade server bay hard disk drive: primary computer storage device which reads and writes to one or more rotating magnetic disk platters
High Performance Computing (HPC) system: computing system which is designed (or assembled), optimized, marketed and sold to execute highly parallel applications for higher performance computing applications NOTE 1: HPC systems support applications including, but not restricted to, deep learning or artificial intelligence. NOTE 2: HPC systems feature multiple clustered nodes to increase computational capability and often featuring high speed inter-processing interconnects as well as large memory capability and bandwidth. SIST EN 303 470 V1.1.1:2019
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 12 NOTE 3: HPC systems may be comprised of multiple clusters of homogenous nodes, for which the clusters may be heterogeneous. hyper-converged server: highly integrated enterprise device which contains the same components as a computer server in addition to the features of a network server and storage server
hypervisor: supervisory system level software that establishes and manages a virtualized environment which enables multiple operating systems to run on a single physical system at the same time ICT equipment: equipment providing data storage, processing and transport services NOTE: A combination of Information Technology Equipment and Network Telecommunications Equipment. idle state: operational state of a computer server in which the operating system and other software have completed loading but is not performing any useful work NOTE 1:
The computer server is capable of completing workload transactions, but no active workload transactions are requested or pending by the system. NOTE 2: For systems where ACPI standards are applicable, idle state correlates only to ACPI System Level S0. I/O device: device which provides data input and output capability between a computer server and other devices NOTE: An I/O device may be integral to the computer server motherboard or may be connected to the motherboard via expansion slots. I/O port: physical circuitry within an I/O device where an independent I/O session can be established NOTE: A port is not the same as a connector receptacle; it is possible that a single connector receptacle can service multiple ports of the same interface. large scale server: resilient/scalable server which ships as a pre-integrated/pre-tested system housed in one or more full frames or racks and that includes a high connectivity input/output subsystem with a minimum of 32 dedicated input/output slots maximum power: peak sustained or root means square power consumption value while operating the worst case functions memory: server component external to the processor in which information is stored for immediate use by the processor motherboard: main circuit board of the server typically accommodating the processor, memory, expansion slots and enabling the attachment of additional circuit boards multi-bay blade server: blade server requiring more than one bay for installation in a blade chassis multi-output power supply unit: power supply unit designed to deliver the majority of its rated output power to more than one primary direct current (d.c.) output for the purpose of powering a computer server NOTE 1:
Multi-output power supply units may offer one or more standby outputs that remain active whenever connected to an input power source. NOTE 2: The total rated power output from any additional power supply unit outputs, other than primary or standby outputs, is greater than or equal to 20 W. network client (testing): computer or computer server that generates workload traffic for transmission to an Equipment Under Test connected via a network switch network server: large network device which contains the same components as a computer server together with more than 11 ports, has a total line rate throughput of greater than or equal to 12 Gb/s and is designed to dynamically reconfigure ports and speed and to support a virtualized network environment, software defined networking NOTE: Supporting features are described by the product's datasheet description and are either accompanied with vendor specific utilities and/or commercially available software supporting these functions. networking equipment: device whose primary function is to pass data among various network interfaces, providing data connectivity among connected devices via the routing of data packets (e.g. routers and switches) SIST EN 303 470 V1.1.1:2019
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 13 normalized performance: relative performance values calibrated to a baseline common to the set of equipment being evaluated pedestal server: self-contained computer server that is designed with power supply units, cooling, input/output devices, and other resources necessary for stand-alone operation within a frame similar to that of a tower client computer power supply unit: self-contained device, physically separable from the motherboard of the computer server, that converts a.c. or d.c. input power to one or more d.c. power outputs for powering the computer server via a removable or hard-wired electrical connection processor: central processing unit of the computer server comprising logic circuitry that responds to and processes the basic instructions that drive the server product category: second-order classification or sub-type within a product group or form-factor that is based on product features and installed components NOTE: Used in the present document to determine qualification and test requirements. quarter-height blade server: blade server requiring one quarter the height of a standard server bay rack-mounted server: computer server that is designed for deployment in a standard 19 inch ICT equipment rack as defined by CENELEC EN 60297 [i.1] or ETSI EN 300 119 [i.4] NOTE: For the purposes of the present document, a blade server is considered under a separate product category and excluded from the rack-mounted product category. reported maximum power: highest maximum power recorded on the eleven SERT™ worklet scores for the two tested configurations resilient server: computer server designed with extensive Reliability, Availability, Serviceability (RAS) and scalability features integrated in the micro architecture of the system, Central Processor Unit (CPU) and chipset
NOTE:
The requirements are listed in annex A. server appliance: server that is not intended to execute user-supplied software, delivers services through one or more networks, is typically managed through a web or command line interface and is bundled with a pre-installed OS and application software that is used to perform a dedicated function or set of tightly coupled functions server processor utilization: ratio of processor computing activity to full-load processor computing activity at a specified voltage and frequency, measured instantaneously or with a short term average of use over a set of active and/or idle cycles
server product family: group of servers sharing one chassis and motherboard combination that may contain multiple hardware and software configurations single output power supply unit: power supply unit designed to deliver the majority of its rated output power to one primary direct current (d.c.) output for the purpose of powering a computer server NOTE 1: Single-output power supply units may offer one or more standby outputs that remain active whenever connected to an input power source. NOTE 2: The total rated power output from any additional power supply units outputs, other than primary and standby outputs, is not greater than 20 W. NOTE 3: Power supply units that offer multiple outputs at the same voltage as the primary output are considered single-output power supply units unless those outputs are generated from separate converters or have separate output rectification stages, or have independent current limits. single-wide blade server: blade server requiring the width of a standard blade server bay solid state drive: storage device that uses memory chips instead of rotating magnetic platters for data storage
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 14 storage device: fully-functional storage system that supplies data storage services to clients and devices attached directly or through a network NOTE 1: A storage product may be composed of integrated storage controllers, storage devices, embedded network elements, software, and other devices. NOTE 2: While storage products may contain one or more embedded processors, these processors do not execute user-supplied software applications but may execute data-specific applications (e.g. data replication, backup utilities, data compression, install agents). NOTE 3: Components and subsystems that are an integral part of the storage product architecture (e.g. to provide internal communications between controllers and disks) are considered to be part of the storage product. NOTE 4: Components that are normally associated with a storage environment (e.g. devices required for operation of an external Storage Area Network) are not considered to be part of the storage product. storage server: enterprise storage device which contains the same components as a computer server together with ≥ 10 storage devices and software (vendor or 3rd party) that supports storage system connectivity, capacity optimization management, virtualized storage environment and software defined storage NOTE: Supporting features are described by the product's datasheet description and are either accompanied with vendor specific utilities and/or commercially available software supporting these functions. Uninterruptible Power Supply (UPS): combination of convertors, switches, and energy storage devices (such as batteries) constituting a power system for maintaining continuity of load power in case of input power failure weighted geometric mean: geometric mean calculated using a predetermined factor for each of the elements prior to aggregation worklet: synthetic software routine, using real application functions focused on a particular type of computing activity, which stresses a particular characteristic of the system NOTE: A floating point and integer performance stress code is an example of a CPU worklet. 3.2 Symbols Void. 3.3 Abbreviations For the purposes of the present document, the following abbreviations apply: a.c., AC Alternating Current ACPI Advanced Configuration and Power Interface APA Auxiliary Processing Accelerator BIOS Basic Input/Output System CENELEC European Committee for Electrotechnical Standardization
CPU Central Processor Unit d.c., DC Direct Current DDR Double Data Rate DIMM Dual In-line Memory Module EEE Energy Efficient Ethernet EUT Equipment Under Test FPGA Field Programmable Gate Array GB GigaByte GPGPU General-Purpose computing on Graphics Processing Units HDD Hard Disk Drive HPC Higher Performance Computing I/O Input/Output ICT Information and Communication Technology ID Identification ISO/IEC
International Organization for Standardization/International Electrotechnical Committee SIST EN 303 470 V1.1.1:2019
ETSI ETSI EN 303 470 V1.1.1 (2019-03) 15 IT Information Technology JVM Java™ Virtual Machine LU Lower-Upper OS Operating System PCI Peripheral Component Interconnect PDU Power Distribution Unit PSU Power Supply Unit RAS Reliability, Availability and Serviceability rms root mean square
SERT™ Server Efficiency Rating Tool SOR Successive Over-Relaxation SPEC Standard Performance Evaluation Corporation SSD Solid State Drive SSJ Server Side Java™ UPS Uninterruptible Power Supply VDC Volts (DC) WCPU Weighting of CPU WMemory Weighting of Memory WStorage Weighting of Storage 4 Server product categories and representative product family configurations 4.1 General Servers are sold in different form factors, processor socket counts, resilience levels and configuration types and different groups of servers will have distinct performance capability and power demands. Servers are categorized as defined in clause 4.4 by specific form factor and configuration parameters to enable the setting of appropriate idle power or active efficiency thresholds and assessment of like products with regards to procurement or market entry requirements. To compare or evaluate systems, the evaluation or metric used shall only be made against like products. Like products are grouped into categories. Products of different categories shall not be compared using the metric of the present document. Even though servers are classified in categories by the type of system, each server is customized by its configuration to best match the application for which they are being sold or purchased. As a result, a product is represented by a fixed set of configurations. For an appropriate evaluation, the category shall be defined and the product family configurations itemized. The product family configuration establishes a single group representing the efficiency of the product, as this then covers the range of configurations of that product that would be sold. Since the products sold are custom-configured, there should be a minimum of 2 configurations that would represent the family of configurations, bounded by the high-end configuration (see clause 4.5.2) and the low-end configuration (see clause 4.5.3). 4.2 Applications and metric applicability Computer servers are architected in such a way as to be configurable to different groups of applications. The full configuration, including logical and physical elements, is optimized to deliver the most effective platform for operating those applications. The software component of the metric is designed to execute typical real world applications and is designed to stress and assess the elements and associated functionality included in the server systems. By stressing the elements in a fashion that replicates real
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.
Loading comments...