IEC TR 63250:2021

Household and similar electrical appliances - Method for measuring performance - Assessment of repeatability, reproducibility and uncertainty

IEC TR 63250:2021 deals with the determination of repeatability and reproducibility of test methods used for assessing the performance characteristics of household and similar electrical appliances. It also provides guidance for carrying out round robin tests (RRT).
It also specifies the uncertainty reporting of measurements of household and similar electrical appliances.
It describes methods to estimate the uncertainty of a measured result and to predict the range of measured values when the same appliance is measured in another laboratory applying the same measurement method.
It does not cover the development of measurement methods. It also does not deal with:
– the production variability of the appliance;
– how closely the measurement method reflects the normal use of appliances in households.

General Information

• Status: Published
• Publication Date: 23-Jun-2021
• ICS: 97.030 - Domestic electrical appliances in general
• Technical Committee: TC 59 - Performance of household and similar electrical appliances
• Drafting Committee: WG 18 - TC 59/WG 18
• Current Stage: PPUB - Publication issued
• Start Date: 24-Jun-2021
• Completion Date: 23-Jul-2021

Overview

IEC TR 63250:2021 - Household and similar electrical appliances - Method for measuring performance - Assessment of repeatability, reproducibility and uncertainty - is a Technical Report from the IEC that provides guidance on assessing the precision and measurement uncertainty of test methods used for household and similar electrical appliances. The report describes how to determine repeatability (intra‑laboratory variability) and reproducibility (inter‑laboratory variability), how to carry out and analyse round robin tests (RRTs), and how to report uncertainty for measured performance characteristics. It does not cover development of measurement methods, production variability of appliances, or how closely methods reflect normal household use.

Key Topics and Requirements

• Repeatability and reproducibility assessment: Procedures for estimating standard deviations that characterise intra‑ and inter‑laboratory variability.
• Round robin tests (RRTs): Guidance on planning, conducting and analysing inter‑laboratory comparisons to derive reproducibility and repeatability figures.
• Uncertainty estimation and reporting: Methods to estimate measurement uncertainty (including expanded uncertainty) and to predict measurement ranges when the same appliance is measured in another laboratory.
• Scrutiny and outlier testing: Techniques for checking consistency and identifying outliers, including graphical and numerical methods such as Mandel’s h and k statistics, Cochran’s C test, and Grubbs’ test.
• Data reporting: Recommended data and formats to report for reproducibility, repeatability and uncertainty assessments.
• Informative annexes: Practical examples (bottom‑up uncertainty analysis), detailed RRT procedures, worked RRT example and sample result expressions to aid implementation.

Applications and Who Uses It

IEC TR 63250 is intended for organisations involved in measurement and conformity assessment of household appliances:

• Testing laboratories conducting performance measurements who need to quantify precision and uncertainty.
• Manufacturers validating test methods and documenting measurement uncertainty for product declarations.
• Regulators and policy makers designing test programs or schemes that rely on inter‑laboratory comparability.
• Standards developers and technical committees using RRTs to set reproducibility/precision requirements for normative test standards.
• Certification bodies and third‑party assessors verifying reported values against declared limits.

Practical uses include designing RRTs, estimating measurement uncertainty for energy/water performance claims, and comparing laboratory results across jurisdictions.

Related Standards (if applicable)

IEC TR 63250 was developed following IEC procedures and complements other international guidance on measurement uncertainty and inter‑laboratory testing. Users should consult relevant IEC and ISO guidance documents and national normative references when applying the report.

Standards Content (Sample)

IEC TR 63250 ®
Edition 1.0 2021-06
TECHNICAL
REPORT
Household and similar electrical appliances – Method of measuring
performance – Assessment of repeatability, reproducibility and uncertainty
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form
or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from
either IEC or IEC's member National Committee in the country of the requester. If you have any questions about IEC
copyright or have an enquiry about obtaining additional rights to this publication, please contact the address below or
your local IEC member National Committee for further information.

IEC Central Office Tel.: [phone]
3, rue de Varembé [e-mail]
CH-1211 Geneva 20 www.iec.ch
Switzerland
About the IEC
The International Electrotechnical Commission (IEC) is the leading global organization that prepares and publishes
International Standards for all electrical, electronic and related technologies.

About IEC publications
The technical content of IEC publications is kept under constant review by the IEC. Please make sure that you have the
latest edition, a corrigendum or an amendment might have been published.

IEC publications search - webstore.iec.ch/advsearchform IEC online collection - oc.iec.ch
The advanced search enables to find IEC publications by a Discover our powerful search engine and read freely all the
variety of criteria (reference number, text, technical publications previews. With a subscription you will always
committee, …). It also gives information on projects, replaced have access to up to date content tailored to your needs.
and withdrawn publications.
Electropedia - www.electropedia.org
IEC Just Published - webstore.iec.ch/justpublished
The world's leading online dictionary on electrotechnology,
Stay up to date on all new IEC publications. Just Published
containing more than 22 000 terminological entries in English
details all new publications released. Available online and
and French, with equivalent terms in 18 additional languages.
once a month by email.
Also known as the International Electrotechnical Vocabulary

(IEV) online.
IEC Customer Service Centre - webstore.iec.ch/csc

If you wish to give us your feedback on this publication or
need further assistance, please contact the Customer Service
Centre: [e-mail].
IEC TR 63250 ®
Edition 1.0 2021-06
TECHNICAL
REPORT
Household and similar electrical appliances – Method of measuring

performance – Assessment of repeatability, reproducibility and uncertainty

INTERNATIONAL
ELECTROTECHNICAL
COMMISSION
ICS 97.030 ISBN 978-2-8322-9939-5

– 2 – IEC TR 63250:2021 © IEC 2021
CONTENTS
FOREWORD . 4
INTRODUCTION . 6
1 Scope . 7
2 Normative references . 7
3 Terms and definitions . 7
4 Determination of standard deviations . 10
4.1 General . 10
4.2 Repeatability standard deviation . 10
4.3 Reproducibility standard deviation . 10
5 Assessment of repeatability, reproducibility, and uncertainty of a measurement
method . 11
5.1 Purpose . 11
5.2 Requirements . 11
5.3 Expression of repeatability and reproducibility . 12
5.4.1 The importance of the uncertainty . 12
5.4.2 Methods to estimate uncertainty . 12
5.4.3 Expanded uncertainty calculation . 13
6 Scrutiny of results for consistency and outliers . 14
6.1 Purpose . 14
6.2 Graphical consistency technique (Mandel's h and k statistics) . 14
6.2.1 Inter-laboratory consistency statistic h . 14
6.2.2 Intra-laboratory consistency statistic k . 14
6.2.3 Evaluation . 14
6.3 Numerical outlier technique . 15
6.3.1 Cochran's C test . 15
6.3.2 Grubbs' test . 15
6.3.3 Evaluation . 15
7 Data to be reported for assessing the repeatability, reproducibility and uncertainty
of a test method. 16
Annex A (informative)  Example of bottom-up analysis . 17
A.1 General . 17
A.2 Temperature measurement system . 17
A.2.1 General . 17
A.2.2 Calibration of thermocouples . 17
A.2.3 Calibration of the DAQ system . 17
A.3 Uncertainty temperature measurement . 17
A.4 Analysis of each component in the uncertainty formulation, example
thermocouple simulator . 19
Annex B (informative)  Guidance on how to conduct round robin tests for household
and similar electrical appliances . 21
B.1 General . 21
B.2 Scope . 21
B.3 Process and responsibilities . 22
B.3.1 Process . 22
B.3.2 Responsibilities . 23
B.4 Testing laboratories . 23

B.4.1 Potential laboratories . 23
B.4.2 Announcement. 23
B.4.3 Selection of laboratories . 24
B.4.4 Final list of laboratories . 24
B.5 Transportation of the product . 24
B.5.1 Logistics . 24
B.5.2 Packaging. 24
B.6 Test . 24
B.6.1 Performance of test . 24
B.6.2 Laboratory visit . 25
B.6.3 Transmission of result . 25
B.7 Analysis, report and termination . 25
B.7.1 Analysis . 25
B.7.2 Report . 25
B.7.3 Termination and publication of final external report . 26
Annex C (informative)  Example of a round robin test and its analysis. 27
C.1 General . 27
C.2 Standard deviations and assessment of repeatability and reproducibility . 27
C.3 Scrutiny of results for consistency and outliers. 29
C.3.1 Example of Mandel's h and k statistics . 29
C.3.2 Example of numerical outlier test . 29
Annex D (informative) Example of expression of results . 32
Bibliography . 33

Figure 1 – Visualisation of straggler and outlier of Cochrane value . 16
Figure C.1 – Mandel's h statistics. 30
Figure C.2 – Mandel's k statistics . 31

Table A.1 – Description of uncertainty parameters . 18
Table A.2 – Measuring a temperature of −23 °C at a climate room temperature
between 16 °C to 32 °C . 19
Table A.3 – Expanded uncertainty in measured temperature (U(T), k = 2) at a room
temperature in the range of 16 °C to 32 °C and a lab temperature between 14 °C and

32 °C . 19
Table A.4 – Calibration results of the simulator . 20
Table C.1 – Measurement results . 27
Table C.2 – Standard deviations, repeatability and reproducibility . 28
Table C.3 – Mandel's h and k statistics . 29
Table C.4 – Cochran's test C and Grubbs' test G . 29
Table C.5 – Cochran's test C and Grubbs' test G summary . 30

– 4 – IEC TR 63250:2021 © IEC 2021
INTERNATIONAL ELECTROTECHNICAL COMMISSION
____________
HOUSEHOLD AND SIMILAR ELECTRICAL APPLIANCES –
METHOD OF MEASURING PERFORMANCE – ASSESSMENT OF
REPEATABILITY, REPRODUCIBILITY AND UNCERTAINTY

FOREWORD
1) The International Electrotechnical Commission (IEC) is a worldwide organization for standardization comprising
all national electrotechnical committees (IEC National Committees). The object of IEC is to promote international
co-operation on all questions concerning standardization in the electrical and electronic fields. To this end and
in addition to other activities, IEC publishes International Standards, Technical Specifications, Technical Reports,
Publicly Available Specifications (PAS) and Guides (hereafter referred to as "IEC Publication(s)"). Their
preparation is entrusted to technical committees; any IEC National Committee interested in the subject dealt with
may participate in this preparatory work. International, governmental and non-governmental organizations liaising
with the IEC also participate in this preparation. IEC collaborates closely with the International Organization for
Standardization (ISO) in accordance with conditions determined by agreement between the two organizations.
2) The formal decisions or agreements of IEC on technical matters express, as nearly as possible, an international
consensus of opinion on the relevant subjects since each technical committee has representation from all
interested IEC National Committees.
3) IEC Publications have the form of recommendations for international use and are accepted by IEC National
Committees in that sense. While all reasonable efforts are made to ensure that the technical content of IEC
Publications is accurate, IEC cannot be held responsible for the way in which they are used or for any
misinterpretation by any end user.
4) In order to promote international uniformity, IEC National Committees undertake to apply IEC Publications
transparently to the maximum extent possible in their national and regional publications. Any divergence between
any IEC Publication and the corresponding national or regional publication shall be clearly indicated in the latter.
5) IEC itself does not provide any attestation of conformity. Independent certification bodies provide conformity
assessment services and, in some areas, access to IEC marks of conformity. IEC is not responsible for any
services carried out by independent certification bodies.
6) All users should ensure that they have the latest edition of this publication.
7) No liability shall attach to IEC or its directors, employees, servants or agents including individual experts and
members of its technical committees and IEC National Committees for any personal injury, property damage or
other damage of any nature whatsoever, whether direct or indirect, or for costs (including legal fees) and
expenses arising out of the publication, use of, or reliance upon, this IEC Publication or any other IEC
Publications.
8) Attention is drawn to the Normative references cited in this publication. Use of the referenced publications is
indispensable for the correct application of this publication.
9) Attention is drawn to the possibility that some of the elements of this IEC Publication may be the subject of patent
rights. IEC shall not be held responsible for identifying any or all such patent rights.
IEC 63250 has been prepared IEC technical committee 59: Performance of household and
similar electrical appliances. It is a Technical Report.
The text of this Technical Report is based on the following documents
Draft Report on voting
59/752/DTR 59/765/RVDTR
Full information on the voting for its approval can be found in the report on voting indicated in
the above table.
The language used for the development of this Technical Report is English.
Words in bold in the text are defined in Clause 3.
This document was drafted in accordance with ISO/IEC Directives, Part 2, and developed in
accordance with ISO/IEC Directives, Part 1 and ISO/IEC Directives, IEC Supplement, available
at www.iec.ch/members_experts/refdocs. The main document types developed by IEC are
described in greater detail at www.iec.ch/standardsdev/publications.

The committee has decided that the contents of this document will remain unchanged until the
stability date indicated on the IEC website under webstore.iec.ch in the data related to the
specific document. At this date, the document will be
• reconfirmed,
• withdrawn,
• replaced by a revised edition, or
• amended.
– 6 – IEC TR 63250:2021 © IEC 2021
INTRODUCTION
To encourage the efficient use of energy and other resources, national governments and
regional authorities have issued regulations that mandate the provision of information to
consumers regarding the energy and water consumption of household appliances and
associated performance characteristics.
Therefore, methods for measuring performance characteristics must be of sufficient accuracy
to provide confidence to governments, consumers and manufacturers.
The accuracy of a test method is expressed in terms of bias and precision. Precision, when
evaluating test methods, is expressed in terms of two measurement concepts: repeatability
(intra-laboratory variability) and reproducibility (inter-laboratory variability). Therefore,
standard procedures are required for determining the repeatability and the reproducibility of
test methods. The determination of levels of repeatability and reproducibility is frequently
done by carrying out round robin tests (RRT). The repeatability of a test method must be
sufficiently accurate for comparative testing. The reproducibility of a test method must be
sufficiently accurate for the determination of values that are declared, and for checking these
declared values. Other ways to assess the uncertainty are possible.
Uncertainty reporting is essential to ensure measured data are interpreted correctly. Especially
when data of measurements are to be compared between laboratories or when normative
requirements are set up, it is necessary to know the uncertainty with which data can be
measured.
In conformity assessment using a binary decision rule, a property of an item is measured, and
the item is accepted as conforming if the measured value of the property lies within a defined
acceptance interval. A measured value outside the acceptance interval leads to rejection of the
item as non-conforming.
The objective of this technical report is to give guidelines for household and similar electrical
appliances within TC 59, but it can also be used for assessing other types of appliances outside
the technical committee 59 and its subcommittees' environment.
It is intended to collate and summarise the information needed for assessing the repeatability,
reproducibility and uncertainty of measurements of performance of household and similar
electrical appliances present in previous IEC publications .
___________
IEC TR 61923, IEC TR 62617 and IEC TR 62970

HOUSEHOLD AND SIMILAR ELECTRICAL APPLIANCES –
METHOD OF MEASURING PERFORMANCE – ASSESSMENT OF
REPEATABILITY, REPRODUCIBILITY AND UNCERTAINTY

1 Scope
This Technical Report deals with the determination of repeatability and reproducibility of test
methods used for assessing the performance characteristics of household and similar electrical
appliances. It also provides guidance for carrying out round robin tests (RRT).
It also specifies the uncertainty reporting of measurements of household and similar electrical
appliances.
It describes methods to estimate the uncertainty of a measured result and to predict the range
of measured values when the same appliance is measured in another laboratory applying the
same measurement method.
It does not cover the development of measurement methods. It also does not deal with:
– the production variability of the appliance;
– how closely the measurement method reflects the normal use of appliances in households.
NOTE 1 Although this technical report does not cover the development of test methods, it can be taken into
consideration for this purpose.
NOTE 2 For the purpose of this technical report production variability includes the variation of the individual
appliances of the same type and model manufactured on the same production line.
NOTE 3 For noise standardisation, some deviating definitions are used (see. IEC 60704-3:2019).
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies.
For undated references, the latest edition of the referenced document (including any
amendments) applies.
ISO 5725-2:2019, Accuracy (trueness and precision) of measurement methods and results –
Part 2: Basic method for the determination of repeatability and reproducibility of a standard
measurement method
ISO 80000-1:2009, Quantities and units – Part 1: General
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following
addresses:
• IEC Electropedia: available at http://www.electropedia.org/
• ISO Online browsing platform: available at http://www.iso.org/obp
3.1
accuracy
closeness of agreement between a test result or measurement result and the true value
Note 1 to entry: In practice, the accepted reference value is substituted for the true value.
Note 2 to entry: The term "accuracy", when applied to a set of test or measurement results, involves a combination
of random components and a common systematic error or bias component.
Note 3 to entry: Accuracy refers to a combination of trueness and precision.

– 8 – IEC TR 63250:2021 © IEC 2021
[SOURCE: ISO 3534-2:2006, 3.3.1, modified – Cross-references have been deleted]
3.2
precision
closeness of agreement between independent test/measurement results obtained under
stipulated conditions
Note 1 to entry: Precision depends only on the distribution of random errors and does not relate to the true value
or the specified value.
Note 2 to entry: The measure of precision is usually expressed in terms of imprecision and computed as a standard
deviation of the test results or measurement. Less precision is reflected by a larger standard deviation.
Note 3 to entry: Quantitative measures of precision depend critically on the stipulated conditions. Repeatability
conditions and reproducibility conditions are particular sets of extreme stipulated conditions.
[SOURCE: ISO 3534-2:2006, 3.3.4, modified – Cross-references have been deleted]
3.3
repeatability
precision under repeatability conditions.
Note 1 to entry: Repeatability can be expressed quantitatively in terms of the dispersion characteristics of the
results.
[SOURCE: ISO 3534-2:2006, 3.3.5, modified – Cross-references have been deleted]
3.4
repeatability conditions
observation conditions where independent test/measurement results are obtained with the
same method on identical test/measurement items in the same test or measuring facility by the
same operator using the same equipment within short intervals of time
Note 1 to entry: Repeatability conditions include:
• the same measurement procedure or test procedure;
• the same operator;
• the same measuring or test equipment used under
• the same conditions;
• the same location;
• repetition over a short period of time.
[SOURCE: ISO 3534-2:2006, 3.3.6, modified – Cross-references have been deleted]
3.5
repeatability standard deviation
standard deviation of test results or measurement results obtained under repeatability
conditions
Note 1 to entry: It is a measure of the dispersion of the distribution of test results or measurement results under
repeatability conditions.
Note 2 to entry: Similarly, "repeatability variance" and "repeatability coefficient of variation" can be defined and
used as measures of the dispersion of test or measurement results under repeatability conditions.
[SOURCE: ISO 3534-2:2006, 3.3.7, modified – Cross-references have been deleted]
3.6
reproducibility
precision under reproducibility conditions
Note 1 to entry: Reproducibility can be expressed quantitatively in terms of the dispersion characteristics of the
results.
Note 2 to entry: Results are usually understood to be corrected results
[SOURCE: ISO 3534-2:2006, 3.3.10, modified – Cross-references have been deleted]

3.7
reproducibility conditions
observation conditions where independent test/measurement results are obtained with the
same method on identical test/measurement items in different test or measurement facilities
with different operators using different equipment
[SOURCE: ISO 3534-2:2006, 3.3.11, modified – Cross-references have been deleted]
3.8
reproducibility standard deviation:
standard deviation of test results or measurement results obtained under reproducibility
conditions
Note 1 to entry: It is a measure of the dispersion of the distribution of test or measurement results under
reproducibility conditions.
Note 2 to entry: Similarly, "reproducibility variance" and "reproducibility coefficient of variation" can be defined and
used as measures of the dispersion of test or measurement results under reproducibility conditions.
[SOURCE: ISO 3534-2:2006, 3.3.12 modified – Cross-references have been deleted]
3.9
outlier
member of a small subset of observations that appears to be inconsistent with the remainder of
a given sample
Note 1 to entry: The classification of an observation or a subset of observations as outlier(s) is relative to the
chosen model for the population from which the data set originates. This or these observations are not to be
considered as genuine members of the main population.
Note 2 to entry: An outlier may originate from a different underlying population, or be the result of incorrect
recording or gross measurement error.
Note 3 to entry: The subset may contain one or more observations.
[SOURCE: ISO 16269-4:2010, 2.2 modified – Cross-references have been deleted]
3.10
statistical uncertainty
repeatability standard deviation obtained in one laboratory under repeatability conditions
3.11
expanded uncertainty
quantity defining an interval about the result of a measurement that may be expected to
encompass a large fraction of the distribution of values that could reasonably be attributed to
the measurand
Note 1 to entry: The fraction may be viewed as the coverage probability or level of confidence of the interval.
Note 2 to entry: To associate a specific level of confidence with the interval defined by the expanded uncertainty
requires explicit or implicit assumptions regarding the probability distribution characterized by the measurement
result and its combined standard uncertainty. The level of confidence that may be attributed to this interval can be
known only to the extent to which such assumptions may be justified.
Note 3 to entry: Expanded uncertainty is termed overall uncertainty in paragraph 5 of Recommendation INC-1
(1980)
[SOURCE: ISO/IEC Guide 98-3:2008, 2.3.5]
3.12
bias
difference between the expectation of a test result or measurement result and a true value
[SOURCE: ISO 3534-2:2006, 3.3.2, modified – Cross-references and Notes have been deleted]

– 10 – IEC TR 63250:2021 © IEC 2021
3.13
round robin testing
RRT
ringtest
process in which one or more items are tested according to a specific protocol by a number of
different laboratories.
4 Determination of standard deviations
4.1 General
Repeatability and reproducibility standard deviations serve as parameters for assessing:
– the suitability of a measurement method;
– the accuracy of a measurement result;
– conformance of measured values to acceptance interval.
Rounding is only applied to reported values in Clause 7. If numbers are rounded, they are
rounded to the nearest number in accordance with ISO 80000-1:2009, Annex B, Rule B. If the
rounding takes place to the right of the comma, the omitted places are not filled with zeros.
4.2 Repeatability standard deviation
The repeatability standard deviation s of a measurement method within laboratory i is
L,i
calculated from Equation (1):
n
i
s ( xx− ) (1)
L ,i ∑ ki i
n−1
i
k =1
i
where
n is the number of measurement results;
i
x is the particular measurement result;
ki
x is the arithmetic mean value of n measurement results x of laboratory i.
i i
k
The average repeatability standard deviation s of a measurement method within p
r
laboratories is calculated from Equation (2):
p
ss=
(2)
∑
r L,i
p
i=1
where
s is the repeatability standard deviation within laboratory i;
L ,i
p is the number of laboratories participating in the inter-laboratory test.
4.3 Reproducibility standard deviation
The reproducibility standard deviation s of a measurement method is calculated from
R
Equations (3), (4), and (5):
=
p
x = x
∑ (3)
mi
p
i=1
p
nn=
(4)
∑ i
p
i=1
p
11n−
s x−+x s
( ) (5)
R ∑ im r
pn−1
i=1
x
where x is the arithmetic mean value of the arithmetic mean values of the participating
i
m
laboratories and n is the number of measurements in all laboratories.
NOTE s is expected to be greater than s .
R r
5 Assessment of repeatability, reproducibility, and uncertainty of a
measurement method
5.1 Purpose
The purpose of assessing the repeatability and the reproducibility is to determine whether a
method of measurement is suitable for comparative testing only or for the determination and
verification of values independent of the laboratory performing the measurement.
A low repeatability standard deviation is necessary to determine whether a method of
measurement is suitable for comparative testing. A low reproducibility standard deviation is
necessary to determine whether this method of measurement is suitable for determination and
verification of values independent of the laboratory performing the measurement. A more
general expression of the reproducibility of a measurement is the uncertainty of the
measurement method. This concept can be either based on the adequate mathematical
description and analysis of the input quantities and their uncertainties (5.4.2 a) or by
measurement of the reproducibility, for example by performing and analysing a round robin
test (5.4.2 b).
In conformity assessment that uses a binary decision rule, a property of an item is measured,
and the item is accepted as conforming if the measured value of the property lies within a
defined acceptance interval. A measured value outside the acceptance interval leads to
rejection of the item as non-conforming. The acceptance interval is either an externally imposed
interval of conforming values or a mutually agreed interval of permissible values.
5.2 Requirements
The following requirements are taken into account when assessing the reproducibility and/or
repeatability of a measurement method:
a) repeatability and reproducibility of a measurement method are assessed by an
interlaboratory test;
NOTE Data obtained in interlaboratory studies can indicate that further effort is needed to improve the
measurement method.
b) the necessary number of measurements and laboratories participating in an interlaboratory
test depends on the type of appliance and are established by the responsible body. With

=
– 12 – IEC TR 63250:2021 © IEC 2021
respect to a statistical evaluation of the measurement results, at least five measurement
results from each of at least five laboratories, excluding any outlier, should be available.
The number of measurements is the same for each laboratory;
c) the test procedures are specified completely and accurately, including the rounding of
values from measurement results, the accuracy of the measuring instruments and the
environmental conditions, as appropriate;
d) wherever possible, precise values of intermediate results (without rounding) should be
recorded and used in subsequent calculations to ensure that the final result is as accurate
as possible;
e) the test laboratories shall adhere to the measurement procedure specified in the standard
or in the test programme;
f) only appliances with low variability shall be used;
g) the reference appliance, if any, shall have the lowest variability.
5.3 Expression of repeatability and reproducibility
Reproducibility standard deviation can be a value of absolute or relative nature. Depending
on the nature of the error, either one or the other shall be preferred. Absolute values of the
reproducibility standard deviation shall be preferred when the measurement error is not likely
to be influenced by the absolute value of the measurand. Relative values are preferred when
the error will grow or diminish with the absolute magnitude of the measurand.
5.4 The approach to uncertainty
5.4.1 The importance of the uncertainty
When a measurement has been performed giving a value as a result for some quantity (i.e. the
measurand), how accurate is the measurand? In other words:
– if the measurement is repeated, will the same value be achieved as the initial result?
– if another group or another laboratory performs the measurement, how close are the results
expected to be?
By means of an uncertainty amount, an uncertainty interval y ± U may be calculated, where y is
the measurement result and U the expanded uncertainty that is determined to give the interval
a high probability (often 95 %) to cover the true value, y, of the measurand. U is said to be the
uncertainty associated with the result y.
The uncertainty interval of a measurement is therefore a basis for qualifying the measurement.
The narrower the confidence interval desired, i.e. the smaller the value of the uncertainty U, the
more care is needed for the measurement method, the measuring equipment, the training of
the operators, and the number of repetitions of the same experiment.
5.4.2 Methods to estimate uncertainty
There are in principle two ways to estimate uncertainty: a bottom-up method and a top-down
method. The two methods should often be used in parallel to achieve a reliable uncertainty
amount.
a) The bottom-up method (Refer to ISO/IEC GUIDE 98-3:2008 and Annex A for an example)
In this method, the measurement result y is expressed as a function of input quantities. This
function is often the formula used for the calculation of the result.
In the case of home laundry appliances, y can represent one of the final measurement
results such as water consumption, energy consumption, washing performance, spin speed,
spin drying performance, programme duration or rinsing efficiency. The input quantities can
be temperature, masses, times, power, etc.
The magnitude of all the uncertainty contributions of each input quantity is estimated.
By combining the uncertainties of the input quantities according to the law of propagation of
uncertainty (see Clause 2 of ISO/IEC GUIDE 98-3:2008), the uncertainty of the result y can
be calculated.
With this calculation, it can be seen how a specific uncertainty contribution from an input
quantity influences the combined uncertainty of the final result, and, therefore, how a
reduction in an uncertainty contribution from an input quantity will influence the combined
uncertainty of the final result.
Uncertainties can usually be reduced at some cost by making more measurements, using
other methods or other equipment. This means that different approaches can be followed to
reduce the uncertainty of the final result in the most cost-effective way.
Bottom-up calculations of uncertainty of results can be checked for consistency through
inter-laboratory comparisons. If, for example, a reference test material is used in the
measurement, its properties should be consistent between laboratories.
NOTE This turned out to not be the case for the carpet used as reference for the dust pick up measurement of
vacuum cleaners. Such mistakes or errors might be found only by measurement in different laboratories, e.g.
through a round robin test or other top-down methods
b) The top-down method (Annex B and Annex C)
In this method, the reproducibility standard deviation is estimated from testing of the
same machine (or the same model) in different laboratories using the same measurement
method. This testing is in general named 'ringtest' or 'round robin test'. The
reproducibility standard deviation of the measurement results can then be seen as the
inherent uncertainty of the measuring method because it can be influenced by remaining
differences in the ambient conditions, test personnel, and whatever else may be different
between different measurements in different laboratories. In principle, it is only valid for the
machine investigated in each ringtest, but results can be also true for similar types of
machines.
Therefore, the two methods 'bottom up' and 'top down' may be used in parallel to achieve a
reliable uncertainty quantification. However, both methods depend on the validity of the
model (for the bottom-up method) or the data (for the top-down method) used.
5.4.3 Expanded uncertainty calculation
The uncertainty of a measured result has two sources:
– the statistical uncertainty of what is measured, as expressed in the repeatability
standard deviation, showing the accuracy of the measurement in the laboratory having
done the measurement;
– the uncertainty of the measuring method itself. This is expressed as expanded uncertainty
where it is common to set the borders at a 95 % confidence interval, which is the minimum
and maximum value range within which the average measured result can be found when the
measurement is re-done at any other laboratory.
To be meaningful, the uncertainty statement must have an associated confidence level: i.e. it
is necessary to state the probability that the true value lies within the range given. The reasons
for choosing a 95 % confidence level in this document are as follows:
– it is established practice throughout much of Europe, North America, and Asia (see ISO/IEC
GUIDE 98-3:2008);
– ISO/IEC GUIDE 98-3:2008 assumes that the combined uncertainty has a distribution that is
a close approximation to a normal distribution. A 95 % confidence level approximates to a
range of 2 times the standard deviation. It is a widely held view that, for most measurement
systems, the approximation to a normal distribution for the distribution of the combined
uncertainty is reliable out to 2 standard deviations, but beyond that the approximation is
less reliable.
If a normal distribution can be assumed, the 95 % confidence interval is given by multiplying
the reproducibility standard deviation by a factor of 2 to calculate the expanded uncertainty
of a measurand.
Like reproducibility standard deviation (5.3), the expanded uncertainty can be a value of
absolute or relative nature. Depending on the nature of the error, either one or the other is
preferred. It is the responsibility of the responsible standardisation committee to conclude in
which way the expanded uncertainty is expressed. If expressed in absolute values, the
expanded uncertainty is given with the term '(abs)', otherwise in '%'.

– 14 – IEC TR 63250:2021 © IEC 2021
Examples of the expression of values are given in Annex D.
6 Scrutiny of results for consistency and outliers
6.1 Purpose
Outliers related to measurement results as well as to test laboratories are scarce but cannot
always be avoided. They are taken into consideration, but they should not lead to a distortion
of the results of a test which has been carried out to assess the repeatability and
reproducibility of a measurement method. An example is shown in Annex C with Table C.1
and Table C.2
A statistical procedure by which suspect measurement results or laboratories are judged is not
specified. The final decision about the treatment of outliers (e.g. repetition of a test or ignoring
results) is taken by the responsible body with justification.
NOTE The scrutiny of measurement results for consistency and outliers is based on 8.3 in ISO 5725-2:2019;
attention is drawn to 8.3.3.2 in ISO 5725-2:2019 concerning the application of Cochran's and Grubb's tests.
6.2 Graphical consistency technique (Mandel's h and k statistics)
6.2.1 Inter-laboratory consistency statistic h
The inter-laboratory consistency statistic h for laboratory i is calculated from Equation (6):

i
x − x
i m
h =
i
p
1 (6)
x − x
( )
∑ i m
p− 1
i=1
6.2.2 Intra-laboratory consistency statistic k
The intra-laboratory consistency statistic k for laboratory i is calculated by Equation (7):

i
s
L ,i
k = (7)
i
s
r
6.2.3 Evaluation
The calculated values are plotted as appropriate (refer to the examples of Table C.3 and
Figure C.1 and Figure C.2). Lines are drawn on the plots corresponding to the indicators given
in Tables 7 and 8 in ISO 5725-2:2019. These indicator lines serve as guidance when examining
patterns in the data.
– Various patterns can appear in the h plots. All laboratories can have both positive and
negative values. Neither of these patterns is unusual nor requires investigation. But if all
the h values for one laboratory are of one sign, and the h values for the other laboratories
are all of the other sign, then the reason should be sought.
– If one laboratory stands out on the k plot presenting many large values, then the reason
should be sought: this indicates that it has a poorer repeatability than the other
laboratories. A laboratory could give rise to consistently small k values because of such
factors as excessive rounding of its data or measurement scale with inadequate resolution.
If the h and k plots indicate that specific laboratories exhibit patterns of results that are markedly
different from the others, this laboratory should be contacted to ascertain the cause of the
different behaviour. The responsible body could:
a) accept the laboratory's data as possible interim result;
b) ask the laboratory to redo the measurement (if feasible);

c) delete the laboratory's data from the study.
6.3 Numerical outlier technique
6.3.1 Cochran's C test
Cochran's criterion applies strictly only when all the standard deviations are derived from the
same number (n) of measurement results obtained under repeatability conditions. It tests only
the highest
...