ISO/TR 24514:2018

Activities relating to drinking water and wastewater services — Examples of the use of performance indicators using ISO 24510, ISO 24511 and ISO 24512 and related methodologies

This document provides several examples of varying complexity which illustrate the use and intent of the performance assessment methodology set out in ISO 24510, ISO 24511 and ISO 24512. The document also provides examples of the parallel and similar practices for measuring performance or establishing benchmarks as found in various institutional circumstances. These examples represent practices in a range of utilities (e.g. small, medium and large water utilities; water utilities from developed and developing countries; water utilities in both the public and private sectors; government and non-governmental agencies; and utilities with and without certified management systems). They are drawn from many geographical locations.

Activités relatives aux services de l'eau potable et de l'assainissement — Exemples d'utilisation d'indicateurs de performance à l'aide l'ISO 24510, l'ISO 24511 et l'ISO 24512 et des méthodologies associées

General Information

• Status: Published
• Publication Date: 24-May-2018
• ICS: 03.080.30 - Services for consumers
• ICS: 13.060.20 - Drinking water
• ICS: 13.060.30 - Sewage water
• Technical Committee: ISO/TC 224 - Drinking water, wastewater and stormwater systems and services
• Drafting Committee: ISO/TC 224 - Drinking water, wastewater and stormwater systems and services
• Current Stage: 6060 - International Standard published
• Completion Date: 25-May-2018

Standards Content (Sample)

TECHNICAL ISO/TR
REPORT 24514
First edition
2018-05
Activities relating to drinking water
and wastewater services — Examples
of the use of performance indicators
using ISO 24510, ISO 24511 and ISO
24512 and related methodologies
Activités relatives aux services de l'eau potable et de
l'assainissement — Exemples d'utilisation d'indicateurs de
performance à l'aide l'ISO 24510, l'ISO 24511 et l'ISO 24512 et des
méthodologies associées
Reference number
ISO/TR 24514:2018(E)
ISO 2018
---------------------- Page: 1 ----------------------
ISO/TR 24514:2018(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2018

All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may

be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting

on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address

below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Fax: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
ii © ISO 2018 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/TR 24514:2018(E)
Contents Page

Foreword ........................................................................................................................................................................................................................................iv

Introduction ..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

4 Format for the examples provided in this document ..................................................................................................... 3

4.1 General ........................................................................................................................................................................................................... 3

4.2 Name of the organization ............................................................................................................................................................... 3

4.3 Background of the organization ............................................................................................................................................... 3

4.4 Objective for making use of performance indicators ............................................................................................. 3

4.5 Performance indicator methodology ................................................................................................................................... 3

4.6 Summary of the experience ......................................................................................................................................................... 3

Annex A (informative) International organizations ............................................................................................................................. 4

Annex B (informative) National regulatory organizations ............................................................................................................ 8

Annex C (informative) Associations or groups of organizations ..........................................................................................23

Annex D (informative) Individual utilities ..................................................................................................................................................41

Bibliography .............................................................................................................................................................................................................................65

© ISO 2018 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/TR 24514:2018(E)
Foreword

ISO (the International Organization for Standardization) is a worldwide federation of national standards

bodies (ISO member bodies). The work of preparing International Standards is normally carried out

through ISO technical committees. Each member body interested in a subject for which a technical

committee has been established has the right to be represented on that committee. International

organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.

ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of

electrotechnical standardization.

The procedures used to develop this document and those intended for its further maintenance are

described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the

different types of ISO documents should be noted. This document was drafted in accordance with the

editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of

any patent rights identified during the development of the document will be in the Introduction and/or

on the ISO list of patent declarations received (see www .iso .org/patents).

Any trade name used in this document is information given for the convenience of users and does not

constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and

expressions related to conformity assessment, as well as information about ISO's adherence to the

World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see the following

URL: www .iso .org/iso/foreword .html.

This document was prepared by Technical Committee ISO/TC 224, Service activities relating to drinking

water supply, wastewater and stormwater systems.

Any feedback or questions on this document should be directed to the user’s national standards body. A

complete listing of these bodies can be found at www .iso .org/members .html.
iv © ISO 2018 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/TR 24514:2018(E)
Introduction
0.1 General

This document is a companion document to ISO 24510, ISO 24511 and ISO 24512 and was originally to

be developed with the thought that many organizations (water utilities, regulators, users’ associations)

would benefit from examples illustrating how the performance indicator methodology and the

performance indicators, such as those described and illustrated in the series of standards on the topic of

activities relating to drinking water and wastewater services produced by ISO/TC 224, have been applied

by a variety of organizations. Please refer to ISO 24510, ISO 24511 and ISO 24512 for guidance on the

[1],[2],[3]

process and concepts . However, there are many similar and parallel methodologies measuring

the performance of organizations, not least of which are several “benchmarking” methodologies, some

of which are described in this document. Therefore, this document provides examples of quantification

practices and processes for measuring organizational performance.

Performance indicators as developed by ISO TC 224 are intended to be used primarily within water

utilities, often over time, to demonstrate progress towards achieving high-level corporate objectives. In

practice, these indicators are also being used to indicate projected benefits that could be achieved with

changes in investment strategies or operations. Benchmarking, on the other hand, is used primarily

to demonstrate the efficiency of operations, particularly by sharing information between comparable

organizations often with the identification of best practices related to the particular operation being

benchmarked. Used in a time series, all can also be applied to demonstrating progress towards meeting

objectives and demonstrating continuous improvement. What is confusing is that both can use exactly

the same metric, i.e. a numerator using one data set, and a denominator using another data set. For

example, energy used/megalitre of water produced.

Figure 1 illustrates the basic similarities in the use of a typical metric for the internal uses linked to

achieving corporate objectives (performance indicators) and how benchmarking (as described in

[11]

ISO 24523 ) enables information sharing on metrics and ultimately sharing of best practices. Both

methodologies are intended to assist in continuous improvement and to measure achievement of

objectives.
Figure 1 — Illustration of uses of performance metrics.

Within the field of organizational assessment or measurement, there are several terms that are regarded

at least informally as being interchangeable. These include “performance indicators”, “performance

measures” and “benchmarks”. The first is the preferred terminology of the series of standards on the

topic of activities relating to drinking water and wastewater services produced by ISO/TC 224. The

terminology describing the process for utilizing “performance indicators” and “performance measures”

© ISO 2018 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/TR 24514:2018(E)

tends to be “assessment”. The terminology describing the development and utilization of “benchmarks”

is “benchmarking”. The end result can be the same: both result in the calculation of a metric that

indicates the status achieved in respect of performance of a particular function or activity. However,

these terms are used to achieve subtly different purposes. Performance indicators are normally used

to measure activities within a single organization where the basis of calculation does not change

from year to year and, at least in the ISO standards referenced, are directly related to organizational

objectives at the corporate level.

Benchmarking is intended to encourage and allow comparison between organizations providing similar

services in a defined context. Such methodologies often serve to measure achievement of operational

objectives (which are necessarily a subset of corporate objectives and which support the achievement of

the corporate objectives). That is, the elements comprising the numerator and the elements comprising

the denominator should be identical between the organizations being compared. The benchmarks will

enable comparison of organizations in common and defined fields, but are not necessarily directly

or closely linked to the high-level corporate objectives. ISO 24523 provides information on the

[11]
benchmarking process .

A distinction between the performance indicator methodology and the benchmarking methodology is

that while performance indicator comparison can be an essential part of benchmarking, performance

assessment is a crucial part of benchmarking. Therefore, benchmarking is a way to apply the ISO 2451X

standards. However, “benchmarking” differentiates from pure performance indicator comparisons

through additional and continuing work steps, in particular “analysis” and “implementation” (see

[12]
ISO 24523, Table 1 ), leading to performance improvement.

The purpose of this document is to provide practical examples based on real life applications for the

consideration of utilities using the guidelines in ISO 24510, ISO 24511 and ISO 24512, and also to

indicate examples of where other metrics are employed for regulatory and other purposes.

This document should encourage and assist utilities, particularly small and medium-sized organizations,

when using ISO 24510, ISO 24511 and ISO 24512, to think and communicate clearly about the meaning

and use of performance indicators.

0.2 Summary of the methodology of the series of standards on the topic of activities relating

to drinking water and wastewater services produced by ISO/TC 224

The series of standards on the topic of activities relating to drinking water and wastewater services

produced by ISO/TC 224 provide guidance for water utilities that wish to demonstrate that they are

meeting their broad social and other objectives as established by top management. These often reflect

objectives established implicitly or explicitly in legislation which may govern the delivery of water

services. The series of standards on the topic of activities relating to drinking water and wastewater

services produced by ISO/TC 224 contemplate a three-step process. Step 1 is to determine water

utilities’ strategic objectives. Such objectives in the case of water and sanitation services explicitly

include promoting public health, protecting the environment and providing for a sustainable service.

Step 2 asks what service criteria are to be used to determine if the objective is being met. The final step

asks what metrics should be used to demonstrate that the criteria are being achieved.

For example, Table 1 may represent these three stages in respect to the objective of promoting

public health.

Table 1 — Example of the performance indicator steps employed within the series of standards on

the topic of activities relating to drinking water and wastewater services produced by ISO/TC 224

Objective Service criteria Indicator
(Step 1 – define) (Step 2 – how to measure) (Step 3 – establish a metric)

Promoting public health Delivering safe drinking water Percentage of delivered water quality

tests that meet regulated requirements
vi © ISO 2018 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/TR 24514:2018(E)
0.3 Purpose of the applications

In many countries, governments require that utilities (both public and private) report on their

performance, in a unified and consistent manner. For example, all countries within the OECD are

committed to such a policy and have established requirements for public sector utilities to publish

annual reports indicating measures of their performance. The purpose of this policy and program is

to provide assurances to the population that these utilities are effective in their activities (typically

showing the economic efficiency of the activity in terms of cost/unit of output. For associations

representing member utilities (sometimes in the public sector but often in the private sector) the

association establishes methodologies for reporting on a uniform basis the performances achieved by

the members. The purpose of this may often be to demonstrate good corporate citizenship and may

include indicators of environmental protection, or consumer relations. For individual utilities (again for

both public and private utilities), the purpose of calculating performance measures may be principally

for internal purposes (e.g. reporting to management, demonstrating continuous improvement, or

demonstrating the need for investment in new technology or for repair of infrastructure).

Regardless of the purpose for which the practice of measuring and reporting performance was

established, benefits are obtained and shared with all stakeholders. In addition, all the metrics

developed can be associated with organizational objectives and purposes.
© ISO 2018 – All rights reserved vii
---------------------- Page: 7 ----------------------
TECHNICAL REPORT ISO/TR 24514:2018(E)
Activities relating to drinking water and wastewater
services — Examples of the use of performance indicators
using ISO 24510, ISO 24511 and ISO 24512 and related
methodologies

IMPORTANT — The examples in this report are included only because they illustrate the use

of the series of standards on the topic of activities relating to drinking water and wastewater

services produced by ISO/TC 224 or similar requirements or practices of assessment or

benchmarking.

Value judgements expressed in these examples relate to the relative benefits for each example and

reflect decisions made specifically by the management of the utilities in the examples. No endorsement

is given by ISO/TC 224 to
a) those decisions,
b) the data used, or

c) their conformity with other ISO standards or non-ISO practices or requirements.

No endorsement is given by ISO/TC 224 to any organization or their practices.

No utility’s particular application of the series of standards on the topic of activities relating to drinking

water and wastewater services produced by ISO/TC 224 or the parallel measurement requirements or

practices is recommended because the management of each utility has selected the application most

suited to its needs. No endorsement is given by ISO/TC 224 of the choices made by individual utilities

or the relative merits of these different applications of the series of standards on the topic of activities

relating to drinking water and wastewater services produced by ISO/TC 224.
1 Scope

This document provides several examples of varying complexity which illustrate the use and intent of

the performance assessment methodology set out in ISO 24510, ISO 24511 and ISO 24512. The document

also provides examples of the parallel and similar practices for measuring performance or establishing

benchmarks as found in various institutional circumstances. These examples represent practices in

a range of utilities (e.g. small, medium and large water utilities; water utilities from developed and

developing countries; water utilities in both the public and private sectors; government and non-

governmental agencies; and utilities with and without certified management systems). They are drawn

from many geographical locations.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— ISO Online browsing platform: available at https: //www .iso .org/obp
— IEC Electropedia: available at http: //www .electropedia .org/
© ISO 2018 – All rights reserved 1
---------------------- Page: 8 ----------------------
ISO/TR 24514:2018(E)
3.1
benchmark

single value, representing an accepted reference value derived either from comparisons among

participants or from literature, used for orientation

Note 1 to entry: The benchmark may be determined collaboratively or individually.

Note 2 to entry: By clustering, different benchmarks can occur for different peer groups.

[SOURCE: ISO 24523, 3.2]
3.2
indicator

parameter, or a value derived from parameters, which provides information about a subject matter

with a significance extending beyond that directly associated with a parameter value

Note 1 to entry: Adapted from OECD works on “Core sets of indicators for environmental performance

[11]
reviews” .

Note 2 to entry: Indicators can refer to context, conditions, means, activities or performances.

3.3
measure, noun
value resulting from measurement and the process used to obtain that value
[SOURCE: ISO 9241-11:1998, 3.11]
3.4
performance
measurable result

Note 1 to entry: Performance can relate either to quantitative or qualitative findings.

Note 2 to entry: Performance can relate to the management of activities, processes, products, services, systems

or organizations.

[SOURCE: ISO/IEC Directives Part 1 and Consolidated Supplement — 2017 (8th edition), Annex SL,

Appendix 2, 1605 3.13]
3.5
performance measure

means used to assess the system performance, typically by diagnostic or relative performance methods

[SOURCE: ISO TR 19358:2002, 2.6, modified — term made singular.]
3.6
process

set of interrelated or interacting activities that use inputs to deliver an intended result

Note 1 to entry: Whether the "intended result" of a process is called an output, product or service depends on the

context of the reference.

Note 2 to entry: Inputs to a process are generally the outputs of other processes and outputs of a process are

generally the inputs to other processes.

Note 3 to entry: Two or more interrelated and interacting processes in series can also be referred to as a process.

Note 4 to entry: Processes in an organization are generally planned and carried out under controlled conditions

to add value.

Note 5 to entry: A process where the conformity of the resulting output cannot be readily or economically

validated is frequently referred to as a "special process".
2 © ISO 2018 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/TR 24514:2018(E)

Note 6 to entry: In benchmarking, organizational and technical processes and combinations of both of them are

considered. A process within the meaning of benchmarking comprises a combination of one task with one plant/

one object (e.g. operate sewer network, treat wastewater, treat drinking water, provide domestic connection,

further train staff, purchase material).
[SOURCE: ISO 9000: 2015, 3.4.1, modified — Note 6 to entry replaced.]
4 Format for the examples provided in this document
4.1 General

Examples illustrating the application of the series of standards on the topic of activities relating

to drinking water and wastewater services produced by ISO/TC 224 or the parallel alternative

methodologies are provided in Annexes A to D. The contents of these examples are generally organized

under the headings given from 4.2 to 4.5. However, where full information on the practice or requirement

is not available at the time of publication of this document, summary information and a bibliographical

reference have been supplied.

Please note, the Annexes distinguish between methodologies and requirements established by

international organizations (Annex A), national regulatory bodies (Annex B), associations or groups of

utilities (Annex C), or individual utilities (Annex D).
4.2 Name of the organization

This section simply provides the identity of the organization offering its experience.

4.3 Background of the organization

This section briefly describes the organization and its history. The purpose is to provide a context

for understanding how the performance assessment methodologies of the series of standards on the

topic of activities relating to drinking water and wastewater services produced by ISO/TC 224, or

similar requirements or practices of assessment or benchmarking, were applied. It is also intended to

provide suitable information for a reader of this document to relate his or her own organization to the

organization described in the example.
4.4 Objective for making use of performance indicators

This section includes a description of the purpose of the organization when implementing the

requirement to develop and report the use of performance indicators whether conforming to the series

of standards on the topic of activities relating to drinking water and wastewater services produced by

ISO/TC 224 or similar requirements or practices of assessment or benchmarking.
4.5 Performance indicator methodology

This section provides a short description of each step followed in the methodology. Tables and figures

may be attached to illustrate the process.
4.6 Summary of the experience

This section includes a summary of any lessons learned by the organization from the production of

performance indicators whether conforming to the series of standards on the topic of activities

relating to drinking water and wastewater services produced by ISO/TC 224 or other similar practices

of assessment or benchmarking applied, as the case may be. It may also include the organization’s

conclusions on its future expanded application of the practice of assessment or benchmarking.

© ISO 2018 – All rights reserved 3
---------------------- Page: 10 ----------------------
ISO/TR 24514:2018(E)
Annex A
(informative)
International organizations
A.1 European Environment Agency — Purposes of performance indicators

European policies are increasingly focused on preserving the Earth's limited resources in a sustainable

manner, while minimizing impacts on the environment. This is included in the resource efficiency

and green economy agendas. In order to obtain knowledge on the actual pressure on the aquatic

environment from water abstractions and emission of pollutants, and for assessing urban water

management, we need to extend the knowledge base beyond compliance with current legislation. With

the aim of improving the European level knowledge base in urban water management in the resource

efficiency context, the European Environment Agency (EEA) hosted an expert meeting jointly organized

with leading water associations in Europe. This event framed the context and discussed topics related

to the exploitation of data already available with water utility associations and benchmarking networks

beyond what is currently reported via institutional frameworks for implementing legislation.

EU-level assessments of the resource efficiency or environmental performance of water utilities are

not currently as holistic as they could be. There is considerable reporting of environmental data

concerning water already in place, from the local to the EU level. However, these reporting obligations

are primarily concerned with the water quality parameters applicable to drinking water and treated

urban wastewater. The parameters are related to compliance with the EU directives pertaining to the

achievement of drinking water standards, urban wastewater collection and treatment requirements,

and receiving water quality objectives.
[5]

As outlined in the EEA report ‘Towards the efficient use of water resources in Europe’ economic

production cannot be sustained if it requires excessive water use and burdens natural resources. It

is thus essential that water uses and efficiencies are also considered in water management practices,

including: the actual pressures in the aquatic environment from water abstractions, the resulting

emissions of pollutants, and the energy consumption/recovery from managing the urban water cycle.

This report follows on from the discussions in the expert meeting on how the organizations and networks

involved in urban water management can share their knowledge bases to support environmental and

resource efficiency policies, and technical improvements. The availability of this knowledge base could

create a more comprehensive approach to assessing Europe's water resources and threats. It could also

enable a comparison of the environmental performance of different water utilities, monitor progress

over time, and aid the implementation of novel environmental technologies.

In its response to the European Citizen's Initiative “Right2Water”, the European Commission committed

to exploring the idea of benchmarking water quality and will cooperate with existing initiatives to

provide a wider set of benchmarks for water services. This significantly contributes to improving the

transparency and accountability of water service providers by giving citizens access to comparable data

on the key economic, technical and quality performance indicators of water operators. The information

provided in this report, although having a specific focus on environmental performance based on data

from voluntary benchmarking exercises, can be a useful contribution to this debate.

1) The following texts (but not the subheadings) are excerpts from the Executive Summary of the EEA Technical

[4]
Report No. 5/2014 .
4 © ISO 2018 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/TR 24514:2018(E)
A.2 International Benchmarking Network of Water and Wastewater Utilities
(IBNET) of the World Bank Group, Global Water Practice (www .ib -net .org)
A.2.1 Background of the organization

The World Bank is a development institution providing financial resources for development in more

than 150 countries and territories. The World Bank Water Global Practice is in charge of development,

implementation, supervision and assessment of projects related to water. The investment directly into

water and sanitation utilities constitutes roughly 60 % of Water Global Practice operations and is just

above one billion dollars a year for the last 10 years.

Performance and search for good utilities was always a key topic of interest of the World Bank activities

in the water sector. In order to do that, the World Bank established the International Benchmarking

Network of Water and Wastewater Utilities (IBNET) within the Bank’s Water and Sanitation Program.

Client governments develop standard indicators and maintain data to efficiently allocate financial

resources and develop sector improvement programs that address water and wastewater services for

all consumers, including the poor. IBNET provides a fact-based framework to measure the performance

of a service provider and set the standard for water sector performance assessment. IBNET provides

governments, utilities and the public with a clear and objective picture of water services and utility

performance.
A.2.2 Objective for makin
...