SIST-TS CEN/TS 17171:2018

Management of observed hydrometric data - Guidance

This European Standard gives recommendations for the management of observed hydrometric data, including raw data
and other data and statistics derived from these observations.

Mangement gemessener hydrometrischer Datensätze - Empfehlungen

Gestion des données hydrométriques observées - Recommandations

Vodenje opazovanih hidrometričnih podatkov - Navodilo

Ta evropski standard podaja priporočila za vodenje opazovanih hidrometričnih podatkov, vključno z neobdelanimi podatki ter drugimi podatki in statistiko, ki izhajajo iz teh opažanj.

General Information

• Status: Published
• Public Enquiry End Date: 31-Jan-2018
• Publication Date: 11-Sep-2018
• ICS: 07.060 - Geology. Meteorology. Hydrology
• ICS: 35.240.70 - IT applications in science
• Technical Committee: IMIN - Measurement instruments
• Current Stage: 6060 - National Implementation/Publication (Adopted Project)
• Start Date: 29-Aug-2018
• Due Date: 03-Nov-2018
• Completion Date: 12-Sep-2018
• Ref Project: CEN/TS 17171:2018 - Management of observed hydrometric data - Guidance

Standards Content (sample)

SLOVENSKI STANDARD
SIST-TS CEN/TS 17171:2018
01-oktober-2018
9RGHQMHRSD]RYDQLKKLGURPHWULþQLKSRGDWNRY1DYRGLOR
Management of observed hydrometric data - Guidance
Mangement gemessener hydrometrischer Datensätze - Empfehlungen
Gestion des données hydrométriques observées - Recommandations
Ta slovenski standard je istoveten z: CEN/TS 17171:2018
ICS:
07.060 Geologija. Meteorologija. Geology. Meteorology.
Hidrologija Hydrology
35.240.70 Uporabniške rešitve IT v IT applications in science
znanosti
SIST-TS CEN/TS 17171:2018 en,fr,de

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------
SIST-TS CEN/TS 17171:2018
---------------------- Page: 2 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171
TECHNICAL SPECIFICATION
SPÉCIFICATION TECHNIQUE
August 2018
TECHNISCHE SPEZIFIKATION
ICS 07.060; 35.240.70
English Version
Management of observed hydrometric data - Guidance

Gestion des données hydrométriques observées - Mangement gemessener hydrometrischer Datensätze -

Recommandations Empfehlungen

This Technical Specification (CEN/TS) was approved by CEN on 26 February 2018 for provisional application.

The period of validity of this CEN/TS is limited initially to three years. After two years the members of CEN will be requested to

submit their comments, particularly on the question whether the CEN/TS can be converted into a European Standard.

CEN members are required to announce the existence of this CEN/TS in the same way as for an EN and to make the CEN/TS

available promptly at national level in an appropriate form. It is permissible to keep conflicting national standards in force (in

parallel to the CEN/TS) until the final decision about the possible conversion of the CEN/TS into an EN is reached.

CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,

Finland, Former Yugoslav Republic of Macedonia, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania,

Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland,

Turkey and United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels

© 2018 CEN All rights of exploitation in any form and by any means reserved Ref. No. CEN/TS 17171:2018 E

worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171:2018 (E)
Contents Page

European foreword ....................................................................................................................................................... 5

Introduction .................................................................................................................................................................... 6

1 Scope .................................................................................................................................................................... 7

2 Normative references .................................................................................................................................... 7

3 Terms and definitions ................................................................................................................................... 7

4 Principles of hydrometric data management ....................................................................................... 8

4.1 The requirement for data management ................................................................................................. 8

4.2 The sequence of data management .......................................................................................................... 8

4.3 Duty of care ....................................................................................................................................................... 9

4.3.1 General ................................................................................................................................................................ 9

4.3.2 Legacy data ........................................................................................................................................................ 9

4.4 Data types and terminology ..................................................................................................................... 10

4.5 Time series data ........................................................................................................................................... 10

4.5.1 General ............................................................................................................................................................. 10

4.5.2 Time of observation .................................................................................................................................... 10

4.5.3 Derived data terminology ......................................................................................................................... 12

4.5.4 Resolution of data storage ........................................................................................................................ 12

4.5.5 Uncertainty ..................................................................................................................................................... 13

4.6 Metadata and descriptive material ........................................................................................................ 13

4.7 Maximizing data utility .............................................................................................................................. 13

4.7.1 Availability of metadata ............................................................................................................................. 13

4.7.2 Data quality control ..................................................................................................................................... 14

4.7.3 Data gap infilling .......................................................................................................................................... 15

4.7.4 Regular data reviews .................................................................................................................................. 15

4.8 Data storage considerations .................................................................................................................... 15

4.8.1 Data storage ................................................................................................................................................... 15

4.8.2 Changes to data ............................................................................................................................................. 15

4.8.3 Data disposal .................................................................................................................................................. 15

4.8.4 Electronic data storage system considerations ................................................................................ 16

4.8.5 Tapes, charts and other data in paper format ................................................................................... 16

4.8.6 Data security .................................................................................................................................................. 16

4.8.7 Data rescue ..................................................................................................................................................... 17

4.9 Archiving hydrometric data: national collections............................................................................ 17

4.10 Electronic transfer of data ........................................................................................................................ 17

5 Metadata .......................................................................................................................................................... 18

5.1 General ............................................................................................................................................................. 18

5.2 Monitoring station/site/point metadata ............................................................................................. 18

5.2.1 General ............................................................................................................................................................. 18

5.2.2 Station identifier .......................................................................................................................................... 19

5.2.3 Station description ...................................................................................................................................... 19

5.2.4 Geographical location ................................................................................................................................. 20

5.2.5 Operating period .......................................................................................................................................... 20

5.3 Observation metadata ................................................................................................................................ 20

5.3.1 General ............................................................................................................................................................. 20

5.3.2 Data set metadata ........................................................................................................................................ 20

---------------------- Page: 4 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171:2018 (E)

5.3.3 Data flags and comments ........................................................................................................................... 20

6 Precipitation data ......................................................................................................................................... 21

6.1 Raw data ........................................................................................................................................................... 21

6.1.1 Data to be recorded...................................................................................................................................... 21

6.1.2 Additional raw data for manual observations ................................................................................... 21

6.2 Resolution of data storage ......................................................................................................................... 21

6.3 Data processing and formatting .............................................................................................................. 21

6.3.1 General ............................................................................................................................................................. 21

6.4 Quality control of precipitation data ..................................................................................................... 21

6.4.1 General ............................................................................................................................................................. 21

6.4.2 Levels of quality control of precipitation data ................................................................................... 21

6.4.3 Methods for quality control of precipitation data ............................................................................ 22

6.4.4 Quality control of frozen precipitation ................................................................................................. 22

6.5 Precipitation specific metadata ............................................................................................................... 22

7 Water level data ............................................................................................................................................ 23

7.1 Raw data ........................................................................................................................................................... 23

7.1.1 Data to be recorded...................................................................................................................................... 23

7.1.2 Types of level data ........................................................................................................................................ 23

7.1.3 Regular measurements and event data ................................................................................................ 23

7.1.4 Additional raw data for continuous measurements ........................................................................ 24

7.1.5 Groundwater levels and water levels in reservoirs ......................................................................... 25

7.2 Quality control of water level data ......................................................................................................... 25

7.2.1 General ............................................................................................................................................................. 25

7.2.2 Levels of quality control of water level data ....................................................................................... 25

7.2.3 Methods for quality control of water level data ................................................................................ 25

7.2.4 Data estimation and infilling .................................................................................................................... 26

7.3 Water level specific metadata .................................................................................................................. 27

8 Velocity and discharge data ...................................................................................................................... 27

8.1 Raw data ........................................................................................................................................................... 27

8.1.1 Data to be recorded...................................................................................................................................... 27

8.1.2 Additional raw data for irregular/spot measurements ................................................................. 28

8.1.3 Additional raw data for regular or fixed site measurements ....................................................... 28

8.2 Data processing ............................................................................................................................................. 29

8.2.1 General ............................................................................................................................................................. 29

8.2.2 Discharge data derived from water levels ........................................................................................... 29

8.2.3 Discharge data from time of flight ultrasonic systems ................................................................... 30

8.2.4 Discharge data from velocity index method ....................................................................................... 31

8.2.5 Discharge data from electromagnetic systems .................................................................................. 31

8.2.6 Combining discharge data from multiple methods .......................................................................... 31

8.3 Resolution of data storage ......................................................................................................................... 31

8.4 Derived data ................................................................................................................................................... 31

8.4.1 Daily mean flow ............................................................................................................................................. 31

8.4.2 Peak discharges ............................................................................................................................................. 32

8.5 Naturalization of discharge data ............................................................................................................. 32

8.6 Quality control of discharge data ............................................................................................................ 32

8.6.1 Levels of quality control of discharge data .......................................................................................... 32

8.6.2 Methods for quality control of discharge data ................................................................................... 32

8.6.3 Review of extreme flows ............................................................................................................................ 33

8.6.4 Data estimation and infilling .................................................................................................................... 34

8.7 Discharge specific metadata ..................................................................................................................... 36

8.7.1 Additional monitoring station/site/point metadata ....................................................................... 36

8.7.2 Additional observation metadata ........................................................................................................... 36

---------------------- Page: 5 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171:2018 (E)

9 Volume data ................................................................................................................................................... 37

9.1 General ............................................................................................................................................................. 37

9.2 Level/storage relationships ..................................................................................................................... 37

9.3 Frequency of sampling and processing ................................................................................................ 37

9.4 Resolution of data storage ........................................................................................................................ 37

9.5 Volume data specific metadata ............................................................................................................... 37

Bibliography ................................................................................................................................................................. 38

---------------------- Page: 6 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171:2018 (E)
European foreword

This document (CEN/TS 17171:2018) has been prepared by Technical Committee CEN/TC 318

“Hydrometry”, the secretariat of which is held by BSI.

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights. CEN shall not be held responsible for identifying any or all such patent rights.

According to the CEN/CENELEC Internal Regulations, the national standards organisations of the

following countries are bound to announce this Technical Specification: Austria, Belgium, Bulgaria,

Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Former Yugoslav Republic of Macedonia,

France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta,

Netherlands, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland,

Turkey and the United Kingdom.
---------------------- Page: 7 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171:2018 (E)
Introduction

Water management decisions and policies ought to be based upon quantitative knowledge of the

hydrological system. Commonly, such knowledge results from observational hydrometric data, the

collection of which is the subject of other standards, e.g. EN ISO 18365. The subsequent management of

such hydrometric data provides the linkage between field measurement and the eventual use of

processed data to address a wide range of strategic and operational water management applications. As

both the demand for and complexity of hydrometric data increase, it is important that the procedures

and processes used to manage these data are standardized to allow greater integration of data and

ensure their protection for future use.

The availability of high-quality observational data are vital to developing an understanding of the

hydrological cycle. Optimizing data management systems helps ensure that the maximum benefits are

achieved from those resources invested in hydrometric monitoring. Effective standardized procedures

for data transmission, manipulation, quality control, expression of uncertainty and storage are vitally

important and their use should be promoted throughout hydrometric observation networks.

Those responsible for hydrometric data management are encouraged by this Technical Specification to

adopt the ethos of professional stewardship and to remember their role as guardians of an important

national, and sometimes international, resource.

This Technical Specification is designed for use by all organisations and individuals collecting,

processing or storing hydrometric data. Some of the clauses contained in standard are only applicable

for those maintaining national or regional collections of hydrometric data (for example, 4.9). However

most recommendations are widely applicable to all users, including organisations, companies or

individuals involved in: hydropower production, water supply, environmental protection, scientific

research or flood risk management.

This Technical Specification is concerned with general aspects of good practice in data management.

Techniques for managing data are recommended, covering metadata collection, data storage and

quality control. This Technical Specification assumes that the raw data have been collected and

transmitted from the field in line with other European Standards for hydrometry, so this Technical

Specification concentrates on the subsequent processing and management of these hydrometric data.

---------------------- Page: 8 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171:2018 (E)
1 Scope

This document gives recommendations for the management of observed hydrometric data, including

raw data and other data as well as statistics derived from these observations. Although the principles of

data management can be applied to all hydrometric observations, particular focus is placed on

measurements of precipitation, water level (including stage), volume and discharge in open channels.

NOTE The range of sites where water levels, and sometimes flow, are measured includes lakes, reservoirs,

rivers, canals, tidal waters, sewers, wells, and boreholes.

The document covers metadata associated with hydrometric data, including recommendations for the

production and management of descriptive, analytical and statistical material relating to sites where

and measuring techniques, by which hydrometric data are collected. The recommendations of this

document can be applied to some forms of data directly derived from observational records (for

example, summary time series of monthly mean river flows). While not primarily designed for the

management of data resulting from more complex numerical models or spatially aggregated data sets

(for example, remotely-sensed data), many of the recommendations are applicable for such types of

data.

This document does not cover the field collection of data or its transmission, but focuses on the

management of data once they have been received in a hydrometric information management system.

2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

EN ISO 772, Hydrometry — Vocabulary and symbols (ISO 772)
3 Terms and definitions

For the purposes of this document, the terms and definitions given in EN ISO 772 and the following

apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— IEC Electropedia: available at http://www.electropedia.org/
— ISO Online browsing platform: available at http://www.iso.org/obp
3.1
data flag
indicator relating to the quality and characteristics of an observation
3.2
derived data
information calculated, or deduced, from raw data (3.5)
3.3
precipitation
water or ice derived from the atmosphere and deposited at ground level

Note 1 to entry: Measured in terms of the depth in millimetres (mm) of its liquid equivalent.

---------------------- Page: 9 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171:2018 (E)
3.4
quality control

process of confirming that the data held are a reliable representation of the variable being measured or

derived
3.5
raw data
data resulting directly from the measurement of variables
3.6
UTC
Coordinated Universal Time
4 Principles of hydrometric data management
4.1 The requirement for data management

Hydrometric data management systems should be designed such that the utility of data to end-users at

all levels is a central consideration. The value of data can be significantly affected by the treatment of

data after collection, so data management systems should be designed to maintain (and where possible

improve) the quality and continuity of records.

As water management decisions and policies are based upon quantitative knowledge of the

hydrological system both now and in the past, the protection of data for future use should be a central

requirement of any data management system. As such, the design of such systems should consider, as

far as possible, both current and future user requirements.
4.2 The sequence of data management

The management of hydrometric data is a continuous sequence (see Figure 1), which starts in the field

at the point of collection and ends if and when data are destroyed. The design of any hydrometric data

management system should consider all stages in this sequence.

The management of data at the point of collection and their subsequent transmission vary depending

on the hydrological variable being recorded, hydrometric methods employed and instrumentation used.

Any data management system used should ensure that adequate data and metadata are recorded at the

point of collection to allow future management and use.

Some data management processes, such as quality control, may occur at more than one stage in the

sequence and be completed by more than one organization. In designing, operating and reviewing data

management systems, all stages in the acquisition, processing and dissemination of data should be

considered.
---------------------- Page: 10 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171:2018 (E)
Figure 1 — Data management sequence
4.3 Duty of care
4.3.1 General

Data often have significant value outside the organization that collected them, and data management

systems should be designed, where possible, to capture all information required for future, sometimes

unforeseen, uses.

Responsibility for the operation of hydrometric monitoring systems may be transferred (for example, to

another organization or other personnel), so data management systems should be capable of preserving

all data and associated metadata required to inform new operators.
4.3.2 Legacy data

The duty of care to manage hydrometric data should be taken to cover any such data currently held by

an organization. In addition to data collected under the direct control of the organization, the duty of

care extends to data collected by predecessors and inherited.

An assessment should be made of all legacy data upon inheritance (or discovery) to determine their

contents and current state. An assessment should be made of their current, and likely future, utility to

the organization and other external parties. While the management (and improvement) of legacy data

can be demanding, the value of the resulting information can be high.

NOTE Long historical records, even those of lower hydrometric quality, can be particularly valuable in

providing an indication of previous hydrological conditions and often improve the value of current data through

extending record lengths.
---------------------- Page: 11 ----------------------
SIST-TS CEN/TS 17171:2018
CEN/TS 17171:2018 (E)

All legacy data should be actively managed. Where possible, such data should be brought into current

data management systems and managed in accordance with 4.5 to 4.10 and Clause 5 to Clause 9.

Where legacy data are not integrated into current data management systems, they should be secured in

archives maintained either by the organization which collected/inherited them or by independent

bodies (for example, national archives, libraries).
Where the quality of legacy data (including
...