SIST-TP CEN/TR 15449-5:2016
(Main)Geographic information - Spatial data infrastructures - Part 5: Validation and testing
Geographic information - Spatial data infrastructures - Part 5: Validation and testing
This part of the Technical Report provides guidance for validation and testing of data, metadata and services, as the main Spatial Data Infrastructure (SDI) components defined in other parts of CEN/TR 15449.
The guidance is given by means of examples of the validation and testing process required to assure conformance with the requirements existing in the relevant standards and guidelines.
Geoinformationen - Geodateninfrastrukturen - Teil 5:Validierung und Tests
Information géographique - Infrastructure de données spatiales - Partie 5 : Validation et essais
Geografske informacije - Infrastruktura za prostorske podatke - 5. del: Validacija in preskušanje
Ta del tehničnega poročila podaja napotke za validacijo in preskušanje podatkov, metapodatkov in storitev kot glavnih komponent infrastrukture za prostorske podatke, določene v drugih delih standarda CEN/TR 15449.
Napotki so podani s primeri validacijskih in preskusnih postopkov, potrebnih za zagotavljanje skladnosti z obstoječimi zahtevami v ustreznih standardih in smernicah.
General Information
Standards Content (Sample)
SLOVENSKI STANDARD
SIST-TP CEN/TR 15449-5:2016
01-december-2016
Geografske informacije - Infrastruktura za prostorske podatke - 5. del: Validacija in
preskušanje
Geographic information - Spatial data infrastructures - Part 5: Validation and testing
Geoinformationen - Geodateninfrastrukturen - Teil 5:Validierung und Tests
Information géographique - Infrastructure de données spatiales - Partie 5 : Validation et
essais
Ta slovenski standard je istoveten z: CEN/TR 15449-5:2015
ICS:
07.040 Astronomija. Geodezija. Astronomy. Geodesy.
Geografija Geography
35.240.70 Uporabniške rešitve IT v IT applications in science
znanosti
SIST-TP CEN/TR 15449-5:2016 en,fr
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST-TP CEN/TR 15449-5:2016
---------------------- Page: 2 ----------------------
SIST-TP CEN/TR 15449-5:2016
TECHNICAL REPORT
CEN/TR 15449-5
RAPPORT TECHNIQUE
TECHNISCHER BERICHT
April 2015
ICS 07.040; 35.240.70
English Version
Geographic information - Spatial data infrastructures - Part 5:
Validation and testing
Information géographique - Infrastructure de données
spatiales - Partie 5 : Validation et essais
This Technical Report was approved by CEN on 21 June 2014. It has been drawn up by the Technical Committee CEN/TC 287.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, Former Yugoslav Republic of Macedonia, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania,
Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and United
Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Avenue Marnix 17, B-1000 Brussels
© 2015 CEN All rights of exploitation in any form and by any means reserved Ref. No. CEN/TR 15449-5:2015 E
worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
Contents Page
Foreword . 4
Introduction . 5
1 Scope . 7
2 Normative references . 7
3 Terms, definitions and abbreviations . 7
3.1 Terms and definitions . 7
3.2 Abbreviations . 7
4 Conformance and testing framework . 8
4.1 General . 8
4.2 Structure of the document . 9
5 Metadata validation . 10
5.1 General . 10
5.2 Validation against XML Schema . 10
5.3 Rule-based validation with Schematron . 11
6 Data validation . 11
6.1 General . 11
6.2 Validation against XML Schema . 12
6.3 Rule-based validation with Schematron . 12
6.4 INSPIRE Abstract Test Suite for Annex I, II and III data themes . 12
7 Service validation . 14
7.1 General . 14
7.2 View services . 15
7.2.1 General . 15
7.2.2 WMS . 15
7.2.3 WMTS . 17
7.3 Download services . 17
7.4 Quality of services . 19
Annex A (informative) Examples of validation processes . 21
A.1 Introduction . 21
A.2 Example of web map service validation . 21
A.3 Example of web feature service validation . 23
Annex B (informative) Example validation tools . 45
B.1 Introduction . 45
B.2 Data and metadata validation with Oxygen . 45
B.3 Validation with XMLSpy . 53
B.4 INSPIRE Validator . 56
B.4.1 Introduction . 56
B.4.2 Supported standards and technologies used . 57
B.4.3 Resource validation process . 58
B.5 OGC CITE . 60
2
---------------------- Page: 4 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
B.6 eENVplus validation service . 60
Annex C (informative) Validation of specification encoding . 61
C.1 Introduction . 61
C.2 Validate Schema . 62
C.3 Check transposition of specification . 62
C.4 Check validatability . 62
C.5 Example of validation of specification encoding . 63
C.5.1 General . 63
C.5.2 Validate schema . 63
C.5.3 Check transposition of specification . 64
C.5.4 Check validatability . 69
C.5.5 Example of test suite for metadata rule-based validation . 75
C.5.6 Example of result of validation of the metadata specification encoding . 81
Bibliography . 84
3
---------------------- Page: 5 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
Foreword
This document (CEN/TR 15449-5:2015) has been prepared by Technical Committee CEN/TC 287
“Geographic information”, the secretariat of which is held by BSI.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. CEN [and/or CENELEC] shall not be held responsible for identifying any or all such patent rights.
4
---------------------- Page: 6 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
Introduction
Spatial data infrastructure (SDI) is a general term for the computerised environment for handling data that
relates to a position on or near the surface of the earth. It may be defined in a range of ways, in different
circumstances, from the local up to the global level.
This Technical Report focuses on the technical aspects of SDIs, thereby limiting the term SDI to mean an
implementation neutral technological infrastructure for geospatial data and services, based upon standards
and specifications. It does not consider an SDI as a carefully designed and dedicated information system;
rather, it is viewed as a collaborative framework of disparate information systems that contain resources that
stakeholders desire to share. The common denominator of SDI resources, which can be data or services, is
their spatial nature. It is understood that the framework is in constant evolution, and that therefore the
requirements for standards and specifications supporting SDI implementations evolve continuously.
SDIs are becoming more and more linked and integrated with systems developed in the context of e-
Government. Important drivers for this evolution are the Digital Agenda for Europe, and related policies (see
Part 1 of this Technical Report). By sharing emerging requirements at an early stage with the standardization
bodies, users of SDIs can help influence the revision of existing or the conception of new standards.
The users of an SDI are considered to be those individuals or organisations that, in the context of their
business processes, need to share and access geo-resources in a meaningful and sustainable way. Based
on platform- and vendor-neutral standards and specifications, an SDI aims at assisting organisations and
individuals in publishing, finding, delivering, and eventually, using geographic information and services over
the internet across borders of information communities in a more cost-effective manner.
Considering the complexity of the subject and the need to capture and formalize different conceptual and
modelling views, CEN/TR 15449 comprises multiple parts. The other parts, published previously, are:
— Part 1: Reference model: This provides a general context model for the other Parts, applying general IT
architecture standards;
— Part 2: Best Practice: This provides best practices guidance for implementing SDI, through the
evaluation of the projects in the frame of the European Union funding programmes.
— Part 3: Data centric view: This addresses the data, which includes application schemas and metadata.
— Part 4: Service centric view: This addresses the concepts of service specifications, the methodology for
developing service specifications through the application of the relevant International Standards, and the
content of such service specifications.
Further parts may be created in the future.
One of the major challenges in the implementation of an SDI is to ensure the conformity of its components
with the requirements specified in the relevant standards and guidelines. This applies to the data
specifications, the derived schemas, the spatial data sets and metadata and the network services. Only if
conformance is ensured, can true interoperability of the harmonized metadata and data by means of network
services be guaranteed. This Part (5) provides guidance for validation and testing of data, metadata and
services, as the main Spatial Data Infrastructure (SDI) components defined in other parts of this Technical
Report.
The intended readers of this document belong to a range of categories:
— technicians engaged in validation and testing of SDI components, who need to find reference material to
use within the validation and testing processes;
5
---------------------- Page: 7 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
— managers who need to assess the complexity of the processes of validation and testing of SDI
components;
— data, metadata and network service providers, aiming at self-validating their own data sets, metadata or
services, who wish to implement validation and testing processes within their organizations;
— designers of data and metadata models, who need to validate their schemas;
— data users interested in acquiring a deeper knowledge about validation and testing processes of SDI
components.
Because the operation of SDIs in Europe is governed by the INSPIRE Directive EC/02/2007 and its relevant
legal and technical documents, this report aims at considering INSPIRE as the reference context, even
though some concepts, wherever possible, are generalized beyond INSPIRE.
Because Validation and Testing is a subject in continuous and rapid evolution, and many different
implementations could exist based on different technical solutions, the topics covered in this report, as well
as the relevant examples provided in the Annexes, cannot be considered complete nor exhaustively
presented. In order to keep updated on the subject, the readers of this report are recommended to follow the
activities and outcomes of the Working Group 5 “Validation and Conformity Testing”, set-up within the
INSPIRE MIG (Maintenance and Implementation Group) and MIF (Maintenance and Implementation
)
1
Framework) .
1) http://inspire.ec.europa.eu/index.cfm/pageid/5160
6
---------------------- Page: 8 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
1 Scope
This part of the Technical Report provides guidance for validation and testing of data, metadata and
services, as the main Spatial Data Infrastructure (SDI) components defined in other parts of the
CEN/TR 15449.
The guidance is given by means of examples of the validation and testing process required to ensure
conformance with the requirements existing in the relevant standards and guidelines.
The National validation and testing context is out of scope of this report.
2 Normative references
The following documents, in whole or in part, are normatively referenced in this document and are
indispensable for its application. For dated references, only the edition cited applies. For undated references,
the latest edition of the referenced document (including any amendments) applies.
EN ISO 19105:2005, Geographic information - Conformance and testing (ISO 19105:2000)
3 Terms, definitions and abbreviations
3.1 Terms and definitions
For the purposes of this document, the terms and definitions of EN ISO 19105:2005 shall apply.
3.2 Abbreviations
ATS: Abstract Test Suite
CRS: Coordinate Reference System
DS: Data Specifications
ESDIN: European Spatial Data Infrastructure with a Best Practice Network — a project supported by
eContent+ programme
ETF: ESDIN Testing Framework
ETS: Executable Test Suite
FE: Filter Encoding
GI: Geographic Information
GML: Geography Markup Language
ISO: International Organization for Standardization
IR: Implementing Rule
MD: Metadata
NA: Not Applicable
NS: Network Services
OGC: Open Geospatial Consortium
PS: Protected Sites
SLA: Service Level Agreement
SOAP: Simple Object Access Protocol
XML: eXtensible Markup Language
7
---------------------- Page: 9 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
xsd: XML Schema Definition
XSLT: Extensible Stylesheet Language Transformations
W3C: World Wide Web Consortium
WFS: Web Feature Service
WMS: Web Map Service
WMTS: Web Map Tiles Services
4 Conformance and testing framework
4.1 General
The implementation of rule-based validation requires the translation of the sometimes textually defined rules
into a machine readable format, in other words you need a formal rules language. The rules language within
a testing environment offers a mechanism for recording conceptual level data management logic. The
required characteristics are given in Table 1.
Table 1 — Characteristics of a rules language for data quality evaluation
Language is naturalistic and easy to learn and
Intuitive
use (by data experts).
Concise grammar for manageability and ease
Compact
of comprehension.
Domain constraints can be expressed
Unambiguous
mathematically.
Reports formal metrics to assert the evaluated
Quantitative level of compliance to data quality rules
(measures).
Logical separation between application domain
and physical implementation models. Allows
rules to be expressed in terms of application
Portable
schema, which is easier for data experts to
relate to and is likely to have a longer life-time
than specific technology implementations.
The language and environment in which it is
Web Enabled used shall be compatible with distributed data,
and web-based interfaces.
Continuous improvement cycles require rules
to evolve with time. It is also unusual to be able
Extensible to capture all data requirements up front and
over time these requirements are likely to be
extended or subject to change.
There are several rules languages which can be used for this purpose. The choice of which to use often
depends on the chosen test engine within the testing environment. Since the data tests are usually executed
on a XML file, two main options are considered:
• a XML database (e.g. BaseX) and a XML based query language (e.g. XPath/XQuery) are used to run
the translated test criteria and perform comprehensive analyses. To develop or extend existing test
criteria an interactive graphical user interface is useful.
• the Schematron language is used to translate the necessary formal constraints.
8
---------------------- Page: 10 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
Testing of geometrical criteria needs additional software components (e.g. JTS – Java Topology Suite, Vivid
Solutions).
In any case it is essential to know which criteria have been tested and which criteria shall be successfully
tested to consider the data set as conformant. Therefore a central repository might be useful to control the
content, the versions and changes of the test criteria.
The following sections and examples of this document relevant to data and metadata validation and testing
refer to the Schematron option above mentioned. More details about schematron can be found at:
• ISO/IEC 19757-3:2006, Information technology - Document Schema Definition Languages (DSDL) –
Part 3: Rule-based validation – Schematron
• www.schematron.com
4.2 Structure of the document
In the following subsections a step by step process is described covering the validation process for:
• data/metadata encoding (according to the specification),
• network services.
For the validation of data/metadata encoding, the following steps are covered:
• Schema Validation: validation of the metadata or data documents against the corresponding XML
schema;
• Schematron Validation: the validation of non-syntactic requirements using semantic rules defined in
Schematron.
Validation approaches alternative to implementations based on XML schema and/or Schematron are
provided as well.
For the validation of network services (view, download, coverage), the following types of services are
covered:
• Functional requirements,
• Quality (performance) aspects.
Examples of validation processes are provided in Annex A, while examples of the use of validation tools are
provided in Annex B.
In Annex C a validation process is provided for metadata and data specification encoding, covering the
following steps:
• Schema Validation: checks that the schema that has been derived from the data model is a valid
)
2
schema according to the W3C XML Schema Recommendation ;
2) The phrase ‘schema validation’ is generally used to describe validating an XML instance against its schema (from the
W3C http://www.w3.org/TR/xmlschema11-1/) In that document, it is clearly distinct from assuring that the schema is itself
valid – that is described in a section headed ‘Errors in Schema Construction and Structure’. In
http://www.w3.org/TR/xmlschema11-1/, any software that goes on to check an XML instance against a schema is
allowed to declare an error if the schema itself is invalid.
9
---------------------- Page: 11 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
• Validation of Transposition: checks that all elements from the Data Specification have been properly
transposed to XML Schema;
• Validatability: checks that all elements from the Data Specification have been transposed to the XML
Schema in a manner that allows for correct semantic interpretation and validation.
The validation process described in Annex C may be useful for those involved in the development of new or
extension of existing Data Specifications.
5 Metadata validation
5.1 General
The metadata validation process encompasses several steps. The validation process for metadata validation
is shown in Figure 1. As a first step, the metadata shall be validated against the metadata schema provided.
As a second step, the metadata shall be validated against formal constraints from the metadata specification
using Schematron. In Figure 1, relevant to an INSPIRE metadata validation, this second step has been split
into two sub-steps, the first relevant to the validation with respect to the constraints related to the core
metadata common to all the INSPIRE themes and then with respect to the theme-specific constraints.
Figure 1 — Metadata validation process
If an error is found and rectified, all validation steps shall be repeated, as the rectification process may have
introduced new errors.
5.2 Validation against XML Schema
Before using an XML document, this shall first be validated against the relevant schema using an XML
validation tool. Any errors identified during schema validation shall be rectified before the data can be used.
The tester shall ensure that the XML document refers to the proper schema; just because a data file is valid
XML does not mean that it is valid according to a specific schema.
10
---------------------- Page: 12 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
As many tools used for the creation and validation of xml metadata may use different approaches, it is
advisable to validate the metadata generated using different validation tools. Thus, further errors in the
metadata can be found and rectified, reducing problems encountered when users attempt to work with faulty
metadata.
5.3 Rule-based validation with Schematron
Once the metadata file has been shown to be formally valid against the schema provided, compliance to
further constraints arising from the data specification shall be checked. This should be done using
Schematron validation; a Schematron rule file shall be provided for each metadata specification. All
metadata provided in the form of XML documents shall pass validation according to the corresponding
Schematron file before it is considered valid. While in most cases the Schematron rules are configured so
that they only display message texts for errors, some also provide informative texts.
6 Data validation
6.1 General
The data validation process encompasses several steps, as shown in Figure 2, relevant to an INSPIRE data
validation. As a first step, the data should be validated against the data schema provided. As a second step,
the data should be validated against formal constraints from the data specification using Schematron, first
with respect to the constraints related to the GML encoding and then with respect to the data-specific
constraints.
Figure 2 — Data validation process
If an error is found and rectified, all validation steps shall be repeated, as the rectification process may have
introduced new errors.
11
---------------------- Page: 13 ----------------------
SIST-TP CEN/TR 15449-5:2016
CEN/TR 15449-5:2015 (E)
6.2 Validation against XML Schema
Before using an XML document, this shall first be validated against the relevant schema for the data file
using an XML validation tool. Any errors identified during schema validation shall be rectified before the data
can be used. The tester shall ensure that the XML document refers to the proper schema; just because a
data file is valid XML does not mean that it is valid according to a specific schema.
As many tools used for the creation and validation of xml data may use different approaches, it is advisable
to validate the data generated using different validation tools. Thus, further errors in the data may be found
and rectified, reducing problems encountered when users attempt to work with faulty data.
6.3 Rule-based validation with Schematron
Once the data file has been shown to be formally valid against the schema provided, compliance to further
constraints arising from the data specification shall be checked. This should be done using Schematron
validation; a Schematron rule file shall be provided for each metadata and data specification. All data
provided in the form of XML documents shall pass validation according to the corresponding Schematron file
before it is considered valid. While in most cases the Schematron rules are configured so that they only
display message texts for errors, some also provide informative texts.
6.4 INSPIRE Abstract Test Suite for Annex I, II and III data themes
The INSPIRE Data Specifications - Technical Guidelines for the Annex I, II and III data themes have an
annex, “Annex A - Abstract Test Suite”, which contains a set of tests to help the conformance testing
process.
The new and updated structure of the Data Specifications contains two different types of requirements:
• the requirements present in the Regulation (Implementing Rules - IR) on interoperability of spatial data
sets and services (IR Requirements)
• the requirements for a specific technical solution proposed in the Technical Guidance for an IR
requirement (TG requirements).
For this reason the Abstract Test Suite (ATS) is composed of two parts:
• Part 1 (normative) - Conformity with Commission Regulation No 1089/2010 (IR Requirements)
• Part 2 (informative) - Conformity with the technical guideline (TG) Requirements
In each part, the requirements to be tested are grouped in several conformance classes and each of these
classes covers a specific aspect.
The ATS contains a detailed list of abstract tests, but for their physical implementation an Executable Test
Suite (ETS) is required. The ETS shall contain operative instructions on how to execute the relevant abstract
test.
Some tests, such as in the Application schema conformance class, may be automated by using xml schema
validation tools. Conver
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.