ISO 19105:2000
(Main)Geographic information — Conformance and testing
Geographic information — Conformance and testing
This International Standard specifies the framework, concepts and methodology for testing and criteria to be achieved to claim conformance to the family of ISO geographic information standards. It provides a framework for specifying abstract test suites (ATS) and for defining the procedures to be followed during conformance testing. Conformance may be claimed for data or software products or services or by specifications including any profile or functional standard. Standardization of test methods and criteria for conformance to geographic information standards will allow verification of conformance to those standards. Verifiable conformance is important to geographic information users, in order to achieve data transfer and sharing. This International Standard is applicable to all the phases of conformance and testing. These phases are characterized by the following major activities: a) the definition of ATS for conformance to the ISO geographic information standards; b) the definition of test methods for conformance to the ISO geographic information standards; c) the conformance assessment process carried out by a testing laboratory for a client, culminating in the production of a conformance test report. This International Standard specifies the requirements for, and gives guidance on, the procedures to be followed in conformance testing for the ISO geographic information standards. It includes only such information as is necessary to meet the following objectives: 1) to achieve confidence in the tests as a measure of conformance; 2) to achieve comparability between the results of corresponding tests applied in different places at different times; 3) to facilitate communication between the parties responsible for the activities described in 1) and 2). This International Standard provides a framework for certification (an administrative procedure which may follow conformance testing) in informative annex B. The following topics are outside the scope of this International Standard. a) The description of requirements for procurement and contracts. b) Testing by means of test methods which are specific to particular applications or systems. c) Acceptance testing, performance testing and robustness testing. The framework established by this International Standard includes the concept of executable test suites (ETS). These, by their very nature, cannot be standardized; consequently, standardization of ETS is outside the scope of this International Standard.
Information géographique — Conformité et essais
Geografske informacije - Ustreznost in preskušanje
General Information
Relations
Standards Content (Sample)
INTERNATIONAL ISO
STANDARD 19105
First edition
2000-12-15
Geographic information — Conformance
and testing
Information géographique — Conformité et essais
Reference number
ISO 19105:2000(E)
©
ISO 2000
---------------------- Page: 1 ----------------------
ISO 19105:2000(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but shall not
be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading this
file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat accepts no liability in this
area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation parameters
were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event
that a problem relating to it is found, please inform the Central Secretariat at the address given below.
© ISO 2000
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic
or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or ISO's member body
in the country of the requester.
ISO copyright office
Case postale 56 � CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.ch
Web www.iso.ch
Printed in Switzerland
ii © ISO 2000 – All rights reserved
---------------------- Page: 2 ----------------------
ISO 19105:2000(E)
Contents Page
Foreword.iv
Introduction.v
1 Scope .1
2 Conformance.2
2.1 Conformance requirements.2
2.2 Abstract test suite.2
3 Terms and definitions .2
4 Abbreviated terms .5
5 General framework of conformance .5
5.1 Introduction.5
5.2 Conformance clause .5
5.3 Conformance requirements.5
5.4 Implementation conformance statement .6
5.5 A conforming implementation.6
6 Conformance testing methodology .6
6.1 Introduction.6
6.2 Types of conformance tests.6
6.3 Implementation Extra Information for Testing.8
6.4 Conformance assessment.8
6.5 Intrinsic properties of the conformance assessment process .10
7 Test methods.11
7.1 Introduction.11
7.2 Approaches to conformance testing .11
7.3 Areas of geographic information for conformance testing.12
8 Abstract test suites and executable test suites .12
8.1 Introduction.12
8.2 Test purposes .13
8.3 Abstract test cases.13
8.4 Executable test cases .13
8.5 Relationship between abstract and executable test cases .13
Annex A (normative) Conformance clauses .14
Annex B (informative) Supporting organizations .19
Bibliography.21
© ISO 2000 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO 19105:2000(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (ISO
member bodies). The work of preparing International Standards is normally carried out through ISO technical
committees. Each member body interested in a subject for which a technical committee has been established has
the right to be represented on that committee. International organizations, governmental and non-governmental, in
liaison with ISO, also take part in the work. ISO collaborates closely with the International Electrotechnical
Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 3.
Draft International Standards adopted by the technical committees are circulated to the member bodies for voting.
Publication as an International Standard requires approval by at least 75 % of the member bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this International Standard may be the subject of
patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
International Standard ISO 19105 was prepared by Technical Committee ISO/TC 211, Geographic
information/Geomatics.
Annex A forms a normative part of this International Standard. Annex B is for information only.
iv © ISO 2000 – All rights reserved
---------------------- Page: 4 ----------------------
ISO 19105:2000(E)
Introduction
The scope of ISO/TC 211 is standardization in the field of digital geographic information. This work aims at
establishing a structured set of International Standards for information concerning objects or phenomena that are
directly or indirectly associated with a location relative to the Earth. These International Standards may specify, for
geographic information, methods, tools and services for data management (including definition and description),
acquiring, processing, analysing, accessing, presenting and transferring such data in digital/electronic form
between different users, systems and locations. The work will be linked to appropriate International Standards for
information technology and data, where possible, and provide a framework for the development of sector-specific
applications using geographic data.
This International Standard provides the framework, concepts, and methodology for testing and the criteria to be
achieved to claim conformance to this family of International Standards. This International Standard is based in part
on ISO 9646-1 which describes conformance and testing in Open Systems Interconnection (OSI), ISO 10303-31
which describes conformance and testing in industrial automation systems and integration, and ISO 10641 which
describes conformance and testing for computer graphics and image processing. While the framework of
conformance testing described in these three International Standards is used in this International Standard, some
concepts have been modified for use in this particular domain.
The objective of standardization in the field of digital geographic information cannot be completely achieved unless
data and systems can be tested to determine whether they conform to the relevant geographic information
standards. Conformance testing is the testing of a candidate product for the existence of specific characteristics
required by an International Standard in order to determine the extent to which that product is a conforming
implementation. It involves testing the capabilities of an implementation against both the conformance requirements
in the relevant International Standard(s) and the statement of the implementation's capabilities.
A framework of an abstract test suite (ATS) is standardized for relevant standards in ISO/TC 211. The
standardization of ATS requires international definition and acceptance of a common test methodology, together
with appropriate test methods and procedures. The purpose of this International Standard is to define this
methodology, to provide a framework for specifying ATS, and to define the procedures to be followed during
conformance testing.
Test methods are also addressed in this International Standard; however, any organization contemplating the use
of test methods defined in this International Standard should carefully consider the constraints on their applicability.
Conformance testing does not include robustness testing, acceptance testing and performance testing, because
the geographic information family of standards does not establish requirements for these areas.
The main body of this International Standard is structured as follows. The general framework of conformance
including the definition of a conforming implementation appears in clause 5. Conformance testing methodology is
described in clause 6. The possible test methods for testing conformance to the ISO geographic information
standards are discussed in clause 7. The relationship between ATS and ETS is presented in clause 8. The
bibliography on conformance testing is given at the end. Guidelines for writing conformance clauses and
associated templates are provided in annex A.
© ISO 2000 – All rights reserved v
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO 19105:2000(E)
Geographic information — Conformance and testing
1 Scope
This International Standard specifies the framework, concepts and methodology for testing and criteria to be
achieved to claim conformance to the family of ISO geographic information standards. It provides a framework for
specifying abstract test suites (ATS) and for defining the procedures to be followed during conformance testing.
Conformance may be claimed for data or software products or services or by specifications including any profile or
functional standard.
Standardization of test methods and criteria for conformance to geographic information standards will allow
verification of conformance to those standards. Verifiable conformance is important to geographic information
users, in order to achieve data transfer and sharing.
This International Standard is applicable to all the phases of conformance and testing. These phases are
characterized by the following major activities:
a) the definition of ATS for conformance to the ISO geographic information standards;
b) the definition of test methods for conformance to the ISO geographic information standards;
c) the conformance assessment process carried out by a testing laboratory for a client, culminating in the
production of a conformance test report.
This International Standard specifies the requirements for, and gives guidance on, the procedures to be followed in
conformance testing for the ISO geographic information standards. It includes only such information as is
necessary to meet the following objectives:
1) to achieve confidence in the tests as a measure of conformance;
2) to achieve comparability between the results of corresponding tests applied in different places at different
times;
3) to facilitate communication between the parties responsible for the activities described in 1) and 2).
This International Standard provides a framework for certification (an administrative procedure which may follow
conformance testing) in informative annex B.
The following topics are outside the scope of this International Standard.
a) The description of requirements for procurement and contracts.
b) Testing by means of test methods which are specific to particular applications or systems.
c) Acceptance testing, performance testing and robustness testing.
The framework established by this International Standard includes the concept of executable test suites (ETS).
These, by their very nature, cannot be standardized; consequently, standardization of ETS is outside the scope of
this International Standard.
© ISO 2000 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO 19105:2000(E)
2 Conformance
2.1 Conformance requirements
This International Standard defines two classes of conformance: class A and class B. Class A concerns
conformance of specifications, including any profile or functional standard, with the series of ISO geographic
information standards as a whole. Class B concerns conformance of conformance clauses as defined by this
International Standard. Further requirements for conformance of profiles in addition to class A are given in
ISO 19106.
NOTE Conformance is defined in annex A.
2.2 Abstract test suite
2.2.1 Test case for conformance class A
a) Test purpose: verify conformance with the ISO geographic information standards.
b) Test method: manually check that all specifications, including profiles and functional standards, claiming
conformance to the ISO geographic information standards have a conformance clause. This conformance
clause shall not exclude conformance with any of the ISO geographic information standards. Conformance
testing shall be performed in accordance with clause 6. The test method used shall be in accordance with
clause 7.
c) Reference: ISO 19105
d) test type: capability test
2.2.2 Test case for conformance class B
a) Test purpose: verify that the conformance clause is written in a correct format.
b) Test method: manually check if the conformance clause is written according to normative annex A.
c) Reference: ISO 19105:2000, annex A.
d) Test type: capability test.
3 Terms and definitions
For the purposes of this International Standard, the following terms and definitions apply.
3.1
abstract test case
generalized test for a particular requirement
NOTE An abstract test case is a formal basis for deriving executable test cases. One or more test purposes are
encapsulated in the abstract test case. An abstract test case is independent of both the implementation and the values. It should
be complete in the sense that it is sufficient to enable a test verdict to be assigned unambiguously to each potentially
observable test outcome (i.e. sequence of test events).
3.2
abstract test method
method for testing implementation independent of any particular test procedure
2 © ISO 2000 – All rights reserved
---------------------- Page: 7 ----------------------
ISO 19105:2000(E)
3.3
abstract test module
set of related abstract test cases
NOTE Abstract test modules may be nested in a hierarchical way.
3.4
ATS
abstract test suite
abstract test module specifying all the requirements to be satisfied for conformance
NOTE Abstract test suites are described in a conformance clause.
3.5
acceptance testing
�user� process of determining whether an implementation satisfies acceptance criteria and enables the user to
determine whether to accept the implementation
NOTE 1 This includes the planning and execution of several kinds of tests (e.g. functional, volume, performance tests) that
demonstrate that the implementation satisfies the user requirements.
NOTE 2 This is not a part of conformance testing.
3.6
basic test
initial capability test intended to identify clear cases of non-conformance
3.7
capability test
test designed to determine whether an IUT conforms to a particular characteristic of an International Standard as
described in the test purpose
3.8
conformance
fulfilment of specified requirements
3.9
conformance assessment process
process for assessing the conformance of an implementation to an International Standard
3.10
conformance clause
clause defining what is necessary in order to meet the requirements of the International Standard
3.11
conformance testing
testing of a product to determine the extent to which the product is a conforming implementation
3.12
conformance test report
summary of the conformance to the International Standard as well as all the details of the testing that supports the
given overall summary
3.13
conforming implementation
implementation which satisfies the requirements
3.14
executable test case
specific test of an implementation to meet particular requirements
NOTE Instantiation of an abstract test case with values.
© ISO 2000 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO 19105:2000(E)
3.15
ETS
executable test suite
set of executable test cases
3.16
fail verdict
test verdict of non-conformance
NOTE Non-conformance may be with respect to either the test purpose or at least one of the conformance requirements of
the relevant standard(s).
3.17
falsification test
test to find errors in the implementation
NOTE If errors are found, one can correctly deduce that the implementation does not conform to the International
Standard; however, the absence of errors does not necessarily imply the converse. The falsification test can only demonstrate
non-conformance. Compare with the verification test. Due to technical and economic problems, in most cases, the falsification
test is adopted as a test method for conformance testing.
3.18
implementation
realization of a specification
NOTE In the context of the ISO geographic information standards, this includes specifications of geographic information
services and datasets.
3.19
ICS
Implementation Conformance Statement
statement of specification options that have been implemented
3.20
IXIT
Implementation eXtra Information for Testing
statement containing all of the information related to the IUT and its corresponding SUT which will enable the
testing laboratory to run an appropriate test suite against that IUT
NOTE IXIT typically provides the details on the organization and storage of concepts in the SUT as well as on the means
of access to and modification of the SUT.
3.21
inconclusive verdict
test verdict when neither a pass verdict nor a fail verdict apply
3.22
non-conformance
failure to fulfil one or more specified requirements
3.23
pass verdict
test verdict of conformance
3.24
performance testing
measurement of the performance characteristics of an Implementation Under Test (IUT), such as its throughput,
responsiveness, etc., under various conditions
NOTE This is not a part of conformance testing.
4 © ISO 2000 – All rights reserved
---------------------- Page: 9 ----------------------
ISO 19105:2000(E)
3.25
robustness testing
process of determining how well an IUT processes data which contains errors
NOTE This is not a part of conformance testing.
3.26
SUT
System Under Test
computer hardware, software and communication network required to support IUT
3.27
testing laboratory
organization that carries out the conformance assessment process
3.28
verification test
test developed to prove rigorously whether an IUT is correct
4 Abbreviated terms
For the purposes of this International Standard, the following abbreviations apply.
ATS: Abstract Test Suite
ETS: Executable Test Suite
ICS: Implementation Conformance Statement
IUT: Implementation Under Test
IXIT: Implementation eXtra Information for Testing
SUT: System Under Test
5 General framework of conformance
5.1 Introduction
In the context of the ISO geographic information standards, an implementation exhibits conformance if it complies
with the conformance requirements of the applicable ISO geographic information standards. The conformance
requirements are stated in the conformance clause of each International Standard.
5.2 Conformance clause
All testable ISO geographic information standards contain a conformance clause. This clause specifies all the
requirements that shall be satisfied to claim conformance to that International Standard. The conformance clause
serves as an entry point for conformance testing.
The requirements for conformance clauses are given in annex A.
5.3 Conformance requirements
Conformance requirements may be classified as
a) mandatory requirements: these shall be observed in all cases;
© ISO 2000 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO 19105:2000(E)
b) conditional requirements: these shall be observed if the conditions set out in the specification apply;
c) optional requirements: these may be selected to suit the implementation, provided that any requirements
applicable to the option are observed.
Furthermore, conformance requirements may be stated
a) positively: they state what is required to be done;
b) negatively: they state what is required not to be done.
To manage a set of options specific to a subset of International Standards, such as profiles, conformance classes
may be provided (see normative annex A).
5.4 Implementation conformance statement
To evaluate the conformance of a particular implementation, there shall be a statement of the options which have
been implemented. This will allow the implementation to be tested for conformance against the relevant
requirements, and against those requirements only. Such a statement is called an Implementation Conformance
Statement (ICS). This statement shall contain only options within the framework of requirements specified in the
relevant ISO geographic information standards. The statement shall not include options outside this framework.
This statement provides the testing laboratory with a better understanding of the System Under Test (SUT) for use
in the conformance assessment process and helps identify the boundaries of the domain of testing.
This statement may be generated by using an ICS proforma. An ICS proforma may be provided by the testing
laboratory. It is a questionnaire used as a framework to document those SUT capabilities necessary to undertake
conformance testing.
[2,3,4] [5]
For further information on ICS, see ISO 9646 and ISO 10303-31 .
5.5 A conforming implementation
A conforming implementation shall satisfy the conformance requirements of the applicable ISO geographic
information standards and shall be consistent with the ICS. Such an implementation shall have passed all tests
constituting the domain of testing, including any optional requirements stated as supported in it. A conforming
implementation may support additional capabilities not described in the appropriate International Standard,
providing those capabilities are not explicitly prohibited in the standard.
6 Conformance testing methodology
6.1 Introduction
This clause describes the conformance testing methodology, the different types of conformance tests that are used,
and additional information that must be provided by the client to the testing laboratory.
6.2 Types of conformance tests
6.2.1 Overview
The objective of conformance testing is to determine whether the implementation being tested conforms to the
requirements stated in the relevant ISO geographic information standard. Two types of tests are identified,
according to the extent to which they provide an indication of conformance.
6 © ISO 2000 – All rights reserved
---------------------- Page: 11 ----------------------
ISO 19105:2000(E)
a) Basic tests, which provide preliminary evidence that an IUT conforms. If the ATS identifies tests to be used as
basic tests, they shall be used at the start of the conformance assessment process. Such tests are
1)
standardized.
b) Capability tests, which check that the observable capabilities of the IUT are in accordance with the capabilities
claimed in the ICS. They endeavour to provide testing which is as comprehensive as possible over the full
range of conformance requirements specified in the appropriate International Standard. Such tests are
standardized.
An ATS given in a conformance clause indicates which capability tests, if any, shall be used as basic tests. An ATS
does not include any basic tests that are additional to the set of capability tests. In simple cases, basic tests may
not be necessary.
6.2.2 Basic tests
Basic tests provide limited testing of an IUT in order to establish whether or not it is appropriate to perform more
thorough testing. Basic tests may be in abstract or executable form. They may be used to determine the degree of
conformance prior to deciding whether or not to proceed with thorough capability testing. Basic tests are simple
capability tests. Conformance clauses should identify those test cases that may be used as basic tests.
Basic tests shall be used
a) for detecting obvious cases of non-conformance, and
b) as a preliminary step in order to decide whether or not to run capability tests.
Basic tests shall not be used alone to
a) determine whether an implementation conforms, or
b) guarantee the determination of causes for failure.
6.2.3 Capability tests
Capability tests, which may be in abstract or executable form, should exercise an implementation as thoroughly as
is practical over the full range of conformance requirements specified in an International Standard. Capability tests
should be provided to check mandatory capabilities and those optional capabilities that are identified in the ICS as
being supported by the IUT.
Capability tests shall be used to
a) check that the capabilities of the IUT are consistent with the conformance requirements, that is, determine
whether or not an implementation conforms, and
b) investigate causes for failure.
Capability tests shall not be used to
a) test in detail the behaviour associated with each capability which has been implemented, or
b) guarantee completeness.
Abstract capability tests are standardized within an ATS.
1) An executable test case may be obtained for these basic tests, the system may be tested before application for the basic
and capability tests to the testing laboratory (the in-house testing). This may also be done for capability tests, but it is more
appropriate for basic tests.
© ISO 2000 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO 19105:2000(E)
6.3 Implementation eXtra Information for Testing
In order to test an implementation, the testing laboratory requires information relating to the IUT and the
environment in which it will be tested. The client submitting the implementation for testing shall provide this
information as an Implementation eXtra Information for Testing (IXIT). The IXIT may be completed using the
question/answer process initiated by the IXIT proforma.
The IXIT shall contain the following:
a) information about the IUT which is needed by the testing laboratory in order to be able to run the appropriate
ETS against that IUT and analyse the results;
b) reference to the related ICS and any other administrative information.
The IXIT shall not conflict with the related ICS; a consistency check shall be performed during preparation for
testing to remove any inconsistencies. While the ICS gives information to the testing laboratory for the definition of
the domain of testing, the IXIT gives information on how to perform the testing. In particular, it provides the details
on the organization and storage of concepts in the SUT and on the means for accessing and modifying the SUT. It
also contains conversion algorithms between the concepts of the IUT and the International Standard. There shall
be one IXIT for each conformance assessment process.
[2,3,4] [5]
...
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.Geografske informacije - Ustreznost in preskušanjeInformation géographique -- Conformité et essaisGeographic information -- Conformance and testing35.240.70Uporabniške rešitve IT v znanostiIT applications in science07.040Astronomija. Geodezija. GeografijaAstronomy. Geodesy. GeographyICS:Ta slovenski standard je istoveten z:ISO 19105:2000SIST ISO 19105:2003en01-november-2003SIST ISO 19105:2003SLOVENSKI
STANDARD
SIST ISO 19105:2003
ReferencenumberISO19105:2000(E)©ISO2000INTERNATIONALSTANDARDISO19105Firstedition2000-12-15Geographicinformation—ConformanceandtestingInformationgéographique—ConformitéetessaisSIST ISO 19105:2003
ISO19105:2000(E)PDFdisclaimerThisPDFfilemaycontainembeddedtypefaces.InaccordancewithAdobe'slicensingpolicy,thisfilemaybeprintedorviewedbutshallnotbeeditedunlessthetypefaceswhichareembeddedarelicensedtoandinstalledonthecomputerperformingtheediting.Indownloadingthisfile,partiesacceptthereintheresponsibilityofnotinfringingAdobe'slicensingpolicy.TheISOCentralSecretariatacceptsnoliabilityinthisarea.AdobeisatrademarkofAdobeSystemsIncorporated.DetailsofthesoftwareproductsusedtocreatethisPDFfilecanbefoundintheGeneralInforelativetothefile;thePDF-creationparameterswereoptimizedforprinting.EverycarehasbeentakentoensurethatthefileissuitableforusebyISOmemberbodies.Intheunlikelyeventthataproblemrelatingtoitisfound,pleaseinformtheCentralSecretariatattheaddressgivenbelow.©ISO2000Allrightsreserved.Unlessotherwisespecified,nopartofthispublicationmaybereproducedorutilizedinanyformorbyanymeans,electronicormechanical,includingphotocopyingandmicrofilm,withoutpermissioninwritingfromeitherISOattheaddressbeloworISO'smemberbodyinthecountryoftherequester.ISOcopyrightofficeCasepostale56CH-1211Geneva20Tel.+41227490111Fax+41227490947E-mailcopyright@iso.chWebwww.iso.chPrintedinSwitzerlandii©ISO2000–AllrightsreservedSIST ISO 19105:2003
ISO19105:2000(E)©ISO2000–AllrightsreservediiiContentsPageForeword.ivIntroduction.v1Scope.12Conformance.22.1Conformancerequirements.22.2Abstracttestsuite.23Termsanddefinitions.24Abbreviatedterms.55Generalframeworkofconformance.55.1Introduction.55.2Conformanceclause.55.3Conformancerequirements.55.4Implementationconformancestatement.65.5Aconformingimplementation.66Conformancetestingmethodology.66.1Introduction.66.2Typesofconformancetests.66.3ImplementationExtraInformationforTesting.86.4Conformanceassessment.86.5Intrinsicpropertiesoftheconformanceassessmentprocess.107Testmethods.117.1Introduction.117.2Approachestoconformancetesting.117.3Areasofgeographicinformationforconformancetesting.128Abstracttestsuitesandexecutabletestsuites.128.1Introduction.128.2Testpurposes.138.3Abstracttestcases.138.4Executabletestcases.138.5Relationshipbetweenabstractandexecutabletestcases.13AnnexA(normative)Conformanceclauses.14AnnexB(informative)Supportingorganizations.19Bibliography.21SIST ISO 19105:2003
ISO19105:2000(E)iv©ISO2000–AllrightsreservedForewordISO(theInternationalOrganizationforStandardization)isaworldwidefederationofnationalstandardsbodies(ISOmemberbodies).TheworkofpreparingInternationalStandardsisnormallycarriedoutthroughISOtechnicalcommittees.Eachmemberbodyinterestedinasubjectforwhichatechnicalcommitteehasbeenestablishedhastherighttoberepresentedonthatcommittee.Internationalorganizations,governmentalandnon-governmental,inliaisonwithISO,alsotakepartinthework.ISOcollaboratescloselywiththeInternationalElectrotechnicalCommission(IEC)onallmattersofelectrotechnicalstandardization.InternationalStandardsaredraftedinaccordancewiththerulesgivenintheISO/IECDirectives,Part3.DraftInternationalStandardsadoptedbythetechnicalcommitteesarecirculatedtothememberbodiesforvoting.PublicationasanInternationalStandardrequiresapprovalbyatleast75%ofthememberbodiescastingavote.AttentionisdrawntothepossibilitythatsomeoftheelementsofthisInternationalStandardmaybethesubjectofpatentrights.ISOandIECshallnotbeheldresponsibleforidentifyinganyorallsuchpatentrights.InternationalStandardISO19105waspreparedbyTechnicalCommitteeISO/TC211,Geographicinformation/Geomatics.AnnexAformsanormativepartofthisInternationalStandard.AnnexBisforinformationonly.SIST ISO 19105:2003
ISO19105:2000(E)©ISO2000–AllrightsreservedvIntroductionThescopeofISO/TC211isstandardizationinthefieldofdigitalgeographicinformation.ThisworkaimsatestablishingastructuredsetofInternationalStandardsforinformationconcerningobjectsorphenomenathataredirectlyorindirectlyassociatedwithalocationrelativetotheEarth.TheseInternationalStandardsmayspecify,forgeographicinformation,methods,toolsandservicesfordatamanagement(includingdefinitionanddescription),acquiring,processing,analysing,accessing,presentingandtransferringsuchdataindigital/electronicformbetweendifferentusers,systemsandlocations.TheworkwillbelinkedtoappropriateInternationalStandardsforinformationtechnologyanddata,wherepossible,andprovideaframeworkforthedevelopmentofsector-specificapplicationsusinggeographicdata.ThisInternationalStandardprovidestheframework,concepts,andmethodologyfortestingandthecriteriatobeachievedtoclaimconformancetothisfamilyofInternationalStandards.ThisInternationalStandardisbasedinpartonISO9646-1whichdescribesconformanceandtestinginOpenSystemsInterconnection(OSI),ISO10303-31whichdescribesconformanceandtestinginindustrialautomationsystemsandintegration,andISO10641whichdescribesconformanceandtestingforcomputergraphicsandimageprocessing.WhiletheframeworkofconformancetestingdescribedinthesethreeInternationalStandardsisusedinthisInternationalStandard,someconceptshavebeenmodifiedforuseinthisparticulardomain.Theobjectiveofstandardizationinthefieldofdigitalgeographicinformationcannotbecompletelyachievedunlessdataandsystemscanbetestedtodeterminewhethertheyconformtotherelevantgeographicinformationstandards.ConformancetestingisthetestingofacandidateproductfortheexistenceofspecificcharacteristicsrequiredbyanInternationalStandardinordertodeterminetheextenttowhichthatproductisaconformingimplementation.ItinvolvestestingthecapabilitiesofanimplementationagainstboththeconformancerequirementsintherelevantInternationalStandard(s)andthestatementoftheimplementation'scapabilities.Aframeworkofanabstracttestsuite(ATS)isstandardizedforrelevantstandardsinISO/TC211.ThestandardizationofATSrequiresinternationaldefinitionandacceptanceofacommontestmethodology,togetherwithappropriatetestmethodsandprocedures.ThepurposeofthisInternationalStandardistodefinethismethodology,toprovideaframeworkforspecifyingATS,andtodefinetheprocedurestobefollowedduringconformancetesting.TestmethodsarealsoaddressedinthisInternationalStandard;however,anyorganizationcontemplatingtheuseoftestmethodsdefinedinthisInternationalStandardshouldcarefullyconsidertheconstraintsontheirapplicability.Conformancetestingdoesnotincluderobustnesstesting,acceptancetestingandperformancetesting,becausethegeographicinformationfamilyofstandardsdoesnotestablishrequirementsfortheseareas.ThemainbodyofthisInternationalStandardisstructuredasfollows.Thegeneralframeworkofconformanceincludingthedefinitionofaconformingimplementationappearsinclause5.Conformancetestingmethodologyisdescribedinclause6.ThepossibletestmethodsfortestingconformancetotheISOgeographicinformationstandardsarediscussedinclause7.TherelationshipbetweenATSandETSispresentedinclause8.Thebibliographyonconformancetestingisgivenattheend.GuidelinesforwritingconformanceclausesandassociatedtemplatesareprovidedinannexA.SIST ISO 19105:2003
SIST ISO 19105:2003
INTERNATIONALSTANDARDISO19105:2000(E)©ISO2000–Allrightsreserved1Geographicinformation—Conformanceandtesting1ScopeThisInternationalStandardspecifiestheframework,conceptsandmethodologyfortestingandcriteriatobeachievedtoclaimconformancetothefamilyofISOgeographicinformationstandards.Itprovidesaframeworkforspecifyingabstracttestsuites(ATS)andfordefiningtheprocedurestobefollowedduringconformancetesting.Conformancemaybeclaimedfordataorsoftwareproductsorservicesorbyspecificationsincludinganyprofileorfunctionalstandard.Standardizationoftestmethodsandcriteriaforconformancetogeographicinformationstandardswillallowverificationofconformancetothosestandards.Verifiableconformanceisimportanttogeographicinformationusers,inordertoachievedatatransferandsharing.ThisInternationalStandardisapplicabletoallthephasesofconformanceandtesting.Thesephasesarecharacterizedbythefollowingmajoractivities:a)thedefinitionofATSforconformancetotheISOgeographicinformationstandards;b)thedefinitionoftestmethodsforconformancetotheISOgeographicinformationstandards;c)theconformanceassessmentprocesscarriedoutbyatestinglaboratoryforaclient,culminatingintheproductionofaconformancetestreport.ThisInternationalStandardspecifiestherequirementsfor,andgivesguidanceon,theprocedurestobefollowedinconformancetestingfortheISOgeographicinformationstandards.Itincludesonlysuchinformationasisnecessarytomeetthefollowingobjectives:1)toachieveconfidenceinthetestsasameasureofconformance;2)toachievecomparabilitybetweentheresultsofcorrespondingtestsappliedindifferentplacesatdifferenttimes;3)tofacilitatecommunicationbetweenthepartiesresponsiblefortheactivitiesdescribedin1)and2).ThisInternationalStandardprovidesaframeworkforcertification(anadministrativeprocedurewhichmayfollowconformancetesting)ininformativeannexB.ThefollowingtopicsareoutsidethescopeofthisInternationalStandard.a)Thedescriptionofrequirementsforprocurementandcontracts.b)Testingbymeansoftestmethodswhicharespecifictoparticularapplicationsorsystems.c)Acceptancetesting,performancetestingandrobustnesstesting.TheframeworkestablishedbythisInternationalStandardincludestheconceptofexecutabletestsuites(ETS).These,bytheirverynature,cannotbestandardized;consequently,standardizationofETSisoutsidethescopeofthisInternationalStandard.SIST ISO 19105:2003
ISO19105:2000(E)2©ISO2000–Allrightsreserved2Conformance2.1ConformancerequirementsThisInternationalStandarddefinestwoclassesofconformance:classAandclassB.ClassAconcernsconformanceofspecifications,includinganyprofileorfunctionalstandard,withtheseriesofISOgeographicinformationstandardsasawhole.ClassBconcernsconformanceofconformanceclausesasdefinedbythisInternationalStandard.FurtherrequirementsforconformanceofprofilesinadditiontoclassAaregiveninISO19106.NOTEConformanceisdefinedinannexA.2.2Abstracttestsuite2.2.1TestcaseforconformanceclassAa)Testpurpose:verifyconformancewiththeISOgeographicinformationstandards.b)Testmethod:manuallycheckthatallspecifications,includingprofilesandfunctionalstandards,claimingconformancetotheISOgeographicinformationstandardshaveaconformanceclause.ThisconformanceclauseshallnotexcludeconformancewithanyoftheISOgeographicinformationstandards.Conformancetestingshallbeperformedinaccordancewithclause6.Thetestmethodusedshallbeinaccordancewithclause7.c)Reference:ISO19105d)testtype:capabilitytest2.2.2TestcaseforconformanceclassBa)Testpurpose:verifythattheconformanceclauseiswritteninacorrectformat.b)Testmethod:manuallycheckiftheconformanceclauseiswrittenaccordingtonormativeannexA.c)Reference:ISO19105:2000,annexA.d)Testtype:capabilitytest.3TermsanddefinitionsForthepurposesofthisInternationalStandard,thefollowingtermsanddefinitionsapply.3.1abstracttestcasegeneralizedtestforaparticularrequirementNOTEAnabstracttestcaseisaformalbasisforderivingexecutabletestcases.Oneormoretestpurposesareencapsulatedintheabstracttestcase.Anabstracttestcaseisindependentofboththeimplementationandthevalues.Itshouldbecompleteinthesensethatitissufficienttoenableatestverdicttobeassignedunambiguouslytoeachpotentiallyobservabletestoutcome(i.e.sequenceoftestevents).3.2abstracttestmethodmethodfortestingimplementationindependentofanyparticulartestprocedureSIST ISO 19105:2003
ISO19105:2000(E)©ISO2000–Allrightsreserved33.3abstracttestmodulesetofrelatedabstracttestcasesNOTEAbstracttestmodulesmaybenestedinahierarchicalway.3.4ATSabstracttestsuiteabstracttestmodulespecifyingalltherequirementstobesatisfiedforconformanceNOTEAbstracttestsuitesaredescribedinaconformanceclause.3.5acceptancetestinguserprocessofdeterminingwhetheranimplementationsatisfiesacceptancecriteriaandenablestheusertodeterminewhethertoaccepttheimplementationNOTE1Thisincludestheplanningandexecutionofseveralkindsoftests(e.g.functional,volume,performancetests)thatdemonstratethattheimplementationsatisfiestheuserrequirements.NOTE2Thisisnotapartofconformancetesting.3.6basictestinitialcapabilitytestintendedtoidentifyclearcasesofnon-conformance3.7capabilitytesttestdesignedtodeterminewhetheranIUTconformstoaparticularcharacteristicofanInternationalStandardasdescribedinthetestpurpose3.8conformancefulfilmentofspecifiedrequirements3.9conformanceassessmentprocessprocessforassessingtheconformanceofanimplementationtoanInternationalStandard3.10conformanceclauseclausedefiningwhatisnecessaryinordertomeettherequirementsoftheInternationalStandard3.11conformancetestingtestingofaproducttodeterminetheextenttowhichtheproductisaconformingimplementation3.12conformancetestreportsummaryoftheconformancetotheInternationalStandardaswellasallthedetailsofthetestingthatsupportsthegivenoverallsummary3.13conformingimplementationimplementationwhichsatisfiestherequirements3.14executabletestcasespecifictestofanimplementationtomeetparticularrequirementsNOTEInstantiationofanabstracttestcasewithvalues.SIST ISO 19105:2003
ISO19105:2000(E)4©ISO2000–Allrightsreserved3.15ETSexecutabletestsuitesetofexecutabletestcases3.16failverdicttestverdictofnon-conformanceNOTENon-conformancemaybewithrespecttoeitherthetestpurposeoratleastoneoftheconformancerequirementsoftherelevantstandard(s).3.17falsificationtesttesttofinderrorsintheimplementationNOTEIferrorsarefound,onecancorrectlydeducethattheimplementationdoesnotconformtotheInternationalStandard;however,theabsenceoferrorsdoesnotnecessarilyimplytheconverse.Thefalsificationtestcanonlydemonstratenon-conformance.Comparewiththeverificationtest.Duetotechnicalandeconomicproblems,inmostcases,thefalsificationtestisadoptedasatestmethodforconformancetesting.3.18implementationrealizationofaspecificationNOTEInthecontextoftheISOgeographicinformationstandards,thisincludesspecificationsofgeographicinformationservicesanddatasets.3.19ICSImplementationConformanceStatementstatementofspecificationoptionsthathavebeenimplemented3.20IXITImplementationeXtraInformationforTestingstatementcontainingalloftheinformationrelatedtotheIUTanditscorrespondingSUTwhichwillenablethetestinglaboratorytorunanappropriatetestsuiteagainstthatIUTNOTEIXITtypicallyprovidesthedetailsontheorganizationandstorageofconceptsintheSUTaswellasonthemeansofaccesstoandmodificationoftheSUT.3.21inconclusiveverdicttestverdictwhenneitherapassverdictnorafailverdictapply3.22non-conformancefailuretofulfiloneormorespecifiedrequirements3.23passverdicttestverdictofconformance3.24performancetestingmeasurementoftheperformancecharacteristicsofanImplementationUnderTest(IUT),suchasitsthroughput,responsiveness,etc.,undervariousconditionsNOTEThisisnotapartofconformancetesting.SIST ISO 19105:2003
ISO19105:2000(E)©ISO2000–Allrightsreserved53.25robustnesstestingprocessofdetermininghowwellanIUTprocessesdatawhichcontainserrorsNOTEThisisnotapartofconformancetesting.3.26SUTSystemUnderTestcomputerhardware,softwareandcommunicationnetworkrequiredtosupportIUT3.27testinglaboratoryorganizationthatcarriesouttheconformanceassessmentprocess3.28verificationtesttestdevelopedtoproverigorouslywhetheranIUTiscorrect4AbbreviatedtermsForthepurposesofthisInternationalStandard,thefollowingabbreviationsapply.ATS:AbstractTestSuiteETS:ExecutableTestSuiteICS:ImplementationConformanceStatementIUT:ImplementationUnderTestIXIT:ImplementationeXtraInformationforTestingSUT:SystemUnderTest5Generalframeworkofconformance5.1IntroductionInthecontextoftheISOgeographicinformationstandards,animplementationexhibitsconformanceifitcomplieswiththeconformancerequirementsoftheapplicableISOgeographicinformationstandards.TheconformancerequirementsarestatedintheconformanceclauseofeachInternationalStandard.5.2ConformanceclauseAlltestableISOgeographicinformationstandardscontainaconformanceclause.ThisclausespecifiesalltherequirementsthatshallbesatisfiedtoclaimconformancetothatInternationalStandard.Theconformanceclauseservesasanentrypointforconformancetesting.TherequirementsforconformanceclausesaregiveninannexA.5.3ConformancerequirementsConformancerequirementsmaybeclassifiedasa)mandatoryrequirements:theseshallbeobservedinallcases;SIST ISO 19105:2003
ISO19105:2000(E)6©ISO2000–Allrightsreservedb)conditionalrequirements:theseshallbeobservediftheconditionssetoutinthespecificationapply;c)optionalrequirements:thesemaybeselectedtosuittheimplementation,providedthatanyrequirementsapplicabletotheoptionareobserved.Furthermore,conformancerequirementsmaybestateda)positively:theystatewhatisrequiredtobedone;b)negatively:theystatewhatisrequirednottobedone.TomanageasetofoptionsspecifictoasubsetofInternationalStandards,suchasprofiles,conformanceclassesmaybeprovided(seenormativeannexA).5.4ImplementationconformancestatementToevaluatetheconformanceofaparticularimplementation,thereshallbeastatementoftheoptionswhichhavebeenimplemented.Thiswillallowtheimplementationtobetestedforconformanceagainsttherelevantrequirements,andagainstthoserequirementsonly.SuchastatementiscalledanImplementationConformanceStatement(ICS).ThisstatementshallcontainonlyoptionswithintheframeworkofrequirementsspecifiedintherelevantISOgeographicinformationstandards.Thestatementshallnotincludeoptionsoutsidethisframework.ThisstatementprovidesthetestinglaboratorywithabetterunderstandingoftheSystemUnderTest(SUT)foruseintheconformanceassessmentprocessandhelpsidentifytheboundariesofthedomainoftesting.ThisstatementmaybegeneratedbyusinganICSproforma.AnICSproformamaybeprovidedbythetestinglaboratory.ItisaquestionnaireusedasaframeworktodocumentthoseSUTcapabilitiesnecessarytoundertakeconformancetesting.ForfurtherinformationonICS,seeISO9646[2,3,4]andISO10303-31[5].5.5AconformingimplementationAconformingimplementationshallsatisfytheconformancerequirementsoftheapplicableISOgeographicinformationstandardsandshallbeconsistentwiththeICS.Suchanimplementationshallhavepassedalltestsconstitutingthedomainoftesting,includinganyoptionalrequirementsstatedassupportedinit.AconformingimplementationmaysupportadditionalcapabilitiesnotdescribedintheappropriateInternationalStandard,providingthosecapabilitiesarenotexplicitlyprohibitedinthestandard.6Conformancetestingmethodology6.1IntroductionThisclausedescribestheconformancetestingmethodology,thedifferenttypesofconformanceteststhatareused,andadditionalinformationthatmustbeprovidedbytheclienttothetestinglaboratory.6.2Typesofconformancetests6.2.1OverviewTheobjectiveofconformancetestingistodeterminewhethertheimplementationbeingtestedconformstotherequirementsstatedintherelevantISOgeographicinformationstandard.Twotypesoftestsareidentified,accordingtotheextenttowhichtheyprovideanindicationofconformance.SIST ISO 19105:2003
ISO19105:2000(E)©ISO2000–Allrightsreserved7a)Basictests,whichprovidepreliminaryevidencethatanIUTconforms.IftheATSidentifiesteststobeusedasbasictests,theyshallbeusedatthestartoftheconformanceassessmentprocess.Suchtestsarestandardized.1)b)Capabilitytests,whichcheckthattheobservablecapabilitiesoftheIUTareinaccordancewiththecapabilitiesclaimedintheICS.TheyendeavourtoprovidetestingwhichisascomprehensiveaspossibleoverthefullrangeofconformancerequirementsspecifiedintheappropriateInternationalStandard.Suchtestsarestandardized.AnATSgiveninaconformanceclauseindicateswhichcapabilitytests,ifany,shallbeusedasbasictests.AnATSdoesnotincludeanybasicteststhatareadditionaltothesetofcapabilitytests.Insimplecases,basictestsmaynotbenecessary.6.2.2BasictestsBasictestsprovidelimitedtestingofanIUTinordertoestablishwhetherornotitisappropriatetoperformmorethoroughtesting.Basictestsmaybeinabstractorexecutableform.Theymaybeusedtodeterminethedegreeofconformancepriortodecidingwhetherornottoproceedwiththoroughcapabilitytesting.Basictestsaresimplecapabilitytests.Conformanceclausesshouldidentifythosetestcasesthatmaybeusedasbasictests.Basictestsshallbeuseda)fordetectingobviouscasesofnon-conformance,andb)asapreliminarystepinordertodecidewhetherornottoruncapabilitytests.Basictestsshallnotbeusedalonetoa)determinewhetheranimplementationconforms,orb)guaranteethedeterminationofcausesforfailure.6.2.3CapabilitytestsCapabilitytests,whichmaybeinabstractorexecutableform,shouldexerciseanimplementationasthoroughlyasispracticaloverthefullrangeofconformancerequirementsspecifiedinanInternationalStandard.CapabilitytestsshouldbeprovidedtocheckmandatorycapabilitiesandthoseoptionalcapabilitiesthatareidentifiedintheICSasbeingsupportedbytheIUT.Capabilitytestsshallbeusedtoa)checkthatthecapabilitiesoftheIUTareconsistentwiththeconformancerequirements,thatis,determinewhetherornotanimplementationconforms,andb)investigatecausesforfailure.Capabilitytestsshallnotbeusedtoa)testindetailthebehaviourassociatedwitheachcapabilitywhichhasbeenimplemented,orb)guaranteecompleteness.AbstractcapabilitytestsarestandardizedwithinanATS.1)Anexecutabletestcasemaybeobtainedforthesebasictests,thesystemmaybetestedbeforeapplicationforthebasicandcapabilityteststothetestinglaboratory(thein-housetesting).Thismayalsobedoneforcapabilitytests,butitismoreappropriateforbasictests.SIST ISO 19105:2003
ISO19105:2000(E)8©ISO2000–Allrightsreserved6.3ImplementationeXtraInformationforTestingInordertotestanimplementation,thetestinglaboratoryrequiresinformationrelatingtotheIUTandtheenvironmentinwhichitwillbetested.TheclientsubmittingtheimplementationfortestingshallprovidethisinformationasanImplementationeXtraInformationforTesting(IXIT).TheIXITmaybecompletedusingthequestion/answerprocessinitiatedbytheIXITproforma.TheIXITshallcontainthefollowing:a)informationabouttheIUTwhichisneededbythetestinglaboratoryinordertobeabletoruntheappropriateETSagainstthatIUTandanalysetheresults;b)referencetotherelatedICSandanyotheradministrativeinformation.TheIXITshallnotconflictwiththerelatedICS;aconsistencycheckshallbeperformedduringpreparationfortestingtoremoveanyinconsistencies.WhiletheICSgivesinformationtothetestinglaboratoryforthedefinitionofthedomainoftesting,theIXITgivesinformationonhowtoperformthetesting.Inparticular,itprovidesthedetailsontheorganizationandstorageofconceptsintheSUTandonthemeansforaccessingandmodifyingtheSUT.ItalsocontainsconversionalgorithmsbetweentheconceptsoftheIUTandtheInternationalStandard.ThereshallbeoneIXITforeachconformanceassessmentprocess.ForfurtherinformationonIXIT,seeISO9646[2,3,4]andISO10303-31[5].6.4Conformanceassessment6.4.1ProcessoverviewTheconformanceassessmentprocessencompassesallconformancetestingactivitiesnecessarytodeterminetheconformanceofanimplementationtotherelevantISOgeographicinformationstandards.Theconformanceassessmentprocessinvolvesfourphases:a)preparationfortesting;b)testcampaign;c)analysisofresults;d)conformancetestreport.TheconformanceassessmentprocessisshowninFigure1.6.4.2PreparationfortestingThepreparationforthetestingphaseshouldinvolvethefollowingsteps:a)productionofadministrativeinformation;b)productionofICSandIXITfortesting;c)identificationoftestmethodandATS;d)anICSreview,conductedbyanalysingtheconformancestatementwithrespecttotherelevantconformancerequirements;e)anIXITreviewwhichincludesaconsistencycheckagainstthecorrespondingICS;f)selectionofinitialabstracttestcasesandassignmentofparametervaluesbasedontheICSandtheIXIT;SIST ISO 19105:2003
ISO19105:2000(E)©ISO2000–Allrightsreserved9g)preparationoftheSUT;NOTEThisenablesaclienttorunexecutabletestcasesontheIUTbeforepresentingtheIUTforthetestcampaign.h)finalabstracttest-caseselection.AnETSisproducedasaresultofselectingabstracttestcasesandassigningparametervalues[stepsf)andh)above].Atthispoint,theIUTandthescopeoftheconformanceassessmentprocessarefixedandcannotbechangedsubsequently.Thisisachievedbyagreementbetweentheclientandthetestinglaboratory.Figure1—Conformanceassessmentprocessoverview6.4.3TestcampaignAtestcampaignistheprocessofexecutingtheETSandrecordingtheobservedtestoutcomeandanyotherrelevantinformationinaconformancelog.TheinputtotheIUTandtheobservedtestoutcomeresultingfromtheexecutionofatestcaseshallberecordedintheconformancelog.TherecordingandretentionofallinformationprovidedbytheIUTduringthetestcampaignisnecessaryfortheanalysisphaseandforauditingpurposes.6.4.4AnalysisofresultsTheanalysisofresultsshallbeperformedbyevaluatingtheobservedtestoutcomeagainsttheverdictcriteriawhichareprescribedbytheabstracttestcase.Althoughthereisacleardistinctionbetweenthetestcampaignandtheanalysisphase,thetwomayoverlapintime.Atestverdictisastatementofpass,failorinconclusive.Passandfailarethetwomajorverdictsbut,inrarecases,averdictofinconclusivemayhavetobeassigned.Ajustificationshallbegivenwitheachfailorinconclusiveverdict;informativemessagesmayalsobeprovided.a)“Passverdict”meansthattheobservedtestoutcomegivesevidenceofconformancetotheconformancerequirementonwhichthetestpurposeisfocused,andisvalidwithrespecttotherelevantInternationalStandardandwithrespecttotheICS.b)“Failverdict”meansthattheobservedtestoutcomedemonstratesnon-conformancewithrespecttoeitheratestpurposeoratleastoneconformancerequirementintherelevantstandard(s).EXAMPLEAbnormaltermination—iftheexecutionoftheexecutabletestcaseisterminatedprematurely,forwhateverreason.c)“Inconclusiveverdict”meansthattheobservedtestoutcomeproducesneitherapassnorafailverdict.Thisshouldoccuronlyinveryrarecircumstances.SIST ISO 19105:2003
ISO19105:2000(E)10©ISO2000–AllrightsreservedEXAMPLETest-caseerror.Thetestverdictshallbeassignedtoaparticulartestoutcomeusingtheverdictcriteriarelevanttothatparticularabstracttestcase.ThetestverdictsassignedshallthenbesynthesizedintoanoverallsummaryfortheIUT.6.4.5ConformancetestreportTheresultsofconformancetestingshallbedocumentedinaconformancetestreport.Thisreportshallbeintwoparts:asummaryanddetailedinformation.Aproformashallbeusedtoproduceeachconformancetestreport.ThefirstpartshallbeanoverallsummaryoftheconformancestatusoftheIUT.Thisoverallsummaryshallprovideanoverviewoftheverdictsthatareassignedtothetestcasesexecutedinthecon
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.