ISO/IEC 25040:2011
(Main)Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Evaluation process
Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Evaluation process
ISO/IEC 25040:2011 contains requirements and recommendations for the evaluation of software product quality and clarifies the general concepts. It provides a process description for evaluating software product quality and states the requirements for the application of this process. The evaluation process can be used for different purposes and approaches. The process can be used for the evaluation of the quality of pre-developed software, commercial-off-the-shelf software or custom software and can be used during or after the development process. ISO/IEC 25040:2011 establishes the relationship of the evaluation reference model to the SQuaRE documents as well as shows how each SQuaRE document should be used during the activities of the evaluation process. It is intended for those responsible for software product evaluation and is appropriate for developers, acquirers and independent evaluators of software products. These three different approaches are detailed in ISO/IEC 14598-3, ISO/IEC 14598-4, and ISO/IEC 14598-5. It is not intended for evaluation of other aspects of software products (such as functional requirements, process requirements, business requirements, etc.).
Ingénierie des systèmes et du logiciel — Exigences de qualité et évaluation des systèmes et du logiciel (SQuaRE) — Modèle de référence d'évaluation et guide
General Information
Relations
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 25040
First edition
2011-03-01
Systems and software engineering —
Systems and software Quality
Requirements and Evaluation
(SQuaRE) — Evaluation process
Ingénierie des systèmes et du logiciel — Exigences de qualité et
évaluation des systèmes et du logiciel (SQuaRE) — Modèle de
référence d'évaluation et guide
Reference number
©
ISO/IEC 2011
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
© ISO/IEC 2011
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2011 – All rights reserved
Contents Page
Foreword .iv
Introduction.v
1 Scope.1
2 Conformance .1
3 Normative references.1
4 Terms and definitions .1
5 Software product quality evaluation reference model .10
5.1 Reference model - general .10
5.2 Reference model - evaluation processes.11
5.3 Roles.13
5.4 Quality in the life cycle.13
5.5 Support for the evaluation.13
6 Software product quality evaluation process .14
6.1 General requirements .14
6.2 Documentation .14
6.3 Establish the evaluation requirements .15
6.4 Specify the evaluation.17
6.5 Design the evaluation .19
6.6 Execute the evaluation.20
6.7 Conclude the evaluation.21
Annex A (informative) Evaluation levels.25
Annex B (informative) Evaluation methods.29
Annex C (informative) Example of Cost-Effectiveness Ranking of Evaluation Methods .34
Annex D (informative) Relationships between software product quality evaluation process
reference model and software and system life cycle processes .35
Annex E (informative) Evaluation report template .37
Annex F (informative) Diagrams of inputs, outcomes, constraints and resources for activities.39
Bibliography.44
© ISO/IEC 2011 – All rights reserved iii
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electro technical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 25040 is part of the SQuaRE series of standards and was prepared by Joint Technical Committee
ISO/IEC JTC 1, Information technology, Subcommittee SC 7, Software and systems engineering.
iv © ISO/IEC 2011 – All rights reserved
Introduction
As the use of information technology grows, the number of critical computer systems also grows. Such
systems include, for example, security critical, life critical, economically critical and safety critical systems. The
quality of software in these systems is particularly important because software faults can lead to serious
consequences.
Evaluation is the systematic determination of the extent to which an entity meets its specified criteria. The
evaluation of software product quality is vital to both the acquisition and development of software. The relative
importance of the various characteristics of software quality depends on the intended usage or objectives of
the system of which the software is a part; software products need to be evaluated to decide whether relevant
quality characteristics meet the requirements of the system.
This document is part of the SQuaRE series of standards and contains general requirements for software
product quality evaluation as well as clarifies the associated general concepts.
The general goal of creating the SQuaRE set of standards is to move to a logically organized, enriched and
unified series covering two main processes: software quality requirements specification and software quality
evaluation, supported by a software quality measurement process. The purpose of the SQuaRE set of
standards is to assist those developing and acquiring software products with the specification and evaluation
of quality requirements. It establishes criteria for the specification of software product quality requirements,
their measurement, and evaluation. It includes a quality model for aligning customer definitions of quality with
attributes of the development process. In addition, the series provides recommended measures of software
product quality attributes that can be used by developers, acquirers, and evaluators.
SQuaRE provides
• terms and definitions,
• reference models,
• general guide,
• individual division guides, and
• standards for requirements specification, planning and management, measurement and evaluation
purposes.
SQuaRE includes International Standards on quality model and measures, as well as on quality requirements
and evaluation.
SQuaRE replaces the current ISO/IEC 9126 series and the ISO/IEC 14598 series.
This International Standard is intended to be used in conjunction with the other parts of the SQuaRE series of
standards, and with the ISO/IEC 14598 series and the ISO/IEC 9126 series until superseded by the
ISO/IEC 25000 series of standards.
The SQuaRE series of standards consists of the following divisions under the general title Systems and
software product Quality Requirements and Evaluation:
• ISO/IEC 2500n - Quality Management Division,
• ISO/IEC 2501n - Quality Model Division,
© ISO/IEC 2011 – All rights reserved v
• ISO/IEC 2502n - Quality Measurement Division,
• ISO/IEC 2503n - Quality Requirements Division, and
• ISO/IEC 2504n - Quality Evaluation Division.
Annex A provides an explanation on levels of evaluation, aspects to be considered when defining evaluation
levels and suggestions on evaluation techniques to be applied according to the rank of evaluation level.
Annex B provides examples of evaluation methods.
Annex C provides a table showing relationships between some evaluation methods, possible cost rank and
effectiveness per software quality characteristics.
Annex D provides relationships between the software product quality evaluation process reference model and
the software and system life cycle processes.
Annex E provides an example template of an evaluation report.
Annex F provides the diagrams of inputs, outcomes, constraints and resources for each evaluation activity.
Figure 1 illustrates the organization of the SQuaRE series representing families of standards, further called
Divisions.
Quality Model
Division
2501n
Quality
Quality
Quality Management
Requirements Evaluation
Division
Division Division
2500n
2503n 2504n
Quality
Measurement Division
2502n
Extension Division
25050 - 25099
2503n 2504n
Figure 1 — Organization of the SQuaRE series of International Standards
The Divisions within the SQuaRE model are as follows.
• ISO/IEC 2500n - Quality Management Division. The International Standards that form this division
define all common models, terms and definitions referred to by all other standards from the SQuaRE
series. Referring paths (guidance through SQuaRE documents) and high-level practical suggestions in
applying proper standards to specific application cases offer help to all types of users. The division also
provides requirements and guidance for a supporting function which is responsible for the management
of software product requirements, specification and evaluation.
vi © ISO/IEC 2011 – All rights reserved
• ISO/IEC 2501n - Quality Model Division. The International Standard that forms this division presents
detailed quality models for software, quality in use and data. Practical guidance on the use of the quality
model is also provided.
• ISO/IEC 2502n - Quality Measurement Division. The International Standards that form this division
include a software product quality measurement reference model, mathematical definitions of quality
measures, and practical guidance for their application. This division presents internal measures of
software quality, external measures of software quality and quality in use measures. Quality measure
elements (QME) forming foundations for the latter measures are defined and presented.
• ISO/IEC 2503n - Quality Requirements Division. The International Standard that forms this division
helps specifying quality requirements. These quality requirements can be used in the process of quality
requirements, elicitation for a software product to be developed or as inputs for an evaluation process.
The requirements definition process is mapped to technical processes defined in ISO/IEC 15288.
• ISO/IEC 2504n - Quality Evaluation Division. The International Standards that form this division provide
requirements, recommendations and guidelines for software product evaluation, whether performed by
independent evaluators, acquirers or developers. The support for documenting a measure as an
evaluation module is also presented.
ISO/IEC 25050 to ISO/IEC 25099 are reserved to be used for SQuaRE extension International Standards
and/or Technical Reports.
This International Standard is part of the 2504n series on quality evaluation division that currently consists of
the following International Standards:
• ISO/IEC 25040 - Evaluation process: contains general requirements for specification and evaluation of
software quality and clarifies the general concepts. Provides a process description for evaluating quality of
software product and states the requirements for the application of thi
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.