ISO/IEC 14102:2008
(Main)Information technology — Guideline for the evaluation and selection of CASE tools
Information technology — Guideline for the evaluation and selection of CASE tools
Within systems and software engineering, Computer-Aided Software Engineering (CASE) tools represent a major part of the supporting technologies used to develop and maintain information technology systems. Their selection must be carried out with careful consideration of both the technical and management requirements. ISO/IEC 14102:2008 defines both a set of processes and a structured set of CASE tool characteristics for use in the technical evaluation and the ultimate selection of a CASE tool. It follows the software product evaluation model defined in ISO/IEC 14598-5:1998. ISO/IEC 14102:2008 adopts the general model of software product quality characteristics and subcharacteristics defined in ISO/IEC 9126-1:2001 and extends these when the software product is a CASE tool; it provides product characteristics unique to CASE tools. This larger set of characteristics is then organized into four groups: characteristics related to life cycle process functionality; characteristics related to CASE tool usage functionality; general quality characteristics; general characteristics not related to quality. This grouping provides a more manageable approach to the overall evaluation and selection process. The technical evaluation can indicate how well a CASE tool meets its user's stated requirements. It can also indicate how well the tool meets its claimed functionality. The objective of the technical evaluation process is to provide quantitative results on which the final selection can be based. Measurement assigns numbers (or other ratings) to attributes of entities; a major activity of evaluation is to obtain these measurements for use in selection. The final selection results should aim to achieve objectivity, repeatability and impartiality. These objectives and the confidence in the outcomes will in part depend on the resources allocated to the overall evaluation and selection process. The user of ISO/IEC 14102:2008 is asked to deal with these issues at an early stage.To be widely acceptable, these CASE tool evaluation and selection processes must be of value to the users of CASE tools, and to the suppliers of CASE to the community at large. The information outlined in ISO/IEC 14102:2008 should lead to more cost effective selections of CASE tools and to a greater uniformity in how CASE tool functions and features are described.
Technologies de l'information — Lignes directrices pour l'évaluation et la sélection d'outils CASE
General Information
Relations
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 14102
Second edition
2008-11-01
Information technology — Guideline for
the evaluation and selection of CASE
tools
Technologies de l'information — Lignes directrices pour l'évaluation et la
sélection d'outils CASE
Reference number
ISO/IEC 14102:2008(E)
©
ISO/IEC 2008
---------------------- Page: 1 ----------------------
ISO/IEC 14102:2008(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but shall
not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the
unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2008
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or ISO's
member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2008 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC 14102:2008(E)
Contents Page
1 Scope .1
2 Normative references .2
3 Terms and definitions .2
4 Abbreviated terms .3
5 Overview of evaluation and selection of CASE tools .3
5.1 Introduction of the evaluation and selection of CASE tools.3
5.2 Overview of the evaluation and selection of CASE tools.4
5.3 General process considerations.5
6 Preparation process .6
6.1 Overview.6
6.2 Goal setting .6
6.3 Establishing selection criteria.7
6.4 Project planning and control.7
7 Structuring process.8
7.1 Overview.8
7.2 Requirements definition.8
7.3 CASE tool information gathering.10
7.4 Identifying final candidate CASE tools.10
8 Evaluation process.11
8.1 Overview.11
8.2 Preparing for evaluation .11
8.3 Evaluating CASE tools .12
8.4 Evaluation reporting.13
9 CASE tool selection process.13
9.1 Overview.13
9.2 Preparing for selection.14
9.3 Applying the selection algorithm.14
9.4 Recommending a selection decision.14
9.5 Validating the selection decision.15
10 CASE tool characteristics.15
10.1 Overview.15
10.2 Characteristics related to life-cycle process functionality.15
10.3 Characteristics related to CASE tool usage functionality.24
10.4 General quality characteristics .27
10.5 General characteristics not related to quality .30
Annex A (informative) Considerations on the use of this International Standard.33
Annex B (informative) Examples of selection algorithms.35
Annex C (informative) Evaluation Report Contents.38
Bibliography .40
© ISO/IEC 2008 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/IEC 14102:2008(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees established
by the respective organization to deal with particular fields of technical activity. ISO and IEC technical
committees collaborate in fields of mutual interest. Other international organizations, governmental and
non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology,
ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 14102 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering.
This second edition cancels and replaces the first edition (ISO/IEC 14102:1995), which has been technically
revised.
iv © ISO/IEC 2008 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC 14102:2008(E)
Introduction
Within systems and software engineering, Computer-Aided Software Engineering (CASE) tools represent a
major part of the supporting technologies used to develop and maintain information technology systems. Their
selection must be carried out with careful consideration of both the technical and management requirements.
This International Standard defines both a set of processes and a structured set of CASE tool characteristics for
use in the technical evaluation and the ultimate selection of a CASE tool. It follows the software product
evaluation model defined in ISO/IEC 14598-5:1998.
This International Standard adopts the general model of software product quality characteristics and
sub-characteristics defined in ISO/IEC 9126-1:2001 and extends these when the software product is a CASE
tool; it provides product characteristics unique to CASE tools. This larger set of characteristics is then organized
into four groups. This grouping provides a more manageable approach to the overall evaluation and selection
process.
The technical evaluation can indicate how well a CASE tool meets its user’s stated requirements. It can also
indicate how well the tool meets its claimed functionality.
The objective of the technical evaluation process is to provide quantitative results on which the final selection
can be based. Measurement assigns numbers (or other ratings) to attributes of entities; a major activity of
evaluation is to obtain these measurements for use in selection. The final selection results should aim to achieve
objectivity, repeatability and impartiality. These objectives and the confidence in the outcomes will in part
depend on the resources allocated to the overall evaluation and selection process. The user of this International
Standard is asked to deal with these issues at an early stage.
To be widely acceptable, these CASE tool evaluation and selection processes must be of value to the users of
CASE tools and to the suppliers of CASE to the community at large. The information outlined in this International
Standard should lead to more cost-effective selections of CASE tools and to a greater uniformity in how CASE
tool functions and features are described.
© ISO/IEC 2008 – All rights reserved v
---------------------- Page: 5 ----------------------
INTERNATIONAL STANDARD ISO/IEC 14102:2008(E)
Information technology — Guideline for the evaluation and
selection of CASE tools
1 Scope
This International Standard gives guidelines for the evaluation and selection of CASE tools, covering a partial or full
portion of the software engineering life cycle. It establishes processes and activities to be applied for the evaluation of
CASE tools and selecting the most appropriate CASE tools from several candidates. These processes are generic,
and organizations must tailor them to meet organizational needs. The CASE tool evaluation and selection processes
should be viewed in the larger context of the organization’s technology adoption process.
This International Standard provides the following:
a) guidance on identifying organizational requirements for CASE tools;
b) guidance on mapping those requirements to CASE tool characteristics to be evaluated;
c) a process for selecting the most appropriate CASE tool from several tools, based on measurements of the
defined characteristics.
Primary users of this International Standard are organizations that intend to adopt CASE tools to support their
software life cycle processes. CASE tool suppliers can also use this International Standard to describe characteristics
of their CASE tools.
This International Standard is not intended to apply to:
a) software engineering frameworks whose purpose is to provide mechanisms for data, control and presentation
integration;
b) general purpose tools (e.g. word processors, spreadsheets) which can be used in software engineering activities,
nor CASE tools of very narrow scope or specific purpose (e.g. a compiler);
c) planning for the implementation of CASE tools within an organization (even though it is recognised that this is an
important subject).
NOTE A user of this International Standard can make the best possible selection of a CASE tool and yet have no guarantee
of a successful implementation. ISO/IEC TR 14471 Adoption of CASE Tools addresses this subject.
This International Standard contains a set of processes, activities, and tasks designed to be tailored. The tailoring
process is the selection of applicable processes, activities and tasks.
Compliance with this International Standard is defined as the performance of the processes, activities, and tasks
selected from this International Standard for the evaluation and selection project. Any organization imposing this
International Standard as a condition of trade is responsible for specifying the minimum set of required processes,
activities, and tasks which constitute compliance for a given application of this International Standard.
© ISO/IEC 2008 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/IEC 14102:2008(E)
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated references,
only the edition cited applies. For undated references, the latest edition of the referenced document (including any
amendments) applies.
ISO/IEC 12207:2008, Systems and software engineering — Software life cycle processes
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1
assessment
action of applying specific documented criteria to a specific software module, package or product for the purpose of
determining acceptance or release of the software module, package or product
3.2
CASE tool
software product that can assist software engineers by providing automated support for software life-cycle activities
as defined in ISO/IEC 12207:2008
NOTE 1 A CASE tool can provide support in only selected functional areas or in a wide variety of functional areas.
NOTE 2 CASE tools can be used in several modes:
— As stand-alone tools; in this case, only compatibility with environment elements should be addressed.
— In small groups that communicate directly with one another; it can be supposed that integration is predefined,
perhaps proprietarily.
— In the presence of a larger framework of the SEE; in this case the ability of the tool to use the relevant services of
the framework should be addressed.
3.3
measurement
use of a metric to assign a value (which may be a number or category) from a scale to an attribute of an entity
[ISO/IEC 14598-1:1999]
NOTE 1 Measurement can be qualitative when using categories. For example, some important attributes of software products,
e.g. the language of a source program (Java, C++, C, COBOL, etc.) are qualitative categories.
NOTE 2 Measurement can apply to metrics other than Software quality metrics.
NOTE 3 An Object can be measured directly, or can be information about or representations of the Object measured indirectly
by the application of metrics.
3.4
metric
defined measurement method and measurement scale
[ISO/IEC 14598-1:1999]
2 © ISO/IEC 2008 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC 14102:2008(E)
3.5
rating
action of mapping the measured value to the appropriate rating level
[ISO/IEC 14598-1:1999]
NOTE 1 Used to determine the rating level associated with the software for a specific quality characteristic.
NOTE 2 Rating and rating levels can be applied to characteristics other than quality characteristics.
3.6
rating level
scale point on an ordinal scale which is used to categorize a measurement scale
[ISO/IEC 14598-1:1999]
NOTE 1 The rating level enables software to be classified (rated) in accordance with the stated or implied needs (see 8.2).
NOTE 2 Appropriate rating levels can be associated with the different views of quality i.e. ‘Users’, ‘Managers’ or ‘Developers’.
3.7
SEE
Software Engineering Environment
environment which provides automated services for the engineering of software systems and related domains
(project management, process management, etc.)
[ISO/IEC 15940:2006]
NOTE 1 It includes the platform, system software, utilities, and CASE tools installed.
NOTE 2 The SEE architecture has two aspects:
— the CASE tools which provide facilities for supporting life-cycle processes, and
— general framework which provides a set of capabilities that offer common services used by the tools.
4 Abbreviated terms
CASE Computer Aided Software Engineering
GUI Graphical User Interface
5 Overview of evaluation and selection of CASE tools
5.1 Introduction of the evaluation and selection of CASE tools
This International Standard defines both a set of processes and a structured set of CASE tool characteristics for use
in the technical evaluation and the ultimate selection of a CASE tool. It follows the Software product evaluation model
defined in ISO/IEC 14598-5:1998.
This International Standard adopts the general model of software product quality characteristics and
sub-characteristics defined in ISO/IEC 9126-1:2001, and extends these when the software product is a CASE tool; it
provides product characteristics unique to CASE tools as described in 10.2 to 10.5. This larger set of characteristics
is then organized into four groups; they are characteristics related to life cycle process functionality, CASE tool usage
functionality, general quality and not related to quality. This grouping provides a more manageable approach to the
overall evaluation and selection process.
© ISO/IEC 2008 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/IEC 14102:2008(E)
The objective of the technical evaluation process is to provide quantitative results on which the final selection can be
based. Measurement assigns numbers (or other ratings) to attributes of entities; a major activity of evaluation is to
obtain these measurements for use in selection. The final selection results should aim to achieve objectivity,
repeatability and impartiality. These objectives and the confidence in the outcomes will in part depend on the
resources allocated to the overall evaluation and selection process. The user of this International Standard is asked
to deal with these issues at an early stage.
NOTE 1 Characteristic: An aspect of a product by which it can be described and evaluated. A characteristic may be refined into
multiple levels of sub-characteristics that bear on its ability to satisfy stated or implied needs.
NOTE 2 Atomic sub-characteristic: The highest level evaluation categories are called characteristics. Characteristics are
usually subdivided into sub-characteristics. Many sub-characteristics may be further subdivided into lower level
sub-characteristics. At the lowest level, when no further subdivision is appropriate, the sub-characteristics are referred to as
atomic sub-characteristics.
5.2 Overview of the evaluation and selection of CASE tools
This sub-clause illustrates an overview of the evaluation and selection of CASE tools discussed in this International
Standard as shown in Figure 1. Evaluation and selection of CASE tools includes four major processes:
⎯ Preparation Process
⎯ Structuring Process
⎯ Evaluation Process
⎯ Selection Process
Project
plan
High level
Preparation
Selection
goals
criteria
Evaluation
Structuring
plan
Evaluation
Structured
report
requirements
Evaluation
List of candidates
Selection
Selection
recommendation
Figure 1 — Overview of evaluation and selection of CASE tools
A key process is the structuring of a set of requirements against which candidate CASE tools are to be evaluated, and
upon which selection decisions will be based. The CASE tool characteristics defined in 10.2 to 10.5 forms the basis
for requirements structuring and play a central role in the overall process.
4 © ISO/IEC 2008 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC 14102:2008(E)
5.3 General process considerations
There are several considerations that apply to the processes described in this International Standard on a global
basis. The intent is for the user of this International Standard to tailor its application in such a way as to maximize the
probability of a successful evaluation and selection process, and minimize its cost and risk. See Annex A.
5.3.1 Sequencing of processes
This International Standard does not impose the sequence of process activities described above and in the following
clauses. It is up to the organization to select the relevant processes and activities needed to meet its evaluation and
selection goals.
The organization will decide which to employ, in what sequence, and with what degree of parallelism. The
sequencing of the processes’ activities is then documented in an evaluation project plan.
5.3.2 Reducing cost and risk
In general, organizations which apply this International Standard will want to minimize the cost of the entire evaluation
and selection process to the extent possible, while maintaining the level of effort necessary to select the most
appropriate CASE tool(s) for their use. These objectives may be addressed by minimizing the number of tools
evaluated, minimizing the cost of evaluating specific tools, and ensuring that the formality of the process is
appropriate to the organization.
The activities of CASE tool information gathering and identifying final candidates for selection (see Clause 9)
effectively allow the user of this International Standard to screen the available tools against the organization’s needs,
and eliminate from consideration tools which do not, or are not likely to, substantially address the organization’s
needs.
NOTE 1 It may be that the organization is unable to find any tool which appear likely to sufficiently meet its needs. In such a
case, the stated needs themselves should be re-examined, and if they are found to accurately reflect the organization’s actual
requirements for technology improvement, the overall evaluation and selection process may be abandoned. Similarly, if the final
candidate tools appear to be marginal in addressing the organization’s needs, the level of detail and formality of the subsequent
activities should be made to reflect the risk factor, and the organization should be prepared to not select a tool if the evaluation
process so indicates, as the typical cost of bringing a new tool into operational use is substantial.
Evaluations of candidate tools may have already been performed and be available to the organization. Such
information may be used to reduce the cost of candidate tool evaluation.
NOTE 2 Previous evaluations which have been performed on a different Version of the candidate tool may still yield useful
information. Similarly, evaluations which addressed a different set of organizational needs may still provide useful information.
This International Standard calls for the development of several plans and reports, and implicitly, for their review by
various personnel within the organization. In addition, activities are required to perform the four processes outlined.
The format and level of detail of the data products is left to the discretion of the organization, as is the level of effort
necessary to perform the activities.
NOTE 3 Some organizations may need to limit the scope, detail and formality of the processes to apply this International
Standard within existing resource constraints.
© ISO/IEC 2008 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/IEC 14102:2008(E)
6 Preparation process
6.1 Overview
The evaluation and selection processes require the agreement of management. In line with this agreement, a set of
goals for the introduction (or enhancement) of CASE technology will be established. A set of CASE tool selection
guidelines will be identified and a project plan developed. The process is shown in Figure 2.
Goal setting
Establishing
Selection
High level
goals
Selection
criteria
Project planning
& control
Project plan
Figure 2 — Overview of preparation process
6.2 Goal setting
The development of a set of realistic goals is a necessary first activity. In developing goals, both a rationale for
acquisition (why acquire a CASE tool) and a general policy for acquisition (what type of tool to acquire and how to do
it) should be developed.
NOTE Goal setting activities, including possibly the identification of selection criteria, may have already been performed as a
part of other efforts prior to formally entering the preparation process of evaluation and selection of CASE tools.
The following tasks should be performed.
a) Develop rationale for acquisition:
1) Review the organization’s current software development process, determining its maturity and areas of
concern.
2) Review the current state of CASE technology and observe trends for consideration as future reference
technology.
3) Compare the organization’s current practices to possible future practices if CASE tools are adopted and
identify areas of potential benefit.
4) Identify probable impacts of CASE tools on the organization; e.g., areas where training and education,
procedure guides, and technical support are needed to effectively deploy CASE technology.
b) Define goals and expectations:
1) Set overall goals (e.g., productivity improvement, quality improvement, enhanced process manageability).
2) Define evaluation and selection constraints (e.g., cost, schedule, resources).
6 © ISO/IEC 2008 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC 14102:2008(E)
3) Quantify and classify expectations (based upon goals).
c) Set general policy for acquisition:
1) Identify constraints on tool acquisition (e.g., implementation cost, schedule, other resources).
2) Develop alternate approaches to introducing/augmenting CASE technology (e.g., buy a tool, modify an
existing tool, develop a new tool).
3) Assess the feasibility of the various alternatives in light of organizational readiness, technical considerations,
performance specifications, and resources.
4) The goals and expectations established here will be used to guide subsequent activities in the overall
process and, finally, to validate the selection decision.
6.3 Establishing selection criteria
Based upon the goals and expectations developed above, selection criteria should be established:
a) Decompose the high level goals into a set of selection criteria to make the (go/no go) selection decision.
NOTE The selection criteria should be objective and quantitative. Each selection criterion should include some defined
threshold specified on which the major go/no go decision will be made during selection.
b) Define the relative importance of the selection criteria.
NOTE The relative importance of the selection criteria will be used to determine the weights assigned to tool
characteristics and sub-characteristics for evaluation.
c) Define the level of detail and the nature of the evaluation activities to be performed.
NOTE The nature of the evaluation activities covers the methods used in collecting the data. Reference, for example,
how the data are measured, collected with predefined criteria, or based upon subjective Observation.
d) Define the evaluation/selection scenario to be performed (see Annex A).
6.4 Project planning and control
Based upon the goals and selection criteria which have been established for the overall evaluation and selection
process, a project plan should be created and a control mechanism implemented. The plan and control mechanism
should be developed in accordance with the organization’s normal planning and control process, and it should
contain the following:
a) A project team organization with assigned responsibilities.
NOTE The skill of the evaluators will have an impact on the results of the evaluation and its applicability to the
organization. The evaluation personnel should be selected with this in mind, and the skill level of evaluators should be a
factor in assessing evaluation results. The evaluation team sho
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.