ISO/IEC 14102:2008

Information technology - Guideline for the evaluation and selection of CASE tools

Within systems and software engineering, Computer-Aided Software Engineering (CASE) tools represent a major part of the supporting technologies used to develop and maintain information technology systems. Their selection must be carried out with careful consideration of both the technical and management requirements. ISO/IEC 14102:2008 defines both a set of processes and a structured set of CASE tool characteristics for use in the technical evaluation and the ultimate selection of a CASE tool. It follows the software product evaluation model defined in ISO/IEC 14598-5:1998. ISO/IEC 14102:2008 adopts the general model of software product quality characteristics and subcharacteristics defined in ISO/IEC 9126-1:2001 and extends these when the software product is a CASE tool; it provides product characteristics unique to CASE tools. This larger set of characteristics is then organized into four groups: characteristics related to life cycle process functionality; characteristics related to CASE tool usage functionality; general quality characteristics; general characteristics not related to quality. This grouping provides a more manageable approach to the overall evaluation and selection process. The technical evaluation can indicate how well a CASE tool meets its user's stated requirements. It can also indicate how well the tool meets its claimed functionality. The objective of the technical evaluation process is to provide quantitative results on which the final selection can be based. Measurement assigns numbers (or other ratings) to attributes of entities; a major activity of evaluation is to obtain these measurements for use in selection. The final selection results should aim to achieve objectivity, repeatability and impartiality. These objectives and the confidence in the outcomes will in part depend on the resources allocated to the overall evaluation and selection process. The user of ISO/IEC 14102:2008 is asked to deal with these issues at an early stage.To be widely acceptable, these CASE tool evaluation and selection processes must be of value to the users of CASE tools, and to the suppliers of CASE to the community at large. The information outlined in ISO/IEC 14102:2008 should lead to more cost effective selections of CASE tools and to a greater uniformity in how CASE tool functions and features are described.

Technologies de l'information — Lignes directrices pour l'évaluation et la sélection d'outils CASE

General Information

• Status: Published
• Publication Date: 22-Oct-2008
• ICS: 35.080 - Software
• Technical Committee: ISO/IEC JTC 1/SC 7 - Software and systems engineering
• Drafting Committee: ISO/IEC JTC 1/SC 7/WG 4 - Tools and environment
• Current Stage: 9060 - Close of review
• Completion Date: 02-Dec-2030

Relations

• Revises: ISO/IEC 14102:1995 - Information technology - Guideline for the evaluation and selection of CASE tools
• Effective Date: 15-Apr-2008

Overview

ISO/IEC 14102:2008 - Information technology: Guideline for the evaluation and selection of CASE tools provides a structured, process-oriented approach for evaluating and selecting Computer‑Aided Software Engineering (CASE) tools within systems and software engineering. The standard defines a set of processes, measurable CASE tool characteristics and a framework for producing quantitative, repeatable and impartial selection decisions. It follows the software product evaluation model (ISO/IEC 14598‑5:1998) and adopts the software quality model of ISO/IEC 9126‑1:2001, extended for CASE tool‑specific characteristics.

Key Topics

• Process model for selection: preparatory, structuring, evaluation and selection processes (planning, requirements, information gathering, candidate identification, evaluation reporting and decision validation).
• CASE tool characteristics: organized into four groups:
  • Characteristics related to life‑cycle process functionality
  • Characteristics related to CASE tool usage functionality
  • General quality characteristics (based on ISO/IEC 9126)
  • General characteristics not related to quality
• Measurement and metrics: methods for assigning numeric or categorical values to attributes, mapping measurements to rating levels, and using these measurements in selection algorithms.
• Objectivity and repeatability: emphasis on quantitative evaluation, impartial reporting and resources planning to increase confidence in outcomes.
• Tailoring and scope: processes are generic and intended to be tailored to organizational needs; standard is focused on CASE tools (not general office tools, very narrow single‑purpose tools, nor implementation planning).

Applications

• Organizations planning to adopt CASE tools for parts or all of the software life cycle can use ISO/IEC 14102:2008 to:
  • Define organizational requirements and map them to measurable CASE tool characteristics.
  • Compare vendor claims against measured tool behaviour and functionality.
  • Apply consistent selection algorithms to produce defensible, repeatable procurement decisions.
  • Produce evaluation reports that support management and procurement.
• Primary users: IT managers, systems and software engineering teams, procurement officers, and CASE tool suppliers (to describe tool features in a standardized way).
• Benefits: more cost‑effective selections, improved comparability between tools, greater uniformity in how CASE features are described, and clearer evidence for procurement decisions.

Related standards

• ISO/IEC 9126‑1:2001 - software product quality model (adopted and extended)
• ISO/IEC 14598‑5:1998 - software product evaluation model (referenced)
• ISO/IEC 12207:2008 - software life cycle processes (normative reference)
• ISO/IEC TR 14471 - adoption of CASE tools (implementation considerations; noted but separate)

Keywords: ISO/IEC 14102:2008, CASE tools evaluation, CASE tool selection, software engineering standards, measurement and metrics, ISO/IEC 9126, ISO/IEC 14598.

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD 14102
Second edition
2008-11-01
Information technology — Guideline for
the evaluation and selection of CASE
tools
Technologies de l'information — Lignes directrices pour l'évaluation et la
sélection d'outils CASE
Reference number
©
ISO/IEC 2008
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but shall
not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the
unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.

©  ISO/IEC 2008
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or ISO's
member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail [e-mail]
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2008 – All rights reserved

Contents Page
1 Scope .1
2 Normative references .2
3 Terms and definitions .2
4 Abbreviated terms .3
5 Overview of evaluation and selection of CASE tools .3
5.1 Introduction of the evaluation and selection of CASE tools.3
5.2 Overview of the evaluation and selection of CASE tools.4
5.3 General process considerations.5
6 Preparation process .6
6.1 Overview.6
6.2 Goal setting .6
6.3 Establishing selection criteria.7
6.4 Project planning and control.7
7 Structuring process.8
7.1 Overview.8
7.2 Requirements definition.8
7.3 CASE tool information gathering.10
7.4 Identifying final candidate CASE tools.10
8 Evaluation process.11
8.1 Overview.11
8.2 Preparing for evaluation .11
8.3 Evaluating CASE tools .12
8.4 Evaluation reporting.13
9 CASE tool selection process.13
9.1 Overview.13
9.2 Preparing for selection.14
9.3 Applying the selection algorithm.14
9.4 Recommending a selection decision.14
9.5 Validating the selection decision.15
10 CASE tool characteristics.15
10.1 Overview.15
10.2 Characteristics related to life-cycle process functionality.15
10.3 Characteristics related to CASE tool usage functionality.24
10.4 General quality characteristics .27
10.5 General characteristics not related to quality .30
Annex A (informative) Considerations on the use of this International Standard.33
Annex B (informative) Examples of selection algorithms.35
Annex C (informative) Evaluation Report Contents.38
Bibliography .40

© ISO/IEC 2008 – All rights reserved iii

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees established
by the respective organization to deal with particular fields of technical activity. ISO and IEC technical
committees collaborate in fields of mutual interest. Other international organizations, governmental and
non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology,
ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 14102 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and systems engineering.
This second edition cancels and replaces the first edition (ISO/IEC 14102:1995), which has been technically
revised.
iv © ISO/IEC 2008 – All rights reserved

Introduction
Within systems and software engineering, Computer-Aided Software Engineering (CASE) tools represent a
major part of the supporting technologies used to develop and maintain information technology systems. Their
selection must be carried out with careful consideration of both the technical and management requirements.
This International Standard defines both a set of processes and a structured set of CASE tool characteristics for
use in the technical evaluation and the ultimate selection of a CASE tool. It follows the software product
evaluation model defined in ISO/IEC 14598-5:1998.
This International Standard adopts the general model of software product quality characteristics and
sub-characteristics defined in ISO/IEC 9126-1:2001 and extends these when the software product is a CASE
tool; it provides product characteristics unique to CASE tools. This larger set of characteristics is then organized
into four groups. This grouping provides a more manageable approach to the overall evaluation and selection
process.
The technical evaluation can indicate how well a CASE tool meets its user’s stated requirements. It can also
indicate how well the tool meets its claimed functionality.
The objective of the technical evaluation process is to provide quantitative results on which the final selection
can be based. Measurement assigns numbers (or other ratings) to attributes of entities; a major activity of
evaluation is to obtain these measurements for use in selection. The final selection results should aim to achieve
objectivity, repeatability and impartiality. These objectives and the confidence in the outcomes will in part
depend on the resources allocated to the overall evaluation and selection process. The user of this International
Standard is asked to deal with these issues at an early stage.
To be widely acceptable, these CASE tool evaluation and selection processes must be of value to the users of
CASE tools and to the suppliers of CASE to the community at large. The information outlined in this International
Standard should lead to more cost-effective selections of CASE tools and to a greater uniformity in how CASE
tool functions and features are described.

© ISO/IEC 2008 – All rights reserved v

INTERNATIONAL STANDARD ISO/IEC 14102:2008(E)

Information technology — Guideline for the evaluation and
selection of CASE tools
1 Scope
This International Standard gives guidelines for the evaluation and selection of CASE tools, covering a partial or full
portion of the software engineering life cycle. It establishes processes and activities to be applied for the evaluation of
CASE tools and selecting the most appropriate CASE tools from several candidates. These processes are generic,
and organizations must tailor them to meet organizational needs. The CASE tool evaluation and selection processes
should be viewed in the larger context of the organization’s technology adoption process.
This International Standard provides the following:
a) guidance on identifying organizational requirements for CASE tools;
b) guidance on mapping those requirements to CASE tool characteristics to be evaluated;
c) a process for selecting the most appropriate CASE tool from several tools, based on measurements of the
defined characteristics.
Primary users of this International Standard are organizations that intend to adopt CASE tools to support their
software life cycle processes. CASE tool suppliers can also use this International Standard to describe characteristics
of their CASE tools.
This International Standard is not intended to apply to:
a) software engineering frameworks whose purpose is to provide mechanisms for data, control and presentation
integration;
b) general purpose tools (e.g. word processors, spreadsheets) which can be used in software engineering activities,
nor CASE tools of very narrow scope or specific purpose (e.g. a compiler);
c) planning for the implementation of CASE tools within an organization (even though it is recognised that this is an
important subject).
NOTE A user of this International Standard can make the best possible selection of a CASE tool and yet have no guarantee
of a successful implementation. ISO/IEC TR 14471 Adoption of CASE Tools addresses this subject.
This International Standard contains a set of processes, activities, and tasks designed to be tailored. The tailoring
process is the selection of applicable processes, activities and tasks.
Compliance with this International Standard is defined as the performance of the processes, activities, and tasks
selected from this International Standard for the evaluation and selection project. Any organization imposing this
International Standard as a condition of trade is responsible for specifying the minimum set of required processes,
activities, and tasks which constitute compliance for a given application of this International Standard.
© ISO/IEC 2008 – All rights reserved 1

2 Normative references
The following referenced documents are indispensable for the application of this document. For dated references,
only the edition cited applies. For undated references, the latest edition of the referenced document (including any
amendments) applies.
ISO/IEC 12207:2008, Systems and software engineering — Software life cycle processes
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1
assessment
action of applying specific documented criteria to a specific software module, package or product for the purpose of
determining acceptance or release of the software module, package or product
3.2
CASE tool
software product that can assist software engineers by providing automated support for software life-cycle activities
as defined in ISO/IEC 12207:2008
NOTE 1 A CASE tool can provide support in only selected functional areas or in a wide variety of functional areas.
NOTE 2 CASE tools can be used in several modes:
— As stand-alone tools; in this case, only compatibility with environment elements should be addressed.
— In small groups that communicate directly with one another; it can be supposed that integration is predefined,
perhaps proprietarily.
— In the presence of a larger framework of the SEE; in this case the ability of the tool to use the relevant services of
the framework should be addressed.
3.3
measurement
use of a metric to assign a value (which may be a number or category) from a scale to an attribute of an entity
[ISO/IEC 14598-1:1999]
NOTE 1 Measurement can be qualitative when using categories. For example, some important attributes of software products,
e.g. the language of a source program (Java, C++, C, COBOL, etc.) are qualitative categories.
NOTE 2 Measurement can apply to metrics other than Software quality metrics.
NOTE 3 An Object can be measured directly, or can be information about or representations of the Object measured indirectly
by the application of metrics.
3.4
metric
defined measurement method and measurement scale
[ISO/IEC 14598-1:1999]
2 © ISO/IEC 2008 – All rights reserved

3.5
rating
action of mapping the measured value to the appropriate rating level
[ISO/IEC 14598-1:1999]
NOTE 1 Used to determine the rating level associated with the software for a specific quality characteristic.
NOTE 2 Rating and rating levels can be applied to characteristics other than quality characteristics.
3.6
rating level
scale point on an ordinal scale which is used to categorize a measurement scale
[ISO/IEC 14598-1:1999]
NOTE 1 The rating level enables software to be classified (rated) in accordance with the stated or implied needs (see 8.2).
NOTE 2 Appropriate rating levels can be associated with the different views of quality i.e. ‘Users’, ‘Managers’ or ‘Developers’.
3.7
SEE
Software Engineering Environment
environment which provides automated services for the engineering of software systems and related domains
(project management, process management, etc.)
[ISO/IEC 15940:2006]
NOTE 1 It includes the platform, system software, utilities, and CASE tools installed.
NOTE 2 The SEE architecture has two aspects:
— the CASE tools which provide facilities for supporting life-cycle processes, and
— general framework which provides a set of capabilities that offer common services used by the tools.
4 Abbreviated terms
CASE Computer Aided Software Engineering
GUI Graphical User Interface
5 Overview of evaluation and selection of CASE tools
5.1 Introduction of the evaluation and selection of CASE tools
This International Standard defines both a set of processes and a structured set of CASE tool characteristics for use
in the technical evaluation and the ultimate selection of a CASE tool. It follows the Software product evaluation model
defined in ISO/IEC 14598-5:1998.
This International Standard adopts the general model of software product quality characteristics and
sub-characteristics defined in ISO/IEC 9126-1:2001, and extends these when the software product is a CASE tool; it
provides product characteristics unique to CASE tools as described in 10.2 to 10.5. This larger set of characteristics
is then organized into four groups; they are characteristics related to life cycle process functionality, CASE tool usage
functionality, general quality and not related to quality. This grouping provides a more manageable approach to the
overall evaluation and selection process.
© ISO/IEC 2008 – All rights reserved 3

The objective of the technical evaluation process is to provide quantitative results on which the final selection can be
based. Measurement assigns numbers (or other ratings) to attributes of entities; a major activity of evaluation is to
obtain these measurements for use in selection. The final selection results should aim to achieve objectivity,
repeatability and impartiality. These objectives and the confidence in the outcomes will in part depend on the
resources allocated to the overall evaluation and selection process. The user of this International Standard is asked
to deal with these issues at an early stage.
NOTE 1 Characteristic: An aspect of a product by which it can be described and evaluated. A characteristic may be refined into
multiple levels of sub-characteristics that bear on its ability to satisfy stated or implied needs.
NOTE 2 Atomic sub-characteristic: The highest level evaluation categories are called characteristics. Characteristics are
usually subdivided into sub-characteristics. Many sub-characteristics may be further subdivided into lower level
sub-characteristics. At the lowest level, when no further subdivision is appropriate, the sub-characteristics are referred to as
atomic sub-characteristics.
5.2 Overview of the evaluation and selection of CASE tools
This sub-clause illustrates an overview of the evaluation and selection of CASE tools discussed in this International
Standard as shown in Figure 1. Evaluation and selection of CASE tools includes four major processes:
⎯ Preparation Process
⎯ Structuring Process
⎯ Evaluation Process
⎯ Selection Process
Project
plan
High level
Preparation
Selection
goals
criteria
Evaluation
Structuring
plan
Evaluation
Structured
report
requirements
Evaluation
List of candidates
Selection
Selection
recommendation
Figure 1 — Overview of evaluation and selection of CASE tools

A key process is the structuring of a set of requirements against which candidate CASE tools are to be evaluated, and
upon which selection decisions will be based. The CASE tool characteristics defined in 10.2 to 10.5 forms the basis
for requirements structuring and play a central role in the overall process.

4 © ISO/IEC 2008 – All rights reserved

5.3 General process considerations
There are several considerations that apply to the processes described in this International Standard on a global
basis. The intent is for the user of this International Standard to tailor its application in such a way as to maximize the
probability of a successful evaluation and selection process, and minimize its cost and risk. See Annex A.
5.3.1 Sequencing of processes
This International Standard does not impose the sequence of process activities described above and in the following
clauses. It is up to the organization to select the relevant processes and activities needed to meet its evaluation and
selection goals.
The organization will decide which to employ, in what sequence, and with what degree of parallelism. The
sequencing of the processes’ activities is then documented in an evaluation project plan.
5.3.2 Reducing cost and risk
In general, organizations which apply this International Standard will want to minimize the cost of the entire evaluation
and selection process to the extent possible, while maintaining the level of effort necessary to select the most
appropriate CASE tool(s) for their use. These objectives may be addressed by minimizing the number of tools
evaluated, minimizing the cost of evaluating specific tools, and ensuring that the formality of the process is
appropriate to the organization.
The activities of CASE tool information gathering and identifying final candidates for selection (see Clause 9)
effectively allow the user of this International Standard to screen the available tools against the organization’s needs,
and eliminate from consideration tools which do not, or are not likely to, substantially address the organization’s
needs.
NOTE 1 It may be that the organization is unable to find any tool which appear likely to sufficiently meet its needs. In such a
case, the stated needs themselves should be re-examined, and if they are found to accurately reflect the organization’s actual
requirements for technology improvement, the overall evaluation and selection process may be abandoned. Similarly, if the final
candidate tools appear to be marginal in addressing the organization’s needs, the level of detail and formality of the subsequent
activities should be made to reflect the risk factor, and the organization should be prepared to not select a tool if the evaluation
process so indicates, as the typical cost of bringing a new tool into operational use is substantial.
Evaluations of candidate tools may have already been performed and be available to the organization. Such
information may be used to reduce the cost of candidate tool evaluation.
NOTE 2 Previous evaluations which have been performed on a different Version of the candidate tool may still yield useful
information. Similarly, evaluations which addressed a different set of organizational needs may still provide useful information.
This International Standard calls for the development of several plans and reports, and implicitly, for their review by
various personnel within the organization. In addition, activities are required to perform the four processes outlined.
The format and level of detail of the data products is left to the discretion of the organization, as is the level of effort
necessary to perform the activities.
NOTE 3 Some organizations may need to limit the scope, detail and formality of the processes to apply this International
Standard within existing resource constraints.

© ISO/IEC 2008 – All rights reserved 5

6 Preparation process
6.1 Overview
The evaluation and selection processes require the agreement of management. In line with this agreement, a set of
goals for the introduction (or enhancement) of CASE technology will be established. A set of CASE tool selection
guidelines will be identified and a project plan developed. The process is shown in Figure 2.
Goal setting
Establishing
Selection
High level
goals
Selection
criteria
Project planning
& control
Project plan
Figure 2 — Overview of preparation process
6.2 Goal setting
The development of a set of realistic goals is a necessary first activity. In developing goals, both a rationale for
acquisition (why acquire a CASE tool) and a general policy for acquisition (what type of tool to acquire and how to do
it) should be developed.
NOTE Goal setting activities, including possibly the identification of selection criteria, may have already been performed as a
part of other efforts prior to formally entering the preparation process of evaluation and selection of CASE tools.
The following tasks should be performed.
a) Develop rationale for acquisition:
1) Review the organization’s current software development process, determining its maturity and areas of
concern.
2) Review the current state of CASE technology and observe trends for consideration as future reference
technology.
3) Compare the organization’s current practices to possible future practices if CASE tools are adopted and
identify areas of potential benefit.
4) Identify probable impacts of CASE tools on the organization; e.g., areas where training and education,
procedure guides, and technical support are needed to effectively deploy CASE technology.
b) Define goals and expectations:
1) Set overall goals (e.g., productivity improvement, quality improvement, enhanced process manageability).
2) Define evaluation and selection constraints (e.g., cost, schedule, resources).
6 © ISO/IEC 2008 – All rights reserved

3) Quantify and classify expectations (based upon goals).
c) Set general policy for acquisition:
1) Identify constraints on tool acquisition (e.g., implementation cost, schedule, other resources).
2) Develop alternate approaches to introducing/augmenting CASE technology (e.g., buy a tool, modify an
existing tool, develop a new tool).
3) Assess the feasibility of the various alternatives in light of organizational readiness, technical considerations,
performance specifications, and resources.
4) The goals and expectations established here will be used to guide subsequent activities in the overall
process and, finally, to validate the selection decision.
6.3 Establishing selection criteria
Based upon the goals and expectations developed above, selection criteria should be established:
a) Decompose the high level goals into a set of selection criteria to make the (go/no go) selection decision.
NOTE The selection criteria should be objective and quantitative. Each selection criterion should include some defined
threshold specified on which the major go/no go decision will be made during selection.
b) Define the relative importance of the selection criteria.
NOTE The relative importance of the selection criteria will be used to determine the weights assigned to tool
characteristics and sub-characteristics for evaluation.
c) Define the level of detail and the nature of the evaluation activities to be performed.
NOTE The nature of the evaluation activities covers the methods used in collecting the data. Reference, for example,
how the data are measured, collected with predefined criteria, or based upon subjective Observation.
d) Define the evaluation/selection scenario to be performed (see Annex A).
6.4 Project planning and control
Based upon the goals and selection criteria which have been established for the overall evaluation and selection
process, a project plan should be created and a control mechanism implemented. The plan and control mechanism
should be developed in accordance with the organization’s normal planning and control process, and it should
contain the following:
a) A project team organization with assigned responsibilities.
NOTE The skill of the evaluators will have an impact on the results of the evaluation and its applicability to the
organization. The evaluation personnel should be selected with this in mind, and the skill level of evaluators should be a
factor in assessing evaluation results. The evaluation team should be representative of the intended tool user group.
b) A set of operational goals obtained by decomposing the overall goals previously established.
c) A set of selection guidelines: weighted selection criteria, definition of level of detail and nature, and an evaluation
and selection scenario (see Annex A).
d) A schedule of activities and their tasks, along with an estimate of resource requirements and a cost estimate.
e) A means of monitoring and controlling the execution of the plan.
f) If developed, the project plan and control mechanism should be updated as the project evolves.
© ISO/IEC 2008 – All rights reserved 7

7 Structuring process
7.1 Overview
The structuring of the evaluation and selection activities can begin when a set of high level goals, selection guidelines,
and a project plan are in place. The structuring process begins with a requirements definition activity which is followed
by two parallel activities: the gathering of information on existing CASE tools, and the preparation of a list of candidate
CASE tools to be evaluated.
The Organization of CASE tool requirements will follow the four groups of CASE tool characteristics as outlined in
10.2 to 10.5. The major activities are shown in Figure 3.

Requirements High level
Definition goals
Selection
criteria
Structured CASE tool
information
Requirements
gathering
Identifying final
CASE tool List of
candidates
candidates
Figure 3 — Overview of structuring process

7.2 Requirements definition
7.2.1 Overview
During requirements definition, the requirements for the CASE tool are collected and organized into the CASE tool
characteristics as noted in 10.2 and 10.3 identify the major CASE specific characteristics, 10.4 identifies general
software quality characteristics, and 10.5 identifies a set of characteristics not related to quality. A comprehensive set
of requirements is necessary to select the most appropriate CASE tool, and the structuring process provides for
greater ease and repeatability in the evaluation process. Three activities are required.
7.2.2 Organizational information gathering
To be able to define a set of detailed requirements to be satisfied by the CASE tool, information about the
organization should be gathered, including:
a) Willingness of the organization to fully fund and implement CASE tool use.
b) Current software engineering environment within the organization, including data describing current hardware,
operating software, and tool use.
8 © ISO/IEC 2008 – All rights reserved

c) Types of software development projects undertaken by the organization include size, domain of application.
d) Characteristics and constraints of the target systems for which software is developed.
e) Specific expected impacts and improvements of CASE technology on the organization.
f) Requirements from potential tool users and end users.
g) Current organizational procurement policies.
This information is necessary to ensure the tool or tools are appropriate for use within the organization, they address
organizational needs, and needs perceived by their future users.
NOTE This information can be gathered in a number of ways, including Surveys and focus groups.
7.2.3 Requirements identification
The tool user’s requirements should deal with the question of what the CASE tool should do as well as its impact on
the existing environment. The following tasks should be performed in building the list of requirements:
a) Analyze the requirements and adjust the level of detail to which requirements are defined and measured.
b) Evaluate the current need for CASE tools while taking into consideration those projects where the CASE tool
may initially be used.
c) Identify desired methodology (e.g., process-oriented, data-oriented, object-oriented).
d) Identify portions of the life-cycle to be supported (e.g., planning, analysis, design).
e) Identify required functions of the CASE tool.
f) Identify required quality characteristics of the CASE tool.
g) Check the consistency of the requirements with the previously established goals.
NOTE These requirements represent the total set of organizational requirements. It is possible that no single CASE tool may
satisfy all of the requirements, but that individual CASE tools may satisfy a sufficient number to justify their use by the organization,
which may continue to search for tools to support remaining requirements.
7.2.4 Requirements structuring
The applicability of the user needs identified in 10.2 to 10.5, and any others which the organization may wish to add,
should be defined. The purpose of this structuring is to organize the requirements in such a way that the evaluation
can proceed more effectively. The tasks include:
a) Categorize the user requirements in terms of the organization of 10.2 to 10.5, and decompose them into detailed
specifications.
b) Select characteristics and specific sub-characteristics from 10.2 to 10.5 which can be evaluated to determine the
extent to which the CASE tool meets the detailed specifications.
NOTE The extent to which a CASE tool supports or implements a specific methodology may be a critical factor, and
should be seriously considered when selecting characteristics and sub-characteristics and weighting those
sub-characteristics.
c) Identify weights for the characteristics and sub-characteristics.
NOTE 1 The weights are applied to the ratings determined during the evaluation as part of the selection process, and reflect
the relative importance of the related selection criterion as determined during the preparation process.
© ISO/IEC 2008 – All rights reserved 9

NOTE 2 The assignment of weights is a subjective task which has a fundamental impact on the outcome of the entire
evaluation and selection process. The assignment of weights should reflect both the organization’s actual requirements and the
ability of the organization to evaluate the characteristic. See Annex B for further discussion.
NOTE 3 ISO/IEC 25051:2006 addresses quality requirements applicable to CASE tools when considered as software
packages, and should be consulted as part of the requirements structuring task. It provides additional guidance on a subset of the
quality requirements of ISO/IEC 9126-1:2001.
7.3 CASE tool information gathering
A general search of potential CASE tools to be evaluated is undertaken based upon the requirements and selection
criteria established. The activities of gathering information and identifying the candidate CASE tools may require
several iterations to quickly and efficiently identify the most promising tools for further evaluation. For the CASE tools
that appear most promising for further evaluation, additional and more detailed data that deal with their potential
acquisition are obtained. This additional information may help to quickly eliminate many tools, allowing attention to be
focused on the remaining candidates. Information to be obtained includes:
a) Vendor general information (e.g., business history, available support, plans & strategies).
b) Vendor’s specific product development strategy.
c) The tool’s cost (e.g., price, maintenance, modifications, training).
d) The hardware and software required to support tool use.
e) The hardware and software required to support final application/product use.
f) The training required for efficient tool use.
g) The tool’s functional capabilities.
h) The tool’s methodology and life-cycle support.
i) How the tool interfaces to external systems.
j) The number of users, existence of a user’s group, the users’ response to the tool.
k) The tool’s license mechanism (e.g., floating license, multi-user licenses, Cross platform licenses).
7.4 Identifying final candidate CASE tools
When the set of potential candidate tools has been identified, the final candidates for selection (those to be
evaluated) may be chosen. This is accomplished through the following tasks:
a) Establish a set of high-priority or critical, requirements to be met by CASE tools.
b) Compare the user’s functional requirements with the CASE tool’s functional capabilities, supporting
methodology, system environment.
c) Compare the managerial requirements with the CASE tool’s cost, available training and support.
d) Analyze the tool vendors’ user base, user response, support and business history.
e) Identify tools satisfying a sufficient number of high-priority or critical requirements which then become the final
candidates for formal evaluation. The results of the previous tasks provide the justification for the list of
candidates.
NOTE The tasks described in this subclause represent a “screening” of possible candidates to allow the organization to
identify the candidates most likely to be acceptable, given the organization’s requirements or suppliers abilities. The identification
of final candidates tan be performed in parallel with CASE tool information gathering, or the two activities may be iterated. The goal
is to reduce the cost of tool evaluation by only considering a screened set of final candidates during the evaluation process.
10 © ISO/IEC 2008 – All rights reserved

8 Evaluation process
8.1 Overview
Evaluation can begin when the structured requirements have been defined and a screened set of final candidates for
selection has been chosen. Final preparations will be made for evaluating the candidate CASE tools, including the
development of an evaluation plan. The evaluation activities are then performed and documented, resulting in a
profile of how each CASE tool measures up to the structured requirements. The objective is to produce the technical
evaluation reports necessary for the selection process, as shown in Figure 4.

Preparing for
Structured
Evaluation
Requirements
List of candidates
Evaluating
Evaluation plan
CASE tools
Evaluation
Evaluation report
Reporting
Figure 4 — Overview of evaluation process

8.2 Preparing for evaluation
To define the necessary level of detail prior to beginning evaluation activities, final preparations are necessary. Based
upon the list of candidate CASE tools and the structured requirements, the following tasks should be accomplished:
a) For each atomic sub-characteristic, define or select one or more metrics and define the details of their use.
NOTE ISO/IEC JTC 1 SC7 Working Group 6 developed the following technical reports regarding metrics:
ISO/IEC TR 9126-2:2003, ISO/IEC TR 9126-3:2003 and ISO/IEC TR 9126-4:2004 (see Bibliography).
b) Set the rating levels and identify the means by which the levels will be generated or computed.
NOTE A measured metric value (e.g., average lines of code per module = 274) must then be assigned a rating value
(e.g., 1.3 on a scale of 0 to 4). The means by which rating levels are obtained from measurements must be identified.
c) Define the assessment characteristics for evaluation, establishing what is acceptable, taking into consideration
the rating levels previously defined and the context of use of the product.
d) Identify and schedule all activities that must be performed as part of the evaluation process.
NOTE Activities include preparing any data sets necessary for the evaluation, obtaining tool documentation and an
instance of the tool to be evaluated, providing evaluators any necessary training in tool use, hands-on tool use, recording of
tool outputs, and analysis of results.
© ISO/IEC 2008 – All rights reserved 11

In some cases, a Bench Mark Test (BMT) may be a part of the evaluation process. The recommended approach for
a BMT includes:
⎯ Identify the required critical tool functions.
⎯ Identify a test project or sample program to be the basis for the BMT.
⎯ Develop a BMT scenario, defining inputs and expected outputs.
To focus evaluation activities and provide for traceability of the evaluation process, develop an evaluation plan that
includes the information above.
8.3 Evaluating CASE tools
8.3.1 Overview
The software is evaluated in comparison with each of the chosen characteristics. Evaluation is a process of
measurement, rating and assessment.
8.3.2 Measurement
Measurements can be made based upon information obtained by examining the CASE tool itself, or information
about it, through the following types of tasks:
a) Examining the vendor-supplied documentation.
b) Examining the source code and other intermediate products, if available.
c) Interviewing actual users of the software.
d) Viewing demonstrations and interviewing demonstrators.
e) Executing test cases.
f) Applying to test projects.
g) Examining results of previous evaluations (whether in-house, third party, or other evaluations).
h) Performing a BMT on the candidate tools and analysing the results.
Measurement values may be binary, based on a continuous scale (quantifiable), or textual. There are both objective
and subjective characteristics.
NOTE Objective characteristics are those which permit independent and repeatable test or metric. Subjective characteristics
are those for which no independent and repeatable test or metric exists (e.g. fitness of the user interface to the culture of the user).
For objective characteristics, the evaluation should be made by a repeatable procedure such that another evaluator
would be able to produce the same results. During evaluation, if test cases are used, a uniform, predefined, and
documented set of cases should be used.
For subjective characteristics, the evaluation should be performed repeatedly by more than one person or group, who
will discuss and agree upon results.
The evaluation results should be recorded in a quantified manner, where possible, together with textual justification,
where applicable.
12 © ISO/IEC 2008 – All rights reserved

8.3.3 Rating
In the rating task, each measured value is rated against the scale of values defined in the evaluation plan. Rating
levels are either directly generated or computed according to previously defined algorithms.
NOTE It is possible that requirements may be revised during the evaluation, and this may require revision of rating scales.
8.3.4 Assessment
Based upon the resulting ratings and the previously defined assessment criteria, assess the sub-characteristics and
characteristics. In accordance with the selection guidelines and the evaluation plan, ratings should be aggregated up
to the characteristic level.
8.4 Evaluation reporting
The end result of th
...