ISO/IEC 25062:2006
(Main)Software engineering — Software product Quality Requirements and Evaluation (SQuaRE) — Common Industry Format (CIF) for usability test reports
Software engineering — Software product Quality Requirements and Evaluation (SQuaRE) — Common Industry Format (CIF) for usability test reports
ISO/IEC 25062:2006 provides a standard method for reporting usability test findings. The format is designed for reporting results of formal usability tests in which quantitative measurements were collected, and is particularly appropriate for summative/comparative testing. The CIF does not indicate how to perform a usability test but provides guidance on how to report the results of a usability test. The CIF targets two audiences: usability professionals and stakeholders in an organization. Stakeholders can use the usability data to help make informed decisions concerning the release of software products or the procurement of such products. The format includes the following elements: the description of the product, the goals of the test, the test participants the tasks the users were asked to perform, the experimental design of the test, the method or process by which the test was conducted, the usability measures and data collection methods, and the numerical results.
Ingénierie du logiciel — Exigences de qualité du produit logiciel et évaluation (SQuaRE) — Format commun de l'industrie (CIF) pour les rapports d'essai de rentabilité
General Information
Relations
Standards Content (Sample)
INTERNATIONAL ISO/IEC
STANDARD 25062
First edition
2006-04-01
Corrected version
2006-10-01
Software engineering — Software product
Quality Requirements and Evaluation
(SQuaRE) — Common Industry Format
(CIF) for usability test reports
Ingénierie du logiciel — Exigences de qualité du produit logiciel et
évaluation (SQuaRE) — Format commun de l'industrie (CIF) pour les
rapports d'essai de rentabilité
Reference number
©
ISO/IEC 2006
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
© ISO/IEC 2006
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2006 – All rights reserved
Contents Page
Foreword. iv
Introduction . v
1 Scope . 1
2 Conformance. 1
3 Normative references . 2
4 Terms and definitions. 2
5 Report Format . 4
5.1 Title page . 4
5.2 Executive summary . 4
5.3 Introduction . 5
5.4 Method . 5
5.5 Results . 11
5.6 Appendices. 14
Annex A (informative) Checklist . 15
Annex B (informative) Glossary. 19
Annex C (informative) Report Template. 23
Annex D (informative) Example . 29
Bibliography . 46
© ISO/IEC 2006 – All rights reserved iii
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 25062 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 7, Software and system engineering.
This corrected version of ISO/IEC 25062 incorporates the following correction:
Figure 1, Appendix C, has been included on page 43.
iv © ISO/IEC 2006 – All rights reserved
Introduction
Usability of software is a key factor in predicting successful deployment of that software. Software
manufacturers subject software to usability testing at various stages in a product’s development; some
companies that purchase software also test products for usability before making purchasing decisions. Testing
often involves (1) subjects who are representative of the target population of users of the software, (2)
representative tasks, and (3) measures of efficiency, effectiveness and subjective satisfaction. When this type
of experimental situation exists, the testing is termed summative, i.e., the results can be expressed as
statistically meaningful measures of central tendency (e.g. mean or median) and variability (e.g. standard
deviation). The Common Industry Format (CIF) for Usability Test Reports is intended for use by usability
professionals to report the results of summative usability testing.
The CIF standardizes the types of information that are captured about testing with users. The level of detail
allows the same or another organization to replicate the test procedure. The major variables are user
demographics, task descriptions, context of the test, including the equipment used, the environment in which
the test is conducted, and the protocol by which the subjects and the test administrator(s) interact, as well as
the particular metrics chosen to code the findings of the study.
The CIF is intended to replace the proprietary formats employed by companies that perform usability testing,
both vendors and purchasers of software. Until now there has been no standard format for reporting usability
testing results. Advantages of using a standardized reporting format include (1) a reduction in training time for
usability staff since an individual only needs to learn to use one form regardless of how many companies he
works for and (2) enhanced potential for increased communication between vendors and purchasing
organizations since readers of CIF-compliant reports will share a common language and expectations.
The purpose of this International Standard is to facilitate incorporation of usability as part of the procurement
decision-making process for interactive software products so that it is easier to judge whether a product meets
usability goals. Examples of decisions include purchasing, upgrading and automating. It provides a common
format for human factors engineers and usability professionals in supplier companies to report the methods
and results of usability tests to customer organizations.
Audience
The Common Industry Format (CIF) is meant to be used by usability professionals within supplier
organizations to generate reports that can be used by customer organizations in the CIF report. The CIF is
also meant to be used by customer organizations to verify that a particular report is CIF-compliant. The
Usability Test Report itself is intended for two types of readers:
— Usability professionals in customer organizations who are evaluating both the technical merit of
usability tests and the usability of the products; and
— Other technical professionals and managers who are using the test results to make business decisions.
The CIF may also be used within a single organization if a formal report of a summative usability test needs to
be generated. In this case additional material such as a list of detailed findings may be included.
The report is in two main sections, an Executive Summary and a main body. The main body contains the
Methods and Results sections and is aimed at the first audience above. These sections (1) describe the test
methodology and results in sufficient technical detail to allow replication by another organization if the test is
repeated, and (2) support application of test data to questions about the product’s expected costs and benefits.
Understanding and interpreting these sections will require technical background in human factors or usability
engineering for optimal use. The second audience is directed to the Introduction, which provides summary
information for non-usability professionals and managers. The Introduction may also be of general interest to
other computing professionals. Decision makers without usability engineering expertise may find the
information in the main body to be useful but should rely on expert interpretation when necessary.
Organization
Clause 1 describes the scope of this specification and the conformance criteria. Clause 4 provides definitions
© ISO/IEC 2006 – All rights reserved v
of the terms used throughout the document. Clause 5 is the main description of the specification.
Additional Information
Annex A provides a checklist that can be used to ensure inclusion of required and recommended information.
A glossary is provided in Annex B to define terminology used in the report format description. A Word template
for report production can be found at: http://www.ncits.org/ref-docs/CIF/CIF_template.dot. A printed version of
the template can be found in Annex C. An example is provided in Annex D illustrating how the format is used
followed by an informative bibliography.
vi © ISO/IEC 2006 – All rights reserved
INTERNATIONAL STANDARD ISO/IEC 25062:2006(E)
Software engineering — Software product Quality
Requirements and Evaluation (SQuaRE) — Common Industry
Format (CIF) for usability test reports
1 Scope
This International Standard is intended to be used to report the measures obtained from a test of usability as
defined in ISO 9241-11: effectiveness, efficiency and satisfaction in a specified context of use.
NOTE Metrics for other more-detailed usability requirements can be found in ISO/IEC 9126 parts 2 and 3.
This International Standard is intended to be used by:
⎯ usability professionals within supplier organizations to generate reports that can be used by customer
organizations;
⎯ customer organizations to verify that a particular report conforms to this International Standard;
⎯ human factors or other usability professionals in customer organizations who are evaluating both the
technical merit of usability tests and the usability of the products; and
⎯ other technical professionals and managers in the customer organization who are using the test results to
make business decisions about product suitability and purchase.
The Executive Summary and Introduction in 5.2 and 5.3 provide summary information for nonusability
professionals and managers.
Subclauses 5.4 and 5.5 describe the test methodology and results in technical detail suitable for replication,
and also support application of test data to questions about the product’s expected costs and benefits.
Understanding and interpreting these sections will require technical background in human factors or usability
engineering for optimal use.
The report format assumes sound practice [1, 2] has been followed in the design and execution of the test.
Test procedures which produce measures that summarize usability should be used, i.e. the test is summative
in nature. Some usability evaluation methods, such as formative tests, are intended to identify problems rather
than produce measures; the format is not structured to support the results of such testing methods.
2 Conformance
A usability test report conforms to this International Standard if it complies with all the requirements in this
International Standard (stated as “shall”). The recommendations (stated as “should”) should be implemented
whenever appropriate.
This International Standard specifies the minimum information that should be provided. Additional information
may be included. For example, if an organization finds that an additional list of findings is useful, the list may
be included even though it is not specified as part of a conformant CIF report.
© ISO/IEC 2006 – All rights reserved 1
ISO/IE
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.