ISO/IEC 4944:2024
(Main)Information technology — User interfaces — Evaluating usability of natural user interfaces
Information technology — User interfaces — Evaluating usability of natural user interfaces
This document provides a framework, requirements and recommendations for evaluating the usability of a natural user interface (NUI) for systems, products or services. Measuring and reporting the usability of the NUI are also described. The usability evaluation focuses on the efficiency, effectiveness and satisfaction of the NUI. This document can also be applied to the usability evaluation of other user interfaces (UIs) for emerging technologies.
Technologie de l'information — Interfaces utilisateur — Cadre pour l'évaluation de l'utilisabilité d'une interface utilisateur naturelle
General Information
Standards Content (Sample)
International
Standard
ISO/IEC 4944
First edition
Information technology — User
2024-09
interfaces — Evaluating usability of
natural user interfaces
Technologie de l'information — Interfaces utilisateur — Cadre
pour l'évaluation de l'utilisabilité d'une interface utilisateur
naturelle
Reference number
© ISO/IEC 2024
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
© ISO/IEC 2024 – All rights reserved
ii
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
3.1 Terms related to natural user .1
3.2 Terms related to usability .2
4 Natural user interfaces (NUI). 2
5 Evaluating usability of a natural user interface (NUI) . 3
5.1 Framework .3
5.2 NUI as a system, product or service being evaluated .4
5.3 Evaluating usability for diverse NUI users .4
5.4 Evaluating usability for goals, tasks and interactions of NUI.5
5.5 Evaluating usability for diverse contexts of use of NUI .5
5.6 Components of usability.6
5.7 Evaluation based on use .6
6 Measuring usability of a NUI . 7
6.1 Purposes of and measures for evaluation .7
6.2 Measures for evaluating the user interface of NUI .8
6.2.1 Evaluating user interface of NUI for accomplishing tasks .8
6.2.2 Effectiveness measures .8
6.2.3 Efficiency measures.9
6.2.4 Satisfaction measures . .10
6.3 Measures for evaluating naturalness of interactions with NUI.10
6.3.1 Evaluating naturalness of interactions with NUI for accomplishing tasks .10
6.3.2 Effectiveness measures .10
6.3.3 Efficiency measures.11
6.3.4 Satisfaction measures . . 12
7 Reporting usability evaluation results .12
7.1 Content elements for reporting usability evaluations . 12
7.2 Format for reporting usability evaluations . 12
Bibliography .13
© ISO/IEC 2024 – All rights reserved
iii
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical activity.
ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations,
governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types
of document should be noted. This document was drafted in accordance with the editorial rules of the ISO/
IEC Directives, Part 2 (see www.iso.org/directives or www.iec.ch/members_experts/refdocs).
ISO and IEC draw attention to the possibility that the implementation of this document may involve the
use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of any
claimed patent rights in respect thereof. As of the date of publication of this document, ISO and IEC had not
received notice of (a) patent(s) which may be required to implement this document. However, implementers
are cautioned that this may not represent the latest information, which may be obtained from the patent
database available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall not be held
responsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO's adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www.iso.org/iso/foreword.html.
In the IEC, see www.iec.ch/understanding-standards.
This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 35, User interfaces.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national-committees.
© ISO/IEC 2024 – All rights reserved
iv
Introduction
As smart devices such as smart phones and smart speakers have become popular, new forms of user
interfaces have been introduced. One aspect of the new interfaces is mimicking natural ways of interacting
with the devices and utilizing speech, gestures, body language, eye gaze and other natural features of
human communication through natural user interfaces (NUIs), which allow for faster, smoother, intuitive,
and smarter interaction. However, while the concept of NUI is well defined, practical applications of NUI still
need to be developed. This is strongly related to the lack of usability of NUI in smart devices.
When specified users of systems, products or services achieve their specified goals with effectiveness,
efficiency and satisfaction in a specified context of use (as defined in ISO/IEC 9241-11), usability can be
evaluated. Effectiveness is considered as the ability of users to complete tasks with the system (e.g. the NUI).
Efficiency measures resources taken by users to complete specified tasks. Satisfaction measures comfort
levels of user experience along with acceptance.
It is important that the usability evaluation of NUI concentrates on recognizing the user's intention and
focuses on multiple modalities of information flows between the users and the devices.
This document describes a framework for evaluating the usability of NUI. Although the framework can be
applied in most general contexts, it is specifically focused on optimizing user experiences in terms of NUI.
© ISO/IEC 2024 – All rights reserved
v
International Standard ISO/IEC 4944:2024(en)
Information technology — User interfaces — Evaluating
usability of natural user interfaces
1 Scope
This document provides a framework, requirements and recommendations for evaluating the usability of
a natural user interface (NUI) for systems, products or services. Measuring and reporting the usability of
the NUI are also described. The usability evaluation focuses on the efficiency, effectiveness and satisfaction
of the NUI. This document can also be applied to the usability evaluation of other user interfaces (UIs) for
emerging technologies.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content constitutes
requirements of this document. For dated references, only the edition cited applies. For undated references,
the latest edition of the referenced document (including any amendments) applies.
ISO 9241-11, Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 9241-11 and the following apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
3.1 Terms related to natural user
3.1.1
user interface
UI
set of all the components of an interactive system that provide information and controls for the user to
accomplish specific tasks with the interactive system
[SOURCE: ISO 9241-110:2020, 3.10]
3.1.2
natural user interface
NUI
user interface where the user operates the interface through a manner related to everyday human
behaviours in a real world
Note 1 to entry: Natural refers to the user's expectations, behaviour and feelings that are involved in interacting with a NUI.
Note 2 to entry: Everyday human behaviours refer to behaviours that the user can be expected to have already learned.
Note 3 to entry: Interactions with NUIs typically model interactions performed in the natural world, including (but not
limited to) touch, gestures or vocal interactions, or a combination thereof.
© ISO/IEC 2024 – All rights reserved
3.1.3
natural user interface interaction
NUI interaction
user-system interaction making use of a natural user interface
3.1.4
naturalness
correspondence with everyday interactions between humans and the real-world objects with which they
interact
Note 1 to entry: Naturalness relates to everyday behaviours that the user can be expected to have already learned.
EXAMPLE To perform a “hello” gesture, the user refers to an object by pointing; to order something the user
issues a vocal command; or to agree the user nods in the affirmative or voices a vocal onomatopoeia for agreement.
3.2 Terms related to usability
3.2.1
interaction
user-system interaction
exchange of information between a user and an interactive system via the user interface to complete the
intended task
Note 1 to entry: Adapted from ISO 9241-110:2020, 3.11 definition of user-system in
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.