IEC PAS 62883:2014
(Main)The universAAL framework for user interaction in multimedia AAL spaces
The universAAL framework for user interaction in multimedia AAL spaces
IEC/PAS 62883:2014(E) specifies a framework for adaptive handling of explicit interaction among humans and AAL spaces. This is based on a differentiation between explicit and implicit interaction as a consequence of the paradigm shift from Human-Computer Interaction to Human-Environment Interaction, further explained in the definition of the latter term. As a framework, a main subject matter of the specification is the identification of relevant areas for further standardization, thereby also looking at the interrelationships among the identified areas. The PAS also provides a first extensible specification in some of those areas.
General Information
Standards Content (Sample)
IEC PAS 62883
Edition 1.0 2014-03
PUBLICLY AVAILABLE
SPECIFICATION
PRE-STANDARD
colour
inside
The universAAL framework for user interaction in multimedia AAL spaces
IEC PAS 62883:2014-03(en)
---------------------- Page: 1 ----------------------
THIS PUBLICATION IS COPYRIGHT PROTECTED
Copyright © 2014 IEC, Geneva, Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form
or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from
either IEC or IEC's member National Committee in the country of the requester. If you have any questions about IEC
copyright or have an enquiry about obtaining additional rights to this publication, please contact the address below or
your local IEC member National Committee for further information.IEC Central Office Tel.: +41 22 919 02 11
3, rue de Varembé Fax: +41 22 919 03 00
CH-1211 Geneva 20 info@iec.ch
Switzerland www.iec.ch
About the IEC
The International Electrotechnical Commission (IEC) is the leading global organization that prepares and publishes
International Standards for all electrical, electronic and related technologies.About IEC publications
The technical content of IEC publications is kept under constant review by the IEC. Please make sure that you have the
latest edition, a corrigenda or an amendment might have been published.IEC Catalogue - webstore.iec.ch/catalogue Electropedia - www.electropedia.org
The stand-alone application for consulting the entire The world's leading online dictionary of electronic and
bibliographical information on IEC International Standards, electrical terms containing more than 30 000 terms and
Technical Specifications, Technical Reports and other definitions in English and French, with equivalent terms in 14
documents. Available for PC, Mac OS, Android Tablets and additional languages. Also known as the International
iPad. Electrotechnical Vocabulary (IEV) online.IEC publications search - www.iec.ch/searchpub IEC Glossary - std.iec.ch/glossary
The advanced search enables to find IEC publications by a More than 55 000 electrotechnical terminology entries in
variety of criteria (reference number, text, technical English and French extracted from the Terms and Definitions
committee,…). It also gives information on projects, replaced clause of IEC publications issued since 2002. Some entries
and withdrawn publications. have been collected from earlier publications of IEC TC 37,
77, 86 and CISPR.IEC Just Published - webstore.iec.ch/justpublished
Stay up to date on all new IEC publications. Just Published IEC Customer Service Centre - webstore.iec.ch/csc
details all new publications released. Available online and If you wish to give us your feedback on this publication or
also once a month by email. need further assistance, please contact the Customer Service
Centre: csc@iec.ch.---------------------- Page: 2 ----------------------
IEC PAS 62883
Edition 1.0 2014-03
PUBLICLY AVAILABLE
SPECIFICATION
PRE-STANDARD
colour
inside
The universAAL framework for user interaction in multimedia AAL spaces
INTERNATIONAL
ELECTROTECHNICAL
COMMISSION
PRICE CODE
ICS 13.180; 33.160 ISBN 978-2-8322-1452-7
Warning! Make sure that you obtained this publication from an authorized distributor.
® Registered trademark of the International Electrotechnical Commission---------------------- Page: 3 ----------------------
– 2 – IEC PAS 62883:2014 IEC 2014
CONTENTS
FOREWORD ........................................................................................................................... 4
INTRODUCTION ..................................................................................................................... 6
1 Scope .............................................................................................................................. 8
2 Normative references ...................................................................................................... 9
3 Terms, definitions and abbreviations ............................................................................... 9
3.1 Terms and definitions .............................................................................................. 9
3.2 Abbreviation .......................................................................................................... 12
4 The Specification of the universAAL UI Framework ........................................................ 12
4.1 Overview............................................................................................................... 12
4.2 Analysis of the relationships between UI Handlers and I/O Channels .................... 13
4.3 Dialog descriptions ............................................................................................... 15
4.4 The Adaptation Concept........................................................................................ 18
4.4.1 Overview ....................................................................................................... 18
4.4.2 Responsibilities of Applications ...................................................................... 18
4.4.3 Responsibilities of UI handlers ....................................................................... 19
4.4.4 Responsibilities on the brokerage layer ......................................................... 20
4.5 Provisions of the UI Framework ............................................................................ 22
4.5.1 Introduction ................................................................................................... 22
4.5.2 The UI Bus and its brokerage protocols ......................................................... 22
4.5.3 The dialog manager and its role in assisting the UI Bus ................................. 28
4.5.4 The Resource Manager ................................................................................. 30
Annex A (informative) Use cases ......................................................................................... 31
A.1 Use Case: Supporting rich human computer interaction ........................................ 31
A.2 Use Case: Healthy Lifestyle Service Package Use Case (universAAL) .................. 32
Annex B (informative) An Overview of the universAAL Project ............................................. 33
Bibliography .......................................................................................................................... 35
Figure 1 – Paradigm shift from HCI to HEI .............................................................................. 6
Figure 2 – logical separation of application and presentation layers ........................................ 7
Figure 3 – The scope of the specified UI framework marked by the green colour .................... 8
Figure 4 – The notion of AAL spaces ...................................................................................... 9
Figure 5 – The need of smart environments to utilize channels for bridging between the
physical world and the virtual realm ...................................................................................... 10
Figure 7 – The notion of a smart environment ....................................................................... 11
Figure 8 – An open system for plugging in applications and UI handlers independently
from each other .................................................................................................................... 13
Figure 9 – Channel binding by I/O devices ............................................................................ 13
Figure 10 – The notion of a driver with the case of a UPNP-aware driver .............................. 14
Figure 11 – The case of a universAAL aware driver .............................................................. 14
Figure 12 – Possible relationship between UI handlers and drivers ....................................... 15
Figure 13 – The dialog package based on the notion of a form ............................................. 16
Figure 14 – A possible graphical visualization of the mapping between dialog types
and the predefined standard groups ...................................................................................... 17
Figure 15 – The universAAL framework for supporting adaptivity, which builds on top of
the universAAL context and service buses (see footnote 4) .................................................. 18
---------------------- Page: 4 ----------------------IEC PAS 62883:2014 IEC 2014 – 3 –
Figure 16 – A model for describing access impairments ........................................................ 20
Figure 17 – Summary of the adaptation parameters .............................................................. 21
Figure 18 – The components comprising the universAAL UI framework ................................. 22
Figure 19 – The main messages exchanged on the UI Bus ................................................... 23
Figure 20 – The notion of a UI request as constructed by applications .................................. 23
Figure 21 – Overview of the sequence of actions when the priority check is positive ............. 24
Figure 22 – The case of switching to a new UI handler when handling changes in the
context .................................................................................................................................. 25
Figure 24 – The abstract class to be extended by applications that want to send UI
requests to the UI bus ........................................................................................................... 28
Figure 25 – The abstract class to be extended by UI handlers that accept UI requests
forwarded by the UI bus for rendering ................................................................................... 28
Figure 26 – The interface of the UI Bus ................................................................................. 28
Figure 27 – Access to the resources managed by the RM ..................................................... 30
Figure B.1 – Project ID Card ................................................................................................. 33
Figure B.2 – The three pillars of the universAAL platform ...................................................... 34
---------------------- Page: 5 ----------------------– 4 – IEC PAS 62883:2014 IEC 2014
INTERNATIONAL ELECTROTECHNICAL COMMISSION
____________
THE UNIVERSAAL FRAMEWORK FOR USER
INTERACTION IN MULTIMEDIA AAL SPACES
FOREWORD
1) The International Electrotechnical Commission (IEC) is a worldwide organization for standardization comprising
all national electrotechnical committees (IEC National Committees). The object of IEC is to promote
international co-operation on all questions concerning standardization in the electrical and electronic fields. To
this end and in addition to other activities, IEC publishes International Standards, Technical Specifications,
Technical Reports, Publicly Available Specifications (PAS) and Guides (hereafter referred to as “IEC
Publication(s)”). Their preparation is entrusted to technical committees; any IEC National Committee interested
in the subject dealt with may participate in this preparatory work. International, governmental and non-
governmental organizations liaising with the IEC also participate in this preparation. IEC collaborates closely
with the International Organization for Standardization (ISO) in accordance with conditions determined by
agreement between the two organizations.2) The formal decisions or agreements of IEC on technical matters express, as nearly as possible, an international
consensus of opinion on the relevant subjects since each technical committee has representation from all
interested IEC National Committees.3) IEC Publications have the form of recommendations for international use and are accepted by IEC National
Committees in that sense. While all reasonable efforts are made to ensure that the technical content of IEC
Publications is accurate, IEC cannot be held responsible for the way in which they are used or for any
misinterpretation by any end user.4) In order to promote international uniformity, IEC National Committees undertake to apply IEC Publications
transparently to the maximum extent possible in their national and regional publications. Any divergence
between any IEC Publication and the corresponding national or regional publication shall be clearly indicated in
the latter.5) IEC itself does not provide any attestation of conformity. Independent certification bodies provide conformity
assessment services and, in some areas, access to IEC marks of conformity. IEC is not responsible for any
services carried out by independent certification bodies.6) All users should ensure that they have the latest edition of this publication.
7) No liability shall attach to IEC or its directors, employees, servants or agents including individual experts and
members of its technical committees and IEC National Committees for any personal injury, property damage or
other damage of any nature whatsoever, whether direct or indirect, or for costs (including legal fees) and
expenses arising out of the publication, use of, or reliance upon, this IEC Publication or any other IEC
Publications.8) Attention is drawn to the Normative references cited in this publication. Use of the referenced publications is
indispensable for the correct application of this publication.9) Attention is drawn to the possibility that some of the elements of this IEC Publication may be the subject of
patent rights. IEC shall not be held responsible for identifying any or all such patent rights.
A PAS is a technical specification not fulfilling the requirements for a standard, but made
available to the public.IEC-PAS 62883 has been processed by IEC technical committee 100: Audio, video and
multimedia systems and equipment.The text of this PAS is based on the This PAS was approved for
following document: publication by the P-members of the
committee concerned as indicated in
the following document
Draft PAS Report on voting
100/2189/PAS 100/2228/RVD
Following publication of this PAS, which is a pre-standard publication, the technical committee
or subcommittee concerned may transform it into an International Standard.---------------------- Page: 6 ----------------------
IEC PAS 62883:2014 IEC 2014 – 5 –
This PAS shall remain valid for an initial maximum period of 3 years starting from the
publication date. The validity may be extended for a single period up to a maximum of 3 years,
at the end of which it shall be published as another type of normative document, or shall be
withdrawn.IMPORTANT – The 'colour inside' logo on the cover page of this publication indicates
that it contains colours which are considered to be useful for the correctunderstanding of its contents. Users should therefore print this document using a
colour printer.---------------------- Page: 7 ----------------------
– 6 – IEC PAS 62883:2014 IEC 2014
INTRODUCTION
Ambient Assisted Living (AAL) strives to ensure the independence, safety, wellbeing and
autonomy of users by using ICT, including multimedia systems and equipment and audio /
video communication, for creating intelligent living environments that react to the needs of
users by providing relevant assistance. Such intelligent environments can be labelled as AAL
Spaces, which are characterized by a number of devices that can be stationary, mobile or
embedded within other objects. Multiple users can find themselves in an AAL space
simultaneously, possibly moving around within the AAL space, and entering and leaving it
dynamically. These characteristics introduce new challenges when it comes to handling
interaction with users in AAL spaces.With the assumption that people are surrounded by highly distributed systems of networked
interactive devices, AAL intensifies the paradigm shift from Human-Computer Interaction (HCI)
to Human-Environment Interaction (HEI). One of the main challenges of HEI is to keep the
multiplicity of functional units hidden to humans while making the functionality provided by
them easily available based on natural ways of interaction. Instead of controlling each device
separately, users should be able to interact with a whole device ensemble as one single unit
and articulate goals instead of looking for functionality at the level of each single device
separately (Figure 1).Figure 1 – Paradigm shift from HCI to HEI
Another important challenge for designers and developers of systems in AAL spaces is that
interaction with applications can take place through a variety of devices at different locations
with different capabilities in terms of serving a single user privately or not, supported
modalities, modality-specific parameters such as screen size and resolution, power
consumption, etc., which implies the need in AAL spaces to logically separate the application
layer from the presentation layer (Figure 2).Consequently, applications have to use abstract user interfaces that are device-, modality-,
and layout-neutral and allow to postpone the rendering of the user interface to the execution-
time, which makes it possible to interact with users in a personalized and situation-aware way.
The separation of concerns also facilitates the creation of clean programming interfaces
based on an open and flexible architecture that have to enable the plug-and-play of both
applications and user interaction handlers (UI handlers), and allows UI handlers to serve
arbitrary applications.The resulted openness complements the openness supported by IEC 62481-2 that enables
the sharing of multimedia content and streams within an ensemble of devices. It adds the
---------------------- Page: 8 ----------------------IEC PAS 62883:2014 IEC 2014 – 7 –
perspective of sharing the input and output channels provided by those devices to the DLNA
perspective of content sharing.Figure 2 – logical separation of application and presentation layers
_____________
This understanding of the term I/O channel is based on the actual roles of devices that enable interaction with
human users: a display provides a visual output channel, a loudspeaker, an audio output channel, and a
microphone, an audio input channel.---------------------- Page: 9 ----------------------
– 8 – IEC PAS 62883:2014 IEC 2014
THE UNIVERSAAL FRAMEWORK FOR USER
INTERACTION IN MULTIMEDIA AAL SPACES
1 Scope
This Publicly Available Specification (PAS) specifies a framework for adaptive handling of
explicit interaction among humans and AAL spaces. This is based on a differentiation
between explicit and implicit interaction as a consequence of the paradigm shift from Human-
Computer Interaction to Human-Environment Interaction, further explained in the definition of
the latter term.As a framework, a main subject matter of the specification is the identification of relevant
areas for further standardization, thereby also looking at the interrelationships among the
identified areas. The PAS also provides a first extensible specification in some of those areas.
The proposed UI framework has been derived from the logical separation of application and
presentation layers as depicted by Figure 2, and encompasses the following elements
(Figure 3):• Analysis of the relationships between UI handlers and I/O devices without specifying
possible languages, models, or abstract APIs for interaction with these devices, as there
are certain international standardization activities that go in this direction ;• the language and model for describing application-specific dialogs / user interfaces as part
of UI requests made by applications to the UI framework;• the adaptation concept and parameters needed to achieve adaptive UI and the way they
affect UI requests; and• Protocols used by the UI framework to broker between UI handlers and applications as
pluggable components.I/O Device
not covered
UI Handler
UI Request
(forwarded)
UI Response
Brokerage
UI Request
UI Response
(forwarded)
Application
Figure 3 – The scope of the specified UI framework marked by the green colour
_____________
For example [3] on representing user input coming from input devices.
---------------------- Page: 10 ----------------------
IEC PAS 62883:2014 IEC 2014 – 9 –
2 Normative references
The following documents, in whole or in part, are normatively referenced in this document and
are indispensable for its application. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including anyamendments) applies.
IEC 62481-2, Digital living network alliance (DLNA) home networked device interoperability
guidelines – Part 2: DLNA media formatsISO/IEC Guide 71:2001, Guidelines for standards developers to address the needs of older
persons and persons with disabilitiesISO 9241-11:1998, Ergonomic requirements for office work with visual display terminals
(VDTs) – Part 11: Guidance on usabilityISO 9241-110:2006, Ergonomics of human-system interaction - Part 110: Dialogue principles
3 Terms, definitions and abbreviationsFor the purposes of this document, the following terms and definitions apply.
3.1 Terms and definitions
3.1.1
ambient assisted living
AAL
products, services, environments and facilities used to support those whose independence,
safety, wellbeing and autonomy are compromised by their physical or mental status
Note 1 to entry: In this PAS, AAL refers to the usage of ICT, including multimedia systems and equipment and
audio / video communication, for creating intelligent living environments that react to the needs of users by
providing relevant assistance.[SOURCE: ISO/IEC GUIDE 71:2001]
3.1.2
AAL service user
person who interacts with an AAL system or is connected with an AAL system
3.1.3
AAL space
denotes a smart environment that provides AAL services to its users
Figure 4 – The notion of AAL spaces
---------------------- Page: 11 ----------------------
– 10 – IEC PAS 62883:2014 IEC 2014
3.1.4
channel
denotes the bridging passage that smart environments need between the physical world and
the virtual realm (Figure 5). Channels are realized by devices. Depending on the kind of
channel opened, a channel might be called a sensing channel (realized by sensors), an acting
channel (realized by actuators), an input channel (realized by microphones, keyboards, etc.),
or an output channel (realized by displays, loudspeakers, etc.). The latter two types of
channels might be referred to as I/O channelsFigure 5 – The need of smart environments to utilize channels
for bridging between the physical world and the virtual realm
3.1.5
human-environment interaction
interaction in smart environments is generally divided into two major areas: implicit and
explicit interactionImplicit interaction is mostly about using sensing channels for observation of happenings, with
or without involvement of humans, in order to recognize in the background relevant situations
to which the environment might be able to react in a desired way.Explicit interaction, on the contrary, is about situations in which a human user seeks the
dialog with the environment or vice versa, for instance when the user instructs that the
brightness of the TV should be increased or when the environment notifies the user that it is
time to take a certain medicine. Explicit interaction takes place by utilizing input and output
channels provided by I/O devices.3.1.6
I/O device
an abbreviation for input and / or output device. A device that provides an input and / or
output channel for facilitating explicit interaction between a smart environment and its human
users. Input devices, such as a microphone, a keyboard, or a mouse, can capture an
instruction or response that is provided by a human user and represent it in terms of data in
the virtual realm (Figure 6). Upon receive of data within the virtual realm that is intended to be
presented to human users, output devices, such as displays and loudspeakers, can make it
perceivable to the addressed humans---------------------- Page: 12 ----------------------
IEC PAS 62883:2014 IEC 2014 – 11 –
Figure 6 – The role of devices in realizing bridging channels
3.1.7
multimodal UI handler
denotes UI handlers that perform the interaction by using multiple channels in parallel,
possibly with a hybrid mix supporting different modalities3.1.8
smart environment
denotes an environment centred on its human users in which a set of embedded networked
artefacts, both hardware (HW) and software (SW), collectively realize the paradigm of
Ambient Intelligence, mainly by providing for context-awareness and personalization, adaptive
reactivity, and anticipatory pro-activityFigure 7 – The notion of a smart environment
3.1.9
user
person who interacts with the product, service or environment
3.1.10
user interface
all components of an interactive system (software or hardware) that provide information
and/or control functions for the user to accomplish specific tasks with the interactive system
---------------------- Page: 13 ----------------------– 12 – IEC PAS 62883:2014 IEC 2014
3.2 Abbreviation
AAL Ambient Assisted Living
API Application programming interface
Cf confer
DLNA Digital Living Network Alliance
e.g. for example
etc. et cetera
GUI Graphical user interface
HCI Human-Computer Interaction
HEI Human-Environment Interaction
HTML Hypertext Markup Language
HW Hardware
ICT Information and communications' technology
i.e. id est, that is to say
I/O channels Input/Output channel
OWL Web Ontology Language
RDF Resource Description Framework
RM Resource Manager
SW Software
UI User Interaction
UI model User Interaction model
UIML User Interface Markup Language
uAAL universAAL
UPNP Universal Plug and Play
W3C World Wide Web Consortium
XForms W3C Standard at http://www.w3.org/TR/xforms/
XML Extensible Markup Language
XPath XML Path Language; W3C Standard at http://www.w3.org/TR/xpath/
4 The Specification of the universAAL UI Framework
4.1 Overview
As depicted in the Introduction to this PAS, the universAAL UI Framework for Multimedia AAL
spaces is based on separating the application layer from the presentation layer by introducing
(1) a new type of software components called UI handlers and (2) a brokerage mechanism
between the application and presentation layers. The goal is to achieve an open system in
which arbitrary applications as well as UI handlers can be plugged in independently from each
other while enabling applications to benefit from support for multimodal interaction both by the
framework itself and by the UI handlers:---------------------- Page: 14 ----------------------
IEC PAS 62883:2014 IEC 2014 – 13 –
Figure 8 – An open system for plugging in applications
and UI handlers independently from each other
4.2 Analysis of the relationships between UI Handlers and I/O Channels
UI handlers are seen responsible for handling requests for interacting with human users. To
do so, they need to utilize the available I/O channels but are not necessarily supposed to take
over the binding and management of those channels. On the other side, the "manager" of a
single input or output channel is usually not able to handle the whole of a UI request sent by
an application because applications often wait for user response in the context of some
information to be presented to the user; as a result. At least one output channel and one input
channel are supposed to be utilized simultaneously by the same UI handler in order to be able
to interpret user input in the context of the output presented to the user.Therefore, it is important to have a more precise look at the relationship between UI handlers
and I/O channels: Section 3.1 states that channels are realized by certain devices but it does
not say anything about their binding and availability in the virtual realm. In the concept map in
Figure 9, those definitions are extended by introducing the concept of Channel Binding; the
same concept can be used for both input and output channels:Figure 9 – Channel binding by I/O devices
The device that realizes a channel has to have an integrated solution for providing access to
the channel. This might be very low-level, at hard- and firmware level to exchange certain bits
and bytes through a physical connector interface (called Embedded Binding) or already
software using higher-level protocols and abstractions (called Driver). Drivers might be
provided by third-party and / or run on third-party devices. Like Figure 10 denotes, a driver
might wrap another driver in order to comply with higher-level abstractions (e.g. a “UPNP
driver”) for binding a special-purpose device that is not readily “UPNP-aware” usually uses
---------------------- Page: 15 ----------------------– 14 – IEC PAS 62883:2014 IEC 2014
“Embedded Bindings” or “Legacy Drivers” for providing wrappers that interact at the UPNP
level of abstraction.In particular, the higher-level abstractions might only make sense in the context of a
framework created for certain purposes (cf. a windowing system that uses a mouse driver and
already interprets the mouse e...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.