IEC PAS 62883:2014

The universAAL framework for user interaction in multimedia AAL spaces

IEC/PAS 62883:2014(E) specifies a framework for adaptive handling of explicit interaction among humans and AAL spaces. This is based on a differentiation between explicit and implicit interaction as a consequence of the paradigm shift from Human-Computer Interaction to Human-Environment Interaction, further explained in the definition of the latter term. As a framework, a main subject matter of the specification is the identification of relevant areas for further standardization, thereby also looking at the interrelationships among the identified areas. The PAS also provides a first extensible specification in some of those areas.

General Information

• Status: Published
• Publication Date: 12-Sep-2021
• ICS: 01 - GENERALITIES. TERMINOLOGY. STANDARDIZATION. DOCUMENTATION
• ICS: 13.180 - Ergonomics
• ICS: 33.160.99 - Other audio, video and audiovisual equipment
• Technical Committee: TA 16 - Active Assisted Living (AAL), wearable electronic devices and technologies, accessibility and user interfaces
• Current Stage: WPUB - Publication withdrawn
• Completion Date: 13-Sep-2021

Standards Content (Sample)

IEC PAS 62883
Edition 1.0 2014-03
PUBLICLY AVAILABLE
SPECIFICATION
PRE-STANDARD
colour
inside
The universAAL framework for user interaction in multimedia AAL spaces
IEC PAS 62883:2014-03(en)
---------------------- Page: 1 ----------------------
THIS PUBLICATION IS COPYRIGHT PROTECTED
Copyright © 2014 IEC, Geneva, Switzerland

All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form

or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from

either IEC or IEC's member National Committee in the country of the requester. If you have any questions about IEC

copyright or have an enquiry about obtaining additional rights to this publication, please contact the address below or

your local IEC member National Committee for further information.
IEC Central Office Tel.: [phone]
3, rue de Varembé Fax: [phone]
CH-1211 Geneva 20 [e-mail]
Switzerland www.iec.ch
About the IEC

The International Electrotechnical Commission (IEC) is the leading global organization that prepares and publishes

International Standards for all electrical, electronic and related technologies.
About IEC publications

The technical content of IEC publications is kept under constant review by the IEC. Please make sure that you have the

latest edition, a corrigenda or an amendment might have been published.
IEC Catalogue - webstore.iec.ch/catalogue Electropedia - www.electropedia.org

The stand-alone application for consulting the entire The world's leading online dictionary of electronic and

bibliographical information on IEC International Standards, electrical terms containing more than 30 000 terms and

Technical Specifications, Technical Reports and other definitions in English and French, with equivalent terms in 14

documents. Available for PC, Mac OS, Android Tablets and additional languages. Also known as the International

iPad. Electrotechnical Vocabulary (IEV) online.

IEC publications search - www.iec.ch/searchpub IEC Glossary - std.iec.ch/glossary

The advanced search enables to find IEC publications by a More than 55 000 electrotechnical terminology entries in

variety of criteria (reference number, text, technical English and French extracted from the Terms and Definitions

committee,…). It also gives information on projects, replaced clause of IEC publications issued since 2002. Some entries

and withdrawn publications. have been collected from earlier publications of IEC TC 37,

77, 86 and CISPR.
IEC Just Published - webstore.iec.ch/justpublished

Stay up to date on all new IEC publications. Just Published IEC Customer Service Centre - webstore.iec.ch/csc

details all new publications released. Available online and If you wish to give us your feedback on this publication or

also once a month by email. need further assistance, please contact the Customer Service

Centre: [e-mail].
---------------------- Page: 2 ----------------------
IEC PAS 62883
Edition 1.0 2014-03
PUBLICLY AVAILABLE
SPECIFICATION
PRE-STANDARD
colour
inside
The universAAL framework for user interaction in multimedia AAL spaces
INTERNATIONAL
ELECTROTECHNICAL
COMMISSION
PRICE CODE
ICS 13.180; 33.160 ISBN 978-2-8322-1452-7

Warning! Make sure that you obtained this publication from an authorized distributor.

® Registered trademark of the International Electrotechnical Commission
---------------------- Page: 3 ----------------------
– 2 – IEC PAS 62883:2014  IEC 2014
CONTENTS

FOREWORD ........................................................................................................................... 4

INTRODUCTION ..................................................................................................................... 6

1 Scope .............................................................................................................................. 8

2 Normative references ...................................................................................................... 9

3 Terms, definitions and abbreviations ............................................................................... 9

3.1 Terms and definitions .............................................................................................. 9

3.2 Abbreviation .......................................................................................................... 12

4 The Specification of the universAAL UI Framework ........................................................ 12

4.1 Overview............................................................................................................... 12

4.2 Analysis of the relationships between UI Handlers and I/O Channels .................... 13

4.3 Dialog descriptions ............................................................................................... 15

4.4 The Adaptation Concept........................................................................................ 18

4.4.1 Overview ....................................................................................................... 18

4.4.2 Responsibilities of Applications ...................................................................... 18

4.4.3 Responsibilities of UI handlers ....................................................................... 19

4.4.4 Responsibilities on the brokerage layer ......................................................... 20

4.5 Provisions of the UI Framework ............................................................................ 22

4.5.1 Introduction ................................................................................................... 22

4.5.2 The UI Bus and its brokerage protocols ......................................................... 22

4.5.3 The dialog manager and its role in assisting the UI Bus ................................. 28

4.5.4 The Resource Manager ................................................................................. 30

Annex A (informative) Use cases ......................................................................................... 31

A.1 Use Case: Supporting rich human computer interaction ........................................ 31

A.2 Use Case: Healthy Lifestyle Service Package Use Case (universAAL) .................. 32

Annex B (informative) An Overview of the universAAL Project ............................................. 33

Bibliography .......................................................................................................................... 35

Figure 1 – Paradigm shift from HCI to HEI .............................................................................. 6

Figure 2 – logical separation of application and presentation layers ........................................ 7

Figure 3 – The scope of the specified UI framework marked by the green colour .................... 8

Figure 4 – The notion of AAL spaces ...................................................................................... 9

Figure 5 – The need of smart environments to utilize channels for bridging between the

physical world and the virtual realm ...................................................................................... 10

Figure 7 – The notion of a smart environment ....................................................................... 11

Figure 8 – An open system for plugging in applications and UI handlers independently

from each other .................................................................................................................... 13

Figure 9 – Channel binding by I/O devices ............................................................................ 13

Figure 10 – The notion of a driver with the case of a UPNP-aware driver .............................. 14

Figure 11 – The case of a universAAL aware driver .............................................................. 14

Figure 12 – Possible relationship between UI handlers and drivers ....................................... 15

Figure 13 – The dialog package based on the notion of a form ............................................. 16

Figure 14 – A possible graphical visualization of the mapping between dialog types

and the predefined standard groups ...................................................................................... 17

Figure 15 – The universAAL framework for supporting adaptivity, which builds on top of

the universAAL context and service buses (see footnote 4) .................................................. 18

---------------------- Page: 4 ----------------------
IEC PAS 62883:2014  IEC 2014 – 3 –

Figure 16 – A model for describing access impairments ........................................................ 20

Figure 17 – Summary of the adaptation parameters .............................................................. 21

Figure 18 – The components comprising the universAAL UI framework ................................. 22

Figure 19 – The main messages exchanged on the UI Bus ................................................... 23

Figure 20 – The notion of a UI request as constructed by applications .................................. 23

Figure 21 – Overview of the sequence of actions when the priority check is positive ............. 24

Figure 22 – The case of switching to a new UI handler when handling changes in the

context .................................................................................................................................. 25

Figure 24 – The abstract class to be extended by applications that want to send UI

requests to the UI bus ........................................................................................................... 28

Figure 25 – The abstract class to be extended by UI handlers that accept UI requests

forwarded by the UI bus for rendering ................................................................................... 28

Figure 26 – The interface of the UI Bus ................................................................................. 28

Figure 27 – Access to the resources managed by the RM ..................................................... 30

Figure B.1 – Project ID Card ................................................................................................. 33

Figure B.2 – The three pillars of the universAAL platform ...................................................... 34

---------------------- Page: 5 ----------------------
– 4 – IEC PAS 62883:2014  IEC 2014
INTERNATIONAL ELECTROTECHNICAL COMMISSION
____________
THE UNIVERSAAL FRAMEWORK FOR USER
INTERACTION IN MULTIMEDIA AAL SPACES
FOREWORD

1) The International Electrotechnical Commission (IEC) is a worldwide organization for standardization comprising

all national electrotechnical committees (IEC National Committees). The object of IEC is to promote

international co-operation on all questions concerning standardization in the electrical and electronic fields. To

this end and in addition to other activities, IEC publishes International Standards, Technical Specifications,

Technical Reports, Publicly Available Specifications (PAS) and Guides (hereafter referred to as “IEC

Publication(s)”). Their preparation is entrusted to technical committees; any IEC National Committee interested

in the subject dealt with may participate in this preparatory work. International, governmental and non-

governmental organizations liaising with the IEC also participate in this preparation. IEC collaborates closely

with the International Organization for Standardization (ISO) in accordance with conditions determined by

agreement between the two organizations.

2) The formal decisions or agreements of IEC on technical matters express, as nearly as possible, an international

consensus of opinion on the relevant subjects since each technical committee has representation from all

interested IEC National Committees.

3) IEC Publications have the form of recommendations for international use and are accepted by IEC National

Committees in that sense. While all reasonable efforts are made to ensure that the technical content of IEC

Publications is accurate, IEC cannot be held responsible for the way in which they are used or for any

misinterpretation by any end user.

4) In order to promote international uniformity, IEC National Committees undertake to apply IEC Publications

transparently to the maximum extent possible in their national and regional publications. Any divergence

between any IEC Publication and the corresponding national or regional publication shall be clearly indicated in

the latter.

5) IEC itself does not provide any attestation of conformity. Independent certification bodies provide conformity

assessment services and, in some areas, access to IEC marks of conformity. IEC is not responsible for any

services carried out by independent certification bodies.

6) All users should ensure that they have the latest edition of this publication.

7) No liability shall attach to IEC or its directors, employees, servants or agents including individual experts and

members of its technical committees and IEC National Committees for any personal injury, property damage or

other damage of any nature whatsoever, whether direct or indirect, or for costs (including legal fees) and

expenses arising out of the publication, use of, or reliance upon, this IEC Publication or any other IEC

Publications.

8) Attention is drawn to the Normative references cited in this publication. Use of the referenced publications is

indispensable for the correct application of this publication.

9) Attention is drawn to the possibility that some of the elements of this IEC Publication may be the subject of

patent rights. IEC shall not be held responsible for identifying any or all such patent rights.

A PAS is a technical specification not fulfilling the requirements for a standard, but made

available to the public.

IEC-PAS 62883 has been processed by IEC technical committee 100: Audio, video and

multimedia systems and equipment.
The text of this PAS is based on the This PAS was approved for
following document: publication by the P-members of the
committee concerned as indicated in
the following document
Draft PAS Report on voting
100/2189/PAS 100/2228/RVD

Following publication of this PAS, which is a pre-standard publication, the technical committee

or subcommittee concerned may transform it into an International Standard.
---------------------- Page: 6 ----------------------
IEC PAS 62883:2014  IEC 2014 – 5 –

This PAS shall remain valid for an initial maximum period of 3 years starting from the

publication date. The validity may be extended for a single period up to a maximum of 3 years,

at the end of which it shall be published as another type of normative document, or shall be

withdrawn.

IMPORTANT – The 'colour inside' logo on the cover page of this publication indicates

that it contains colours which are considered to be useful for the correct

understanding of its contents. Users should therefore print this document using a

colour printer.
---------------------- Page: 7 ----------------------
– 6 – IEC PAS 62883:2014  IEC 2014
INTRODUCTION

Ambient Assisted Living (AAL) strives to ensure the independence, safety, wellbeing and

autonomy of users by using ICT, including multimedia systems and equipment and audio /

video communication, for creating intelligent living environments that react to the needs of

users by providing relevant assistance. Such intelligent environments can be labelled as AAL

Spaces, which are characterized by a number of devices that can be stationary, mobile or

embedded within other objects. Multiple users can find themselves in an AAL space

simultaneously, possibly moving around within the AAL space, and entering and leaving it

dynamically. These characteristics introduce new challenges when it comes to handling

interaction with users in AAL spaces.

With the assumption that people are surrounded by highly distributed systems of networked

interactive devices, AAL intensifies the paradigm shift from Human-Computer Interaction (HCI)

to Human-Environment Interaction (HEI). One of the main challenges of HEI is to keep the

multiplicity of functional units hidden to humans while making the functionality provided by

them easily available based on natural ways of interaction. Instead of controlling each device

separately, users should be able to interact with a whole device ensemble as one single unit

and articulate goals instead of looking for functionality at the level of each single device

separately (Figure 1).
Figure 1 – Paradigm shift from HCI to HEI

Another important challenge for designers and developers of systems in AAL spaces is that

interaction with applications can take place through a variety of devices at different locations

with different capabilities in terms of serving a single user privately or not, supported

modalities, modality-specific parameters such as screen size and resolution, power

consumption, etc., which implies the need in AAL spaces to logically separate the application

layer from the presentation layer (Figure 2).

Consequently, applications have to use abstract user interfaces that are device-, modality-,

and layout-neutral and allow to postpone the rendering of the user interface to the execution-

time, which makes it possible to interact with users in a personalized and situation-aware way.

The separation of concerns also facilitates the creation of clean programming interfaces

based on an open and flexible architecture that have to enable the plug-and-play of both

applications and user interaction handlers (UI handlers), and allows UI handlers to serve

arbitrary applications.

The resulted openness complements the openness supported by IEC 62481-2 that enables

the sharing of multimedia content and streams within an ensemble of devices. It adds the

---------------------- Page: 8 ----------------------
IEC PAS 62883:2014  IEC 2014 – 7 –

perspective of sharing the input and output channels provided by those devices to the DLNA

perspective of content sharing.
Figure 2 – logical separation of application and presentation layers
_____________

This understanding of the term I/O channel is based on the actual roles of devices that enable interaction with

human users: a display provides a visual output channel, a loudspeaker, an audio output channel, and a

microphone, an audio input channel.
---------------------- Page: 9 ----------------------
– 8 – IEC PAS 62883:2014  IEC 2014
THE UNIVERSAAL FRAMEWORK FOR USER
INTERACTION IN MULTIMEDIA AAL SPACES
1 Scope

This Publicly Available Specification (PAS) specifies a framework for adaptive handling of

explicit interaction among humans and AAL spaces. This is based on a differentiation

between explicit and implicit interaction as a consequence of the paradigm shift from Human-

Computer Interaction to Human-Environment Interaction, further explained in the definition of

the latter term.

As a framework, a main subject matter of the specification is the identification of relevant

areas for further standardization, thereby also looking at the interrelationships among the

identified areas. The PAS also provides a first extensible specification in some of those areas.

The proposed UI framework has been derived from the logical separation of application and

presentation layers as depicted by Figure 2, and encompasses the following elements

(Figure 3):

• Analysis of the relationships between UI handlers and I/O devices without specifying

possible languages, models, or abstract APIs for interaction with these devices, as there

are certain international standardization activities that go in this direction ;

• the language and model for describing application-specific dialogs / user interfaces as part

of UI requests made by applications to the UI framework;

• the adaptation concept and parameters needed to achieve adaptive UI and the way they

affect UI requests; and

• Protocols used by the UI framework to broker between UI handlers and applications as

pluggable components.
I/O Device
not covered
UI Handler
UI Request
(forwarded)
UI Response
Brokerage
UI Request
UI Response
(forwarded)
Application
Figure 3 – The scope of the specified UI framework marked by the green colour
_____________
For example [3] on representing user input coming from input devices.
---------------------- Page: 10 ----------------------
IEC PAS 62883:2014  IEC 2014 – 9 –
2 Normative references

The following documents, in whole or in part, are normatively referenced in this document and

are indispensable for its application. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any
amendments) applies.

IEC 62481-2, Digital living network alliance (DLNA) home networked device interoperability

guidelines – Part 2: DLNA media formats

ISO/IEC Guide 71:2001, Guidelines for standards developers to address the needs of older

persons and persons with disabilities

ISO 9241-11:1998, Ergonomic requirements for office work with visual display terminals

(VDTs) – Part 11: Guidance on usability

ISO 9241-110:2006, Ergonomics of human-system interaction - Part 110: Dialogue principles

3 Terms, definitions and abbreviations
For the purposes of this document, the following terms and definitions apply.
3.1 Terms and definitions
3.1.1
ambient assisted living
AAL

products, services, environments and facilities used to support those whose independence,

safety, wellbeing and autonomy are compromised by their physical or mental status

Note 1 to entry: In this PAS, AAL refers to the usage of ICT, including multimedia systems and equipment and

audio / video communication, for creating intelligent living environments that react to the needs of users by

providing relevant assistance.
[SOURCE: ISO/IEC GUIDE 71:2001]
3.1.2
AAL service user
person who interacts with an AAL system or is connected with an AAL system
3.1.3
AAL space
denotes a smart environment that provides AAL services to its users
Figure 4 – The notion of AAL spaces
---------------------- Page: 11 ----------------------
– 10 – IEC PAS 62883:2014  IEC 2014
3.1.4
channel

denotes the bridging passage that smart environments need between the physical world and

the virtual realm (Figure 5). Channels are realized by devices. Depending on the kind of

channel opened, a channel might be called a sensing channel (realized by sensors), an acting

channel (realized by actuators), an input channel (realized by microphones, keyboards, etc.),

or an output channel (realized by displays, loudspeakers, etc.). The latter two types of

channels might be referred to as I/O channels
Figure 5 – The need of smart environments to utilize channels
for bridging between the physical world and the virtual realm
3.1.5
human-environment interaction

interaction in smart environments is generally divided into two major areas: implicit and

explicit interaction

Implicit interaction is mostly about using sensing channels for observation of happenings, with

or without involvement of humans, in order to recognize in the background relevant situations

to which the environment might be able to react in a desired way.

Explicit interaction, on the contrary, is about situations in which a human user seeks the

dialog with the environment or vice versa, for instance when the user instructs that the

brightness of the TV should be increased or when the environment notifies the user that it is

time to take a certain medicine. Explicit interaction takes place by utilizing input and output

channels provided by I/O devices.
3.1.6
I/O device

an abbreviation for input and / or output device. A device that provides an input and / or

output channel for facilitating explicit interaction between a smart environment and its human

users. Input devices, such as a microphone, a keyboard, or a mouse, can capture an

instruction or response that is provided by a human user and represent it in terms of data in

the virtual realm (Figure 6). Upon receive of data within the virtual realm that is intended to be

presented to human users, output devices, such as displays and loudspeakers, can make it

perceivable to the addressed humans
---------------------- Page: 12 ----------------------
IEC PAS 62883:2014  IEC 2014 – 11 –
Figure 6 – The role of devices in realizing bridging channels
3.1.7
multimodal UI handler

denotes UI handlers that perform the interaction by using multiple channels in parallel,

possibly with a hybrid mix supporting different modalities
3.1.8
smart environment

denotes an environment centred on its human users in which a set of embedded networked

artefacts, both hardware (HW) and software (SW), collectively realize the paradigm of

Ambient Intelligence, mainly by providing for context-awareness and personalization, adaptive

reactivity, and anticipatory pro-activity
Figure 7 – The notion of a smart environment
3.1.9
user
person who interacts with the product, service or environment
3.1.10
user interface

all components of an interactive system (software or hardware) that provide information

and/or control functions for the user to accomplish specific tasks with the interactive system

---------------------- Page: 13 ----------------------
– 12 – IEC PAS 62883:2014  IEC 2014
3.2 Abbreviation
AAL Ambient Assisted Living
API Application programming interface
Cf confer
DLNA Digital Living Network Alliance
e.g. for example
etc. et cetera
GUI Graphical user interface
HCI Human-Computer Interaction
HEI Human-Environment Interaction
HTML Hypertext Markup Language
HW Hardware
ICT Information and communications' technology
i.e. id est, that is to say
I/O channels Input/Output channel
OWL Web Ontology Language
RDF Resource Description Framework
RM Resource Manager
SW Software
UI User Interaction
UI model User Interaction model
UIML User Interface Markup Language
uAAL universAAL
UPNP Universal Plug and Play
W3C World Wide Web Consortium
XForms W3C Standard at http://www.w3.org/TR/xforms/
XML Extensible Markup Language
XPath XML Path Language; W3C Standard at http://www.w3.org/TR/xpath/
4 The Specification of the universAAL UI Framework
4.1 Overview

As depicted in the Introduction to this PAS, the universAAL UI Framework for Multimedia AAL

spaces is based on separating the application layer from the presentation layer by introducing

(1) a new type of software components called UI handlers and (2) a brokerage mechanism

between the application and presentation layers. The goal is to achieve an open system in

which arbitrary applications as well as UI handlers can be plugged in independently from each

other while enabling applications to benefit from support for multimodal interaction both by the

framework itself and by the UI handlers:
---------------------- Page: 14 ----------------------
IEC PAS 62883:2014  IEC 2014 – 13 –
Figure 8 – An open system for plugging in applications
and UI handlers independently from each other
4.2 Analysis of the relationships between UI Handlers and I/O Channels

UI handlers are seen responsible for handling requests for interacting with human users. To

do so, they need to utilize the available I/O channels but are not necessarily supposed to take

over the binding and management of those channels. On the other side, the "manager" of a

single input or output channel is usually not able to handle the whole of a UI request sent by

an application because applications often wait for user response in the context of some

information to be presented to the user; as a result. At least one output channel and one input

channel are supposed to be utilized simultaneously by the same UI handler in order to be able

to interpret user input in the context of the output presented to the user.

Therefore, it is important to have a more precise look at the relationship between UI handlers

and I/O channels: Section 3.1 states that channels are realized by certain devices but it does

not say anything about their binding and availability in the virtual realm. In the concept map in

Figure 9, those definitions are extended by introducing the concept of Channel Binding; the

same concept can be used for both input and output channels:
Figure 9 – Channel binding by I/O devices

The device that realizes a channel has to have an integrated solution for providing access to

the channel. This might be very low-level, at hard- and firmware level to exchange certain bits

and bytes through a physical connector interface (called Embedded Binding) or already

software using higher-level protocols and abstractions (called Driver). Drivers might be

provided by third-party and / or run on third-party devices. Like Figure 10 denotes, a driver

might wrap another driver in order to comply with higher-level abstractions (e.g. a “UPNP

driver”) for binding a special-purpose device that is not readily “UPNP-aware” usually uses

---------------------- Page: 15 ----------------------
– 14 – IEC PAS 62883:2014  IEC 2014

“Embedded Bindings” or “Legacy Drivers” for providing wrappers that interact at the UPNP

level of abstraction.

In particular, the higher-level abstractions might only make sense in the context of a

framework created for certain purposes (cf. a windowing system that uses a mouse driver and

already interprets the mouse e
...