e-Competence performance indicators and common metrics

The aim of this document is to enable unbiased and consistent use of indicators and measurements to enable verification of an individual’s competence to the EN 16234-1 (e-CF) to facilitate its consistent application.
The document addresses the assessment of competence as articulated within the EN 16234-1 (e-CF), regardless of where, when and how the competence was attained or developed.
The aim is to provide guidance on the use of indicators and measurements to support the assessment and/or verification of an IT professional’s competence.
Guidance is confined to possible indicators and how they may be applied to achieve consistency and transparency for the verification of an e-CF competence at a specific level (1-5).
This document guides readers through objective assessment of e-CF competence to avoid possible influence from personal feelings, interpretations or prejudice.
Finally, this document aims to offer, at least, examples of indicators and metrics for each of the e-competences listed in EN 16234-1 (e-CF).

Leistungsindikatoren für E-Kompetenz und gemeinsame Metriken

e-Compétences : indicateurs de performance et métriques communes

Kazalniki učinkovitosti e-usposobljenosti in skupne meritve

General Information

Status
Published
Public Enquiry End Date
16-Feb-2022
Publication Date
02-May-2022
Technical Committee
Current Stage
6060 - National Implementation/Publication (Adopted Project)
Start Date
21-Apr-2022
Due Date
26-Jun-2022
Completion Date
03-May-2022

Buy Standard

Technical report
SIST-TP CEN/TR 17802:2022 - BARVE na PDF-str 10,18,24,28,33
English language
66 pages
sale 10% off
Preview
sale 10% off
Preview
e-Library read for
1 day
Technical report
kSIST-TP FprCEN/TR 17802:2022 - BARVE na PDF-str 10,18,24,27,32
English language
66 pages
sale 10% off
Preview
sale 10% off
Preview
e-Library read for
1 day

Standards Content (sample)

SLOVENSKI STANDARD
SIST-TP CEN/TR 17802:2022
01-junij-2022
Kazalniki učinkovitosti e-usposobljenosti in skupne meritve
e-Competence performance indicators and common metrics
Leistungsindikatoren für E-Kompetenz und gemeinsame Metriken
e-Compétences : indicateurs de performance et métriques communes
Ta slovenski standard je istoveten z: CEN/TR 17802:2022
ICS:
35.020 Informacijska tehnika in Information technology (IT) in
tehnologija na splošno general
SIST-TP CEN/TR 17802:2022 en,fr,de

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------
SIST-TP CEN/TR 17802:2022
---------------------- Page: 2 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802
TECHNICAL REPORT
RAPPORT TECHNIQUE
April 2022
TECHNISCHER BERICHT
ICS 35.020
English Version
e-Competence performance indicators and common
metrics

e-Compétences : indicateurs de performance et Leistungsindikatoren für E-Kompetenz und

métriques communes gemeinsame Metriken

This Technical Report was approved by CEN on 27 March 2022. It has been drawn up by the Technical Committee CEN/TC 428.

CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,

Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,

Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and

United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels

© 2022 CEN All rights of exploitation in any form and by any means reserved Ref. No. CEN/TR 17802:2022 E

worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
Contents Page

European foreword ...................................................................................................................................................... 3

Introduction .................................................................................................................................................................... 4

1 Scope .................................................................................................................................................................... 6

2 Normative references .................................................................................................................................... 6

3 Terms and definitions ................................................................................................................................... 6

4 Using EN 16234-1 for recognition and validation of e-Competences .......................................... 7

4.1 e-Competences from EN 16234-1 (e-CF): abilities of ICT professionals ..................................... 7

4.2 Recognition and validation of e-Competences ..................................................................................... 8

5 Assessment of e-Competences in different contexts ........................................................................ 11

5.1 Context assessment: general considerations ...................................................................................... 11

5.2 Customization of EN 16234-1 (e-CF) ...................................................................................................... 12

5.3 Typical use cases for validation and recognition of e-Competences .......................................... 13

6 Guidelines for development and application of indicators and metrics .................................. 16

6.1 The process for deciding if a person is proficient in an e-Competence at a certain level .. 16

6.2 How to derive indicators and metrics from EN 16234-1 (e-CF) ................................................... 17

6.3 Adapting recognition and validation mechanisms to use cases .................................................. 29

6.4 Application of the assessment process to a candidate .................................................................... 30

Annex A (informative) The EN 16234-1 (e-CF) adoption Maturity Model ............................................. 35

Annex B (informative) Methods for recognition and validation of e-Competences .......................... 39

Annex C (informative) Examples of Indicators ............................................................................................... 42

Bibliography ................................................................................................................................................................. 65

---------------------- Page: 4 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
European foreword

This document (CEN/TR 17802:2022) has been prepared by Technical Committee CEN/TC 428 “ICT

Professionalism and Digital Competences” the secretariat of which is held by UNI.

Attention is drawn to the possibility that some of the elements of this document may be the subject of

patent rights. CEN shall not be held responsible for identifying any or all such patent rights.

Any feedback and questions on this document should be directed to the users’ national standards body.

A complete listing of these bodies can be found on the CEN website.
---------------------- Page: 5 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
Introduction

Recognition of competences is not a new field, many contributions have explored this topic and have

developed assessment tools and methods to pursue clarity and precision, trying to be as unambiguous

and non-judgmental as possible. There is an urgent need in the ICT field to find practical solutions related

to the recognition and assessment of capabilities of ICT professionals whenever/wherever such

competences have been acquired or developed.

This document is not intended as a general guideline as many valuable and authoritative papers already

exist (e.g. from CEDEFOP and SFIA). This document is focused on providing guidance and a method for

applying the EN 16234-1, e-Competence Framework (e-CF) - A common European Framework for ICT

Professionals in all industry sectors — Part 1: Framework and related documents to the process for

competence assessment of ICT professionals.

Therefore, this document provides guidance to e-CF users by supporting the use of indicators, metrics

and criteria that support the assessment of an individual’s e- competence at a specific level of proficiency

as specified in EN 16234-1 (e-CF).
This document provides:
a) Guidelines to derive indicators and metrics from the reference documents.

Most information, related to e-Competences, is described in EN 16234-1 (e-CF) its user guide and

other associated documents such as CWA 16458-1 (ICT Profiles). Complementary to this

information, e-CF users may seek the additional guidance and a methodology offered in this

document to identify indicators and metrics in support of assessment.

The EN 16234-1 (e-CF) and the CWA 16458-1 (ICT Profiles) are references provided for all

stakeholders and users in Europe and worldwide. They are flexible structures and are adaptable to

meet requirements across a broad and in-depth ICT Professional environment. Therefore, it is

impossible to create a complete and exhaustive catalogue of all possible indicators and metrics that

are context specific across for all possible roles and competences. In support of this application

flexibility, this document offers a range of examples which may be applied or used as inspiration for

specific or unique requirements.

b) Guidelines to generate a customized set of indicators, metrics, criteria and methods to collect pieces

of evidence.

Once a satisfactory catalogue of indicators or metrics have been defined to start the assessment, the

next task is to customize them for a specific use case. This customization usually entails the selection

of methods for the collection of supporting evidence and indicators as well as the criteria used to

determine the ICT professional’s proficiency level for each situation. Depending on the use case (see

a list of typical use cases in 5.3) the assessor or organization (e.g. an organization planning to recruit

or address workforce development) may select and adapt:
— the methods used to collect information;

— the criteria used to evaluate if an individual is proficient at a specific e-CF competence level.

This adaptation process is assisted by the guidelines in this document. Furthermore, the document

guides the process by highlighting the weaknesses and strengths of each option by providing

examples of adaptation to specific situations.
---------------------- Page: 6 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)

c) Awareness on the typical assessment process to evaluate the level of a candidate’s e-Competences.

When all of the elements for an assessment are ready (indicators, metrics, criteria, methods for

collecting pieces of evidence, etc.), the process of assessing e-Competences can start. The process

may involve collection of information in the form of documents, samples, etc. or the collection of

information through direct interaction with the ICT professional or other informed persons using

interviews, forms, tests, exercises, observation, etc. An illustrative catalogue of methods for collecting

information is described in Annex B. This document offers examples of pieces of evidence and the

contribution they may provide. The assessor is offered guidelines to manage the assessment process

and to generate a set of assessment results and this document shows examples of results to illustrate

the process.

Finally, it is important to underline that the aim of deploying the e-CF as a competence assessment tool

is to provide objectivity (i.e. unbiased, based on facts and not influenced by personal feelings,

interpretations, or prejudice). This is naturally limited by the experience and capability of the individual

assessor and this is the reasoning has inspired the development of present document designed to offer

pragmatic guidance and inspiration.
This document is structured by six Clauses.

Clauses 1, 2 and 3 describe the scope, the normative references and the relevant terms and definitions

used.

Clause 4 analyses the content of EN 16234-1 (e-CF) from the perspective of the identification of elements

useful in assessing e-competences and proficiency levels.

Clause 5 illustrates the importance of the context of an assessment and describes the factors that affect

the methods, indicators and metrics to be used.

Clause 6 contains a description of a typical process for the assessment and a methodology to identify and

consistently map the indicators.
---------------------- Page: 7 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
1 Scope

The aim of this document is to enable unbiased and consistent use of indicators and measurements to

enable verification of an individual’s competence to the EN 16234-1 (e-CF) to facilitate its consistent

application.

The document addresses the assessment of competence as articulated within the EN 16234-1 (e-CF),

regardless of where, when and how the competence was attained or developed.

The aim is to provide guidance on the use of indicators and measurements to support the assessment

and/or verification of an IT professional’s competence.

Guidance is confined to possible indicators and how they can be applied to achieve consistency and

transparency for the verification of an e-CF competence at a specific level (1-5).

This document guides readers through objective assessment of e-CF competence to avoid possible

influence from personal feelings, interpretations or prejudice.

Finally, this document aims to offer, at least, examples of indicators and metrics for each of the e-

competences listed in EN 16234-1 (e-CF).
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

EN 16234 (all parts), e-Competence Framework (e-CF) - A common European Framework for ICT

Professionals in all industry sectors
3 Terms and definitions

For the purposes of this document, the terms and definitions given in EN 16234 (all parts) and the

following apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— IEC Electropedia: available at https://www.electropedia.org/
— ISO Online browsing platform: available at https://www.iso.org/obp
3.1
indicator

observable aspect or characteristic of an e-Competence that indicates its presence in an individual at a

particular proficiency level

Note 1 to entry: The observation or measurement of an indicator results in one or several qualitative or

quantitative values.
3.2
metric

total or partial value of an indicator where the measured entity is a person (ICT professional), the

attribute is mainly an e-Competence at a competence level and is determined according to well-defined

rules

Note 1 to entry: A metric could be qualitative or quantitative, objective or subjective, and direct or indirect. It may

express value in a range of ways from very basic scales like a classification (e.g. classify an entity according to a mere

type) to sophisticated quantitative metrics (like number of years).
---------------------- Page: 8 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
3.3
criterion

principle used to decide if an ICT professional has acquired an e-Competence at an e-CF level of

proficiency

Note 1 to entry: It can be determined by a value such as the threshold of a metric or of an indicator or possibly a

combination of values such as absolute numbers, percentages, etc.
4 Using EN 16234-1 for recognition and validation of e-Competences
4.1 e-Competences from EN 16234-1 (e-CF): abilities of ICT professionals

EN 16234-1 (e-CF) has established a common language for e-Competences that is independent of

education systems and qualification structures. In order to effectively use the e-CF for e-competence

assessment, a crucial factor is the users comprehensive and holistic understanding of the concept of

competence in e-CF: “Competence is a demonstrated ability to apply knowledge, skills and attitudes for

achieving observable results” (see EN 16234-1:2019, 3.5). For organizations deploying ICT professionals

these “abilities” which are articulated as 41 e-competences within EN 16234-1 are key from the following

three perspectives. They are:

— Formulated from an organizational perspective within Dimension 2. In this way, the relationship

between the competences and the capabilities and processes of enterprises are self-evident and

obvious on one hand. On the other hand, however, if we take for example two ICT professionals but

working in different organizations, with the e-Competence A.2 “Service Level Management” they are

likely to perform different tasks. In our example, one ICT professional works in the service

department of a large telecommunication provider alongside many colleagues and another is solely

responsible for service management in a games app. start-up company. The activities they perform

will differ significantly as a result of their organizational environment however, it is conceivable that

both possess the competence A.2.

— Comprehensive descriptions of capability which include attitudes, reflecting the way ICT

professionals act in a contextually appropriate manner, and behavioural skills, which are used to

successfully engage with situations in the workplace. Behavioural skills may refer to work quality,

social interaction or emotion (for details, see CEN/TR 16234-3). Furthermore, they reflect the

experience of the ICT professional. Experience enables them to act in varied, complex situations, in a

team and with customers, in a goal-oriented but also flexible manner and to behave appropriately.

Especially at higher proficiency levels, competence related experience enables ICT professionals to

deal with challenging unknown problems, to find completely new solutions and approaches and to

independently evaluate the results. Therefore, for example, “Ensures the achievement of planned

results” (Competence A.2 level 4) can result in very different actions from different ICT

professionals; one may focus on motivating employees by addressing them personally, whilst

another may prefer to improve the overall process. The same competence may be acquired and

achieved in many different ways.

— Dynamic and ever changing are characteristics of the ICT professional environment. Focusing on

competence facilitates organizational changes making them as feasible as personnel and individual

development. Owing to rapid evolution, ICT organizations often place less importance on the current

knowledge of their ICT professionals than on their ability to learn and to further develop specific e-

Competences. A design feature of EN 16234-1 (e-CF) is that competence, composed of many variable

elements is flexible yet has an enduring structure; this is an essential advantage of the 41 e-

Competences. So for an ICT professional it is part of being e-competent in “ensuring the achievement

of planned results” (A.2 “Service Level Management”, level 4) to learn, in which environments it is

appropriate to improve the process and in which situation it is appropriate to motivate the

employees and in which situation other actions are required to reach the goal.
---------------------- Page: 9 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)

The comprehensive and holistic concept of competence in EN 16234-1 (e-CF) is one of the most important

characteristics of the e-CF and ensures its flexibility and adaptability. However, these features create

challenges for the identification and validation of e-Competences. The next clause is devoted to showing

how the content of EN 16234-1 may be explored to reveal and enable recognition and validation of e-

Competences.
4.2 Recognition and validation of e-Competences

EN 16234-1 (e-CF) competence is defined as a demonstrated ability to apply knowledge, skills and

attitudes for achieving observable results (see EN 16234-1:2019, 3.5).

This definition of e-Competence can be broken down and used as the starting point to identify the key

elements that enable the recognition and validation of e-Competences. Three aspects can support the

analysis of EN 16234-1 (e-CF):

1. The “ability to apply”, can be related to activities and experience. The definition implies a focus

on actions described in the EN 16234-1 (e-CF) either explicitly or indirectly mentioned as verbs.

References to actions can be found in dimension 2 e-Competences descriptions, in dimension 3

proficiency levels and also in dimension 4, examples of knowledge and skills.

2. “Knowledge and skills” are elements explicitly listed in dimension 4 of the e-CF. The content

includes examples but they are not exhaustive, consequently there are many knowledge and skills items

that may be relevant to this e-CF dimension. In particular, EN 17748-1 (ICT BoK) articulates knowledge

required and deployed by ICT professionals and is designed to enhance the e-CF and therefore can be

used as a significant knowledge reference. Attitudes are integrated into the transversal aspects of the

EN 16234-1 (e-CF) which in turn are associated with knowledge and skills.

3. “Observable results” can be identified from documents and deliverables mentioned in

EN 16234-1 (e-CF) within the descriptions of dimensions 2, 3 or 4. Although this is the primary source

for ‘observable results’ as it comes from normative information, a catalogue can be developed and

enriched by using information from Annex B of CEN/TR 16234-2:2021 (Examples of deliverables related

to e-CF competences) or CWA 16458-2 (ICT Professional Profiles). Within the European ICT Professional

Role Profiles each role profile includes typical deliverables and indicates associated responsibility

(accountable, responsible, contributor).

The relationship between the e-Competence definition and these three key elements is illustrated in

Figure 1.

Figure 1 — Relationship between definition of e-Competence and elements of EN 16234-1

The foundations for assessing if an individual is e-competent at a specific proficiency level are based on

the description of e-Competences which describe the key elements for each of the 41 e-Competences.

Analysis of the content of each dimension provides the base for building a methodology to develop the

mechanisms, guidelines and elements required for the assessment and recognition of competence at a

specific e-competence level of proficiency.
---------------------- Page: 10 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
Dimension 1: e-Competence area

The function of the five areas PLAN – BUILD – RUN – ENABLE – MANAGE is to serve as a navigation aid

and entry point to e-competences (and to relate them to capabilities and processes of the enterprises).

These general areas may be used to consider the context of an ICT professional’s experience, but add little

value in individual assessment or verification.
Dimension 2: e-Competence

EN 16234-1 (e-CF) states: “Each e-Competence is specified by a title and a generic description of the

competence. A total of 41 e-Competences have been identified. […] The e-Competences defined within

the standard are not exhaustive; nonetheless they provide a basic, clear, and sound orientation for

individuals and organizations. [...] The comprehensive descriptions articulated in Dimension 2 provide

primary e-Competence reference points for application of the framework.”

The 41 e-Competences are a normative part of EN 16234-1. The descriptions are short statements of the

most typical actions (understood as abilities, including experience) and results which are achievable

within the addressed e-Competence.

In the example “A.2. Service Level Management” actions can be identified as follows:

“Defines, validates and makes applicable service level agreements (SLAs) and underpinning contracts

tailored to services offered. Negotiates service performance levels taking into account the needs and

capacity of stakeholders and business.”
The observable result is obvious: “service level agreements (SLAs)”.

The corresponding abilities, including experience as well as attitudes, are further specified by describing

and embedding the activities, e.g. as “taking into account the needs and capacity of stakeholders and

business.”

In this way the key elements actions, abilities and observable results for assessment and for building

indicators are identifiable in the Dimension 2 description. Furthermore, it is possible to describe and/or

discuss, what these key elements look like in a specific environment and for a specific person, possibly to

identify the e-Competence at a specific proficiency level, by referencing the e-CF level descriptors of

Table 3.
Dimension 3: e-CF proficiency levels

As previously stated, the scope of this document is to create assessment criteria and indicators for e-CF

e-competence at defined levels to meet different evaluation requirements. Therefore, the assessment of

e-Competence needs to be targeted to define proficiency in an e-Competence at a specific level. Dimension

3 is an essential element for defining proficiency levels and has to be referenced alongside dimension 2

in the assessment process.

EN 16234-1 (e-CF) states: “In Dimension 3, specific proficiency levels are assigned for each e-Competence

described in Dimension 2. The level specifications of this standard encompass e-Competence levels e-1

to e-5. These levels define proficiency criteria and describe the degree of mastery required by an ICT

professional to meet different levels of performance in each competence. The levels are characterized by

a combination of levels of influence within a community, context complexity, autonomy, and typical

behaviour expressed by examples of action verbs.”

The level parameters are illustrated in Annex A of EN 16234-1:2019 and reproduced in Table 3 of this

document.

The five proficiency levels are a normative part of EN 16234-1. The level descriptions are holistic and

incorporate a combination of influence, complexity (of the context), autonomy, behaviour and a

summarizing “level descriptor”. Key elements, including abilities/activities on a specific level can be

derived from Table 3.
---------------------- Page: 11 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)

It can be observed from Table 3 that individuals possessing an e-competence at proficiency level e-4 are

able to:
— provide executive leadership in;
— unpredictable and/or unstructured environments/processes/…;

— act on a level of autonomy on which he/she demonstrates leadership and innovation in unfamiliar,

complex and unpredictable environments and addresses issues involving many interacting factors;

— shows typical behaviour like conceiving, transforming, innovating, finding creative solutions by

application of a wide range of technical and/or management principles.

Level e-4 proficiency is relevant to the previously applied example “A.2. Service Level Management”. The

condensed descriptor for this level is derived from stakeholder agreed examples of competence

performed at this level. The descriptor reads: “Negotiates revision of SLAs, in accordance with the overall

objectives. Ensures the achievement of planned results.” (EN 16234-1, e-Competence A.2, Dimension 3:

e-CF proficiency level).
From this we can observe:
— typical actions are negotiating, ensuring and planning; and
— typical results are SLAs and plans.

The corresponding abilities, including experience and even attitudes, are specified in the description and

embedding of activities, e.g. as “Negotiates revision of SLAs, in accordance with the overall objectives.”

This description is supported by the general level description and characteristics given in the level table.

In this way the key elements, actions, abilities and observable results for assessment and for building

indicators are identifiable from the Dimension 3 descriptions. Using Table 3 it is possible to describe

and/or discuss, what these key elements look like in a specific environment and for a specific person.

When referencing the level descriptions in Dimension 3 and the characteristics defined in Table 3, it is

important to consider context when identifying and assessing competence. Complexity, autonomy and

influence are obviously related to the context and the environment in which an ICT professional operates.

Therefore, it is crucial to always consider context when assessing competence and associated levels and

when determining indicators and criteria beforehand.

It should be noted that assessment needs to be related to a specific level of proficiency of an e-

Competence, it is not feasible to apply a global assessment for all levels of proficiency. This raises a

question on the aggregation of proficiency levels. In general, it is not possible to guarantee that a

candidate demonstrating competence at an upper level (e.g. level 4) of an e-Competence is automatically

competent at lower levels.
DIMENSION 4: knowledge and skills examples

EN 16234-1 (e-CF) states: “Examples of knowledge and skills relate to the e-Competences in Dimension

2. These examples are provided to add value to the competence descriptor and are not intended to be

exhaustive. They offer inspiration and orientation for the identification of further context specific

knowledge and skills assignment.”

These key elements are provided to facilitate understanding of e-competence. They may also inspire the

recognition and validation of e-competence.

Within EN 16234-1, the number of knowledge and skills items per e-competence ranges from 4 to more

than 10 items, depending on the e-Competence. CEN/TR 17748-2 (ICT BoK) also articulates knowledge

required and deployed by ICT professionals and provides an additional knowledge reference.

---------------------- Page: 12 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)

In the example “A.2. Service Level Management” five knowledge and also five skills examples are listed,

from “K1 SLA documentation” to “K5 impact of service level non-compliance on business performance”

and from “S1 analyse service provision records” to “S5 anticipate and mitigate against potential service

disruptions”.

Consequently, typical knowledge and skills (K&S), applied by an ICT professional, are identifiable,

however the lists provided are informative, not normative nor exhaustive.
In summary:

— Dimension 1 describes five e-competence areas: plan, build, run, enable and manage. This dimension

is of limited use for assessment.

— Dimension 2 describes each e-competence with concise sentences that explain the key features of the

competence, describing activities and using verbs such as “responds”, “assures”, “evaluates”. Within

each description, references are made to results, outcomes, and deliverables.
— Dimension 3 sub-divides each e-competence into e-CF proficiency levels 1
...

SLOVENSKI STANDARD
kSIST-TP FprCEN/TR 17802:2022
01-februar-2022
Kazalniki učinkovitosti e-usposobljenosti in skupne meritve
e-Competence performance indicators and common metrics
Leistungsindikatoren für E-Kompetenz und gemeinsame Metriken
e-Compétences : indicateurs de performance et métriques communes
Ta slovenski standard je istoveten z: FprCEN/TR 17802
ICS:
35.020 Informacijska tehnika in Information technology (IT) in
tehnologija na splošno general
kSIST-TP FprCEN/TR 17802:2022 en,fr,de

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------
kSIST-TP FprCEN/TR 17802:2022
---------------------- Page: 2 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FINAL DRAFT
TECHNICAL REPORT
FprCEN/TR 17802
RAPPORT TECHNIQUE
TECHNISCHER BERICHT
November 2021
ICS 35.020
English Version
e-Competence performance indicators and common
metrics

e-Compétences : indicateurs de performance et Leistungsindikatoren für E-Kompetenz und

métriques communes gemeinsame Metriken

This draft Technical Report is submitted to CEN members for Vote. It has been drawn up by the Technical Committee CEN/TC

428.

CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,

Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,

Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and

United Kingdom.

Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are

aware and to provide supporting documentation.

Warning : This document is not a Technical Report. It is distributed for review and comments. It is subject to change without

notice and shall not be referred to as a Technical Report.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels

© 2021 CEN All rights of exploitation in any form and by any means reserved Ref. No. FprCEN/TR 17802:2021 E

worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
Contents Page

European foreword ...................................................................................................................................................... 3

Introduction .................................................................................................................................................................... 4

1 Scope .................................................................................................................................................................... 6

2 Normative references .................................................................................................................................... 6

3 Terms and definitions ................................................................................................................................... 6

4 Using EN 16234-1 for Recognition and validation of e-Competences .......................................... 7

4.1 e-Competences from EN 16234-1 (e-CF): abilities of ICT professionals ..................................... 7

4.2 Recognition and validation of e-Competences ..................................................................................... 8

5 Assessment of e-Competences in different contexts ........................................................................ 11

5.1 Context Assessment: general considerations ..................................................................................... 11

5.2 Customization of EN 16234-1 (e-CF) ...................................................................................................... 12

5.3 Typical use cases for validation and recognition of e-Competences .......................................... 13

6 Guidelines for development and application of indicators and metrics .................................. 16

6.1 The process for deciding if a person is proficient in an e-Competence at a certain level .. 16

6.2 How to derive indicators and metrics from EN 16234-1 (e-CF) ................................................... 17

6.3 Adapting recognition and validation mechanisms to use cases .................................................. 28

6.4 Application of the assessment process to a candidate .................................................................... 29

Annex A (informative) The EN 16234-1 (e-CF) adoption Maturity Model ............................................. 35

Annex B (informative) Methods for recognition and validation of e-Competences .......................... 39

Annex C (informative) Examples of Indicators ............................................................................................... 42

Bibliography ................................................................................................................................................................. 65

---------------------- Page: 4 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
European foreword

This document (FprCEN/TR 17802:2021) has been prepared by Technical Committee CEN/TC 428 “ICT

Professionalism and Digital Competences” the secretariat of which is held by UNI.

This document is currently submitted to the Vote on TR.
---------------------- Page: 5 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
Introduction

Recognition of competences is not a new field, many contributions have explored this topic and have

developed assessment tools and methods to pursue clarity and precision, trying to be as unambiguous

and non-judgmental as possible. There is an urgent need in the ICT field to find practical solutions related

to the recognition and assessment of capabilities of ICT professionals whenever/wherever such

competences have been acquired or developed.

This document is not intended as a general guideline as many valuable and authoritative papers already

exist (e.g. from CEDEFOP and SFIA). This document is focused on providing guidance and a method for

applying the EN 16234-1, e-Competence Framework (e-CF) - A common European Framework for ICT

Professionals in all industry sectors — Part 1: Framework and related documents to the process for

competence assessment of ICT professionals.

Therefore, this document provides guidance to e-CF users by supporting the use of indicators, metrics

and criteria that support the assessment of an individual’s e- competence at a specific level of proficiency

as specified in EN 16234-1 (e-CF).
This document provides:
a) Guidelines to derive indicators and metrics from the reference documents.

Most information, related to e-Competences, is described in EN 16234-1 (e-CF) its user guide and

other associated documents such as CWA 16458-1 (ICT Profiles). Complementary to this

information, e-CF users may seek the additional guidance and a methodology offered in this

document to identify indicators and metrics in support of assessment.

The EN 16234-1 (e-CF) and the CWA 16458-1 (ICT Profiles) are references provided for all

stakeholders and users in Europe and worldwide. They are flexible structures and are adaptable to

meet requirements across a broad and in-depth ICT Professional environment. Therefore, it is

impossible to create a complete and exhaustive catalogue of all possible indicators and metrics that

are context specific across for all possible roles and competences. In support of this application

flexibility, this document offers a range of examples which may be applied or used as inspiration for

specific or unique requirements.

b) Guidelines to generate a customized set of indicators, metrics, criteria and methods to collect pieces

of evidence.

Once a satisfactory catalogue of indicators or metrics have been defined to start the assessment, the

next task is to customize them for a specific use case. This customization usually entails the selection

of methods for the collection of supporting evidence and indicators as well as the criteria used to

determine the ICT professional’s proficiency level for each situation. Depending on the use case (see

a list of typical use cases in 5.3) the assessor or organization (e.g. an organization planning to recruit

or address workforce development) may select and adapt:
— the methods used to collect information,

— the criteria used to evaluate if an individual is proficient at a specific e-CF competence level.

This adaptation process is assisted by the guidelines in this document. Furthermore, the document

guides the process by highlighting the weaknesses and strengths of each option by providing

examples of adaptation to specific situations.
---------------------- Page: 6 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)

c) Awareness on the typical assessment process to evaluate the level of a candidate’s e-Competences.

When all of the elements for an assessment are ready (indicators, metrics, criteria, methods for

collecting pieces of evidence, etc.), the process of assessing e-Competences can start. The process

may involve collection of information in the form of documents, samples, etc. or the collection of

information through direct interaction with the ICT professional or other informed persons using

interviews, forms, tests, exercises, observation, etc. An illustrative catalogue of methods for collecting

information is described in Annex B. This document offers examples of pieces of evidence and the

contribution they may provide. The assessor is offered guidelines to manage the assessment process

and to generate a set of assessment results and this document shows examples of results to illustrate

the process.

Finally, it is important to underline that the aim of deploying the e-CF as a competence assessment tool

is to provide objectivity (i.e. unbiased, based on facts and not influenced by personal feelings,

interpretations, or prejudice). This is naturally limited by the experience and capability of the individual

assessor and this is the reasoning has inspired the development of present document designed to offer

pragmatic guidance and inspiration.
This document is structured by six Clauses.

Clauses 1, 2 and 3 describe the scope, the normative references and the relevant terms and definitions

used.

Clause 4 analyses the content of EN 16234-1 (e-CF) from the perspective of the identification of elements

useful in assessing e-competences and proficiency levels.

Clause 5 illustrates the importance of the context of an assessment and describes the factors that affect

the methods, indicators and metrics to be used.

Clause 6 contains a description of a typical process for the assessment and a methodology to identify and

consistently map the indicators.
---------------------- Page: 7 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
1 Scope

The aim of this document is to enable unbiased and consistent use of indicators and measurements to

enable verification of an individual’s competence to the EN 16234-1 (e-CF) to facilitate its consistent

application.

The document addresses the assessment of competence as articulated within the EN 16234-1 (e-CF),

regardless of where, when and how the competence was attained or developed.

The aim is to provide guidance on the use of indicators and measurements to support the assessment

and/or verification of an IT professional’s competence.

Guidance is confined to possible indicators and how they can be applied to achieve consistency and

transparency for the verification of an e-CF competence at a specific level (1-5).

This document guides readers through objective assessment of e-CF competence to avoid possible

influence from personal feelings, interpretations or prejudice.

Finally, this document aims to offer, at least, examples of indicators and metrics for each of the e-

competences listed in EN 16234-1 (e-CF).
2 Normative references

The following documents are referred to in the text in such a way that some or all of their content

constitutes requirements of this document. For dated references, only the edition cited applies. For

undated references, the latest edition of the referenced document (including any amendments) applies.

EN 16234 (all parts), e-Competence Framework (e-CF) - A common European Framework for ICT

Professionals in all industry sectors
3 Terms and definitions

For the purposes of this document, the terms and definitions given in EN 16234 (all parts) and the

following apply.

ISO and IEC maintain terminological databases for use in standardization at the following addresses:

— IEC Electropedia: available at https://www.electropedia.org/
— ISO Online browsing platform: available at https://www.iso.org/obp
3.1
indicator

observable aspect or characteristic of an e-Competence that indicates its presence in an individual at a

particular proficiency level

Note 1 to entry: The observation or measurement of an indicator results in one or several qualitative or

quantitative values.
3.2
metric

total or partial value of an indicator where the measured entity is a person (ICT professional), the

attribute is mainly an e-Competence at a competence level and is determined according to well-defined

rules

Note 1 to entry: A metric could be qualitative or quantitative, objective or subjective, and direct or indirect. It may

express value in a range of ways from very basic scales like a classification (e.g. classify an entity according to a mere

type) to sophisticated quantitative metrics (like number of years).
---------------------- Page: 8 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
3.3
criterion

principle used to decide if an ICT professional has acquired an e-Competence at an e-CF level of

proficiency

Note 1 to entry: It can be determined by a value such as the threshold of a metric or of an indicator or possibly a

combination of values such as absolute numbers, percentages, etc.
4 Using EN 16234-1 for Recognition and validation of e-Competences
4.1 e-Competences from EN 16234-1 (e-CF): abilities of ICT professionals

EN 16234-1 (e-CF) has established a common language for e-Competences that is independent of

education systems and qualification structures. In order to effectively use the e-CF for e-competence

assessment, a crucial factor is the users comprehensive and holistic understanding of the concept of

competence in e-CF: “Competence is a demonstrated ability to apply knowledge, skills and attitudes for

achieving observable results” (see EN 16234-1:2019, 3.5). For organizations deploying ICT professionals

these “abilities” which are articulated as 41 e-competences within EN 16234-1 are key from the following

three perspectives. They are:

— Formulated from an organizational perspective within Dimension 2. In this way, the relationship

between the competences and the capabilities and processes of enterprises are self-evident and

obvious on one hand. On the other hand, however, if we take for example two ICT professionals but

working in different organizations, with the e-Competence A.2 “Service Level Management” they are

likely to perform different tasks. In our example, one ICT professional works in the service

department of a large telecommunication provider alongside many colleagues and another is solely

responsible for service management in a games app. start-up company. The activities they perform

will differ significantly as a result of their organizational environment however, it is conceivable that

both possess the competence A.2.

— Comprehensive descriptions of capability which include attitudes, reflecting the way ICT

professionals act in a contextually appropriate manner, and behavioural skills, which are used to

successfully engage with situations in the workplace. Behavioural skills may refer to work quality,

social interaction or emotion (for details, see CEN/TR 16234-3). Furthermore, they reflect the

experience of the ICT professional. Experience enables them to act in varied, complex situations, in a

team and with customers, in a goal-oriented but also flexible manner and to behave appropriately.

Especially at higher proficiency levels, competence related experience enables ICT professionals to

deal with challenging unknown problems, to find completely new solutions and approaches and to

independently evaluate the results. Therefore, for example, “Ensures the achievement of planned

results” (Competence A.2 level 4) can result in very different actions from different ICT

professionals; one may focus on motivating employees by addressing them personally, whilst

another may prefer to improve the overall process. The same competence may be acquired and

achieved in many different ways.

— Dynamic and ever changing are characteristics of the ICT professional environment. Focusing on

competence facilitates organizational changes making them as feasible as personnel and individual

development. Owing to rapid evolution, ICT organizations often place less importance on the current

knowledge of their ICT professionals than on their ability to learn and to further develop specific e-

Competences. A design feature of EN 16234-1 (e-CF) is that competence, composed of many variable

elements is flexible yet has an enduring structure; this is an essential advantage of the 41 e-

Competences. So for an ICT professional it is part of being e-competent in “ensuring the achievement

of planned results” (A.2 “Service Level Management”, level 4) to learn, in which environments it is

appropriate to improve the process and in which situation it is appropriate to motivate the

employees and in which situation other actions are required to reach the goal.
---------------------- Page: 9 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)

The comprehensive and holistic concept of competence in EN 16234-1 (e-CF) is one of the most important

characteristics of the e-CF and ensures its flexibility and adaptability. However, these features create

challenges for the identification and validation of e-Competences. The next clause is devoted to showing

how the content of EN 16234-1 may be explored to reveal and enable recognition and validation of e-

Competences.
4.2 Recognition and validation of e-Competences

EN 16234-1 (e-CF) competence is defined as a demonstrated ability to apply knowledge, skills and

attitudes for achieving observable results (see EN 16234-1:2019, 3.5).

This definition of e-Competence can be broken down and used as the starting point to identify the key

elements that enable the recognition and validation of e-Competences. Three aspects can support the

analysis of EN 16234-1 (e-CF):

1. The “ability to apply”, can be related to activities and experience. The definition implies a focus

on actions described in the EN 16234-1 (e-CF) either explicitly or indirectly mentioned as verbs.

References to actions can be found in dimension 2 e-Competences descriptions, in dimension 3

proficiency levels and also in dimension 4, examples of knowledge and skills.

2. “Knowledge and skills” are elements explicitly listed in dimension 4 of the e-CF. The content

includes examples but they are not exhaustive, consequently there are many knowledge and skills items

that may be relevant to this e-CF dimension. In particular, prEN 17748-1 (ICT BoK) articulates knowledge

required and deployed by ICT professionals and is designed to enhance the e-CF and therefore can be

used as a significant knowledge reference. Attitudes are integrated into the transversal aspects of the

EN 16234-1 (e-CF) which in turn are associated with knowledge and skills.

3. “Observable results” can be identified from documents and deliverables mentioned in

EN 16234-1 (e-CF) within the descriptions of dimensions 2, 3 or 4. Although this is the primary source

for ‘observable results’ as it comes from normative information, a catalogue can be developed and

enriched by using information from Annex B of CEN/TR 16234-2:2021 (Examples of deliverables related

to e-CF competences) or CWA 16458-2 (ICT Professional Profiles). Within the European ICT Professional

Role Profiles each role profile includes typical deliverables and indicates associated responsibility

(accountable, responsible, contributor) .

The relationship between the e-Competence definition and these three key elements is illustrated in

Figure 1.

Figure 1 — Relationship between definition of e-Competence and elements of EN 16234-1

The foundations for assessing if an individual is e-competent at a specific proficiency level are based on

the description of e-Competences which describe the key elements for each of the 41 e-Competences.

Analysis of the content of each dimension provides the base for building a methodology to develop the

mechanisms, guidelines and elements required for the assessment and recognition of competence at a

specific e-competence level of proficiency.
---------------------- Page: 10 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
Dimension 1: e-Competence area

The function of the five areas PLAN – BUILD – RUN – ENABLE – MANAGE is to serve as a navigation aid

and entry point to e-competences (and to relate them to capabilities and processes of the enterprises).

These general areas may be used to consider the context of an ICT professional’s experience, but add little

value in individual assessment or verification.
Dimension 2: e-Competence

EN 16234-1 (e-CF) states: “Each e-Competence is specified by a title and a generic description of the

competence. A total of 41 e-Competences have been identified. […] The e-Competences defined within

the standard are not exhaustive; nonetheless they provide a basic, clear, and sound orientation for

individuals and organizations. [...] The comprehensive descriptions articulated in Dimension 2 provide

primary e-Competence reference points for application of the framework.”

The 41 e-Competences are a normative part of EN 16234-1. The descriptions are short statements of the

most typical actions (understood as abilities, including experience) and results which are achievable

within the addressed e-Competence.

In the example “A.2. Service Level Management” actions can be identified as follows:

“Defines, validates and makes applicable service level agreements (SLAs) and underpinning contracts

tailored to services offered. Negotiates service performance levels taking into account the needs and

capacity of stakeholders and business.”
The observable result is obvious: “service level agreements (SLAs)”.

The corresponding abilities, including experience as well as attitudes, are further specified by describing

and embedding the activities, e.g. as “taking into account the needs and capacity of stakeholders and

business.”

In this way the key elements actions, abilities and observable results for assessment and for building

indicators are identifiable in the Dimension 2 description. Furthermore, it is possible to describe and/or

discuss, what these key elements look like in a specific environment and for a specific person, possibly to

identify the e-Competence at a specific proficiency level, by referencing the e-CF level descriptors of

Table 3.
Dimension 3: e-CF proficiency levels

As previously stated, the scope of this document is to create assessment criteria and indicators for e-CF

e-competence at defined levels to meet different evaluation requirements. Therefore, the assessment of

e-Competence needs to be targeted to define proficiency in an e-Competence at a specific level. Dimension

3 is an essential element for defining proficiency levels and has to be referenced alongside dimension 2

in the assessment process.

EN 16234-1 (e-CF) states: “In Dimension 3, specific proficiency levels are assigned for each e-Competence

described in Dimension 2. The level specifications of this standard encompass e-Competence levels e-1

to e-5. These levels define proficiency criteria and describe the degree of mastery required by an ICT

professional to meet different levels of performance in each competence. The levels are characterized by

a combination of levels of influence within a community, context complexity, autonomy, and typical

behaviour expressed by examples of action verbs.”

The level parameters are illustrated in Annex A of EN 16234-1:2019 and reproduced in Table 3 of this

document.

The five proficiency levels are a normative part of EN 16234-1. The level descriptions are holistic and

incorporate a combination of influence, complexity (of the context), autonomy, behaviour and a

summarizing “level descriptor”. Key elements, including abilities/activities on a specific level can be

derived from Table 3.
---------------------- Page: 11 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)

It can be observed from Table 3 that individuals possessing an e-competence at proficiency level e-4 are

able to:
— provide executive leadership in;
— unpredictable and/or unstructured environments/processes/…;

— act on a level of autonomy on which he/she demonstrates leadership and innovation in unfamiliar,

complex and unpredictable environments and addresses issues involving many interacting factors;

— shows typical behaviour like conceiving, transforming, innovating, finding creative solutions by

application of a wide range of technical and/or management principles.

Level e-4 proficiency is relevant to the previously applied example “A.2. Service Level Management”. The

condensed descriptor for this level is derived from stakeholder agreed examples of competence

performed at this level. The descriptor reads: “Negotiates revision of SLAs, in accordance with the overall

objectives. Ensures the achievement of planned results.” (EN 16234-1, e-Competence A.2, Dimension 3:

e-CF proficiency level).
From this we can observe:
— typical actions are negotiating, ensuring and planning and
— typical results are SLAs and plans.

The corresponding abilities, including experience and even attitudes, are specified in the description and

embedding of activities, e.g. as “Negotiates revision of SLAs, in accordance with the overall objectives.”

This description is supported by the general level description and characteristics given in the level table.

In this way the key elements, actions, abilities and observable results for assessment and for building

indicators are identifiable from the Dimension 3 descriptions. Using Table 3 it is possible to describe

and/or discuss, what these key elements look like in a specific environment and for a specific person.

When referencing the level descriptions in Dimension 3 and the characteristics defined in Table 3, it is

important to consider context when identifying and assessing competence. Complexity, autonomy and

influence are obviously related to the context and the environment in which an ICT professional operates.

Therefore, it is crucial to always consider context when assessing competence and associated levels and

when determining indicators and criteria beforehand.

It should be noted that assessment needs to be related to a specific level of proficiency of an e-

Competence, it is not feasible to apply a global assessment for all levels of proficiency. This raises a

question on the aggregation of proficiency levels. In general, it is not possible to guarantee that a

candidate demonstrating competence at an upper level (e.g. level 4) of an e-Competence is automatically

competent at lower levels.
DIMENSION 4: knowledge and skills examples

EN 16234-1 (e-CF) states: “Examples of knowledge and skills relate to the e-Competences in Dimension

2. These examples are provided to add value to the competence descriptor and are not intended to be

exhaustive. They offer inspiration and orientation for the identification of further context specific

knowledge and skills assignment.”

These key elements are provided to facilitate understanding of e-competence. They may also inspire the

recognition and validation of e-competence.

Within EN 16234-1, the number of knowledge and skills items per e-competence ranges from 4 to more

than 10 items, depending on the e-Competence. FprCEN/TR 17748-2 (ICT BoK) also articulates

knowledge required and deployed by ICT professionals and provides an additional knowledge reference.

---------------------- Page: 12 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)

In the example “A.2. Service Level Management” five knowledge and also five skills examples are listed,

from “K1 SLA documentation” to “K5 impact of service level non-compliance on business performance”

and from “S1 analyse service provision records” to “S5 anticipate and mitigate against potential service

disruptions”.

Consequently, typical knowledge and skills (K&S), applied by an ICT professional, are identifiable,

however the lists provided are informative, not normative nor exhaustive.
In summary:

— Dimension 1 describes five e-competence areas: plan, build, run, enable and manage. This dimension

is of limited use for assessment.

— Dimension 2 describes each e-competence with concise sentences that explain the key features of the

competence, describing activities and using verbs such as “responds”, “assures”, “

...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.