SIST-TP CEN/TR 17802:2022
(Main)e-Competence performance indicators and common metrics
e-Competence performance indicators and common metrics
The aim of this document is to enable unbiased and consistent use of indicators and measurements to enable verification of an individual’s competence to the EN 16234-1 (e-CF) to facilitate its consistent application.
The document addresses the assessment of competence as articulated within the EN 16234-1 (e-CF), regardless of where, when and how the competence was attained or developed.
The aim is to provide guidance on the use of indicators and measurements to support the assessment and/or verification of an IT professional’s competence.
Guidance is confined to possible indicators and how they may be applied to achieve consistency and transparency for the verification of an e-CF competence at a specific level (1-5).
This document guides readers through objective assessment of e-CF competence to avoid possible influence from personal feelings, interpretations or prejudice.
Finally, this document aims to offer, at least, examples of indicators and metrics for each of the e-competences listed in EN 16234-1 (e-CF).
Leistungsindikatoren für E-Kompetenz und gemeinsame Metriken
e-Compétences : indicateurs de performance et métriques communes
Kazalniki učinkovitosti e-usposobljenosti in skupne meritve
Namen tega dokumenta je omogočiti nepristransko in dosledno uporabo kazalnikov ter meritev za preverjanje posameznikove usposobljenosti v skladu s standardom EN 16234-1 (e-CF).
Dokument obravnava oceno usposobljenosti, kot je opredeljena v standardu EN 16234-1 (e-CF), pri čemer ni pomembno kje, kdaj in kako je bila usposobljenost dosežena ali razvita.
Cilj je zagotoviti smernice za uporabo kazalnikov in meritev za podporo ocenjevanju in/ali preverjanju usposobljenosti strokovnjaka IT.
Smernice so omejene na možne kazalnike in načine, kako jih je mogoče uporabiti za dosledno in pregledno preverjanje usposobljenosti e-CF na določeni ravni (1–5).
Ta dokument vodi bralce skozi objektivno oceno usposobljenosti e-CF, da bi se izognili možnemu vplivu osebnih nazorov, interpretacij ali predsodkov.
Cilj tega dokumenta je med drugim ponuditi vsaj primere kazalnikov in meritev za vsako od e-usposobljenosti, ki so navedene v standardu EN 16234-1 (e-CF).
General Information
Standards Content (Sample)
SLOVENSKI STANDARD
SIST-TP CEN/TR 17802:2022
01-junij-2022
Kazalniki učinkovitosti e-usposobljenosti in skupne meritve
e-Competence performance indicators and common metrics
Leistungsindikatoren für E-Kompetenz und gemeinsame Metriken
e-Compétences : indicateurs de performance et métriques communes
Ta slovenski standard je istoveten z: CEN/TR 17802:2022
ICS:
35.020 Informacijska tehnika in Information technology (IT) in
tehnologija na splošno general
SIST-TP CEN/TR 17802:2022 en,fr,de
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
SIST-TP CEN/TR 17802:2022
---------------------- Page: 2 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802
TECHNICAL REPORT
RAPPORT TECHNIQUE
April 2022
TECHNISCHER BERICHT
ICS 35.020
English Version
e-Competence performance indicators and common
metrics
e-Compétences : indicateurs de performance et Leistungsindikatoren für E-Kompetenz und
métriques communes gemeinsame Metriken
This Technical Report was approved by CEN on 27 March 2022. It has been drawn up by the Technical Committee CEN/TC 428.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,
Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and
United Kingdom.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2022 CEN All rights of exploitation in any form and by any means reserved Ref. No. CEN/TR 17802:2022 E
worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
Contents Page
European foreword . 3
Introduction . 4
1 Scope . 6
2 Normative references . 6
3 Terms and definitions . 6
4 Using EN 16234-1 for recognition and validation of e-Competences . 7
4.1 e-Competences from EN 16234-1 (e-CF): abilities of ICT professionals . 7
4.2 Recognition and validation of e-Competences . 8
5 Assessment of e-Competences in different contexts . 11
5.1 Context assessment: general considerations . 11
5.2 Customization of EN 16234-1 (e-CF) . 12
5.3 Typical use cases for validation and recognition of e-Competences . 13
6 Guidelines for development and application of indicators and metrics . 16
6.1 The process for deciding if a person is proficient in an e-Competence at a certain level . 16
6.2 How to derive indicators and metrics from EN 16234-1 (e-CF) . 17
6.3 Adapting recognition and validation mechanisms to use cases . 29
6.4 Application of the assessment process to a candidate . 30
Annex A (informative) The EN 16234-1 (e-CF) adoption Maturity Model . 35
Annex B (informative) Methods for recognition and validation of e-Competences . 39
Annex C (informative) Examples of Indicators . 42
Bibliography . 65
2
---------------------- Page: 4 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
European foreword
This document (CEN/TR 17802:2022) has been prepared by Technical Committee CEN/TC 428 “ICT
Professionalism and Digital Competences” the secretariat of which is held by UNI.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. CEN shall not be held responsible for identifying any or all such patent rights.
Any feedback and questions on this document should be directed to the users’ national standards body.
A complete listing of these bodies can be found on the CEN website.
3
---------------------- Page: 5 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
Introduction
Recognition of competences is not a new field, many contributions have explored this topic and have
developed assessment tools and methods to pursue clarity and precision, trying to be as unambiguous
and non-judgmental as possible. There is an urgent need in the ICT field to find practical solutions related
to the recognition and assessment of capabilities of ICT professionals whenever/wherever such
competences have been acquired or developed.
This document is not intended as a general guideline as many valuable and authoritative papers already
exist (e.g. from CEDEFOP and SFIA). This document is focused on providing guidance and a method for
applying the EN 16234-1, e-Competence Framework (e-CF) - A common European Framework for ICT
Professionals in all industry sectors — Part 1: Framework and related documents to the process for
competence assessment of ICT professionals.
Therefore, this document provides guidance to e-CF users by supporting the use of indicators, metrics
and criteria that support the assessment of an individual’s e- competence at a specific level of proficiency
as specified in EN 16234-1 (e-CF).
This document provides:
a) Guidelines to derive indicators and metrics from the reference documents.
Most information, related to e-Competences, is described in EN 16234-1 (e-CF) its user guide and
other associated documents such as CWA 16458-1 (ICT Profiles). Complementary to this
information, e-CF users may seek the additional guidance and a methodology offered in this
document to identify indicators and metrics in support of assessment.
The EN 16234-1 (e-CF) and the CWA 16458-1 (ICT Profiles) are references provided for all
stakeholders and users in Europe and worldwide. They are flexible structures and are adaptable to
meet requirements across a broad and in-depth ICT Professional environment. Therefore, it is
impossible to create a complete and exhaustive catalogue of all possible indicators and metrics that
are context specific across for all possible roles and competences. In support of this application
flexibility, this document offers a range of examples which may be applied or used as inspiration for
specific or unique requirements.
b) Guidelines to generate a customized set of indicators, metrics, criteria and methods to collect pieces
of evidence.
Once a satisfactory catalogue of indicators or metrics have been defined to start the assessment, the
next task is to customize them for a specific use case. This customization usually entails the selection
of methods for the collection of supporting evidence and indicators as well as the criteria used to
determine the ICT professional’s proficiency level for each situation. Depending on the use case (see
a list of typical use cases in 5.3) the assessor or organization (e.g. an organization planning to recruit
or address workforce development) may select and adapt:
— the methods used to collect information;
— the criteria used to evaluate if an individual is proficient at a specific e-CF competence level.
This adaptation process is assisted by the guidelines in this document. Furthermore, the document
guides the process by highlighting the weaknesses and strengths of each option by providing
examples of adaptation to specific situations.
4
---------------------- Page: 6 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
c) Awareness on the typical assessment process to evaluate the level of a candidate’s e-Competences.
When all of the elements for an assessment are ready (indicators, metrics, criteria, methods for
collecting pieces of evidence, etc.), the process of assessing e-Competences can start. The process
may involve collection of information in the form of documents, samples, etc. or the collection of
information through direct interaction with the ICT professional or other informed persons using
interviews, forms, tests, exercises, observation, etc. An illustrative catalogue of methods for collecting
information is described in Annex B. This document offers examples of pieces of evidence and the
contribution they may provide. The assessor is offered guidelines to manage the assessment process
and to generate a set of assessment results and this document shows examples of results to illustrate
the process.
Finally, it is important to underline that the aim of deploying the e-CF as a competence assessment tool
is to provide objectivity (i.e. unbiased, based on facts and not influenced by personal feelings,
interpretations, or prejudice). This is naturally limited by the experience and capability of the individual
assessor and this is the reasoning has inspired the development of present document designed to offer
pragmatic guidance and inspiration.
This document is structured by six Clauses.
Clauses 1, 2 and 3 describe the scope, the normative references and the relevant terms and definitions
used.
Clause 4 analyses the content of EN 16234-1 (e-CF) from the perspective of the identification of elements
useful in assessing e-competences and proficiency levels.
Clause 5 illustrates the importance of the context of an assessment and describes the factors that affect
the methods, indicators and metrics to be used.
Clause 6 contains a description of a typical process for the assessment and a methodology to identify and
consistently map the indicators.
5
---------------------- Page: 7 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
1 Scope
The aim of this document is to enable unbiased and consistent use of indicators and measurements to
enable verification of an individual’s competence to the EN 16234-1 (e-CF) to facilitate its consistent
application.
The document addresses the assessment of competence as articulated within the EN 16234-1 (e-CF),
regardless of where, when and how the competence was attained or developed.
The aim is to provide guidance on the use of indicators and measurements to support the assessment
and/or verification of an IT professional’s competence.
Guidance is confined to possible indicators and how they can be applied to achieve consistency and
transparency for the verification of an e-CF competence at a specific level (1-5).
This document guides readers through objective assessment of e-CF competence to avoid possible
influence from personal feelings, interpretations or prejudice.
Finally, this document aims to offer, at least, examples of indicators and metrics for each of the e-
competences listed in EN 16234-1 (e-CF).
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
EN 16234 (all parts), e-Competence Framework (e-CF) - A common European Framework for ICT
Professionals in all industry sectors
3 Terms and definitions
For the purposes of this document, the terms and definitions given in EN 16234 (all parts) and the
following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at https://www.electropedia.org/
— ISO Online browsing platform: available at https://www.iso.org/obp
3.1
indicator
observable aspect or characteristic of an e-Competence that indicates its presence in an individual at a
particular proficiency level
Note 1 to entry: The observation or measurement of an indicator results in one or several qualitative or
quantitative values.
3.2
metric
total or partial value of an indicator where the measured entity is a person (ICT professional), the
attribute is mainly an e-Competence at a competence level and is determined according to well-defined
rules
Note 1 to entry: A metric could be qualitative or quantitative, objective or subjective, and direct or indirect. It may
express value in a range of ways from very basic scales like a classification (e.g. classify an entity according to a mere
type) to sophisticated quantitative metrics (like number of years).
6
---------------------- Page: 8 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
3.3
criterion
principle used to decide if an ICT professional has acquired an e-Competence at an e-CF level of
proficiency
Note 1 to entry: It can be determined by a value such as the threshold of a metric or of an indicator or possibly a
combination of values such as absolute numbers, percentages, etc.
4 Using EN 16234-1 for recognition and validation of e-Competences
4.1 e-Competences from EN 16234-1 (e-CF): abilities of ICT professionals
EN 16234-1 (e-CF) has established a common language for e-Competences that is independent of
education systems and qualification structures. In order to effectively use the e-CF for e-competence
assessment, a crucial factor is the users comprehensive and holistic understanding of the concept of
competence in e-CF: “Competence is a demonstrated ability to apply knowledge, skills and attitudes for
achieving observable results” (see EN 16234-1:2019, 3.5). For organizations deploying ICT professionals
these “abilities” which are articulated as 41 e-competences within EN 16234-1 are key from the following
three perspectives. They are:
— Formulated from an organizational perspective within Dimension 2. In this way, the relationship
between the competences and the capabilities and processes of enterprises are self-evident and
obvious on one hand. On the other hand, however, if we take for example two ICT professionals but
working in different organizations, with the e-Competence A.2 “Service Level Management” they are
likely to perform different tasks. In our example, one ICT professional works in the service
department of a large telecommunication provider alongside many colleagues and another is solely
responsible for service management in a games app. start-up company. The activities they perform
will differ significantly as a result of their organizational environment however, it is conceivable that
both possess the competence A.2.
— Comprehensive descriptions of capability which include attitudes, reflecting the way ICT
professionals act in a contextually appropriate manner, and behavioural skills, which are used to
successfully engage with situations in the workplace. Behavioural skills may refer to work quality,
social interaction or emotion (for details, see CEN/TR 16234-3). Furthermore, they reflect the
experience of the ICT professional. Experience enables them to act in varied, complex situations, in a
team and with customers, in a goal-oriented but also flexible manner and to behave appropriately.
Especially at higher proficiency levels, competence related experience enables ICT professionals to
deal with challenging unknown problems, to find completely new solutions and approaches and to
independently evaluate the results. Therefore, for example, “Ensures the achievement of planned
results” (Competence A.2 level 4) can result in very different actions from different ICT
professionals; one may focus on motivating employees by addressing them personally, whilst
another may prefer to improve the overall process. The same competence may be acquired and
achieved in many different ways.
— Dynamic and ever changing are characteristics of the ICT professional environment. Focusing on
competence facilitates organizational changes making them as feasible as personnel and individual
development. Owing to rapid evolution, ICT organizations often place less importance on the current
knowledge of their ICT professionals than on their ability to learn and to further develop specific e-
Competences. A design feature of EN 16234-1 (e-CF) is that competence, composed of many variable
elements is flexible yet has an enduring structure; this is an essential advantage of the 41 e-
Competences. So for an ICT professional it is part of being e-competent in “ensuring the achievement
of planned results” (A.2 “Service Level Management”, level 4) to learn, in which environments it is
appropriate to improve the process and in which situation it is appropriate to motivate the
employees and in which situation other actions are required to reach the goal.
7
---------------------- Page: 9 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
The comprehensive and holistic concept of competence in EN 16234-1 (e-CF) is one of the most important
characteristics of the e-CF and ensures its flexibility and adaptability. However, these features create
challenges for the identification and validation of e-Competences. The next clause is devoted to showing
how the content of EN 16234-1 may be explored to reveal and enable recognition and validation of e-
Competences.
4.2 Recognition and validation of e-Competences
EN 16234-1 (e-CF) competence is defined as a demonstrated ability to apply knowledge, skills and
attitudes for achieving observable results (see EN 16234-1:2019, 3.5).
This definition of e-Competence can be broken down and used as the starting point to identify the key
elements that enable the recognition and validation of e-Competences. Three aspects can support the
analysis of EN 16234-1 (e-CF):
1. The “ability to apply”, can be related to activities and experience. The definition implies a focus
on actions described in the EN 16234-1 (e-CF) either explicitly or indirectly mentioned as verbs.
References to actions can be found in dimension 2 e-Competences descriptions, in dimension 3
proficiency levels and also in dimension 4, examples of knowledge and skills.
2. “Knowledge and skills” are elements explicitly listed in dimension 4 of the e-CF. The content
includes examples but they are not exhaustive, consequently there are many knowledge and skills items
that may be relevant to this e-CF dimension. In particular, EN 17748-1 (ICT BoK) articulates knowledge
required and deployed by ICT professionals and is designed to enhance the e-CF and therefore can be
used as a significant knowledge reference. Attitudes are integrated into the transversal aspects of the
EN 16234-1 (e-CF) which in turn are associated with knowledge and skills.
3. “Observable results” can be identified from documents and deliverables mentioned in
EN 16234-1 (e-CF) within the descriptions of dimensions 2, 3 or 4. Although this is the primary source
for ‘observable results’ as it comes from normative information, a catalogue can be developed and
enriched by using information from Annex B of CEN/TR 16234-2:2021 (Examples of deliverables related
to e-CF competences) or CWA 16458-2 (ICT Professional Profiles). Within the European ICT Professional
Role Profiles each role profile includes typical deliverables and indicates associated responsibility
(accountable, responsible, contributor).
The relationship between the e-Competence definition and these three key elements is illustrated in
Figure 1.
Figure 1 — Relationship between definition of e-Competence and elements of EN 16234-1
The foundations for assessing if an individual is e-competent at a specific proficiency level are based on
the description of e-Competences which describe the key elements for each of the 41 e-Competences.
Analysis of the content of each dimension provides the base for building a methodology to develop the
mechanisms, guidelines and elements required for the assessment and recognition of competence at a
specific e-competence level of proficiency.
8
---------------------- Page: 10 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
Dimension 1: e-Competence area
The function of the five areas PLAN – BUILD – RUN – ENABLE – MANAGE is to serve as a navigation aid
and entry point to e-competences (and to relate them to capabilities and processes of the enterprises).
These general areas may be used to consider the context of an ICT professional’s experience, but add little
value in individual assessment or verification.
Dimension 2: e-Competence
EN 16234-1 (e-CF) states: “Each e-Competence is specified by a title and a generic description of the
competence. A total of 41 e-Competences have been identified. […] The e-Competences defined within
the standard are not exhaustive; nonetheless they provide a basic, clear, and sound orientation for
individuals and organizations. [.] The comprehensive descriptions articulated in Dimension 2 provide
primary e-Competence reference points for application of the framework.”
The 41 e-Competences are a normative part of EN 16234-1. The descriptions are short statements of the
most typical actions (understood as abilities, including experience) and results which are achievable
within the addressed e-Competence.
In the example “A.2. Service Level Management” actions can be identified as follows:
“Defines, validates and makes applicable service level agreements (SLAs) and underpinning contracts
tailored to services offered. Negotiates service performance levels taking into account the needs and
capacity of stakeholders and business.”
The observable result is obvious: “service level agreements (SLAs)”.
The corresponding abilities, including experience as well as attitudes, are further specified by describing
and embedding the activities, e.g. as “taking into account the needs and capacity of stakeholders and
business.”
In this way the key elements actions, abilities and observable results for assessment and for building
indicators are identifiable in the Dimension 2 description. Furthermore, it is possible to describe and/or
discuss, what these key elements look like in a specific environment and for a specific person, possibly to
identify the e-Competence at a specific proficiency level, by referencing the e-CF level descriptors of
Table 3.
Dimension 3: e-CF proficiency levels
As previously stated, the scope of this document is to create assessment criteria and indicators for e-CF
e-competence at defined levels to meet different evaluation requirements. Therefore, the assessment of
e-Competence needs to be targeted to define proficiency in an e-Competence at a specific level. Dimension
3 is an essential element for defining proficiency levels and has to be referenced alongside dimension 2
in the assessment process.
EN 16234-1 (e-CF) states: “In Dimension 3, specific proficiency levels are assigned for each e-Competence
described in Dimension 2. The level specifications of this standard encompass e-Competence levels e-1
to e-5. These levels define proficiency criteria and describe the degree of mastery required by an ICT
professional to meet different levels of performance in each competence. The levels are characterized by
a combination of levels of influence within a community, context complexity, autonomy, and typical
behaviour expressed by examples of action verbs.”
The level parameters are illustrated in Annex A of EN 16234-1:2019 and reproduced in Table 3 of this
document.
The five proficiency levels are a normative part of EN 16234-1. The level descriptions are holistic and
incorporate a combination of influence, complexity (of the context), autonomy, behaviour and a
summarizing “level descriptor”. Key elements, including abilities/activities on a specific level can be
derived from Table 3.
9
---------------------- Page: 11 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
It can be observed from Table 3 that individuals possessing an e-competence at proficiency level e-4 are
able to:
— provide executive leadership in;
— unpredictable and/or unstructured environments/processes/…;
— act on a level of autonomy on which he/she demonstrates leadership and innovation in unfamiliar,
complex and unpredictable environments and addresses issues involving many interacting factors;
— shows typical behaviour like conceiving, transforming, innovating, finding creative solutions by
application of a wide range of technical and/or management principles.
Level e-4 proficiency is relevant to the previously applied example “A.2. Service Level Management”. The
condensed descriptor for this level is derived from stakeholder agreed examples of competence
performed at this level. The descriptor reads: “Negotiates revision of SLAs, in accordance with the overall
objectives. Ensures the achievement of planned results.” (EN 16234-1, e-Competence A.2, Dimension 3:
e-CF proficiency level).
From this we can observe:
— typical actions are negotiating, ensuring and planning; and
— typical results are SLAs and plans.
The corresponding abilities, including experience and even attitudes, are specified in the description and
embedding of activities, e.g. as “Negotiates revision of SLAs, in accordance with the overall objectives.”
This description is supported by the general level description and characteristics given in the level table.
In this way the key elements, actions, abilities and observable results for assessment and for building
indicators are identifiable from the Dimension 3 descriptions. Using Table 3 it is possible to describe
and/or discuss, what these key elements look like in a specific environment and for a specific person.
When referencing the level descriptions in Dimension 3 and the characteristics defined in Table 3, it is
important to consider context when identifying and assessing competence. Complexity, autonomy and
influence are obviously related to the context and the environment in which an ICT professional operates.
Therefore, it is crucial to always consider context when assessing competence and associated levels and
when determining indicators and criteria beforehand.
It should be noted that assessment needs to be related to a specific level of proficiency of an e-
Competence, it is not feasible to apply a global assessment for all levels of proficiency. This raises a
question on the aggregation of proficiency levels. In general, it is not possible to guarantee that a
candidate demonstrating competence at an upper level (e.g. level 4) of an e-Competence is automatically
competent at lower levels.
DIMENSION 4: knowledge and skills examples
EN 16234-1 (e-CF) states: “Examples of knowledge and skills relate to the e-Competences in Dimension
2. These examples are provided to add value to the competence descriptor and are not intended to be
exhaustive. They offer inspiration and orientation for the identification of further context specific
knowledge and skills assignment.”
These key elements are provided to facilitate understanding of e-competence. They may also inspire the
recognition and validation of e-competence.
Within EN 16234-1, the number of knowledge and skills items per e-competence ranges from 4 to more
than 10 items, depending on the e-Competence. CEN/TR 17748-2 (ICT BoK) also articulates knowledge
required and deployed by ICT professionals and provides an additional knowledge reference.
10
---------------------- Page: 12 ----------------------
SIST-TP CEN/TR 17802:2022
CEN/TR 17802:2022 (E)
In the example “A.2. Service Level Management” five knowledge and also five skills examples are listed,
from “K1 SLA documentation” to “K5 impact of service level non-compliance on business performance”
and from “S1 analyse service provision records” to “S5 anticipate and mitigate against potential service
disruptions”.
Consequently, typical knowledge and skills (K&S), applied by an ICT professional, are identifiable,
however the lists provided are informative, not normative nor exhaustive.
In summary:
— Dimension 1 describes five e-competence areas: plan, build, run, enable and manage. This dimension
is of limited use for assessment.
— Dimension 2 describes each e-competence with concise sentences that explain the key features of the
competence, describing activities and using verbs such as “responds”, “assures”, “evaluates”. Within
each description, references are made to results, outcomes, and deliverables.
— Dimension 3 sub-divides each e-competence into e-CF proficiency levels 1
...
SLOVENSKI STANDARD
kSIST-TP FprCEN/TR 17802:2022
01-februar-2022
Kazalniki učinkovitosti e-usposobljenosti in skupne meritve
e-Competence performance indicators and common metrics
Leistungsindikatoren für E-Kompetenz und gemeinsame Metriken
e-Compétences : indicateurs de performance et métriques communes
Ta slovenski standard je istoveten z: FprCEN/TR 17802
ICS:
35.020 Informacijska tehnika in Information technology (IT) in
tehnologija na splošno general
kSIST-TP FprCEN/TR 17802:2022 en,fr,de
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.
---------------------- Page: 1 ----------------------
kSIST-TP FprCEN/TR 17802:2022
---------------------- Page: 2 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FINAL DRAFT
TECHNICAL REPORT
FprCEN/TR 17802
RAPPORT TECHNIQUE
TECHNISCHER BERICHT
November 2021
ICS 35.020
English Version
e-Competence performance indicators and common
metrics
e-Compétences : indicateurs de performance et Leistungsindikatoren für E-Kompetenz und
métriques communes gemeinsame Metriken
This draft Technical Report is submitted to CEN members for Vote. It has been drawn up by the Technical Committee CEN/TC
428.
CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway,
Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and
United Kingdom.
Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of which they are
aware and to provide supporting documentation.
Warning : This document is not a Technical Report. It is distributed for review and comments. It is subject to change without
notice and shall not be referred to as a Technical Report.
EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION
EUROPÄISCHES KOMITEE FÜR NORMUNG
CEN-CENELEC Management Centre: Rue de la Science 23, B-1040 Brussels
© 2021 CEN All rights of exploitation in any form and by any means reserved Ref. No. FprCEN/TR 17802:2021 E
worldwide for CEN national Members.
---------------------- Page: 3 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
Contents Page
European foreword . 3
Introduction . 4
1 Scope . 6
2 Normative references . 6
3 Terms and definitions . 6
4 Using EN 16234-1 for Recognition and validation of e-Competences . 7
4.1 e-Competences from EN 16234-1 (e-CF): abilities of ICT professionals . 7
4.2 Recognition and validation of e-Competences . 8
5 Assessment of e-Competences in different contexts . 11
5.1 Context Assessment: general considerations . 11
5.2 Customization of EN 16234-1 (e-CF) . 12
5.3 Typical use cases for validation and recognition of e-Competences . 13
6 Guidelines for development and application of indicators and metrics . 16
6.1 The process for deciding if a person is proficient in an e-Competence at a certain level . 16
6.2 How to derive indicators and metrics from EN 16234-1 (e-CF) . 17
6.3 Adapting recognition and validation mechanisms to use cases . 28
6.4 Application of the assessment process to a candidate . 29
Annex A (informative) The EN 16234-1 (e-CF) adoption Maturity Model . 35
Annex B (informative) Methods for recognition and validation of e-Competences . 39
Annex C (informative) Examples of Indicators . 42
Bibliography . 65
2
---------------------- Page: 4 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
European foreword
This document (FprCEN/TR 17802:2021) has been prepared by Technical Committee CEN/TC 428 “ICT
Professionalism and Digital Competences” the secretariat of which is held by UNI.
This document is currently submitted to the Vote on TR.
3
---------------------- Page: 5 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
Introduction
Recognition of competences is not a new field, many contributions have explored this topic and have
developed assessment tools and methods to pursue clarity and precision, trying to be as unambiguous
and non-judgmental as possible. There is an urgent need in the ICT field to find practical solutions related
to the recognition and assessment of capabilities of ICT professionals whenever/wherever such
competences have been acquired or developed.
This document is not intended as a general guideline as many valuable and authoritative papers already
exist (e.g. from CEDEFOP and SFIA). This document is focused on providing guidance and a method for
applying the EN 16234-1, e-Competence Framework (e-CF) - A common European Framework for ICT
Professionals in all industry sectors — Part 1: Framework and related documents to the process for
competence assessment of ICT professionals.
Therefore, this document provides guidance to e-CF users by supporting the use of indicators, metrics
and criteria that support the assessment of an individual’s e- competence at a specific level of proficiency
as specified in EN 16234-1 (e-CF).
This document provides:
a) Guidelines to derive indicators and metrics from the reference documents.
Most information, related to e-Competences, is described in EN 16234-1 (e-CF) its user guide and
other associated documents such as CWA 16458-1 (ICT Profiles). Complementary to this
information, e-CF users may seek the additional guidance and a methodology offered in this
document to identify indicators and metrics in support of assessment.
The EN 16234-1 (e-CF) and the CWA 16458-1 (ICT Profiles) are references provided for all
stakeholders and users in Europe and worldwide. They are flexible structures and are adaptable to
meet requirements across a broad and in-depth ICT Professional environment. Therefore, it is
impossible to create a complete and exhaustive catalogue of all possible indicators and metrics that
are context specific across for all possible roles and competences. In support of this application
flexibility, this document offers a range of examples which may be applied or used as inspiration for
specific or unique requirements.
b) Guidelines to generate a customized set of indicators, metrics, criteria and methods to collect pieces
of evidence.
Once a satisfactory catalogue of indicators or metrics have been defined to start the assessment, the
next task is to customize them for a specific use case. This customization usually entails the selection
of methods for the collection of supporting evidence and indicators as well as the criteria used to
determine the ICT professional’s proficiency level for each situation. Depending on the use case (see
a list of typical use cases in 5.3) the assessor or organization (e.g. an organization planning to recruit
or address workforce development) may select and adapt:
— the methods used to collect information,
— the criteria used to evaluate if an individual is proficient at a specific e-CF competence level.
This adaptation process is assisted by the guidelines in this document. Furthermore, the document
guides the process by highlighting the weaknesses and strengths of each option by providing
examples of adaptation to specific situations.
4
---------------------- Page: 6 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
c) Awareness on the typical assessment process to evaluate the level of a candidate’s e-Competences.
When all of the elements for an assessment are ready (indicators, metrics, criteria, methods for
collecting pieces of evidence, etc.), the process of assessing e-Competences can start. The process
may involve collection of information in the form of documents, samples, etc. or the collection of
information through direct interaction with the ICT professional or other informed persons using
interviews, forms, tests, exercises, observation, etc. An illustrative catalogue of methods for collecting
information is described in Annex B. This document offers examples of pieces of evidence and the
contribution they may provide. The assessor is offered guidelines to manage the assessment process
and to generate a set of assessment results and this document shows examples of results to illustrate
the process.
Finally, it is important to underline that the aim of deploying the e-CF as a competence assessment tool
is to provide objectivity (i.e. unbiased, based on facts and not influenced by personal feelings,
interpretations, or prejudice). This is naturally limited by the experience and capability of the individual
assessor and this is the reasoning has inspired the development of present document designed to offer
pragmatic guidance and inspiration.
This document is structured by six Clauses.
Clauses 1, 2 and 3 describe the scope, the normative references and the relevant terms and definitions
used.
Clause 4 analyses the content of EN 16234-1 (e-CF) from the perspective of the identification of elements
useful in assessing e-competences and proficiency levels.
Clause 5 illustrates the importance of the context of an assessment and describes the factors that affect
the methods, indicators and metrics to be used.
Clause 6 contains a description of a typical process for the assessment and a methodology to identify and
consistently map the indicators.
5
---------------------- Page: 7 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
1 Scope
The aim of this document is to enable unbiased and consistent use of indicators and measurements to
enable verification of an individual’s competence to the EN 16234-1 (e-CF) to facilitate its consistent
application.
The document addresses the assessment of competence as articulated within the EN 16234-1 (e-CF),
regardless of where, when and how the competence was attained or developed.
The aim is to provide guidance on the use of indicators and measurements to support the assessment
and/or verification of an IT professional’s competence.
Guidance is confined to possible indicators and how they can be applied to achieve consistency and
transparency for the verification of an e-CF competence at a specific level (1-5).
This document guides readers through objective assessment of e-CF competence to avoid possible
influence from personal feelings, interpretations or prejudice.
Finally, this document aims to offer, at least, examples of indicators and metrics for each of the e-
competences listed in EN 16234-1 (e-CF).
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
EN 16234 (all parts), e-Competence Framework (e-CF) - A common European Framework for ICT
Professionals in all industry sectors
3 Terms and definitions
For the purposes of this document, the terms and definitions given in EN 16234 (all parts) and the
following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at https://www.electropedia.org/
— ISO Online browsing platform: available at https://www.iso.org/obp
3.1
indicator
observable aspect or characteristic of an e-Competence that indicates its presence in an individual at a
particular proficiency level
Note 1 to entry: The observation or measurement of an indicator results in one or several qualitative or
quantitative values.
3.2
metric
total or partial value of an indicator where the measured entity is a person (ICT professional), the
attribute is mainly an e-Competence at a competence level and is determined according to well-defined
rules
Note 1 to entry: A metric could be qualitative or quantitative, objective or subjective, and direct or indirect. It may
express value in a range of ways from very basic scales like a classification (e.g. classify an entity according to a mere
type) to sophisticated quantitative metrics (like number of years).
6
---------------------- Page: 8 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
3.3
criterion
principle used to decide if an ICT professional has acquired an e-Competence at an e-CF level of
proficiency
Note 1 to entry: It can be determined by a value such as the threshold of a metric or of an indicator or possibly a
combination of values such as absolute numbers, percentages, etc.
4 Using EN 16234-1 for Recognition and validation of e-Competences
4.1 e-Competences from EN 16234-1 (e-CF): abilities of ICT professionals
EN 16234-1 (e-CF) has established a common language for e-Competences that is independent of
education systems and qualification structures. In order to effectively use the e-CF for e-competence
assessment, a crucial factor is the users comprehensive and holistic understanding of the concept of
competence in e-CF: “Competence is a demonstrated ability to apply knowledge, skills and attitudes for
achieving observable results” (see EN 16234-1:2019, 3.5). For organizations deploying ICT professionals
these “abilities” which are articulated as 41 e-competences within EN 16234-1 are key from the following
three perspectives. They are:
— Formulated from an organizational perspective within Dimension 2. In this way, the relationship
between the competences and the capabilities and processes of enterprises are self-evident and
obvious on one hand. On the other hand, however, if we take for example two ICT professionals but
working in different organizations, with the e-Competence A.2 “Service Level Management” they are
likely to perform different tasks. In our example, one ICT professional works in the service
department of a large telecommunication provider alongside many colleagues and another is solely
responsible for service management in a games app. start-up company. The activities they perform
will differ significantly as a result of their organizational environment however, it is conceivable that
both possess the competence A.2.
— Comprehensive descriptions of capability which include attitudes, reflecting the way ICT
professionals act in a contextually appropriate manner, and behavioural skills, which are used to
successfully engage with situations in the workplace. Behavioural skills may refer to work quality,
social interaction or emotion (for details, see CEN/TR 16234-3). Furthermore, they reflect the
experience of the ICT professional. Experience enables them to act in varied, complex situations, in a
team and with customers, in a goal-oriented but also flexible manner and to behave appropriately.
Especially at higher proficiency levels, competence related experience enables ICT professionals to
deal with challenging unknown problems, to find completely new solutions and approaches and to
independently evaluate the results. Therefore, for example, “Ensures the achievement of planned
results” (Competence A.2 level 4) can result in very different actions from different ICT
professionals; one may focus on motivating employees by addressing them personally, whilst
another may prefer to improve the overall process. The same competence may be acquired and
achieved in many different ways.
— Dynamic and ever changing are characteristics of the ICT professional environment. Focusing on
competence facilitates organizational changes making them as feasible as personnel and individual
development. Owing to rapid evolution, ICT organizations often place less importance on the current
knowledge of their ICT professionals than on their ability to learn and to further develop specific e-
Competences. A design feature of EN 16234-1 (e-CF) is that competence, composed of many variable
elements is flexible yet has an enduring structure; this is an essential advantage of the 41 e-
Competences. So for an ICT professional it is part of being e-competent in “ensuring the achievement
of planned results” (A.2 “Service Level Management”, level 4) to learn, in which environments it is
appropriate to improve the process and in which situation it is appropriate to motivate the
employees and in which situation other actions are required to reach the goal.
7
---------------------- Page: 9 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
The comprehensive and holistic concept of competence in EN 16234-1 (e-CF) is one of the most important
characteristics of the e-CF and ensures its flexibility and adaptability. However, these features create
challenges for the identification and validation of e-Competences. The next clause is devoted to showing
how the content of EN 16234-1 may be explored to reveal and enable recognition and validation of e-
Competences.
4.2 Recognition and validation of e-Competences
EN 16234-1 (e-CF) competence is defined as a demonstrated ability to apply knowledge, skills and
attitudes for achieving observable results (see EN 16234-1:2019, 3.5).
This definition of e-Competence can be broken down and used as the starting point to identify the key
elements that enable the recognition and validation of e-Competences. Three aspects can support the
analysis of EN 16234-1 (e-CF):
1. The “ability to apply”, can be related to activities and experience. The definition implies a focus
on actions described in the EN 16234-1 (e-CF) either explicitly or indirectly mentioned as verbs.
References to actions can be found in dimension 2 e-Competences descriptions, in dimension 3
proficiency levels and also in dimension 4, examples of knowledge and skills.
2. “Knowledge and skills” are elements explicitly listed in dimension 4 of the e-CF. The content
includes examples but they are not exhaustive, consequently there are many knowledge and skills items
that may be relevant to this e-CF dimension. In particular, prEN 17748-1 (ICT BoK) articulates knowledge
required and deployed by ICT professionals and is designed to enhance the e-CF and therefore can be
used as a significant knowledge reference. Attitudes are integrated into the transversal aspects of the
EN 16234-1 (e-CF) which in turn are associated with knowledge and skills.
3. “Observable results” can be identified from documents and deliverables mentioned in
EN 16234-1 (e-CF) within the descriptions of dimensions 2, 3 or 4. Although this is the primary source
for ‘observable results’ as it comes from normative information, a catalogue can be developed and
enriched by using information from Annex B of CEN/TR 16234-2:2021 (Examples of deliverables related
to e-CF competences) or CWA 16458-2 (ICT Professional Profiles). Within the European ICT Professional
Role Profiles each role profile includes typical deliverables and indicates associated responsibility
(accountable, responsible, contributor) .
The relationship between the e-Competence definition and these three key elements is illustrated in
Figure 1.
Figure 1 — Relationship between definition of e-Competence and elements of EN 16234-1
The foundations for assessing if an individual is e-competent at a specific proficiency level are based on
the description of e-Competences which describe the key elements for each of the 41 e-Competences.
Analysis of the content of each dimension provides the base for building a methodology to develop the
mechanisms, guidelines and elements required for the assessment and recognition of competence at a
specific e-competence level of proficiency.
8
---------------------- Page: 10 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
Dimension 1: e-Competence area
The function of the five areas PLAN – BUILD – RUN – ENABLE – MANAGE is to serve as a navigation aid
and entry point to e-competences (and to relate them to capabilities and processes of the enterprises).
These general areas may be used to consider the context of an ICT professional’s experience, but add little
value in individual assessment or verification.
Dimension 2: e-Competence
EN 16234-1 (e-CF) states: “Each e-Competence is specified by a title and a generic description of the
competence. A total of 41 e-Competences have been identified. […] The e-Competences defined within
the standard are not exhaustive; nonetheless they provide a basic, clear, and sound orientation for
individuals and organizations. [.] The comprehensive descriptions articulated in Dimension 2 provide
primary e-Competence reference points for application of the framework.”
The 41 e-Competences are a normative part of EN 16234-1. The descriptions are short statements of the
most typical actions (understood as abilities, including experience) and results which are achievable
within the addressed e-Competence.
In the example “A.2. Service Level Management” actions can be identified as follows:
“Defines, validates and makes applicable service level agreements (SLAs) and underpinning contracts
tailored to services offered. Negotiates service performance levels taking into account the needs and
capacity of stakeholders and business.”
The observable result is obvious: “service level agreements (SLAs)”.
The corresponding abilities, including experience as well as attitudes, are further specified by describing
and embedding the activities, e.g. as “taking into account the needs and capacity of stakeholders and
business.”
In this way the key elements actions, abilities and observable results for assessment and for building
indicators are identifiable in the Dimension 2 description. Furthermore, it is possible to describe and/or
discuss, what these key elements look like in a specific environment and for a specific person, possibly to
identify the e-Competence at a specific proficiency level, by referencing the e-CF level descriptors of
Table 3.
Dimension 3: e-CF proficiency levels
As previously stated, the scope of this document is to create assessment criteria and indicators for e-CF
e-competence at defined levels to meet different evaluation requirements. Therefore, the assessment of
e-Competence needs to be targeted to define proficiency in an e-Competence at a specific level. Dimension
3 is an essential element for defining proficiency levels and has to be referenced alongside dimension 2
in the assessment process.
EN 16234-1 (e-CF) states: “In Dimension 3, specific proficiency levels are assigned for each e-Competence
described in Dimension 2. The level specifications of this standard encompass e-Competence levels e-1
to e-5. These levels define proficiency criteria and describe the degree of mastery required by an ICT
professional to meet different levels of performance in each competence. The levels are characterized by
a combination of levels of influence within a community, context complexity, autonomy, and typical
behaviour expressed by examples of action verbs.”
The level parameters are illustrated in Annex A of EN 16234-1:2019 and reproduced in Table 3 of this
document.
The five proficiency levels are a normative part of EN 16234-1. The level descriptions are holistic and
incorporate a combination of influence, complexity (of the context), autonomy, behaviour and a
summarizing “level descriptor”. Key elements, including abilities/activities on a specific level can be
derived from Table 3.
9
---------------------- Page: 11 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
It can be observed from Table 3 that individuals possessing an e-competence at proficiency level e-4 are
able to:
— provide executive leadership in;
— unpredictable and/or unstructured environments/processes/…;
— act on a level of autonomy on which he/she demonstrates leadership and innovation in unfamiliar,
complex and unpredictable environments and addresses issues involving many interacting factors;
— shows typical behaviour like conceiving, transforming, innovating, finding creative solutions by
application of a wide range of technical and/or management principles.
Level e-4 proficiency is relevant to the previously applied example “A.2. Service Level Management”. The
condensed descriptor for this level is derived from stakeholder agreed examples of competence
performed at this level. The descriptor reads: “Negotiates revision of SLAs, in accordance with the overall
objectives. Ensures the achievement of planned results.” (EN 16234-1, e-Competence A.2, Dimension 3:
e-CF proficiency level).
From this we can observe:
— typical actions are negotiating, ensuring and planning and
— typical results are SLAs and plans.
The corresponding abilities, including experience and even attitudes, are specified in the description and
embedding of activities, e.g. as “Negotiates revision of SLAs, in accordance with the overall objectives.”
This description is supported by the general level description and characteristics given in the level table.
In this way the key elements, actions, abilities and observable results for assessment and for building
indicators are identifiable from the Dimension 3 descriptions. Using Table 3 it is possible to describe
and/or discuss, what these key elements look like in a specific environment and for a specific person.
When referencing the level descriptions in Dimension 3 and the characteristics defined in Table 3, it is
important to consider context when identifying and assessing competence. Complexity, autonomy and
influence are obviously related to the context and the environment in which an ICT professional operates.
Therefore, it is crucial to always consider context when assessing competence and associated levels and
when determining indicators and criteria beforehand.
It should be noted that assessment needs to be related to a specific level of proficiency of an e-
Competence, it is not feasible to apply a global assessment for all levels of proficiency. This raises a
question on the aggregation of proficiency levels. In general, it is not possible to guarantee that a
candidate demonstrating competence at an upper level (e.g. level 4) of an e-Competence is automatically
competent at lower levels.
DIMENSION 4: knowledge and skills examples
EN 16234-1 (e-CF) states: “Examples of knowledge and skills relate to the e-Competences in Dimension
2. These examples are provided to add value to the competence descriptor and are not intended to be
exhaustive. They offer inspiration and orientation for the identification of further context specific
knowledge and skills assignment.”
These key elements are provided to facilitate understanding of e-competence. They may also inspire the
recognition and validation of e-competence.
Within EN 16234-1, the number of knowledge and skills items per e-competence ranges from 4 to more
than 10 items, depending on the e-Competence. FprCEN/TR 17748-2 (ICT BoK) also articulates
knowledge required and deployed by ICT professionals and provides an additional knowledge reference.
10
---------------------- Page: 12 ----------------------
kSIST-TP FprCEN/TR 17802:2022
FprCEN/TR 17802:2021 (E)
In the example “A.2. Service Level Management” five knowledge and also five skills examples are listed,
from “K1 SLA documentation” to “K5 impact of service level non-compliance on business performance”
and from “S1 analyse service provision records” to “S5 anticipate and mitigate against potential service
disruptions”.
Consequently, typical knowledge and skills (K&S), applied by an ICT professional, are identifiable,
however the lists provided are informative, not normative nor exhaustive.
In summary:
— Dimension 1 describes five e-competence areas: plan, build, run, enable and manage. This dimension
is of limited use for assessment.
— Dimension 2 describes each e-competence with concise sentences that explain the key features of the
competence, describing activities and using verbs such as “responds”, “assures”, “
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.