ETSI GR ENI 041 V4.1.1 (2025-12)

Experiential Networked Intelligence (ENI); Knowledge-Enhanced Network LLMs

DGR/ENI-0041v411_NKMELMNOAM

General Information

• Status: Not Published
• Technical Committee: ENI - Experiential Networked Intelligence
• Current Stage: 12 - Citation in the OJ (auto-insert)
• Due Date: 27-Dec-2025
• Completion Date: 09-Dec-2025

Standards Content (Sample)

GROUP REPORT
Experiential Networked Intelligence (ENI);
Knowledge-Enhanced Network LLMs
Disclaimer
The present document has been produced and approved by the Experiential Networked Intelligence (ENI) ETSI Industry
Specification Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.

2 ETSI GR ENI 041 V4.1.1 (2025-12)

Reference
DGR/ENI-0041v411_NKMELMNOAM
Keywords
knowledge-enhanced, network large language
model
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: [phone]  Fax: [phone]

Siret N° 348 623 562 00017 - APE 7112B
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° w061004871

Important notice
The present document can be downloaded from the
ETSI Search & Browse Standards application.
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format on ETSI deliver repository.
Users should be aware that the present document may be revised or have its status changed,
this information is available in the Milestones listing.
If you find errors in the present document, please send your comments to
the relevant service listed under Committee Support Staff.
If you find a security vulnerability in the present document, please report it through our
Coordinated Vulnerability Disclosure (CVD) program.
Notice of disclaimer & limitation of liability
The information provided in the present deliverable is directed solely to professionals who have the appropriate degree of
experience to understand and interpret its content in accordance with generally accepted engineering or
other professional standard and applicable regulations.
No recommendation as to products and services or vendors is made or should be implied.
No representation or warranty is made that this deliverable is technically accurate or sufficient or conforms to any law
and/or governmental rule and/or regulation and further, no representation or warranty is made of merchantability or fitness
for any particular purpose or against infringement of intellectual property rights.
In no event shall ETSI be held liable for loss of profits or any other incidental or consequential damages.

Any software contained in this deliverable is provided "AS IS" with no warranties, express or implied, including but not
limited to, the warranties of merchantability, fitness for a particular purpose and non-infringement of intellectual property
rights and ETSI shall not be held liable in any event for any damages whatsoever (including, without limitation, damages
for loss of profits, business interruption, loss of information, or any other pecuniary loss) arising out of or related to the use
of or inability to use the software.
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and
microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.

© ETSI 2025.
All rights reserved.
ETSI
3 ETSI GR ENI 041 V4.1.1 (2025-12)
Contents
Intellectual Property Rights . 4
Foreword . 4
Modal verbs terminology . 4
Introduction . 4
1 Scope . 5
2 References . 5
2.1 Normative references . 5
2.2 Informative references . 5
3 Definition of terms, symbols and abbreviations . 6
3.1 Terms . 6
3.2 Symbols . 6
3.3 Abbreviations . 6
4 Background and Motivation . 7
4.1 Background . 7
4.2 Motivation . 7
4.3 Enhancing LLMs with External Knowledge . 9
4.3.1 Characteristics . 9
4.3.2 Types of External Knowledge Sources . 9
4.3.3 RAG Systems. 9
5 Creating Network Knowledges to Enhance LLMs . 10
5.1 Overview . 10
5.2 Creating a Knowledge Graph . 11
5.3 Creating a Knowledge Base . 14
6 Training of Knowledge-Enhanced LLMs . 15
6.1 Introduction . 15
6.2 Training Objectives . 16
6.3 Data Engineering . 16
6.3.1 Data Sources . 16
6.3.2 Data Engineering stage . 17
6.4 Training of LLM . 17
6.4.1 Model Selection . 17
6.4.2 Model Training . 18
6.4.3 Model Evaluation and Optimization . 18
6.4.3.1 Model Evaluation . 18
6.4.3.2 Model Optimization . 20
6.5 Deployment and Inference: Operationalizing a RAG System . 20
7 Application scenarios . 21
7.1 Knowledge Q&A . 21
7.2 Content recommendation . 22
7.3 Prediction . 22
7.4 Content generation . 23
8 Summary and Recommendations . 23
8.1 Summary . 23
8.2 Recommendations . 24
Annex B: Change history . 25
History . 26

ETSI
4 ETSI GR ENI 041 V4.1.1 (2025-12)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The declarations
pertaining to these essential IPRs, if any, are publicly available for ETSI members and non-members, and can be
found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to
ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the
ETSI IPR online database.
Pursuant to the ETSI Directives including the ETSI IPR Policy, no investigation regarding the essentiality of IPRs,
including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not
referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become,
essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
DECT™, PLUGTESTS™, UMTS™ and the ETSI logo are trademarks of ETSI registered for the benefit of its
Members. 3GPP™, LTE™ and 5G™ logo are trademarks of ETSI registered for the benefit of its Members and of the
3GPP Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of ®
the oneM2M Partners. GSM and the GSM logo are trademarks registered and owned by the GSM Association.
Foreword
This Group Report (GR) has been produced by ETSI Industry Specification Group (ISG) Experiential Networked
Intelligence (ENI).
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
In the field of communication networks, large language models face issues such as knowledge cutoff, high training
costs, long development and training cycles, a tendency to hallucinate, difficulty in providing deep knowledge in
specialized areas, and the complexity in ingesting and generating real-time network operational data. The present
document describes the development of knowledge-enhanced network Large Language Models. The present document
explains the motivation for developing a Knowledge-Enhanced Large Language Model, followed by recommendations
for how a Large Language Model is trained and then used for network Operation, Administration, Maintenance, and
Performance (OAMP) operations.
ETSI
5 ETSI GR ENI 041 V4.1.1 (2025-12)
1 Scope
The present document describes the development of knowledge-enhanced network Large Language Models.
The present document will explain the motivation for developing a knowledge-enhanced network Large Language
Model, followed by recommendations for how a network Large Language Model is trained and then used for network
Operation, Administration, Maintenance, and Performance (OAMP) operations.
2 References
2.1 Normative references
Normative references are not applicable in the present document.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long-term validity.
The following referenced documents may be useful in implementing an ETSI deliverable or add to the reader's
understanding, but are not required for conformance to the present document.
[i.1] ETSI GR ENI 004 (V3.1.1): "Experiential Networked Intelligence (ENI); ENI terminology".
[i.2] ETSI GR ENI 016 (V2.1.1): "Experiential Networked Intelligence (ENI); Functional Concepts for
Modular System Operation".
[i.3] ETSI GS ENI 005 (V3.1.1): "Experiential Networked Intelligence (ENI); ENI System
Architecture".
[i.4] ETSI GS ENI 030 (V4.1.1): "Experiential Networked Intelligence (ENI); Transformer
Architecture for Policy Translation; Knowledge-based Reasoning using the Functionalities of
Transformers and Knowledge Graphs to Generate Policies".
[i.5] ETSI GR ENI 031 (V4.1.1): "Experiential Networked Intelligence (ENI); Construction and
application of fault maintenance network knowledge graphs".
[i.6] Anwar, M. et al.: "Understanding Misunderstandings: Evaluating LLMs on Networking
Questions". In Proceedings of the ACM SIGCOMM2024Conference.
[i.7] R. Wang et al.: "Role Prompting Guided Domain Adaptation with General Capability Preserve for
Large Language Models". In Findings of the Association for Computational Linguistics: NAACL
2024.
[i.8] D. Edge et al.: "From Local to Global: A GraphRAG Approach to Query-Focused
Summarization", April 2024 (latest version is February 2025).
ETSI
6 ETSI GR ENI 041 V4.1.1 (2025-12)
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the terms given in ETSI GR ENI 004 [i.1], ETSI GS ENI 005 [i.3], ETSI
GS ENI 030 [i.4] and the following apply:
knowledge: analysis of data and information, resulting in an understanding of what the data and information mean
NOTE 1: Knowledge represents a set of patterns that are used to explain, as well as predict, what has happened, is
happening, or is possible to happen in the future; it is based on acquisition of data, information, and skills
through experience and education.
NOTE 2: While "analysis" is key, the process of gaining knowledge sometimes also involves synthesis,
interpretation, and reflection.
1) inferred knowledge: knowledge created based on reasoning using evidence provided
2) measured knowledge: knowledge resulting from the analysis of data and information that was measured or
reported
3) propositional knowledge: knowledge of a proposition, along with a set of facts that prove (or disprove) the
proposition
NOTE 1: This is available in ETSI GS ENI 005 [i.3].
NOTE 2: The standard philosophical definition of propositional knowledge, often called the "Justified True
Belief" (JTB) account, has three conditions:
1) Belief: the proposition is believed;
2) Truth: the proposition is true;
3) Justification: there is a good reason or justification for believing it is true.
knowledge-enhanced LLM: LLM that is systematically augmented with structured external knowledge sources to
improve factual accuracy, reasoning depth, and interpretability
pipeline: end-to-end construct that orchestrates a flow of events and data in response to a trigger
NOTE: This is available in ETSI GS ENI 030 [i.4].
Retrieval-Augmented Generation (RAG) pipeline: end-to-end construct comprising retrieval and generation modules
that collaboratively and dynamically enhance language model outputs by leveraging external knowledge bases
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the abbreviations given in ETSI GR ENI 004 [i.1], ETSI GS ENI 005 [i.3],
ETSI GR ENI 016 [i.2] and the following apply:
ChatGPT Chat Generative Pre-trained Transformer
COT Chain-Of-Thought
CPT Continual Pre-Training
DAPT Domain Adaptive Pre-Training
GPT Generative Pre-Trained transformer
ICL In-Context Learning
IRCOT Interleaved Retrieval with Chain-Of-Thought
ETSI
7 ETSI GR ENI 041 V4.1.1 (2025-12)
LLM Large Language Model
KELLM Knowledge-Enhanced Large Language Model
NER Named Entity Recognition
NLP Natural Language Processing
NKELLM Network Knowledge-Enhanced Large Language Model
RAFT Retrieval-Augmented Fine-Tuning
RAG Retrieval-Augmented Generation
SFT Supervised Fine-tuning
TSFT Task-Specific Fine-Tuning
4 Background and Motivation
4.1 Background
In the field of communication networks, Large Language Models (LLMs) typically have a "knowledge cutoff," which
occurs as a consequence of static training datasets and the prohibitive cost of continuous retraining. This is the point in
time up to which the model has been trained with data. Any information or events that happened after the cutoff date are
unknown to the model. The knowledge cutoff exists due to high training costs and long development and training
cycles. In addition, additional work needs to be done to mitigate hallucinations, provide deep knowledge in specialized
areas, and ingest and generate real-time network operational data.
There are a number of approaches to addressing the knowledge cutoff problem. One approach is to use a hybrid fine-
tuning method such as Domain Adaptive Pre-Training (DAPT) with task-specific tuning using curated datasets to
preserve generalizability. However, DAPT is typically cost-prohibitive.
Another approach is to use In-Context Learning (ICL) with Retrieval-Augmented Generation (RAG). During inference,
a combination of ICL and an advanced RAG solution will provide appropriate up-to-date domain-specific knowledge.
This combination directly addresses the need for "up-to-date domain-specific knowledge" without the high cost of
retraining the entire model.
Typical challenges include:
1) training data pipelines often include outdated documents, causing discrepancies between reported and effective
cutoffs; and
2) semantic duplicates during pretraining can skew knowledge recency.
4.2 Motivation
The rapid growth of LLMs has brought significant advancements in natural language understanding, generation, and
decision-making. Networking environments, characterized by rapid changes, diverse data sources, and complex
topologies, require models that can effectively manage and utilize knowledge relevant to these systems. However,
deploying LLMs in specialized fields like networking often highlights the limitations of these models in leveraging
domain-specific knowledge. For example, while domain adaptation mechanisms such as Continual Pre-Training (CPT)
and Retrieval-Augmented Fine-Tuning (RAFT) work well on most domains, studies show the performance of LLMs in
the networking domain is often inconsistent and unreliable for professional use without enhancement [i.6]. This stems
from their tendency to prioritize statistical patterns over causal reasoning - a critical requirement in root-cause analysis.
A fundamental limitation arises during domain adaptation. When a general-purpose LLM is fine-tuned on a narrow
domain like networking, it often loses its broad linguistic capabilities. This is a fundamental challenge because the
process of improving specialized performance can degrade general performance, challenging the viability of a single
model for hybrid tasks ([i.7]). This is called "catastrophic forgetting." More specifically, this is caused by the model's
parameters being updated to get better at that specific domain. This in turn causes the model to overwrite (or "forget")
the information it previously learned, leading to a degradation in its general capabilities (like summarization, general
conversation, etc.). This trade-off challenges the viability of single-model solutions for hybrid tasks requiring both
technical and linguistic proficiency [i.7].
ETSI
8 ETSI GR ENI 041 V4.1.1 (2025-12)
The motivation for developing a "Knowledge-Enhanced LLM" (KELLM) stems from the unique challenges posed by
modern networking environments and the limitations of traditional LLMs in handling domain-specific and complex
scenarios. Key motivating factors include:
1) Temporal Dynamics
Networking environments evolve at millisecond timescales, with traffic patterns, device states, and security threats
changing dynamically. Current LLM architectures, which rely on static pre-training corpora and batch-oriented updates,
struggle to maintain situational awareness. For example:
a) Latency Sensitivity: Edge-deployed LLMs are required to process network telemetry within 50 ms to 100 ms
to support real-time decisions like traffic rerouting. However, even optimized models like GPT-3.5-turbo
exhibit median inference latencies of 320 ms on standard GPU hardware, exceeding acceptable thresholds for
time-sensitive operations.
b) Concept Drift: Network configurations and threat landscapes change unpredictably. Without continuous online
learning, LLM accuracy degrades by 2 % to 4 % weekly in production environments.
2) Security Concerns
Deploying LLMs in critical network infrastructure introduces novel attack vectors:
a) Adversarial Prompting: Malicious actors can exploit LLM vulnerabilities to generate harmful configurations.
In controlled tests, researchers induced BGP route leaks 37 % of the time by crafting subtle prompt variations.
b) Data Leakage Risks: LLMs processing network logs sometimes inadvertently memorize and expose sensitive
information like IP addresses or authentication tokens. Differential privacy methods reduce this risk but
increase model perplexity by 30 % to 40 %, harming task performance.
3) Multi-source knowledge integration
Networking involves specialized protocols, configurations, and problem-solving methods. Typical training recipes for
existing LLMs do not include network telemetry, logs, and other OAMP examples. Since existing LLMs have not seen
enough examples of these types of data, the LLM has not learned the statistical patterns, relationships, and vocabulary
of the networking domain. Therefore, it cannot generate accurate or reliable responses for tasks like fault detection,
routing optimization, or security analysis. Enhancing an LLM with networking domain-specific knowledge bridges this
gap, enabling it to address complex technical challenges effectively. However, this can be enhanced using multi-agent
frameworks like Microsoft's AutoGen and LangChain, which enable collaborative problem-solving by delegating
subtasks to specialized agents. For example, one agent might parse network logs, another analyse traffic anomalies, and
a third generate mitigation strategies. This also enables data fusion to holistically combine the results of these tasks to
look for related knowledge. These systems often leverage RAG to ground responses in real-time data from vector
databases.
4) Reduce the hallucination of LLMs
Hallucinations are incorrect or fabricated outputs. They pose significant risks in networking, where an erroneous
configuration suggestion or a misdiagnosed fault can disrupt operations. Generic LLMs, often trained on data that lacks
specialized context for networking operations, can generate ungrounded (i.e. not based on facts) responses. To mitigate
this, RAG pipelines can be used to ground LLM responses to a set of comprehensive knowledge sources. Each
knowledge source contains essential static information such as device configurations, network topology documents, and
architectural standards as well as dynamic data that need to be continuously updated with the outputs from network
monitoring tools. By retrieving relevant, up-to-date information before generating a response, the LLM's outputs
become anchored in fact, demonstrably reducing hallucinations. Another complementary technique is Chain-of-Thought
(CoT) prompting, see ETSI GS ENI 030 [i.4] or similar mechanisms, which guide the model to break down a problem
and generate a sequence of reasoning steps. While these steps are not typically verified by external tools in real-time,
they make the model's logical pathway transparent and can improve the quality of the final output. For true interactive
verification, AI Agent-based systems can be used, where guardrails cross-reference proposed actions (e.g. generated
CLI commands) against device APIs or operational policies before execution.
5) Improve the accuracy rate of LLMs
Networking problems often require solutions tailored to specific conditions, such as topology, traffic patterns, and
historical data. An LLM enhanced with knowledge can provide more context-aware, precise, and actionable insights,
improving operational outcomes.
ETSI
9 ETSI GR ENI 041 V4.1.1 (2025-12)
To improve learning effectiveness, a structured training approach, sometimes called curriculum learning, can be
beneficial. This involves first exposing the model to foundational concepts, such as the principles of individual network
protocols or the functions of basic device types. However, a simplistic hierarchical strategy is insufficient on its own.
True network intelligence requires understanding the complex, non-hierarchical interdependencies between different
domains. For example, how a security policy can influence routing decisions, or how application performance needs
dictate quality of service configurations. Therefore, the training process needs to progress from foundational knowledge
to modelling these intricate, cross-layer and cross-technology relationships, which are often better represented as a
dense knowledge graph rather than a simple hierarchy.
6) Enhancing human-computer interaction
Networking teams face challenges in interpreting and acting upon dense technical data. An LLM can serve as an
intermediary, simplifying complex information, enabling better decision-making, and improving collaboration between
human operators and automated systems. By analysing the intent of user input in depth, it is possible to understand
exactly what the requirements are. Whether it is a simple query or a complex task description, the model captures key
information better and provides a more tailored response to the user's needs. For example, in the intelligent customer
service scenario, can quickly understand the user enquiry product problems, and give targeted answers.
4.3 Enhancing LLMs with External Knowledge
4.3.1 Characteristics
A Network Knowledge-Enhanced Large Language Model (NKELLM) is characterized by its ability to:
1) access and reason over external knowledge sources beyond its parametric memory;
2) maintain up-to-date information without retraining; and
3) perform domain-specific tasks with contextual accuracy.
Certain types of NKELLMs provide the ability to mitigate hallucinations through evidence-based generation and/or
types of logical reasoning, see ETSI GS ENI 030 [i.4]. The most effective systems employ hybrid retrieval from
multiple validated sources while implementing rigorous consistency checks.
4.3.2 Types of External Knowledge Sources
The most common external source is a Knowledge Graph, for the descriptions of Knowledge Graph can see ETSI
GS ENI 005 [i.3] and ETSI GS ENI 030 [i.4], since this provides a structured and efficient way to represent and utilize
knowledge. Other forms of external knowledge sources include structured knowledge bases (e.g. Wikidata and
DBPedia), unstructured or semi-structured corpora (e.g. academic paper repository or technical documentation),
multimedia archives (e.g. video transcripts or image-text pairs, such as LAION-5B), and dynamic knowledge sources
(e.g. network telemetry and trouble tickets). However, the dynamic knowledge sources listed cannot be directly ingested
by any type of LLM. These types of sources, though very valuable, require specialized preprocessing and infrastructure,
including protocol-specific decoders for raw data normalization, stream processing engines for temporal
alignment/aggregation, and domain-optimized Natural Language Processing (NLP) pipelines for ticket analysis.
The present document focusses on different types of knowledge, including knowledge base and knowledge graph to
serve as the external knowledge source. The present document will also examine mechanisms to increase knowledge
enhancement when using RAG, such as in-context learning. Examples of using a Knowledge Graph to create an
NKELLM are defined in ETSI GS ENI 030 [i.4] and are not covered in the present document.
4.3.3 RAG Systems
There are a large number of RAG systems, each with different architectures optimized for different problems:
1) Auto-RAG use LLMs to autonomously decide when/what to retrieve using reinforcement learning.
2) Modular RAG decouples retrieval, reranking, and generation into interchangeable components, enabling
customized pipelines for domain-specific needs.
ETSI
10 ETSI GR ENI 041 V4.1.1 (2025-12)
3) Retrieval-Augmented Fine-Tuning (RAFT) fine-tunes LLMs on datasets where answers depend on retrieved
documents, teaching models to identify irrelevant contexts and synthesize multi-document evidence.
4) Recursive RAG retrieves hierarchical document summaries before drilling into granular chunks, ensuring
comprehensive context capture.
5) Iterative RAG systems improve upon standard RAG by performing multiple cycles of refinement. It iteratively
improves the quality of output by repeating the retrieval process in a loop, refining the search criteria each
time based on the previous results until the desired outcome is achieved.
6) Interleaved Retrieval with Chain-of-Thought (IRCOT) dynamically integrates external information retrieval
with structured reasoning. IRCOT transforms RAG from a static 'retrieve-then-answer' process into a dynamic,
multi-step reasoning loop. Instead of retrieving all potentially relevant information at the start, the model
follows a 'think-search-assimilate' cycle. It begins reasoning (the 'Chain-of-Thought'), identifies a specific
unknown, triggers a targeted retrieval for just that piece of information, and then integrates the new fact to
continue its reasoning process. This iterative approach mimics how a human expert solves a problem, leading
to more focused and efficient information gathering.
For RAG systems enhanced with networking knowledge, it is recommended to start with a Modular RAG architecture
due to its flexibility. For specific, high-value application scenarios like automated fault diagnosis or advanced network
optimization, it is recommended implementing IRCOT-style reasoning loops within that modular framework. These
systems can effectively handle advanced strategy design, network architecture consulting, and similar tasks.
Every RAG system needs to have the following functions:
1) Query parsing: Tokenization, Named Entity Recognition (NER), keyword extraction, and multi-turn context
tracking.
2) Retrieval: Variations include sparse retrieval, dense vector retrieval, and optionally hybrid retrieval.
3) Result Processing: Reranking of retrieved documents, context assembly with prompt templating.
4) Generation: Response generation via LLMs (e.g. ChatGPT, LLaMA, Gemini), source attribution, and answer
formatting.
5) Data pipeline: Document upload, chunking, cleaning, embedding generation, and managing vector/index
databases.
5 Creating Network Knowledges to Enhance LLMs
5.1 Overview
Creating a KELLM requires external knowledge, which often comes from operation manuals, regulations, equipment
manuals, papers, third-party databases, etc. One way to do this is to develop a knowledge graph-enhanced LLM (or
transformer), which incorporates the knowledge graph during the pre-training and inference phases of the LLM (or
transformer). For this to work, knowledge needs to be stored in a format that can be understood by the LLM (or
transformer). Therefore, extracting knowledge from an LLM (or transformer) to construct a knowledge graph or
database has become a common practice.
The LLM can be pre-trained by using the entity and relationship information in the knowledge graph ETSI
GS ENI 030 [i.4]. This is called the "knowledge injection" approach, which puts semantic information into the model's
architecture or input. While the knowledge becomes part of the model's core reasoning, this is not an optimal approach.
This is because it is extremely expensive, inflexible (if the knowledge graph changes, the LLM needs to be retrained),
and suffers from catastrophic forgetting.
A better approach is to use a modified RAG system that is integrated with a knowledge graph. In this approach, the
LLM first understands the user query and identifies key entities and relationships. Then, instead of a simple vector
search, it does a graph-based retrieval, that returns a subgraph. The subgraph is then passed to the LLM as context.
This approach is known as GraphRAG [i.8].
ETSI
11 ETSI GR ENI 041 V4.1.1 (2025-12)
The knowledge in the knowledge graph (e.g. nodes, edges, properties of both, and constraints), when grounded by a set
of ontologies see ETSI GS ENI 030 [i.4], can be used as the basis and reference for large language model reasoning. An
ontology provides:
1) The schema and vocabulary for the knowledge graph to use. For example, it defines what types of entities can
exist (e.g. Router, Firewall, VLAN), what properties they can have (ipAddress, firmwareVersion), and what
types of relationships can link them (connectsTo, isMonitoredBy). Without this, the graph does not have
consistent semantics.
2) It enables reliable querying: For an LLM to "call on the relevant knowledge," it needs to know how to ask for
it. The ontology provides the consistent vocabulary for querying. Without it, the LLM would have to guess
whether to ask for a firewall, a security_appliance, or a packet_filter. An ontology ensures there is one,
unambiguous term.
3) It enables logical inference: This is the most critical point. A good ontology includes axioms and constraints
(e.g. "a Router is a subclass of NetworkDevice"; "every Interface needs to be connectedTo something"). This
enables the system to infer new facts that are not explicitly stated in the graph, which is the foundation of true
reasoning.
The LLM can call on the relevant knowledge in the knowledge base for logical analysis and deduction, so as to get a
more reasonable answer. This is typically done using either RAG, GraphRAG, and/or prompt augmentation, where
relevant facts from the knowledge base are retrieved and supplied as context for the LLM's response generation.
According to different application scenarios and user needs, a specific knowledge base can be constructed, so that the
LLM can be customized in specific areas. For example, in an intelligent customer service scenario, a knowledge base is
built for product knowledge and common problems of users, so that the model can better provide accurate help for
users; in the field of education, a subject knowledge base is built that supports models for teaching and answering
questions.
NOTE: An LLM is still a statistical model, even when using a knowledge graph as described above. For example,
it does not perform causal reasoning. However, with targeted training on causal rules or with new
causal-aware architectures, an LLM learns to approximate some aspects of causal reasoning and apply
causal principles in limited contexts.
5.2 Creating a Knowledge Graph
The construction process of knowledge graph refers to ETSI GR ENI 031 [i.5] on network fault maintenance
knowledge graph construction. For content related to knowledge graph function management and capability
orchestration, refer to ETSI GS ENI 030 [i.4].The construction process is as follows:
1) Data acquisition and processing
Data sources include two categories: knowledge data and network operation data.
The scope of knowledge data includes but is not limited to:
a) World knowledge, such as general web pages, books, encyclopaedias, code and other data.
b) Industry data related to network, such as network-related standards and specifications, papers, patents, books,
etc.
c) Work order data for networks, such as customer complaint data, fault work orders, activation work orders,
service incidents, etc.
Network operation data includes but is not limited to:
a) Network operation status data, such as configuration data, alarm data, fault data, performance data, log data,
DPI data, network topology data, command data; etc.
b) Inspection and testing-type data, such as operation data, drive test data, etc.
The data above needs to be regularly collected, processed (e.g. perform tasks such as data filtering, correlation,
cleansing, and deduplication) and normalized. After processing, the data is stored for further processing.
ETSI
12 ETSI GR ENI 041 V4.1.1 (2025-12)
2) Knowledge Extraction
Knowledge extraction transforms unstructured or semi-structured network data into structured knowledge suitable for
inclusion in a knowledge graph. This process requires sophisticated techniques to handle the domain's inherent
complexity and consists of three main subtasks: entity extraction, relationship extraction, and attribute extraction.
Entity Extraction involves identifying and categorizing key entities from diverse documents and data sources. The
primary goal is to recognize concepts critical to network operations. Exemplary entities include network devices
(e.g. routers, switches, firewalls), logical constructs (e.g. VLANs, VRFs), protocols (e.g. OSPF, BGP, SNMP),
technical concepts (e.g. Quality of Service, MPLS), and operational artifacts (e.g. trouble tickets, configuration files). A
significant challenge in the networking domain is ambiguity that requires deep contextual understanding. For example,
the term "port 22" needs to be disambiguated to determine if it refers to TCP/22 (SSH), UDP/22, or a physical interface
labelled '22' on a device. Similarly, an alert like "link down" requires disambiguation to distinguish between an
administrative shutdown (intentional) and a physical layer failure (unintentional). Standard NER models need to be
fine-tuned with network-specific data to resolve these cases correctly. Once extracted and disambiguated, entities need
be linked to a canonical entry in the knowledge graph to ensure consistency and prevent data duplication.
Relationship Extraction identifies and classifies the semantic relationships between the extracted entities. There are
two types of relationships: explicit and implicit. Explicit relationships are stated clearly in text, such as in design
documents ("Router-A is connected to Switch-B") or technical specifications ("BGP is a type of exterior gateway
protocol"). Transformer-based workflows are effective at identifying these explicit links. Implicit relationships are not
directly stated but are implied by the data. For instance, a specific CLI command in a configuration file (interface
GigabitEthernet0/1; ip address 10.1.1.1 255.255.255.0) implicitly creates a hasIPAddress relationship between the
interface and the IP address. Likewise, a sequence of log messages can imply a causal relationship between a power
spike and a subsequent device reboot. Extracting these implicit links requires advanced natural language processing
(NLP) models capable of understanding command syntax, log semantics, and operational cause-and-effect, moving
beyond simple text analysis.
Attribute Extraction retrieves descriptive properties or characteristics of entities, populating the knowledge graph with
detailed, queryable data. It starts with finding specific attribute-value pairs associated with an entity. For a network
device, this can include its model number, serial number, firmware version, or number of ports. For a protocol, it can be
its administrative distance or timer values. Then, a combination of pattern matching (using regular expressions for
structured data like IP addresses or MAC addresses) and NLP-based slot filling is typically used to extract attributes
from both structured configuration files and unstructured text. Then, extracted attributes are normalized to a consistent
format (e.g. standardizing units, date formats) and validated against schema constraints or authoritative sources to
maintain data quality within the knowledge graph.
3) Knowledge Fusion
Knowledge fusion is a crucial step in constructing a knowledge graph related to network knowledge. It integrates
information and knowledge from multiple, often heterogeneous, sources into a single, unified and consistent
representation, such as a knowledge graph. It involves resolving conflicts and redundancies between sources to create a
coherent dataset, which can then be used to infer new knowledge and generate a more holistic understanding of a
domain. The main processes are as follows:
1. Schema Integration
Before merging data, it is necessary to align the schemas of the different sources. This involves mapping
equivalent classes, properties, and constraints to ensure that entities and relationships from different sources
are interpreted consistently within the unified knowledge graph. The use of a formal ontology is critical here,
as it provi
...