IEC TR 61850-90-12:2015

Communication networks and systems for power utility automation - Part 90-12: Wide area network engineering guidelines

IEC TR 61850-90-12:2015(E) provides definitions, guidelines, and recommendations for the engineering of WANs, in particular for protection, control and monitoring based on IEC 61850 and related standards. It addresses substation-to-substation communication, substation-to-control centre and control centre-to-control centre communication. In particular, this Technical Report addresses the most critical aspects of IEC 61850 such as protection related data transmission via GOOSE and SMVs, and the multicast transfer of large volumes of synchrophasor data. The Technical Report addresses issues such as topology, redundancy, traffic latency and quality of service, traffic management, clock synchronization, security and maintenance of the network. This Technical Report contains use cases that show how utilities tackle their WAN engineering. This Technical Report is intended for an audience familiar with electrical power automation based on IEC 61850 and particularly for data network engineers and system integrators. It is intended to help them to understand the technologies, configure a wide area network, define requirements, write specifications, select components and conduct tests.

General Information

• Status: Published
• Publication Date: 22-Jul-2015
• ICS: 33.200 - Telecontrol. Telemetering
• Technical Committee: TC 57 - Power systems management and associated information exchange
• Drafting Committee: WG 10 - TC 57/WG 10
• Current Stage: DELPUB - Deleted Publication
• Start Date: 28-Jul-2020
• Completion Date: 26-Oct-2025

Relations

• Revised: IEC TR 61850-90-12:2020 - Communication networks and systems for power utility automation - Part 90-12: Wide area network engineering guidelines
• Effective Date: 05-Sep-2023

Overview

IEC TR 61850-90-12:2015 - "Communication networks and systems for power utility automation - Part 90‑12: Wide area network engineering guidelines" - provides engineering guidance for designing, specifying and testing wide area networks (WANs) that carry IEC 61850-based protection, control and monitoring traffic. The Technical Report focuses on substation-to-substation, substation-to-control-centre and control-centre-to-control-centre communications and addresses critical real‑time traffic such as GOOSE, Sampled Measured Values (SMV) and high‑volume synchrophasor (WAMS) multicast streams.

Key topics and technical requirements

The report covers practical engineering topics and measurable network requirements, including:

• Traffic types & QoS: classification of protection, telecontrol and monitoring traffic with Quality of Service considerations for packet‑switched and TDM networks.
• Latency, jitter and symmetry: latency components, jitter definitions and the impact of asymmetric paths on protection schemes.
• Clock synchronization & time accuracy: requirements for precise timestamps used by synchrophasors and protection functions.
• Redundancy & availability: topology choices, redundancy principles, single‑point of failure analysis and recovery delay.
• Security & maintenance: network security measures, operations and maintenance practices for utility WANs.
• Traffic management & multicast: multicast handling, tunneling and strategies for high‑volume synchrophasor distribution.
• Layered technologies: guidance on Layer 1–4 technologies and overlays such as fiber, microwave, PLC, SDH/SONET, Carrier Ethernet, MPLS, VPNs, and tunneling of GOOSE/SMV.
• Testing and specification: examples and metrics to help define requirements, select components and conduct acceptance tests.
• Use cases and real utility examples (e.g., ENDESA, Hydro‑Quebec) illustrate practical deployments.

Applications and who uses it

IEC TR 61850-90-12 is intended for:

• Data network engineers and system integrators designing utility WANs that must support IEC 61850 traffic.
• Protection and control engineers who need to translate protection/telecontrol requirements (latency, symmetry, integrity) into network specs.
• Utility architects and operators implementing WAMS/WAMPAC, teleprotection, telecontrol, DER/wind farm connectivity and control center interconnections.

Practical uses:

• Specifying WAN performance (latency, jitter, availability) for substation-to-substation GOOSE/SMV links.
• Designing multicast distribution for synchrophasor systems and WAMPAC.
• Selecting physical and transport technologies (fiber, microwave, MPLS, Carrier Ethernet) and validating redundancy and security approaches.

Related standards

• IEC 61850 family (substation automation models and requirements)
• IEC 61850‑5 (referenced latency/jitter classes)
• IEC TR 62357 (TC57 architecture mapping)
• IEEE C37.118 (synchrophasor/C37.118 example shown in report)

Keywords: IEC TR 61850-90-12, wide area network engineering, WAN for utilities, IEC 61850, GOOSE, SMV, synchrophasor, WAMS, teleprotection, QoS, latency, redundancy, network engineers.

Standards Content (Sample)

IEC TR 61850-90-12 ®
Edition 1.0 2015-07
TECHNICAL
REPORT
colour
inside
Communication networks and systems for power utility automation –
Part 90-12: Wide area network engineering guidelines
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form
or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from
either IEC or IEC's member National Committee in the country of the requester. If you have any questions about IEC
copyright or have an enquiry about obtaining additional rights to this publication, please contact the address below or
your local IEC member National Committee for further information.

IEC Central Office Tel.: [phone]
3, rue de Varembé Fax: [phone]
CH-1211 Geneva 20 [e-mail]
Switzerland www.iec.ch
About the IEC
The International Electrotechnical Commission (IEC) is the leading global organization that prepares and publishes
International Standards for all electrical, electronic and related technologies.

About IEC publications
The technical content of IEC publications is kept under constant review by the IEC. Please make sure that you have the
latest edition, a corrigenda or an amendment might have been published.

IEC Catalogue - webstore.iec.ch/catalogue Electropedia - www.electropedia.org
The stand-alone application for consulting the entire The world's leading online dictionary of electronic and
bibliographical information on IEC International Standards, electrical terms containing more than 30 000 terms and
Technical Specifications, Technical Reports and other definitions in English and French, with equivalent terms in 15
documents. Available for PC, Mac OS, Android Tablets and additional languages. Also known as the International
iPad. Electrotechnical Vocabulary (IEV) online.

IEC publications search - www.iec.ch/searchpub IEC Glossary - std.iec.ch/glossary
The advanced search enables to find IEC publications by a More than 60 000 electrotechnical terminology entries in
variety of criteria (reference number, text, technical English and French extracted from the Terms and Definitions
committee,…). It also gives information on projects, replaced clause of IEC publications issued since 2002. Some entries
and withdrawn publications. have been collected from earlier publications of IEC TC 37,

77, 86 and CISPR.
IEC Just Published - webstore.iec.ch/justpublished

Stay up to date on all new IEC publications. Just Published IEC Customer Service Centre - webstore.iec.ch/csc
details all new publications released. Available online and If you wish to give us your feedback on this publication or
also once a month by email. need further assistance, please contact the Customer Service
Centre: [e-mail].
IEC TR 61850-90-12 ®
Edition 1.0 2015-07
TECHNICAL
REPORT
colour
inside
Communication networks and systems for power utility automation –

Part 90-12: Wide area network engineering guidelines

INTERNATIONAL
ELECTROTECHNICAL
COMMISSION
ICS 33.200 ISBN 978-2-8322-2806-7

– 2 – IEC TR 61850-90-12:2015 © IEC 2015
CONTENTS
FOREWORD . 11
INTRODUCTION . 13
1 Scope . 15
2 Normative references . 15
3 Terms, definitions, abbreviations, acronyms and symbols . 20
3.1 Terms and definitions . 20
3.2 Abbreviations and acronyms . 25
3.3 Network diagram symbols . 32
4 Wide Area Communication in electrical utilities . 34
4.1 Executive summary . 34
4.2 Use Case: ENDESA, Andalusia (Spain) . 36
4.3 Typical interface between a substation and the WAN . 38
4.4 WAN characteristics and actors . 39
4.5 SGAM Mapping . 40
4.6 Network elements and voltage level . 42
4.7 WAN interfaces in substation automation (IEC 61850-5) . 43
4.8 Logical interfaces and protocols in the TC57 Architecture IEC TR 62357 . 44
4.9 Network traffic and ownership . 45
5 WAN overall requirements and data transmission metrics . 45
5.1 Traffic types . 45
5.2 Quality of Service (QoS) of TDM and PSN . 46
5.3 Latency calculation . 46
5.3.1 Latency components . 46
5.3.2 Propagation delay . 46
5.3.3 Residence delay . 47
5.3.4 Latency accumulation . 47
5.3.5 Example: latency of a microwave system . 47
5.3.6 Latency and determinism . 47
5.3.7 Latency classes in IEC 61850-5 . 48
5.4 Jitter . 50
5.4.1 Jitter definition . 50
5.4.2 Jitter classes in IEC 61850 . 51
5.5 Latency symmetry and path congruency . 51
5.6 Medium asymmetry . 51
5.7 Communication speed symmetry . 52
5.8 Recovery delay . 52
5.9 Time accuracy . 52
5.9.1 Time accuracy definition . 52
5.9.2 Time accuracy classes. 53
5.10 Tolerance against failures . 54
5.10.1 Failure . 54
5.10.2 Reliability. 54
5.10.3 Redundancy principles. 55
5.10.4 Redundancy and reliability . 55
5.10.5 Redundancy checking . 56
5.10.6 Redundant layout: single point of failure . 57

5.10.7 Redundant layout: cross-redundancy . 57
5.10.8 Maintainability . 58
5.10.9 Availability . 58
5.10.10 Integrity . 60
5.10.11 Dependability . 62
5.10.12 Example: Dependability of GOOSE transmission . 62
6 Applications analysis . 62
6.1 Application kinds . 62
6.2 Teleprotection (IF2 & IF11) . 63
6.2.1 Teleprotection schemes . 63
6.2.2 Teleprotection data kinds . 64
6.2.3 Teleprotection requirements for latency . 64
6.2.4 Teleprotection requirements for latency asymmetry . 64
6.2.5 Teleprotection requirements for integrity . 64
6.2.6 Teleprotection summary . 65
6.3 Telecontrol (IF1, IF6) . 65
6.4 Substation to control centre (IF10) . 66
6.5 CMD (IF7) . 67
6.5.1 CMD overview . 67
6.5.2 CMD communication requirements . 67
6.6 Control Centre to Control Centre (IF12) . 67
6.7 Wide Area Monitoring System (IF13) . 68
6.7.1 WAMS overview . 68
6.7.2 WAMS topology . 68
6.7.3 WAMS communication requirements . 70
6.8 Wide area monitoring, protection and control (WAMPAC) IF13 . 71
6.8.1 WAMPAC overview . 71
6.8.2 WAMPAC communication requirements . 71
6.8.3 Use case WAMPAC . 73
6.9 Wind turbines and wind virtual power plants . 74
6.10 Distributed Energy and Renewables (DER) . 74
6.11 Summary of communication requirements for WAN . 74
7 Wide-area and real-time network technologies. 75
7.1 Introduction . 75
7.2 Topology . 75
7.3 Overview. 76
7.4 Layer 1 (physical) transmission media . 78
7.4.1 Summary . 78
7.4.2 Installation guidelines . 78
7.4.3 Metallic lines . 79
7.4.4 Power line carrier (PLC) . 80
7.4.5 Radio transmission . 84
7.4.6 Fiber optics. 89
7.4.7 Layer 1 redundancy . 96
7.4.8 Use case: Diverse redundancy against extreme contingencies (Hydro-
Quebec) . 97
7.4.9 Layer 1 security . 98
7.5 Layer 1,5 (physical) multiplexing . 98
7.6 Layer 2 (link) technologies . 99

– 4 – IEC TR 61850-90-12:2015 © IEC 2015
7.6.1 Telephony technologies . 99
7.6.2 SDH/SONET . 101
7.6.3 Optical Transport Network . 111
7.6.4 Ethernet . 112
7.6.5 Ethernet over TDM . 122
7.6.6 Carrier Ethernet . 123
7.6.7 Audio-Video Bridging . 125
7.6.8 Provider Backbone Bridge (PBB) . 125
7.6.9 Multiprotocol Label Switching (MPLS) . 127
7.7 Layer 3 (network) technologies . 135
7.7.1 Internet Protocol (IP) . 135
7.7.2 IP QoS. 148
7.7.3 IP multicast. 151
7.7.4 IP redundancy . 152
7.7.5 IP security . 152
7.7.6 IP communication for utilities . 154
7.7.7 IP summary . 156
7.8 Layer 4 (transport) protocols . 157
7.8.1 Transport layer encapsulation . 157
7.8.2 UDP . 157
7.8.3 TCP . 158
7.8.4 Layer 4 redundancy . 159
7.8.5 Layer 4 security . 159
7.9 Layer 5 (session) and higher . 159
7.9.1 Session layer . 159
7.9.2 Routable GOOSE and SMV . 160
7.9.3 Example: C37.118 transmission . 160
7.9.4 Session protocol for voice and video transmission . 161
7.9.5 Application interface redundancy . 161
7.9.6 Application device redundancy . 162
7.10 Protocol overlay – tunneling . 162
7.10.1 Definitions . 162
7.10.2 Tunneling principle . 163
7.10.3 Tunneling Layer 2 over Layer 3 . 163
7.10.4 Use Case: Tunneling GOOSE and SMV in IEC 61850 . 164
7.10.5 Circuit emulation service (CES) . 165
7.11 Virtual Private Networks (VPNs) . 169
7.11.1 VPN principles . 169
7.11.2 L2VPNs . 169
7.11.3 L2VPN multicast on MPLS . 171
7.11.4 L3VPN . 171
7.11.5 VPN mapping to application . 173
7.12 Cyber Security . 176
7.12.1 Security circles . 176
7.12.2 Network security . 177
7.12.3 Access Control . 180
7.12.4 Threat detection and mitigation . 180
7.12.5 Security architecture . 184
7.12.6 Application (end-to-end) communication security . 185

7.12.7 Security for synchrophasor (PMU) networks (IEC TR 61850-90-5) . 186
7.12.8 Additional recommendations . 187
7.13 QoS and application-specific engineering. 187
7.13.1 General . 187
7.13.2 SDH/SONET QoS and SLA . 187
7.13.3 PSN QoS and SLA . 187
7.13.4 Application and priority . 188
7.13.5 QoS chain between networks . 188
7.13.6 QoS mapping between networks . 189
7.13.7 QoS engineering . 190
7.13.8 Customer restrictions . 191
7.13.9 Clock services . 191
7.14 Configuration and OAM . 191
7.14.1 Network configuration . 191
7.14.2 OAM . 191
7.15 Time synchronization . 193
7.15.1 Oscillator stability . 193
7.15.2 Mutual synchronization . 193
7.15.3 Direct synchronization . 194
7.15.4 Radio synchronization . 194
7.15.5 GNSS synchronization . 195
7.15.6 Frequency distribution . 195
7.15.7 Time distribution . 196
7.15.8 PTP telecommunication profiles . 203
7.15.9 PTP over MPLS . 203
7.15.10 Comparison of time distribution profiles based on IEC 61588 . 203
7.15.11 Use Case: Synchrophasor time synchronization . 205
7.15.12 Use case: Atomic Clock Hierarchy . 205
8 Use cases . 206
8.1 Use case: Current differential teleprotection system (Japan) . 206
8.2 Use case: SDH / MPLS network (Japan) . 210
8.3 Use Case: Wide area stabilizing control system (Japan) . 211
8.4 Use Case: experimental PMU-based WAMPAC system . 213
Bibliography . 216

Figure 1 – Symbols . 33
Figure 2 – Substation locations in Andalusia . 36
Figure 3 – Topology of the Andalusia network . 37
Figure 4 – Cabinet of the substation edge node . 38
Figure 5 – Communication interfaces in a SEN . 39
Figure 6 – Communicating entities . 40
Figure 7 – SGAM communication model. 41
Figure 8 – Principle of grid voltage level and network technology . 42
Figure 9 – Communication paths and interfaces . 43
Figure 10 – IEC TR 62357 Interfaces, protocols and applications . 44
Figure 11 – Composition of end-to-end latency in a microwave relay . 47
Figure 12 – Example of latency in function of traffic . 48

– 6 – IEC TR 61850-90-12:2015 © IEC 2015
Figure 13 – Jitter for two communication delay types. . 50
Figure 14 – Precision and accuracy definitions . 52
Figure 15 – Redundancy of redundant systems . 56
Figure 16 – Redundancy calculation . 56
Figure 17 – Redundancy layout with single point of failure . 57
Figure 18 – Redundancy layout with cross-coupling . 58
Figure 19 – Availability definitions . 59
Figure 20 – Residual error rate in function of the BER . 61
Figure 21 – Principle of synchrophasor transmission . 69
Figure 22 – Target phenomena for WAMPAC . 71
Figure 23 – Example of main function and general information flow . 72
Figure 24 – PMUs and data flow between TSO and regional data hubs . 73
Figure 25 – Network topology (Carrier Ethernet) . 76
Figure 26 – Phase-to-ground coupling for PLC . 80
Figure 27 – HV PLC coupling with suspended line traps . 81
Figure 28 – Phase to phase signal coupling for PLC . 81
Figure 29 – Phase-to-phase signal coupling . 82
Figure 30 – Power line carrier, line traps . 83
Figure 31 – Terrestrial microwave link . 85
Figure 32 – Layer 2 transport on radio systems . 88
Figure 33 – Radio network in feeder automation . 89
Figure 34 – ADSS fiber cable . 90
Figure 35 – ADSS installation with splicing box . 91
Figure 36 – OPGW in ground cable . 91
Figure 37 – OPGW with two “C”-tubes with each 32 fibers . 92
Figure 38 – OPGW fibers . 93
Figure 39 – Splicing box . 94
Figure 40 – WDM over one fiber . 95
Figure 41 – OCh optical components . 95
Figure 42 – Optical link with microwave back-up . 97
Figure 43 – Picture of partially destroyed 735 kV line . 98
Figure 44 – E1 and E2 channel . 100
Figure 45 – Digital Transmission Hierarchy (T – Standards) . 100
Figure 46 – Digital Transmission Hierarchy (E-standard) . 101
Figure 47 – Example of an SDH network for utilities . 102
Figure 48 – SONET multiplexing hierarchy . 103
Figure 49 – SDH multiplexing hierarchy . 103
Figure 50 – SDH/SONET with point-to-point topology . 105
Figure 51 – SDH/SONET with linear topology . 105
Figure 52 – BLSR/BSHR topology in normal conditions (from A to D) . 107
Figure 53 – BLSR/BSHR topology in failure conditions . 107
Figure 54 – UPSR/USHR topology in normal conditions . 108
Figure 55 – UPSR/USHR topology in failure conditions . 109

Figure 56 – Example of information flow relationship in OTN . 112
Figure 57 – IEEE 802.3 (Ethernet) frame format . 113
Figure 58 – IEEE 802.3 (Ethernet) topology with RSTP switches (IEC TR 61850-90-4) . 114
Figure 59 – IEEE 802.1Q-tagged Ethernet frame format . 115
Figure 60 – Direct Ethernet with VLAN in substation-to-substation transmission . 116
Figure 61 – Substation-to-substation Layer 2 transmission tunneled over IP . 117
Figure 62 – PRP structure (within and outside a substation) . 118
Figure 63 – HSR ring connecting substations and control centre . 118
Figure 64 – MACsec frame format . 120
Figure 65 – IEEE 802.1X principle . 121
Figure 66 – Ethernet for substation-to-substation communication. 122
Figure 67 – Packets over TDM . 123
Figure 68 – IEEE 802.1Q/ad/ah network configuration . 126
Figure 69 – Case of IEEE 802.1Q/ad network for utility . 127
Figure 70 – Basic MPLS architecture . 128
Figure 71 – Example of MPLS frame format with IPv4 payload . 129
Figure 72 – MPLS building blocks . 130
Figure 73 – MPLS network architecture for utilities . 131
Figure 74 – IP/MPLS and MPLS-TP features . 132
Figure 75 – MPLS-TP redundant routing . 134
Figure 76 – Ethernet frame with IP network header . 136
Figure 77 – Mapping of IPv4 to Ethernet frames . 137
Figure 78 – Mapping of IPv6 to Ethernet frames . 140
Figure 79 – IPv6 unicast address structure . 141
Figure 80 – IPv6 ULA address structure . 142
Figure 81 – IPv6 link local address structure . 142
Figure 82 – Mapping of IPv4 to IPv6 addresses . 145
Figure 83 – IPv6 evolution . 147
Figure 84 – IEC 61850 stack with IPv4 and IPv6 . 148
Figure 85 – DiffServ codepoint field . 150
Figure 86 – Unidirectional protocol independent multicast . 151
Figure 87 – Bidirectional protocol independent multicast . 152
Figure 88 – Frame format for IPsec (authenticated) . 153
Figure 89 – Frame format for IPsec (encrypted) . 153
Figure 90 – Layer 3 direct connection within same address space . 154
Figure 91 – Connecting substations to SCADA by a NAT . 155
Figure 92 – Substation to SCADA connection over ALG . 156
Figure 93 – Ethernet frame with UDP transport layer . 157
Figure 94 – UDP header . 158
Figure 95 – TCP header . 158
Figure 96 – Session and presentation layers for MMS . 160
Figure 97 – Session and presentation layers for R-GOOSE . 160
Figure 98 – IEEE C37.118 frame over UDP . 161

– 8 – IEC TR 61850-90-12:2015 © IEC 2015
Figure 99 – Redundant network transmission handled by the application layer . 161
Figure 100 – Tunneling in IEC TR 61850-90-1 . 163
Figure 101 – L2TP transporting Layer 2 frames over IP . 164
Figure 102 – Tunneling GOOSE over IP in IEC TR 61850-90-5 . 165
Figure 103 – Pseudo-wire principle . 166
Figure 104 – Non-IP voice communication over PSN . 167
Figure 105 – Circuit emulation over PSN . 168
Figure 106 – L2VPNs VPWS and VPLS . 170
Figure 107 – L3VPN . 171
Figure 108 – Emulation of L3VPN by L2VPN and global router . 172
Figure 109 – Tele-protection over VPWS, . 174
Figure 110 – WAMS over VPLS . 175
Figure 111 – VPN for IP-based SCADA/EMS traffic . 176
Figure 112 – VPN deployment options . 179
Figure 113 – IP network separator . 181
Figure 114 – Security architecture (using segmentation and perimeter security) . 185
Figure 115 – QoS chain . 189
Figure 116 – Timing pulse transmission methods of legacy teleprotection devices . 194
Figure 117 – SyncE application . 195
Figure 118 – Synchronous Ethernet Architecture . 196
Figure 119 – SNTP clock synchronization and network delay measurement . 197
Figure 120 – Model of GMC, two BCs in series and SC over Layer 3 . 200
Figure 121 – Timing diagram of PTP (end-to-end, 2-step, BCs) . 200
Figure 122 – Timing diagram of PTP (peer-to-peer, 2-step TCs) . 201
Figure 123 – Substations synchronization over WAN . 205
Figure 124 – Example of synchronization network . 206
Figure 125 – Current differential 1:1 configuration . 207
Figure 126 – Network configuration for centralized multi-terminal line protection . 207
Figure 127 – Network configuration for distributed multi-terminal line protection . 208
Figure 128 – Current differential teleprotection for HV multi-terminal transmission line
using Layer 2 network . 208
Figure 129 – Configuration of wide area current differential primary and backup
teleprotection system employing Carrier Ethernet and IEC 61588 time synchronization . 210
Figure 130 – Achieving protection for teleprotection services . 211
Figure 131 – System configuration for wide area stabilizing control system . 212
Figure 132 – Appearance of typical CCE cabinet . 213
Figure 133 – Configuration for PMU-based WAMPAC system . 214

Table 1 – Latency classes in IEC 61850-5 . 49
Table 2 – Latency classes in IEC TR 61850-90-1 . 49
Table 3 – Latency classes for WANs . 50
Table 4 – Jitter classes in IEC TR 61850-90-1 . 51
Table 5 – Jitter classes for WAN . 51
Table 6 – Recovery delay classes for WAN . 52

Table 7 – IEC TR 61850-90-1 time accuracy classes . 53
Table 8 – IEC 61850-5 time accuracy classes for IED synchronization . 53
Table 9 – WAN time synchronization classes . 54
Table 10 – Latency for line protection . 64
Table 11 – Summary of operational requirements of line protection . 65
Table 12 – Summary of communication requirements for teleprotection . 65
Table 13 – Communication requirements for CC to SS/PS . 66
Table 14 – Latency and timing requirements from IEC TR 61850-90-2 . 66
Table 15 – Communication requirements for CMD . 67
Table 16 – Communication requirements for inter-control centre communications . 68
Table 17 – Summary of synchrophasor requirements . 70
Table 18 – Summary of communication requirements for wide area monitoring . 71
Table 19 – Typical communication requirements for WAMPAC . 73
Table 20 – Classification of communication requirements . 74
Table 21 – Communication requirements of wide-area applications . 74
Table 22 – Communication technologies . 77
Table 23 – Physical communication media . 78
Table 24 – DSL communication over twisted pairs . 79
Table 25 – Trade-offs in copper cable communication . 80
Table 26 – PLC communication technologies . 83
Table 27 – PLC communication advantages and disadvantages. 84
Table 28 – Microwave link performance . 86
Table 29 – Terrestrial microwave advantages and disadvantages . 86
Table 30 – Public mobile radio technologies . 87
Table 31 – Terrestrial radio advantages and disadvantages . 87
Table 32 – Satellite radio advantages and disadvantages . 87
Table 33 – Optical fibers: advantages and disadvantages . 96
Table 34 – SONET and SDH hierarchies . 104
Table 35 – Summary of SDH/SONET . 111
Table 36 – Ethernet physical layers .
...

IEC TR 61850-90-12 ®
Edition 1.0 2015-07
TECHNICAL
REPORT
colour
inside
Communication networks and systems for power utility automation –
Part 90-12: Wide area network engineering guidelines

All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form

or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from
either IEC or IEC's member National Committee in the country of the requester. If you have any questions about IEC
copyright or have an enquiry about obtaining additional rights to this publication, please contact the address below or

your local IEC member National Committee for further information.

IEC Central Office Tel.: [phone]
3, rue de Varembé Fax: [phone]

CH-1211 Geneva 20 [e-mail]
Switzerland www.iec.ch
About the IEC
The International Electrotechnical Commission (IEC) is the leading global organization that prepares and publishes
International Standards for all electrical, electronic and related technologies.

About IEC publications
The technical content of IEC publications is kept under constant review by the IEC. Please make sure that you have the
latest edition, a corrigenda or an amendment might have been published.

IEC Catalogue - webstore.iec.ch/catalogue Electropedia - www.electropedia.org
The stand-alone application for consulting the entire The world's leading online dictionary of electronic and
bibliographical information on IEC International Standards, electrical terms containing more than 30 000 terms and
Technical Specifications, Technical Reports and other definitions in English and French, with equivalent terms in 15
documents. Available for PC, Mac OS, Android Tablets and additional languages. Also known as the International
iPad. Electrotechnical Vocabulary (IEV) online.

IEC publications search - www.iec.ch/searchpub IEC Glossary - std.iec.ch/glossary
The advanced search enables to find IEC publications by a More than 60 000 electrotechnical terminology entries in
variety of criteria (reference number, text, technical English and French extracted from the Terms and Definitions
committee,…). It also gives information on projects, replaced clause of IEC publications issued since 2002. Some entries
and withdrawn publications. have been collected from earlier publications of IEC TC 37,

77, 86 and CISPR.
IEC Just Published - webstore.iec.ch/justpublished

Stay up to date on all new IEC publications. Just Published IEC Customer Service Centre - webstore.iec.ch/csc
details all new publications released. Available online and If you wish to give us your feedback on this publication or
also once a month by email. need further assistance, please contact the Customer Service
Centre: [e-mail].
IEC TR 61850-90-12 ®
Edition 1.0 2015-07
TECHNICAL
REPORT
colour
inside
Communication networks and systems for power utility automation –

Part 90-12: Wide area network engineering guidelines

INTERNATIONAL
ELECTROTECHNICAL
COMMISSION
ICS 33.200 ISBN 978-2-8322-2806-7

– 2 – IEC TR 61850-90-12:2015 © IEC 2015

CONTENTS
FOREWORD . 11

INTRODUCTION . 13

1 Scope . 15

2 Normative references . 15

3 Terms, definitions, abbreviations, acronyms and symbols . 20

3.1 Terms and definitions . 20

3.2 Abbreviations and acronyms . 25
3.3 Network diagram symbols . 32
4 Wide Area Communication in electrical utilities . 34
4.1 Executive summary . 34
4.2 Use Case: ENDESA, Andalusia (Spain) . 36
4.3 Typical interface between a substation and the WAN . 38
4.4 WAN characteristics and actors . 39
4.5 SGAM Mapping . 40
4.6 Network elements and voltage level . 42
4.7 WAN interfaces in substation automation (IEC 61850-5) . 43
4.8 Logical interfaces and protocols in the TC57 Architecture IEC TR 62357 . 44
4.9 Network traffic and ownership . 45
5 WAN overall requirements and data transmission metrics . 45
5.1 Traffic types . 45
5.2 Quality of Service (QoS) of TDM and PSN . 46
5.3 Latency calculation . 46
5.3.1 Latency components . 46
5.3.2 Propagation delay . 46
5.3.3 Residence delay . 47
5.3.4 Latency accumulation . 47
5.3.5 Example: latency of a microwave system . 47
5.3.6 Latency and determinism . 47
5.3.7 Latency classes in IEC 61850-5 . 48
5.4 Jitter . 50
5.4.1 Jitter definition . 50
5.4.2 Jitter classes in IEC 61850 . 51

5.5 Latency symmetry and path congruency . 51
5.6 Medium asymmetry . 51
5.7 Communication speed symmetry . 52
5.8 Recovery delay . 52
5.9 Time accuracy . 52
5.9.1 Time accuracy definition . 52
5.9.2 Time accuracy classes. 53
5.10 Tolerance against failures . 54
5.10.1 Failure . 54
5.10.2 Reliability. 54
5.10.3 Redundancy principles. 55
5.10.4 Redundancy and reliability . 55
5.10.5 Redundancy checking . 56
5.10.6 Redundant layout: single point of failure . 57

5.10.7 Redundant layout: cross-redundancy . 57

5.10.8 Maintainability . 58

5.10.9 Availability . 58

5.10.10 Integrity . 60

5.10.11 Dependability . 62

5.10.12 Example: Dependability of GOOSE transmission . 62

6 Applications analysis . 62

6.1 Application kinds . 62

6.2 Teleprotection (IF2 & IF11) . 63

6.2.1 Teleprotection schemes . 63
6.2.2 Teleprotection data kinds . 64
6.2.3 Teleprotection requirements for latency . 64
6.2.4 Teleprotection requirements for latency asymmetry . 64
6.2.5 Teleprotection requirements for integrity . 64
6.2.6 Teleprotection summary . 65
6.3 Telecontrol (IF1, IF6) . 65
6.4 Substation to control centre (IF10) . 66
6.5 CMD (IF7) . 67
6.5.1 CMD overview . 67
6.5.2 CMD communication requirements . 67
6.6 Control Centre to Control Centre (IF12) . 67
6.7 Wide Area Monitoring System (IF13) . 68
6.7.1 WAMS overview . 68
6.7.2 WAMS topology . 68
6.7.3 WAMS communication requirements . 70
6.8 Wide area monitoring, protection and control (WAMPAC) IF13 . 71
6.8.1 WAMPAC overview . 71
6.8.2 WAMPAC communication requirements . 71
6.8.3 Use case WAMPAC . 73
6.9 Wind turbines and wind virtual power plants . 74
6.10 Distributed Energy and Renewables (DER) . 74
6.11 Summary of communication requirements for WAN . 74
7 Wide-area and real-time network technologies. 75
7.1 Introduction . 75
7.2 Topology . 75

7.3 Overview. 76
7.4 Layer 1 (physical) transmission media . 78
7.4.1 Summary . 78
7.4.2 Installation guidelines . 78
7.4.3 Metallic lines . 79
7.4.4 Power line carrier (PLC) . 80
7.4.5 Radio transmission . 84
7.4.6 Fiber optics. 89
7.4.7 Layer 1 redundancy . 96
7.4.8 Use case: Diverse redundancy against extreme contingencies (Hydro-
Quebec) . 97
7.4.9 Layer 1 security . 98
7.5 Layer 1,5 (physical) multiplexing . 98
7.6 Layer 2 (link) technologies . 99

– 4 – IEC TR 61850-90-12:2015 © IEC 2015

7.6.1 Telephony technologies . 99

7.6.2 SDH/SONET . 101

7.6.3 Optical Transport Network . 111

7.6.4 Ethernet . 112

7.6.5 Ethernet over TDM . 122

7.6.6 Carrier Ethernet . 123

7.6.7 Audio-Video Bridging . 125

7.6.8 Provider Backbone Bridge (PBB) . 125

7.6.9 Multiprotocol Label Switching (MPLS) . 127

7.7 Layer 3 (network) technologies . 135

7.7.1 Internet Protocol (IP) . 135
7.7.2 IP QoS. 148
7.7.3 IP multicast. 151
7.7.4 IP redundancy . 152
7.7.5 IP security . 152
7.7.6 IP communication for utilities . 154
7.7.7 IP summary . 156
7.8 Layer 4 (transport) protocols . 157
7.8.1 Transport layer encapsulation . 157
7.8.2 UDP . 157
7.8.3 TCP . 158
7.8.4 Layer 4 redundancy . 159
7.8.5 Layer 4 security . 159
7.9 Layer 5 (session) and higher . 159
7.9.1 Session layer . 159
7.9.2 Routable GOOSE and SMV . 160
7.9.3 Example: C37.118 transmission . 160
7.9.4 Session protocol for voice and video transmission . 161
7.9.5 Application interface redundancy . 161
7.9.6 Application device redundancy . 162
7.10 Protocol overlay – tunneling . 162
7.10.1 Definitions . 162
7.10.2 Tunneling principle . 163
7.10.3 Tunneling Layer 2 over Layer 3 . 163
7.10.4 Use Case: Tunneling GOOSE and SMV in IEC 61850 . 164

7.10.5 Circuit emulation service (CES) . 165
7.11 Virtual Private Networks (VPNs) . 169
7.11.1 VPN principles . 169
7.11.2 L2VPNs . 169
7.11.3 L2VPN multicast on MPLS . 171
7.11.4 L3VPN . 171
7.11.5 VPN mapping to application . 173
7.12 Cyber Security . 176
7.12.1 Security circles . 176
7.12.2 Network security . 177
7.12.3 Access Control . 180
7.12.4 Threat detection and mitigation . 180
7.12.5 Security architecture . 184
7.12.6 Application (end-to-end) communication security . 185

7.12.7 Security for synchrophasor (PMU) networks (IEC TR 61850-90-5) . 186

7.12.8 Additional recommendations . 187

7.13 QoS and application-specific engineering. 187

7.13.1 General . 187

7.13.2 SDH/SONET QoS and SLA . 187

7.13.3 PSN QoS and SLA . 187

7.13.4 Application and priority . 188

7.13.5 QoS chain between networks . 188

7.13.6 QoS mapping between networks . 189

7.13.7 QoS engineering . 190

7.13.8 Customer restrictions . 191
7.13.9 Clock services . 191
7.14 Configuration and OAM . 191
7.14.1 Network configuration . 191
7.14.2 OAM . 191
7.15 Time synchronization . 193
7.15.1 Oscillator stability . 193
7.15.2 Mutual synchronization . 193
7.15.3 Direct synchronization . 194
7.15.4 Radio synchronization . 194
7.15.5 GNSS synchronization . 195
7.15.6 Frequency distribution . 195
7.15.7 Time distribution . 196
7.15.8 PTP telecommunication profiles . 203
7.15.9 PTP over MPLS . 203
7.15.10 Comparison of time distribution profiles based on IEC 61588 . 203
7.15.11 Use Case: Synchrophasor time synchronization . 205
7.15.12 Use case: Atomic Clock Hierarchy . 205
8 Use cases . 206
8.1 Use case: Current differential teleprotection system (Japan) . 206
8.2 Use case: SDH / MPLS network (Japan) . 210
8.3 Use Case: Wide area stabilizing control system (Japan) . 211
8.4 Use Case: experimental PMU-based WAMPAC system . 213
Bibliography . 216

Figure 1 – Symbols . 33
Figure 2 – Substation locations in Andalusia . 36
Figure 3 – Topology of the Andalusia network . 37
Figure 4 – Cabinet of the substation edge node . 38
Figure 5 – Communication interfaces in a SEN . 39
Figure 6 – Communicating entities . 40
Figure 7 – SGAM communication model. 41
Figure 8 – Principle of grid voltage level and network technology . 42
Figure 9 – Communication paths and interfaces . 43
Figure 10 – IEC TR 62357 Interfaces, protocols and applications . 44
Figure 11 – Composition of end-to-end latency in a microwave relay . 47
Figure 12 – Example of latency in function of traffic . 48

– 6 – IEC TR 61850-90-12:2015 © IEC 2015

Figure 13 – Jitter for two communication delay types. . 50

Figure 14 – Precision and accuracy definitions . 52

Figure 15 – Redundancy of redundant systems . 56

Figure 16 – Redundancy calculation . 56

Figure 17 – Redundancy layout with single point of failure . 57

Figure 18 – Redundancy layout with cross-coupling . 58

Figure 19 – Availability definitions . 59

Figure 20 – Residual error rate in function of the BER . 61

Figure 21 – Principle of synchrophasor transmission . 69
Figure 22 – Target phenomena for WAMPAC . 71
Figure 23 – Example of main function and general information flow . 72
Figure 24 – PMUs and data flow between TSO and regional data hubs . 73
Figure 25 – Network topology (Carrier Ethernet) . 76
Figure 26 – Phase-to-ground coupling for PLC . 80
Figure 27 – HV PLC coupling with suspended line traps . 81
Figure 28 – Phase to phase signal coupling for PLC . 81
Figure 29 – Phase-to-phase signal coupling . 82
Figure 30 – Power line carrier, line traps . 83
Figure 31 – Terrestrial microwave link . 85
Figure 32 – Layer 2 transport on radio systems . 88
Figure 33 – Radio network in feeder automation . 89
Figure 34 – ADSS fiber cable . 90
Figure 35 – ADSS installation with splicing box . 91
Figure 36 – OPGW in ground cable . 91
Figure 37 – OPGW with two “C”-tubes with each 32 fibers . 92
Figure 38 – OPGW fibers . 93
Figure 39 – Splicing box . 94
Figure 40 – WDM over one fiber . 95
Figure 41 – OCh optical components . 95
Figure 42 – Optical link with microwave back-up . 97
Figure 43 – Picture of partially destroyed 735 kV line . 98

Figure 44 – E1 and E2 channel . 100
Figure 45 – Digital Transmission Hierarchy (T – Standards) . 100
Figure 46 – Digital Transmission Hierarchy (E-standard) . 101
Figure 47 – Example of an SDH network for utilities . 102
Figure 48 – SONET multiplexing hierarchy . 103
Figure 49 – SDH multiplexing hierarchy . 103
Figure 50 – SDH/SONET with point-to-point topology . 105
Figure 51 – SDH/SONET with linear topology . 105
Figure 52 – BLSR/BSHR topology in normal conditions (from A to D) . 107
Figure 53 – BLSR/BSHR topology in failure conditions . 107
Figure 54 – UPSR/USHR topology in normal conditions . 108
Figure 55 – UPSR/USHR topology in failure conditions . 109

Figure 56 – Example of information flow relationship in OTN . 112

Figure 57 – IEEE 802.3 (Ethernet) frame format . 113

Figure 58 – IEEE 802.3 (Ethernet) topology with RSTP switches (IEC TR 61850-90-4) . 114

Figure 59 – IEEE 802.1Q-tagged Ethernet frame format . 115

Figure 60 – Direct Ethernet with VLAN in substation-to-substation transmission . 116

Figure 61 – Substation-to-substation Layer 2 transmission tunneled over IP . 117

Figure 62 – PRP structure (within and outside a substation) . 118

Figure 63 – HSR ring connecting substations and control centre . 118

Figure 64 – MACsec frame format . 120
Figure 65 – IEEE 802.1X principle . 121
Figure 66 – Ethernet for substation-to-substation communication. 122
Figure 67 – Packets over TDM . 123
Figure 68 – IEEE 802.1Q/ad/ah network configuration . 126
Figure 69 – Case of IEEE 802.1Q/ad network for utility . 127
Figure 70 – Basic MPLS architecture . 128
Figure 71 – Example of MPLS frame format with IPv4 payload . 129
Figure 72 – MPLS building blocks . 130
Figure 73 – MPLS network architecture for utilities . 131
Figure 74 – IP/MPLS and MPLS-TP features . 132
Figure 75 – MPLS-TP redundant routing . 134
Figure 76 – Ethernet frame with IP network header . 136
Figure 77 – Mapping of IPv4 to Ethernet frames . 137
Figure 78 – Mapping of IPv6 to Ethernet frames . 140
Figure 79 – IPv6 unicast address structure . 141
Figure 80 – IPv6 ULA address structure . 142
Figure 81 – IPv6 link local address structure . 142
Figure 82 – Mapping of IPv4 to IPv6 addresses . 145
Figure 83 – IPv6 evolution . 147
Figure 84 – IEC 61850 stack with IPv4 and IPv6 . 148
Figure 85 – DiffServ codepoint field . 150
Figure 86 – Unidirectional protocol independent multicast . 151

Figure 87 – Bidirectional protocol independent multicast . 152
Figure 88 – Frame format for IPsec (authenticated) . 153
Figure 89 – Frame format for IPsec (encrypted) . 153
Figure 90 – Layer 3 direct connection within same address space . 154
Figure 91 – Connecting substations to SCADA by a NAT . 155
Figure 92 – Substation to SCADA connection over ALG . 156
Figure 93 – Ethernet frame with UDP transport layer . 157
Figure 94 – UDP header . 158
Figure 95 – TCP header . 158
Figure 96 – Session and presentation layers for MMS . 160
Figure 97 – Session and presentation layers for R-GOOSE . 160
Figure 98 – IEEE C37.118 frame over UDP . 161

– 8 – IEC TR 61850-90-12:2015 © IEC 2015

Figure 99 – Redundant network transmission handled by the application layer . 161

Figure 100 – Tunneling in IEC TR 61850-90-1 . 163

Figure 101 – L2TP transporting Layer 2 frames over IP . 164

Figure 102 – Tunneling GOOSE over IP in IEC TR 61850-90-5 . 165

Figure 103 – Pseudo-wire principle . 166

Figure 104 – Non-IP voice communication over PSN . 167

Figure 105 – Circuit emulation over PSN . 168

Figure 106 – L2VPNs VPWS and VPLS . 170

Figure 107 – L3VPN . 171
Figure 108 – Emulation of L3VPN by L2VPN and global router . 172
Figure 109 – Tele-protection over VPWS, . 174
Figure 110 – WAMS over VPLS . 175
Figure 111 – VPN for IP-based SCADA/EMS traffic . 176
Figure 112 – VPN deployment options . 179
Figure 113 – IP network separator . 181
Figure 114 – Security architecture (using segmentation and perimeter security) . 185
Figure 115 – QoS chain . 189
Figure 116 – Timing pulse transmission methods of legacy teleprotection devices . 194
Figure 117 – SyncE application . 195
Figure 118 – Synchronous Ethernet Architecture . 196
Figure 119 – SNTP clock synchronization and network delay measurement . 197
Figure 120 – Model of GMC, two BCs in series and SC over Layer 3 . 200
Figure 121 – Timing diagram of PTP (end-to-end, 2-step, BCs) . 200
Figure 122 – Timing diagram of PTP (peer-to-peer, 2-step TCs) . 201
Figure 123 – Substations synchronization over WAN . 205
Figure 124 – Example of synchronization network . 206
Figure 125 – Current differential 1:1 configuration . 207
Figure 126 – Network configuration for centralized multi-terminal line protection . 207
Figure 127 – Network configuration for distributed multi-terminal line protection . 208
Figure 128 – Current differential teleprotection for HV multi-terminal transmission line
using Layer 2 network . 208

Figure 129 – Configuration of wide area current differential primary and backup
teleprotection system employing Carrier Ethernet and IEC 61588 time synchronization . 210
Figure 130 – Achieving protection for teleprotection services . 211
Figure 131 – System configuration for wide area stabilizing control system . 212
Figure 132 – Appearance of typical CCE cabinet . 213
Figure 133 – Configuration for PMU-based WAMPAC system . 214

Table 1 – Latency classes in IEC 61850-5 . 49
Table 2 – Latency classes in IEC TR 61850-90-1 . 49
Table 3 – Latency classes for WANs . 50
Table 4 – Jitter classes in IEC TR 61850-90-1 . 51
Table 5 – Jitter classes for WAN . 51
Table 6 – Recovery delay classes for WAN . 52

Table 7 – IEC TR 61850-90-1 time accuracy classes . 53

Table 8 – IEC 61850-5 time accuracy classes for IED synchronization . 53

Table 9 – WAN time synchronization classes . 54

Table 10 – Latency for line protection . 64

Table 11 – Summary of operational requirements of line protection . 65

Table 12 – Summary of communication requirements for teleprotection . 65

Table 13 – Communication requirements for CC to SS/PS . 66

Table 14 – Latency and timing requirements from IEC TR 61850-90-2 . 66

Table 15 – Communication requirements for CMD . 67
Table 16 – Communication requirements for inter-control centre communications . 68
Table 17 – Summary of synchrophasor requirements . 70
Table 18 – Summary of communication requirements for wide area monitoring . 71
Table 19 – Typical communication requirements for WAMPAC . 73
Table 20 – Classification of communication requirements . 74
Table 21 – Communication requirements of wide-area applications . 74
Table 22 – Communication technologies . 77
Table 23 – Physical communication media . 78
Table 24 – DSL communication over twisted pairs . 79
Table 25 – Trade-offs in copper cable communication . 80
Table 26 – PLC communication technologies . 83
Table 27 – PLC communication advantages and disadvantages. 84
Table 28 – Microwave link performance . 86
Table 29 – Terrestrial microwave advantages and disadvantages . 86
Table 30 – Public mobile radio technologies . 87
Table 31 – Terrestrial radio advantages and disadvantages . 87
Table 32 – Satellite radio advantages and disadvantages . 87
Table 33 – Optical fibers: advantages and disadvantages . 96
T
...