ISO/IEC 13961:2000

Information technology — Scalable Coherent Interface (SCI)

The scalable coherent interface (SCI) provides computer-bus-like services but, instead of a bus, uses a collection of fast point-to-point unidirectional links to provide the far higher throughput needed for high-performance multiprocessor systems. SCI supports distributed, shared memory with optional cache coherence for tightly coupled systems, and message-passing for loosely coupled systems. Initial SCI links are defined at 1 Gbyte/s (16-bit parallel) and 1 Gb/s (serial). For applications requiring modular packaging, an interchangeable module is specified along with connector and power. The packets and protocols that implement transactions are defined and their formal specification is provided in the form of computer programs. In addition to the usual read-and-write transactions, SCI supports efficient multiprocessor lock transactions. The distributed cache-coherence protocols are efficient and can recover from an arbitrary number of transmission failures. SCI protocols ensure forward progress despite multiprocessor conflicts (no deadlocks or starvation).

Technologies de l'information — Interface cohérente atteignable

General Information

• Status: Published
• Publication Date: 09-Aug-2000
• ICS: 35.160 - Microprocessor systems
• Technical Committee: ISO/IEC JTC 1/SC 25 - Interconnection of information technology equipment
• Drafting Committee: ISO/IEC JTC 1/SC 25 - Interconnection of information technology equipment
• Current Stage: 9093 - International Standard confirmed
• Completion Date: 13-Jul-2018

Standards Content (Sample)

INTERNATIONAL ISO/IEC
STANDARD
13961
IEEE
Std 1596
First edition
2000-07
Information technology –
Scalable Coherent Interface (SCI)
Reference number
ISO/IEC 13961:2000(E)
IEEE Std 1596, 1998 Edition

---------------------- Page: 1 ----------------------
Abstract: The scalable coherent interface (SCI) provides computer-bus-like services but,
instead of a bus, uses a collection of fast point-to-point unidirectional links to provide the far
higher throughput needed for high-performance multiprocessor systems. SCI supports
distributed, shared memory with optional cache coherence for tightly coupled systems, and
message-passing for loosely coupled systems. Initial SCI links are defined at 1 Gbyte/s (16-bit
parallel) and 1 Gb/s (serial). For applications requiring modular packaging, an interchangeable
module is specified along with connector and power. The packets and protocols that implement
transactions are defined and their formal specification is provided in the form of computer
programs. In addition to the usual read-and-write transactions, SCI supports efficient
multiprocessor lock transactions. The distributed cache-coherence protocols are efficient and
can recover from an arbitrary number of transmission failures. SCI protocols ensure forward
progress despite multiprocessor conflicts (no deadlocks or starvation).
Keywords: bus architecture, bus standard, cache coherence, distributed memory, fiber optic,
interconnect, I/O system, link, mesh, multiprocessor, network, packet protocol, ring, seamless
distributed computer, shared memory, switch, transaction set.

––––––––––
The Institute of Electrical and Electronics Engineers, Inc.
345 East 47th Street, New York, NY 10017-2394, USA
Copyright © 1998 by the Institute of Electrical and Electronics Engineers, Inc.
All rights reserved. First published in 1998.
ISBN 2-8318-5375-3
No part of this publication may be reproduced in any form, in an electronic retrieval system or otherwise, without
the prior written permission of the publisher.

---------------------- Page: 2 ----------------------
INTERNATIONAL ISO/IEC
STANDARD
13961
IEEE
Std 1596
First edition
2000-07
Information technology –
Scalable Coherent Interface (SCI)
Sponsor
Microprocessor and Microcomputer Standards Subcommittee
of the IEEE Computer Society
PRICE CODE
XF
For price, see current catalogue

---------------------- Page: 3 ----------------------
– 2 – ISO/IEC 13961:2000(E)
IEEE Std 1596, 1998 Edition
CONTENTS
Page
FOREWORD .12
Clause
1 Introduction. 19
1.1 Document structure. 19
1.2 SCI overview . 20
1.2.1 Scope and directions. 20
1.2.2 The SCI approach . 21
1.2.3 System configurations. 22
1.2.4 Initial physical models. 23
1.2.5 SCI node model . 24
1.2.6 Architectural parameters . 25
1.2.7 A common CSR architecture . 25
1.2.8 Structure of the specification. 26
1.3 Interconnect topologies. 26
1.3.1 Bridged systems . 26
1.3.2 Scalable systems . 27
1.3.3 Interconnected systems . 27
1.3.4 Backplane rings . 27
1.3.5 Interconnected rings . 28
1.3.6 Rectangular grid interconnects. 29
1.3.7 Butterfly switches. 30
1.3.8 Vendor-dependent switches . 31
1.4 Transactions . 31
1.4.1 Packet formats. 32
1.4.2 Input and output queues . 33
1.4.3 Request and response queues . 34
1.4.4 Switch queues. 36
1.4.5 Subactions. 36
1.4.6 Remote transactions (through agents). 39
1.4.7 Move transactions. 41
1.4.8 Broadcast moves . 42
1.4.9 Broadcast passing by agents . 43
1.4.10 Transaction types. 44
1.4.11 Message passing . 45
1.4.12 Global clocks . 45
1.4.13 Allocation protocols. 46
1.4.14 Queue allocation. 47
1.5 Cache coherence. 49
1.5.1 Interconnect constraints . 49
1.5.2 Distributed directories . 49
1.5.3 Standard optimizations. 50
1.5.4 Future extensions . 50
1.5.5 TLB purges . 53
Copyright  1998 IEEE. All rights reserved.

---------------------- Page: 4 ----------------------
ISO/IEC 13961:2000(E) – 3 –
IEEE Std 1596, 1998 Edition
Clause Page
1.6 Reliability, availability, and support (RAS). 54
1.6.1 RAS overview . 54
1.6.2 Autoconfiguration. 54
1.6.3 Control and status registers . 54
1.6.4 Transmission-error detection and isolation . 55
1.6.5 Error containment . 55
1.6.6 Hardware fault retry (ringlet-local, physical layer option) . 56
1.6.7 Software fault recovery (end-to-end) . 56
1.6.8 System debugging . 57
1.6.9 Alternate routing . 57
1.6.10 Online replacement. 57
2 References, glossary, and notation . 58
2.1 Normative references. 58
2.2 Conformance levels . 58
2.3 Terms and definitions . 59
2.4 Bit and byte ordering. 66
2.5 Numerical values . 68
2.6 C code. 68
3 Logical protocols and formats . 68
3.1 Packet formats. 68
3.1.1 Packet types . 68
3.2 Send and echo packet formats. 69
3.2.1 Request-send packet format . 69
3.2.2 Request-echo packet format . 72
3.2.3 Response-send packet. 74
3.2.4 Standard status codes . 76
3.2.5 Response-echo packet format. 78
3.2.6 Interconnect-affected fields . 79
3.2.7 Init packets . 80
3.2.8 Cyclic redundancy code (CRC) . 81
3.2.9 Parallel 16-bit CRC calculations. 82
3.2.10 CRC stomping. 84
3.2.11 Idle symbols. 85
3.3 Logical packet encodings. 86
3.3.1 Flag coding . 86
3.4 Transaction types . 89
3.4.1 Transaction commands . 89
3.4.2 Lock subcommands . 92
3.4.3 Unaligned DMA transfers . 94
3.4.4 Aligned block-transfer hints. 95
3.4.5 Move transactions. 97
3.4.6 Global time synchronization . 98
3.5 Elastic buffers. 99
3.5.1 Elasticity models . 99
3.5.2 Idle-symbol insertions . 100
3.5.3 Idle-symbol deletions . 101
Copyright  1998 IEEE. All rights reserved.

---------------------- Page: 5 ----------------------
– 4 – ISO/IEC 13961:2000(E)
IEEE Std 1596, 1998 Edition
Clause Page
3.6 Bandwidth allocation. 101
3.6.1 Fair bandwidth allocation . 102
3.6.2 Setting ringlet priority. 104
3.6.3 Bandwidth partitioning. 106
3.6.4 Types of transmission protocols . 108
3.6.5 Pass-transmission protocol . 108
3.6.6 Low-transmission protocol. 111
3.6.7 Idle insertions . 114
3.6.8 High-transmission protocol. 114
3.7 Queue allocation. 116
3.7.1 Queue reservations. 116
3.7.2 Multiple active sends. 118
3.7.3 Unfair reservations. 119
3.7.4 Queue-selection protocols. 119
3.7.5 Re-send priorities. 119
3.8 Transaction errors. 120
3.8.1 Requester timeouts (response-expected packets) . 120
3.8.2 Time-of-death timeout (optional, all nodes) . 120
3.8.3 Responder-processing errors . 122
3.9 Transmission errors . 123
3.9.1 Error isolation . 123
3.9.2 Scrubber maintenance . 125
3.9.3 Producer-detected errors . 126
3.9.4 Consumer-detected errors. 128
3.10 Address initialization. 129
3.10.1 Transaction addressing. 129
3.10.2 Reset types. 131
3.10.3 Unique node identifiers . 132
3.10.4 Ringlet initialization. 133
3.10.5 Simple-subset ringlet resets. 135
3.10.6 Ringlet resets. 135
3.10.7 Ringlet clears (optional) . 137
3.10.8 Inserting initialization packets . 138
3.10.9 Address initialization . 139
3.11 Packet encoding . 140
3.11.1 Common encoding features (L18) . 140
3.11.2 Parallel encoding with 18 signals (P18). 141
3.11.3 Serial encoding with 20-bit symbols (S20). 141
3.12 SCI-specific control and status registers . 144
3.12.1 SCI transaction sets. 144
3.12.2 SCI resets. 145
3.12.3 SCI-dependent fields within standard CSRs . 145
3.12.4 SCI-dependent CSRs. 148
3.12.5 SCI-dependent ROM. 151
3.12.6 Interrupt register formats. 155
3.12.7 Interleaved logical addressing. 157
Copyright  1998 IEEE. All rights reserved.

---------------------- Page: 6 ----------------------
ISO/IEC 13961:2000(E) – 5 –
IEEE Std 1596, 1998 Edition
Clause Page
4 Cache-coherence protocols. 158
4.1 Introduction. 158
4.1.1 Objectives. 158
4.1.2 SCI transaction components . 158
4.1.3 Physical addressing . 159
4.1.4 Coherence directory overview . 159
4.1.5 Memory and cache tags . 160
4.1.6 Instruction-execution model . 161
4.1.7 Coherence document structure . 162
4.2 Coherence update sequences. 163
4.2.1 List prepend. 163
4.2.2 List-entry deletion . 165
4.2.3 Update actions. 167
4.2.4 Cache-line locks . 167
4.2.5 Stable sharing lists. 168
4.3 Minimal-set coherence protocols. 171
4.3.1 Sharing-list updates . 171
4.3.2 Cache fetching. 171
4.3.3 Cache rollouts. 173
4.3.4 Instruction-execution model . 174
4.4 Typical-set coherence protocols. 175
4.4.1 Sharing-list updates . 175
4.4.2 Read-only fetch. 175
4.4.3 Read-write fetch. 177
4.4.4 Data modifications . 178
4.4.5 Mid and head deletions . 179
4.4.6 DMA reads and writes . 181
4.4.7 Instruction-execution model . 183
4.5 Full-set coherence protocols . 184
4.5.1 Full-set option summary. 184
4.5.2 CLEAN-list creation. 184
4.5.3 Sharing-list additions . 185
4.5.4 Cache washing . 187
4.5.5 Cache flushing . 189
4.5.6 Cache cleansing . 191
4.5.7 Pairwise sharing . 192
4.5.8 Pairwise-sharing faults. 196
4.5.9 QOLB sharing . 197
4.5.10 Cache-access properties. 200
4.5.11 Instruction-execution model . 201
4.6 C-code naming conventions. 202
4.7 Coherent read and write transactions. 203
4.7.1 Extended mread transactions. 204
4.7.2 Cache cread and cwrite64 transactions. 205
4.7.3 Smaller tag sizes . 206
Copyright  1998 IEEE. All rights reserved.

---------------------- Page: 7 ----------------------
– 6 – ISO/IEC 13961:2000(E)
IEEE Std 1596, 1998 Edition
Clause Page
5 C-code structure . 207
5.1 Node structure . 207
5.1.1 Signals within a node . 207
5.1.2 Packet transfers among node components . 208
5.1.3 Transfer-cloud components. 208
5.2 A node's linc component . 210
5.2.1 A linc's subcomponents. 210
5.2.2 A linc's elastic buffer . 212
5.2.3 Other linc components . 213
5.3 Other node components. 213
5.3.1 A node's core component. 213
5.3.2 A node's memory component . 213
5.3.3 A node's exec component . 214
5.3.4 A node's proc component. 215
6 Physical layers. 216
6.1 Type 1 module . 217
6.1.1 Module characteristics . 217
6.1.2 Module compatibility considerations . 217
6.1.3 Module size. 218
6.1.4 Warpage, bowing, and deflection . 224
6.1.5 Cooling . 225
6.1.6 Connector . 226
6.1.7 Power and ground connection . 227
6.1.8 Pin allocation for backplane parallel 18-signal encoding. 229
6.1.9 Slot-identification signals . 231
6.2 Type 18-DE-500 signals and power control . 232
6.2.1 SCI differential signals . 233
6.2.2 Status lines. 233
6.2.3 Serial Bus signals . 233
6.2.4 Signal levels and skew. 233
6.2.5 Power-conversion control. 236
6.3 Type 18-DE-500 module extender cable . 238
6.4 Type 18-DE-500 cable-link. 240
6.5 Serial interconnection . 242
6.5.1 Serial interface Type 1-SE-1250, single-ended electrical. 243
6.5.2 Optical interface, fiber-optic signal type 1-FO-1250. 249
6.5.3 Test methods . 252
Annex A (informative) Ringlet initialization . 254
Annex B (informative) SCI design models. 257
B.1 Fast counters . 257
B.2 Translation-lookaside-buffer coherence . 257
B.3 Coherent lock models . 261
B.4 Coherence-performance models . 263
Bibliography . 265
Copyright  1998 IEEE. All rights reserved.

---------------------- Page: 8 ----------------------
ISO/IEC 13961:2000(E) – 7 –
IEEE Std 1596, 1998 Edition
Page
Figure 1 – Physical-layer alternatives. 23
Figure 2 – SCI node model. 24
Figure 3 – 64-bit-fixed addressing . 25
Figure 4 – Bridged systems . 26
Figure 5 – Backplane rings. 28
Figure 6 – Interconnected rings . 29
Figure 7 – 2-D processor grids . 29
Figure 8 – Butterfly ringlets . 30
Figure 9 – Switch interface. 31
Figure 10 – Subactions. 32
Figure 11 – Send-packet format, simplified. 32
Figure 12 – Responder queues. 34
Figure 13 – Logical requester/responder queues .
...