With StarFabric as an on-ramp, the PCI Express Advanced Switching is ready

Basically, StarFabric and AS are open standard serial switched interconnect architectures designed for high-performance applications. Their features and capabilities make them ideal for a broad section of the communication, storage, server, and embedded marketplaces.

StarFabric components have been shipping in production since the first half of 2002 wit h 10s of 1000s of StarFabric ports having been shipped to over 50 system suppliers worldwide.

AS, at 4 times the speed of StarFabric, is currently in specification development, with first products anticipated in 2004. It builds upon the strengths of PCI Express Base while adding the elements necessary to support distributed multiprocessing systems. It provides the required scalability, high-availability, and Quality of Service (QoS) attributes necessary to establish itself as the ubiquitous industry standard interconnect for communication, storage, and embedded applications. As such, it is gaining wide backing by many of the major system vendors in these markets.

PCI Express Base is the emerging serial interconnect technology for desktop, mobile, and server applications. It is optimized for high-speed chip-to-chip connections. Similar to StarFabric's legacy mode, it preserves the PCI programming model. PCI Express Base supports a single processor subsystem with scalable I/O capabilit y. The system topology is a tree hierarchy with the host CPU being the top of the tree. PCI Express Base switches look like PCI-to-PCI bridges to system-level software.

Essentially, PCI Express AS will take advantage of the physical and link layer of PCI Express Base. The communication and embedded computing enhancements are added at the transaction layer.

The basis for the high level of commonality between StarFabric and AS is the shared vision of the original architects of what the next-generation interconnect required. Both groups knew that shared bus-based architectures were reaching their upper limits of usefulness. In order to reach new levels of performance and scalability, a new serial switched-based architecture was required.

Both groups appreciated the importance of the huge installed base of PCI hardware and software. It was strongly belie ved that a new architecture could not abandon this base and still become a ubiquitous standard. An evolutionary approach, which protected the investment by hundreds of companies in PCI hardware and software, was required.

Both groups also understood the unique requirements of distributed processing in communications, storage, and embedded applications. Thus, they created advanced features addressing high-availability, multiple types and classes of traffic, and flexible scalability. The belief was that the next-generation ubiquitous interconnect standard technology must be an open standard, which would create a vibrant ecosystem of the widest range of participants. This would stimulate innovation and provide system designers with the widest range of low-cost components.

For efficient support of distributed multiprocessing applications, both StarFabric and AS support path routing. Path routing eliminates the need for every switch to have look-up tables and specific software to route pack ets. The source of the packet specifies the path that the packet will travel through the fabric to the destination. Path routing is also reversible. When a destination node receives a packet it can easily compute the return path to the source. This enables switches to notify the source of any failures without software intervention. The hardware-based rapid response greatly increases system robustness.

Path routing also supports protection. Since a node can uniquely determine the origin of every packet, it can accept or refuse access based on the originating node.Multicast capability is also supported, which enables a single source packet to be sent to many destination nodes.

Protocol- agnostic

Both architectures are protocol-agnostic. Essentially, AS is an encapsulation architecture where each encapsulated protocol is given a unique Protocol Encapsulation Interface (PEI) number. There are total of 255 PEIs. PEI 0 through 7 are used for fabric managem ent. The PCI-SIG will define PEIs 8 through 223 based on standard protocols. PEIs 224 through 254 can be defined by end-users.

Additionally, both StarFabric and AS support multiple classes of service. StarFabric defines eight types of traffic: provisioning, high-priority isochronous, isochronous, high-priority asynchronous, asynchronous, multicast, address-routed, and special. AS defines eight classes of services as well. AS provides an additional degree of freedom where each TC can be mapped to a specific virtual channel (VC) at run time. For example, if a device only supported 4 VCs, the system designer can map the 8 TCs into the 4 VCs in a manner that fine-tunes the system traffic.

Each of the classes of service noted above has its own credit pool in each device. Congestion in one class will not affect other traffic. For example, a burst in asynchronous data will not affect a stream of isochronous data. Additionally, unused bandwidth of one class can be utilized by other classes .

Credit-based flow control is used in both architectures. This technique guarantees progress through the fabric. A frame or packet is only sent if there is sufficient buffer space in the next node. Frames are never dropped due to congestion, and re-transmission is only required when a failure has occurred.

Both StarFabric and AS are robust fabrics and are considered loss-less in the absence of failures. Each packet is CRC protected and the CRC is checked at every link. Failures are detected in hardware and are reported and managed. Errors are explicitly routed to the fabric management node. Recovery from errors can be accomplished in both hardware and software.

The architectures were designed from the onset with high-availability (HA) in mind. While HA in general is a multi-layer issue, these interconnect technologies have a number of hardware-bas ed HA features and additional primitives for the upper layers to use.

The use of redundant links with hardware recovery is one of the essential HA silicon-based features. Designers can architect a system with parallel redundant fabrics and each end node would have an interface to both the primary and the secondary fabric. In the event that the primary link fails, the silicon would automatically switch to the backup path.

StarFabric and AS both offer advanced feature support that is accessible with the addition of some software to provide basic fabric set-up, operation, maintenance, and event handling. For these features, StarFabric and AS can utilize the same low-level load-store programming model. The StarFabric software model is a layered software approach that can hide the hardware implementation details from higher-level software. It contains bus drivers, fabric-primitives libraries, fabric-distributed libraries, and fabric management layers.

The fabric driver is respo nsible for the run-time fabric control and management policies. It interacts with architecture-defined capabilities of each node. It performs fabric discovery at initialization where each node is identified, and the fabric topology is determined. It is responsible for connection creation and tear down, provisioning bandwidth, resource allocation, and synchronization. It also handles fabric events, such as hot plug, power management, and fault management.

The Fabric Primitives Library (FPL) is a highly-portable operating system-independent layer. It provides a consistent API for low-level fabric routines and hides the register level complexities from the system designer. The FPL is typically linked with the fabric driver.

The device driver controls the implementation-specific capabilities of each node and prepares the node for run-time operations. It translates local interrupts into fabric events.

Common traits

Because the software model and the functionality supported by StarFabric and AS are similar, there is a high-degree of commonality built in. Application level software and some of the high-level fabric modules can be transportable between StarFabric and AS. Investment in this level of software on StarFabric can easily migrate to AS when available.

While the two architectures are highly aligned there is one major difference and that is the speed and cost of the physical layer. StarFabric links are based on 622 Mbit/second Low Voltage Differential Signaling (LVDS) base technology. Each link uses 4 pairs to make a single 2.5Gbit/sec link. This technology is very low-cost and easy to implement at the system level. Standard connectors can be used within a chassis, and inexpensive CAT5 cable can be used in chassis-to-chassis applications. PCI Express uses 2.5Gbit/sec signaling per differential pair with a 4X link being 10Gbit/sec or four times faster than StarFabric. PCI Express has also specified even higher speed links of up to 32 differential pairs. At these speeds, the connectors and cables required are naturally more expensive.

Communication equipment vendors are seeking a long-term solution to their system interconnect needs. They are demanding broad-based industry standards with the scalability, HA, and QoS attributes they require.

StarFabric is suited for 2.5Gbit/sec and below applications. AS will provide similar capabilities as StarFabric but at higher performance. It will be available starting in 2004 and is suited for 10Gbit/sec and above applications