Testing 10GE in Transport Environments
Guylain Barlow, Innocor Ltd.
Apr 27, 2004 (6:00 AM)
Figure 1: Diagram showing the IEEE 802.3ah specification, which defines 10GE LAN and WAN operation.
10 Gigabit Ethernet (10GE) interfaces are finding their way in numerous applications. From short-reach requirements for enterprise users to carrier applications, 10GE is being seen as a key enabling technology. But, to truly penetrate the transport environment, engineers need an effective method to test 10GE interface designs.
In this article, we'll look at the protocol related 10GE test considerations that designers must consider when developing transport systems. The article will focus on the 10GE protocol and related developments as opposed to optical transmission effects such as loss and dispersion. Additionally, the article will look at the impact of error correction techniques on long-haul 10GE connectivity.
A number of physical interfaces have been defined for 10GE to address different applications. The most significant difference is the distinction between 10GE WAN (10GBase-W) that operates at 9.953 Gbit/s and 10GE LAN (10Gbase-R) that runs at a rate of 10.313 Gbit/s (Figure 1). 10GE LAN is the natural evolution of the Ethernet standard from Gigabit Ethernet 10GE WAN, on the other hand, provides OC-192/STM-64 Sonet/SDH functions at the WAN interface sublayer (WIS) for integration into existing carrier networks.
In addition to the fundamental differences between 10GE LAN and WAN, the 802.3ah specification defines several wavelengths. These wavelengths are defined because of the properties of optical fibers and the loss and dispersion experienced at different wavelengths. Consequently, the interfaces and wavelengths used depend on the system application.
Table 1 shows the potential deployment of 10GE interfaces according to interface type and wavelength.
Table 1: 10GE Serial Interfaces
Sonet/SDH infrastructure is mature and so are the associated test methodologies. System performance indicators are built into the infrastructure and indicators are readily available to report bit and block errors. A relatively large number of overhead bytes and associated alarms, for example the B1, B2, B3 bytes and metrics like error free seconds (EFS) are used to report and detect system defects.
Since 10GE WAN is basically 10GE carried as Sonet/SDH payload, the transport test methodologies are similar. For the particular case where 10GE LAN connects into an OC-192/STM-64 core (i.e. 10GbE WAN), it is interesting to note that system-level testing may involve flow control as the 10GE WAN operates at a lower rate and cannot sustain the 10GE LAN rate over long periods of time. When it comes to 10GE LAN compared to WAN, the transport indicators differ and new concepts need to be investigated.
In the absence of bit interleaved parity (BIP) overhead bytes, different means must be identified to detect 10GE LAN errors. Several approaches may be taken by examining the sublayers illustrated in Figure 1 above.
At the PMD layer, basic tests involve the evaluation of transmitter jitter and the measurement of stressed eye sensitivity at the receiver. Jitter, which is basically a timing error caused by amplitude to phase conversion and phase noise can impact signal sampling and cause bit detection errors. The testing of these parameters is performed during the design and characterization stages of product development. Depending on the test strategy adopted, these parameters may or may not be evaluated at the manufacturing stage. Jitter testing involves high costs, especially in manufacturing operations that imply volume testing. There are other potential approaches where vendors rely on their suppliers and perform specific tests to capture errors and defects through other means.
A different approach is close examination of the PCS sublayer. The PCS sublayer performs line encoding to reliably transport 10GbE information. It provides robust bit and block error detection. The PCS encodes each stream of 64 bits of information into 66 bits and applies scrambling to form block codes (Figure 2).
Figure 2: Diagram showing the PCS layer of a 10GE WAN system.
The PCS sublayer implicitly contains information to reflect the vast majority of errors that occur on a transmission line. Bit errors can be detected whether they take the form of block payload errors, where received codes are not recognized as valid 64-bit codewords after descrambling, or invalid sync bits.
Based on this information, metrics can be established based on the number of errored blocks relative to the total number of blocks received. Such metrics are synchronous as they do not depend on the MAC layer where data is sent asynchronously. This is because the PCS continuously sends and receives information whether data information is present or not. This is one of the differences compared to standard data networking media access control (MAC) layer tests that identify frame loss relative to a fixed traffic load and frame size.
Based on the premise of transport testing, the PCS block testing approach has the benefit of providing general coverage and a generic measurement. For example, an out of service traffic source can take the form of random length MAC frames with a pseudo-random bit sequence (PRBS) payload or even a PCS-encoded PRBS bit stream without MAC. This simulates traffic and provides a relatively quick and efficient manufacturing test.
To further stress the system under test, the test may involve the detection of block errors at different signalling speeds by adjusting the transmission clock rate across the supported range which is +/-100 ppm for 10GbE LAN. Testing should extend beyond 100 ppm. It's important to note that 10GE LAN systems offer fewer alarms and indicators than 10GE WAN systems. High bit error rate (BER) and PCS loss of sync are indicatives of sync bit errors while remote and local fault indicators indicate far- and near-end system or line problems.
A rough parallel for local and remote faults can be established against Sonet/SDH alarm indication signal (AIS) and remote defect indication (RDI) alarms respectively. A development to provide additional indicators are the operations, administration and maintenance (OAM) definitions (ref: IEEE 802.3ah) from the Ethernet in the First Mile (EFM) task force. This OAM information, provided at the MAC sublayer, provides additional link monitoring information including remote failures, loopback, and link monitoring. These definitions can be implemented and used in the testing of 10GE LAN.
When considering long-distance transport, the topic of forward error correction (FEC) arises. With FEC, redundant information is added to the transmitted frames to improve error performance via symbol correction, and enable longer optical spans.
In the case of Sonet/SDH, members of the telecom sector have standardized approaches for carrying FEC under the ITU-T G.709 specification. For 10GE WAN that runs at the OC-192/STM-64 rate, G.709 defines the OTU-2 interface at a line rate of 10.709 Gbit/s. OTU-2 therefore represents a conventional way to carry 10GbE WAN over very long spans.
Figure 3 illustrates the encapsulation of 10GE WAN into OTU-2. The OTU-2 frame is composed of three main parts: an overhead area for operation, administration and maintenance functions, a payload area which in this case contains 10GbE WAN, and the FEC block to provide error correction.
Figure 3: Diagram illustrating the enca[psulation og 10GbE WAN traffic into OTU-2.
Unlike on the WAN side, there are currently no explicit standards that outline techniques for carrying FEC to extend the reach of 10GE LAN. As a result, equipment manufacturers are developing proprietary interfaces to carry 10GE LAN with FEC for metro applications.
There are primarily two rates being targeted: 10.709 and 11.095 Gbit/s. The 10.709-Gbit/s rate uses the same transmission rate as OTU-2 with a more compact digital wrapper and FEC. The 11.095-Gbit/s rate applies the same digital wrapper and FEC from OTU-2 onto 10GbE LAN. This lack of explicit standard raises the question of interoperability and third party testing. This situation is bound to change, especially at 11.095 Gbit/s where overhead is exactly the same as in G.709, as more vendors are investing into FEC at 11.095 Gbit/s.
OTU-2 provides a wealth of information analogous to Sonet/SDH and there is an opportunity to monitor transport information. Examples include the monitoring of ODU and OTU BIP bytes. These bytes provide bit parity information covering both the payload (i.e. 10GE WAN) and digital wrapper prior to appending the FEC bytes.
For a closer look at line level errors, a good indicator is FEC error monitoring. The standard G.709 Reed Solomon FEC provides error correction for up to 8 symbol errors per FEC block. Additionally, when configured for error detection, the FEC can find up to 16 symbol errors. The FEC provides efficient burst error protection since it is organized into sixteen FEC blocks based on sixteen interleaved sub-rows that compose each OTU-2 row.
As in Figure 3, an OTU-2 frame is composed of four rows. Transmission testing can therefore be performed based on the number of corrected (or detected) FEC errors relative to the number of received bits. In a test environment, error sensitivity can be verified relative to clock rate variations for FEC protected 10GbE WAN and LAN.
As more 10GE LAN (and WAN) interfaces become available, transmission test scenarios are being developed. One of the challenges for 10GE LAN testing is that the rich set of transport indicators from Sonet/SDH is absent. Yet, there is an opportunity to capture transmission errors independently of the upper layers.
The standard data tests used to evaluate frame transfer capacity remain an important tool to evaluate system and network performance. However, the information available at the PCS layer for 10GE LAN and even WAN is a valuable test tool in a transport environment. In the case of long-haul transport where a digital wrapper and FEC are used, the monitoring of symbol errors becomes a reliable indicator. The standardization of FEC for 10GE LAN clients can only help in formalizing tests and interoperability between vendors.
About the Author
Guylain Barlow is the product manager for Innocor's Test and Measurement division. He has a degree in electrical engineering from the University of Sherbrooke and can be reached at firstname.lastname@example.org.
Copyright © 2003 CMP Media, LLC | Privacy Statement