There is a need for increased network bandwidth to meet the global IP traffic demand, which is growing at a rate of 30% per year according to a recent Cisco forecast.
In order to meet this demand, there will be a need to upgrade to 40G Ethernet links for switch to server and storage area network connections in data centers and 100G Ethernet links for core switching and routing connections in the backbone.
The biggest market for 40G Ethernet (40 GbE) is in data centers for interconnection links with servers and storage area networks. The market for 100G Ethernet (100 GbE) will be driven by high bandwidth switching and routing for core network aggregation of 10G and 40G Ethernet links. The market for 40 GbE and 100 GbE will evolve over the next three to seven years as products become less expensive and more available over time. Figure 1 shows the natural progression of higher speed Ethernet as a function of time to meet the needs of server I/O and core networking bandwidth.
IEEE published the IEEE 802.3ba standard for 40 Gigabit and 100 Gigabit Ethernet in June 2010. Table 1 illustrates the capabilities of different grades of multimode optical fiber (OM1, OM2, OM3 and OM4) to support different Ethernet applications. Only the laser optimized multimode fiber (grades OM3 and OM4) are capable of supporting 40G and 100G Ethernet.
This white paper will focus on the cabling requirements for 40GBASE-SR4 and 100GBASE-SR10 and provide guidance on an effective migration strategy to transition from 10G to 40 Gigabit and 100 Gigabit Ethernet as the need arises in the near future.
40G Ethernet and 100G Ethernet over multimode fiber uses parallel optics at 10 Gb/s per lane. One lane uses 1 fiber for each direction of transmission. 40G Ethernet requires 8 fibers. 100G Ethernet requires 20 fibers. The concept of parallel transmission at 10 Gigabits per lane is illustrated in Figure 2.
The minimum performance that is needed to support 40 GbE and 100 GbE over multimode fiber is OM3 fiber for a distance of 100 meters. Cabling with OM4 fiber provides the capability to extend the reach up to 150 meters.
The channel specification for 40 Gigabit Ethernet and 100 Gigabit Ethernet over multimode fiber is shown in Figure 3. The insertion loss budget for connectors is highlighted in red. OM4 fiber has a tight budget allocation for connector losses, a total of 1 dB or 0.5 dB maximum per connector.
The media dependent interface (MDI) is the physical interface that connects the cabling media to the network equipment. For multimode fiber, the media dependent interface is the MPO adapter that meets the dimensional specifications of IEC 61754-7 interface 7-3.
The corresponding MPO female plug on the optical fiber cable uses a flat interface that meets the dimensional specifications of IEC 61754-7 interface 7-4. Figure 4 illustrates an MPO female plug connector on the patch cord and an MPO male receptacle at the equipment interface.
Both 40 GbE and 100 GbE use the MPO connector interface at the MDI. The lane assignments for transmit and receive fibers are illustrated in Figure 5.
40 GbE uses a 12 position MPO connector interface that aligns 12 fibers is a single row. Four transmit fibers are used on one side and four receive fibers are used on the opposite side of the MPO connector, for a total of eight fibers. The middle four fiber positions are not used.
The recommended equipment interface for 100 GbE is a 24 position MPO connector with two rows of 12 fibers. Ten receive fibers are used in the top row and ten transmit fibers are used in the bottom row for a total of 20 multimode fibers. Alternatively, two 12-fiber MPO interfaces, either side-by-side or top-and-bottom, are also allowed as an equipment interface in the IEEE standard for 100 GbE.
Migrating from 10 GbE (that uses two fibers in either a SC Duplex or a LC Duplex connector) to 40 GbE and 100 GbE will require a lot more fibers and a different type of connector. The way that optical fiber cabling is deployed for 10 GbE can facilitate an easier migration path to 40 GbE and 100 GbE, in the future. An effective migration strategy needs to provide a smooth transition to the higher Ethernet speeds with minimum disruption and without wholesale replacement of existing cabling and connectivity components.
Optical fiber cabling is commonly deployed for backbone cabling in data centers for switch to switch connections and also for horizontal cabling for switch to server and storage area network connections. The use of pre-terminated optical fiber cabling can facilitate the migration path to 40G and 100G Ethernet, in the future. Figure 6 illustrates a pre-terminated cable assembly containing 24 OM4 multimode fibers with two 12-fiber MPO connectors at both ends. This fiber cable assembly plugs into the back of a breakout cassette that splits the 24 fibers into 12 LC Duplex connectors at the front of the cassette.
Four of these cassettes are mounted in a one rack unit (1U) patch panel to provide up to forty-eight 10G equipment connections using LC Duplex patch cords. The Belden FiberExpress Ultra HD 1U patch panel with four LC Duplex cassettes is illustrated on the left hand photo in Figure 7.
Let’s say in the next three years it is necessary to provide some 40G connections, either as a replacement of or as an addition to the existing 10G connections.
Replacement Scenario - if upgrading from 10G to 40 G, one or more of the LC Duplex cassette(s) can be replaced with 12 MPO adapters. The MPO adapters are designed to fit in the same opening as the cassettes. Belden also offers a high density 18 MPO adapter with the same overall physical dimensions as 12 MPO Adapter as shown in Figure 8. The right hand photo in Figure 7 (See previous page) illustrates the case where all four cassettes are replaced with four high density 18 MPO adapters. Figures 7 and 8 illustrate an upgrade path from 10G to 40G that does not require any additional space and reuses the same patch panels. The 12 LC Duplex cassette(s) are replaced with either 12 MPO or 18 MPO adapter(s) as needed. Additional 24-fiber cable assemblies (or any fiber counts in multiples of 12 fibers) are provided as needed for backbone or horizontal cabling.
Addition Scenario – if it is required to add some 40G connections while retaining the 10G connections, Belden also offers a high density cassette containing 18 LC Duplex connections in the same space as a 12 LC Duplex cassette (see Figure 8). Three of the 12 duplex cassettes can be replaced with three 18 LC Duplex cassettes, thus maintaining the 48 10G connections while freeing space for either a 12 MPO or 18 MPO adapter providing up to 18 additional 40G connections. The requisite number of additional fiber cable assemblies in multiples of 12 fibers are provided as needed.
Belden FiberExpress Ultra HD patch panels are available in sizes of 1U, 2U and 4U. The capacity of these patch panels are shown in Figure 9. The FX Ultra HD can accommodate up to 288 LC Duplex or 288 MPO connections in a space of 4 rack units. These patch panels provide the greatest flexibility in migrating to 40G connectivity for your network.
The modular design accommodates either pre-terminated cassettes or field terminated adapter frames.
The maximum insertion loss for 40 Gigabit and 100 Gigabit Ethernet for a 100 meter channel is 1.9 dB using either OM3 or OM4 multimode fiber and 1.5 dB for a 150 meter channel using OM4 multimode fiber. Using worst case MPO connectors with a maximum loss of 0.75 dB per connector (as specified in TIA 568-C.3) and a multimode optical fiber cable with a loss of 3 dB/km (or 0.3 dB/100m) allows a maximum of two connectors for a 100 meter channel. Belden also offers pre-terminated cable assemblies that use low loss MPO connectors with a maximum loss of 0.35 dB per connector. The use of low loss connectors allows up to four connectors for a 100 meter channel. The optical loss calculations for a 100 meter channel are shown in Figure 10.
The IEEE 802.3ba standard requires the use of low loss MPO connectors (0.5 dB loss) when using OM4 multimode fiber for longer channel lengths up to 150 meters. The optical loss calculations for 150 meter channels using Belden’s low loss connectors and OM4 fiber are shown in Figure 11. Up to three, low loss, Belden MPO connectors can be accommodated for a 150 channel using pre-terminated cable assemblies.
The TIA 568-C.0 standard outlines sample methods A and B for maintaining the polarity of parallel array systems. Both methods achieve the same end result, that is to create an optical path from the transmit port of one device to the receive port of another device. Unless otherwise specified, Belden uses Method B polarity scheme for two connector channels using Type B (female to female) OM4 low-loss patch cords; Type B (male to male) OM4 low loss array connector cable assemblies; and Type A (key up to key down) MPO adapters as illustrated in Figure 12.
In order to make a parallel array connection using an MPO adapter, it is important to note that one plug is pinned and the other plug is unpinned.
The MPO receptacle at the transceiver is pinned. Type B patch cords are unpinned on both ends. The Type B parallel array connector cable (also called trunk cable assembly) is pinned at both ends.
The polarity configuration for a 3-connector channel that would be representative of a data center with a Zone Distribution Area (ZDA) is illustrated in Figure 13. The components are the same as for a two connector channel, except that a Type A array connector extension cable is used between the second connector at the ZDA and the third connector at the Equipment Distribution Area (EDA).
The polarity configuration for a 3-connector channel that would be representative of a data center with a cross-connect at the Horizontal Distribution Area (HDA) is illustrated in Figure 14. The components are the same as for a two connector channel, except that a Type A array connector extension cable is used between the network equipment and the first connector termination at the HDA cross-connect. A Type-B patch cord is used to make cross-connections between the first and second connector at the HDA cross-connect. A Type B array connector cable is used between the second connector at the HDA and the third connector at the EDA. A Type-B patch cord is used for the equipment connection at the EDA.
The polarity configuration for a 4-connector channel is illustrated in Figure 15 and is the same as a two connector channel using the same Type B array connector cables and cords and Type A adapters.
The good news is that all the components that are designed to migrate from 10 Gigabit to 40 Gigabit Ethernet are the same components that are used for 100 Gigabit Ethernet. The only difference is that a 100 Gigabit Ethernet connection is established using a Type B array patch cord that interconnects two, 12-fiber MPO connectors in the FX Ultra HD patch panels to one, 24-fiber MPO connector at the transceiver. The 100 Gigabit Ethernet, Type B array patch cord polarity is illustrated in Figure 16. No changes to the patch panels or cabling are required. The 100 Gigabit channels can be added by using spare positions in the MPO adapter frames or by replacing the appropriate number of 40 Gigabit channels
Paul Kish is the Director of Systems and Standards with Belden. Paul Kish is a key contributor to the development of cabling standards with TIA, ISO and IEEE and also serves on the BICSI Technical Information & Methods Committee.