2004 Internet2 IPLS Abilene backbone instrumentation

In 2002 NLANR and Internet2 made the first attempt at instrumenting the Abilene backbone at the Indianapolis router node. The network then was operating at a core link speed of 2.5 Gigabits/second (OC48c Packet-over-SONET). The installation process proved reasonably sophisticated and as a result only a pair of OC48MONs became operational, providing first packet header traces from the core of a high-performance Internet for use by the general public. At the time the installation was also challenged by window-of-opportunity, the Abilene backbone was scheduled to be upgraded to 10 Gigabits/second in late 2002. As the link upgrade progressed, the OC48MON instrumentation became void, a downside of passive network measurements, which require high performance measurement equipment matching the physical link layer characteristics of the target network.

Over the past two years, NLANR's Measurement and Network Analysis team has been working jointly with Endace to develop the DAG6 series of measurement cards, which address the need for passive measurement equipment at 10 Gigabit/second speeds. First traces became available to the public in early 2004, and various real time applications are also under development.

Having gone through the experience two years ago, Internet2 and NLANR made the best use of the new opportunity to instrument the current 10 Gigabit backbone. The plan was cut to bring in three monitors, two OC192MONs plus one OC48MON, to instrument all three backbone links from Indianapolis (IPLS) to Chicago (CHIN), Kansas City (KSCY) and Atlanta (ATLA).

This time, thanks to the concerted effort of all the involved parties, everything went very smoothly and the reward for the research community is tremendous.

Acknowledgements

This project is an excellent example how the concerted effort of multidisciplinary teams can achieve something unique and challenging.

This work has seen the undivided approval, backing and support by the Abilene Backbone Engineering team. We would like to thank in person:

• Steve Corbató, Director Backbone Engineering at UCAID/Internet2
• Rick Summerhill, Associate Director Backbone Engineering at UCAID/Internet2
• Matt Zekauskas, Chair of the Internet2 Measurement Working Group
• Guy Almes, Chief Engineer for Internet2
• Caroline Carver, I2 Technical Staff at Indiana University
• Chris Small, I2 Technical Staff at Indiana University
• John Hicks, TransPAC and Indiana University

Please refer to the bottom of this page for use of any materials provided herein.

Technical Specifications

The fiber optic taps are NetOptics 96142-30 70:30@1300 nanometers rack mountable singlemode splitters. The OC48MON and OC192MON are based on Dell PowerEdge 2650 2U rack mountable servers. The operating system is Debian Linux. The network monitoring cards are Endace' DAG6.1 OC192c PoS / 10 GigE LAN/WAN and Dag4.2 OC48c passive measurement cards. Time is provided by an EndRun Technologies Praecis Ct CDMA time receiver, redistributed to the DAG cards via an Endace TDS-24 Time Distribution Server box. Please do not hesitate to contact us if you have further questions.

Caroline Carver from the Indiana GlobalNOC I2 backbone team working on one of the -48VDC power supply cables.

Chris Small, also from GlobalNOC, crimping cables for the legacy network connections.

(picture to the left) The Abilene rack in the Qwest POP. The front of the rack actually faces the cage, this picture from the back. On the lower right the Juniper T640, the Abilene IPLS router node. Above various components belonging to Abilene operations, terminal servers, Ethernet hubs, etc. At the top a row of Sentry power management servers. To the left top the four 2U NMS server machines, followed by two 1U Dells for Planetlab. Below the 1U NLANR AMP IPLS node, and the Sentry servicing the NLANR PMA installation, presently the three Dell 2650 machines (see also the picture above).

Not visible in those pictures is the row of fiber optic splitters (above the T640) and the CDMA time receiver unit (Praecis Ct from EndRun Technologies), which in turn pulses a TDS-24 Time Distribution Server from Endace, suppling a synchronized 1PPS signal to all the DAG network measurement cards in the PMA monitors.

LIM side of the Juniper T640 (pictures from the left and the right). At the top left the two OC192c PoS links to Kansas City (KSCY) and Chicago (CHIN). Next to the right are the two OC48c PoS connections to OARNet (not instrumented) and the backbone link to Atlanta (ATLA). The instrumented links are easily spotted by their remarkable yellow plastic cover (NetOptics). To the right and below a pair of OC3c and a pair of OC12c as well as four Gigabit Ethernet links towards I2 connectors and some for rack local connectivity.