The NZIX-II data set is a collection of long GPS-syncronized IP header traces captured by the WAND research group with a DAG3.2E system at the New Zealand Internet Exchange hosted by the ITS department at the University of Waikato.
| Trace (click for graphs) | Run length | Size (gzip) | Download http | Download ftp | MD5 checksum |
|---|---|---|---|---|---|
| 20000705-152900 | 02:30:59 | 853MB | 0705-1529.gz | 0705-1529.gz | 8496ba564ae24bae740a8fb86239a558 |
| 20000705-180000 | 06:00:00 | 1482MB | 0705-1800.gz | 0705-1800.gz | 379edc3e8ef60ecc65569b974dd7bec7 |
| 20000706-000000 | 06:00:00 | 587MB | 0706-0000.gz | 0706-0000.gz | ca6cd20a0f01cfd56e3b7b32c4d9c9c9 |
| 20000706-060000 | 06:00:00 | 1197MB | 0706-0600.gz | 0706-0600.gz | 187387b6fb4a2b48fc674ea524d493e0 |
| 20000706-120000 | 06:00:00 | 2013MB | 0706-1200.gz | 0706-1200.gz | 56b093510c6a943a3c530e86aa084fda |
| 20000706-180000 | 06:00:00 | 1473MB | 0706-1800.gz | 0706-1800.gz | 7e95368bcb48ad2d5077aeae2ee6c467 |
| 20000707-000000 | 06:00:00 | 581MB | 0707-0000.gz | 0707-0000.gz | a8248110418b17b7fa2796eab4119d2b |
| 20000707-060000 | 06:00:00 | 1151MB | 0707-0600.gz | 0707-0600.gz | 19e9a8fc137e0181f241438fe81d3e5a |
| 20000707-120000 | 06:00:00 | 1883MB | 0707-1200.gz | 0707-1200.gz | 13341ee5286d83def47355050bb045c2 |
| 20000707-180000 | 06:00:00 | 1365MB | 0707-1800.gz | 0707-1800.gz | 900120ab9692814ccf451e6f981e55bc |
| 20000708-000000 | 06:00:00 | 613MB | 0708-0000.gz | 0708-0000.gz | 17358b5c636f4a1d7447697aebd4cf62 |
| 20000708-060000 | 06:00:00 | 698MB | 0708-0600.gz | 0708-0600.gz | 33373a9fcaa091a29295ebdc4c09521e |
| 20000708-120000 | 06:00:00 | 1221MB | 0708-1200.gz | 0708-1200.gz | 9ba20b27e60a1e0f687d2ce6d6588441 |
| 20000708-180000 | 06:00:00 | 1149MB | 0708-1800.gz | 0708-1800.gz | ad308401f447774ba16e0d46d6aabfc2 |
| 20000709-000000 | 06:00:00 | 531MB | 0709-0000.gz | 0709-0000.gz | a7ffefa7b877575d17aaa6886074510c |
| 20000709-060000 | 06:00:00 | 637MB | 0709-0600.gz | 0709-0600.gz | 1cc5ce5eb38586ef13aea9ad18b913f8 |
| 20000709-120000 | 06:00:00 | 1223MB | 0709-1200.gz | 0709-1200.gz | f0f43e8742414106dae94976a1f04c13 |
| 20000709-180000 | 06:00:00 | 1337MB | 0709-1800.gz | 0709-1800.gz | 768e09f7c226d71e829a3e0f905fb375 |
| 20000710-000000 | 06:00:00 | 475MB | 0710-0000.gz | 0710-0000.gz | 8e97e56fd4a80d8f4b9c57f70d1dd540 |
| 20000710-060000 | 06:00:00 | 1230MB | 0710-0600.gz | 0710-0600.gz | d7330816f488cbda0f2072cf19b15e6e |
| 20000710-120000 | 02:56:27 | 990MB | 0710-1200.gz | 0710-1200.gz | 2350c2bd9752b4a5aa0059cbe1f0951d |
When downloading the traces, please read the disclaimers. When using these traces for research and publications, please give appropriate credits for the NZIX-II data set towards NLANR MOAT and the WAND research group. The WAND group wishes to thank Dennis Su of Waikato's ITS for the support of this measurement point.
The traces are in DAG format, which currently is a fixed 64 bytes record format with 40 bytes of IP header (usually covering most, if not all, of the TCP/IP and UDP/IP headers). To start off, a simple ASCII converter is in d3h2asc.pl, although the timestamp output is inaccurate. This is just in case you need something to get started.
For more sophisticated analysis we recommend using the dagtools package available from the DAG software website. The tool dagdump is a simple converter into ASCII and serves as a good starting point when you wish to import DAG traces into a toolset of your own. Also, CAIDA's CoralReef package should have support for native DAG traces shortly.
At the time of the capture, NZIX served as a peering point among a number of major New Zealand ISP's:
The DAG monitor is connected to a Switched Port Analyzer (SPAN) port via 100BaseTX FastEthernet. Consequently, timestamps are skewed compared to their arrival or departure times at the input/output ports of the switch as the total capacity of the switch is higher than the monitoring uplink and packets arriving from different ports at the same time need to be queued (or dropped) before being delivered to the SPAN port. It is actually possible that traffic gets lost before being monitored, but we estimate that this chance is fairly low as the total bandwidth at the switch currently peaks at around 10-12 MBits/sec. Further analysis of the trace files may reveal more detailed information about the behaviour of the monitor configuration.
The collection is dominated by a contiguous 5-day trace starting on Wednesday 5th of July (in the list shown as 11 individual trace files), with a total size of 24GB/59GB (compressed, uncompressed) and containing approximately 843 million IP headers.
Traces are in DAG standard format (see Section 11, p.11 of the DAG Installation Guide. Trace records are of 64 bytes fixed length, containing full TCP/IP and UDP/IP headers in most cases. The original packet length has been preserved as part of the overhead in the DAG trace record. The traces have been sanitized for IP addresses in a non-reversible fashion and all non-(ICMP/UDP/TCP) traffic has been discarded. The amount of background traffic is known to be non-zero, but low most of the time.
Processing of the traces can be done with the forthcoming dagtools package available from the DAG software website.
Jörg Micheel, Ian Graham, Nevil Brownlee: The Auckland data set: an access link observed, submitted paper to the 14th ITC Specialist Seminar on Access Networks and Systems, April 25th-27th, 2001, Barcelona, Spain.
H Stele Martin, Anthony J McGregor, John G Cleary: Analysis of Internet Delay Times, Proceedings of the First Passive and Active Measurement Workshop, PAM2000, April 3rd-4th 2000, Hamilton, New Zealand, p.141ff.
Sarah K Joyce: Traffic on the Internet - A Study of Internet Games Traffic, BCMS 420 Honours project report, University of Waikato, Hamilton, New Zealand, 2000.
Darryl Veitch, Lidong Huang, EMUlab, University of Melbourne, and Patrice Abry, Ecole Normale Superieure de Lyon, Laboratoire de Physique: Studies of Long Range Dependencies and Self-Similiarity in Internet traffic patterns using Wavelet analysis. Darryl Veitch's home page.
If you are about to write a paper using the above data set, please kindly let us know.
The main home for this data set is at the WAND research group. Graphs include packets, bandwidth and new connections per each trace. The pointers are:
http://wand.cs.waikato.ac.nz/wand/wits/ http://wand.cs.waikato.ac.nz/wand/wits/nzix/2/
The traces online here are the files comprising the continous 5 days trace.