MAWI Working Group Traffic Archive


Packet traces from WIDE backbone

This is a traffic data repository maintained by the MAWI Working Group of the WIDE Project.

Currently, traffic traces are collected at the following sampling points:

samplepoint-F
daily trace at the transit link of WIDE (150Mbps) to the upstream ISP, in operation since 2006/07/01: 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014.
longer traces: 48-hour-long traces on 2007/01/09-11, 72-hour-long traces on 2008/03/18-20, 96-hour-long traces on 2009/03/30-04/02, 83-hour-long traces on 2010/04/13-16, 63-hour-long traces on 2012/03/30-04/01, 72-hour-long traces on 2013/06/25-27, and 24-hour-long traces on 2014/10/02, as part of a Day in the Life of the Internet project.
The link was upgraded from 100Mbps to 150Mbps on June 1 2007.

Note about a large amount of ICMP traffic is in the traces, probing the entire IPv4 space by the USC ANT project. The probing started in September 2011 with sporadic probing, but changed to constant higher-rate probing since March 27, 2013.

Older traces:

samplepoint-A (terminated in Nov 2000)
daily trace of a trans-Pacific T1 line (one of them): 1999, 2000
Y2K roll-over
samplepoint-B
daily trace of another trans-Pacific line (18Mbps CAR on 100Mbps link) This link was terminated on 2006/07/01, and replaced by Samplepoint-F: 2000, 2001, 2002, 2003, 2004, 2005, 2006
107-hour-long traces from another US-Japan link taken from 1999/05/10 10pm to 1999/05/15 7am.
The link was upgraded from 4Mbps to 10Mbps on 5/13 around 15:30.
24-hour-long traces on 2003/02/27, 2005/01/07, 2005/09/22, and 2006/03/03.
plot graphs by aguri: port numbers: 2001, 2002, 2003, 2004, 2005, 2006 (wide-members only: addresses: 2001, 2002, 2003, 2004, 2005, 2006)
samplepoint-C
daily trace at an IPv6 line connected to 6Bone: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009
samplepoint-D
daily trace at an IPv6 line connected to WIDE-6Bone: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009
samplepoint-E
yet another US-Japan line (OC-3) which came into operation in Dec 2001:
plot graphs by aguri: port numbers: 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 (wide-members only: addresses: 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009)

Traffic traces are made by tcpdump, and then, IP addresses in the traces are scrambled by a modified version of tcpdpriv.
The soure code of a set of tools used to create our archive is also available.

MAWILab is a database that assists researchers to evaluate their traffic anomaly detection methods. It consists of a set of labels locating traffic anomalies in the MAWI archive (samplepoints B and F). The labels are obtained using an advanced graph-based methodology that compares and combines different and independent anomaly detectors. The data set is daily updated to include new traffic from upcoming applications and anomalies.

You may use WIDE traffic data for only research purposes. Actions that trespass upon users' privacy are prohibited.

Here is guidelines for protecting user privacy in WIDE traffic archive. (japanese version).

Related Papers

  1. R. Fontugne, P. Borgnat, P. Abry, K. Fukuda.
    "MAWILab: Combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking".
    ACM CoNEXT 2010. Philadelphia, PA. December 2010.
  2. Pierre Borgnat, Guillaume Dewaele, Kensuke Fukuda, Patrice Abry, Kenjiro Cho.
    "Seven Years and One Day: Sketching the Evolution of Internet Traffic."
    INFOCOM2009. Rio de Janeiro, Brazil. April 2009.
  3. Guilaume Dewaele, Kensuke Fukuda, Pierre Borgnat, Patrice Abry, Kenjiro Cho.
    "Extracting Hidden Anomalies using Sketch and Non Gaussian Multiresolution Statistical Detection Procedures"
    SIGCOMM2007 LSAD Workshop, Kyoto Japan. August 2007..
  4. Kenjiro Cho, Ryo Kaizaki and Akira Kato.
    "Aguri: An Aggregation-based Traffic Profiler"
    In Proceedings of QofIS2001 (published by Springer-Verlag in the LCNS series). September 2001.
  5. Kenjiro Cho, Koushirou Mitsuya and Akira Kato.
    "Traffic Data Repository at the WIDE Project"
    USENIX 2000 FREENIX Track, San Diego, CA, June 2000. (HTML version)
  6. Akira KATO, Jun MURAI, Satoshi KATSUNO and Tohru ASAMI.
    "An Internet Traffic Data Repository: The Architecture and the Design Policy"
    INET99, San Jose, CA, June 1999.

Links