Sound Internet research requires comprehensive network measurements. While our measurement techniques are continually improving, however, the visible portion of the Internet, and thus the representativeness of our measurement, is conversely shrinking. Although some distributed research platforms offer many vantage points in the network, much of the Internet’s growth occurs in domains beyond their reach, such as behind NAT boxes and firewalls or in regions of the Internet not exposed by public BGP feeds. Providing access to universal network information would seem to require a widespread deployment of measurement agents in the edge of the network, introducing regular test traffic at sufficient scale to drive general experimentation. While a straightforward implementation of this vision at a global scale appears economically and socially infeasible, we posit that much of this traffic is already being generated and can be opportunistically measured.
The goal of EdgeScope is to leverage the network view of end systems located at the edge of the network. To expose this view, we exploit the fact that peers in large-scale peer-to-peer systems (P2P) have a unique and valuable perspective on network conditions, one to which today’s researchers, network operators and users have limited or no access. Through extensions to a popular BitTorrent client, we have collected diagnostic information that not only validates our design goals but also provides a rich source of data regarding edge system network views. Combined, these datasets cover hundreds of thousands of users located in hundreds of counties and thousands of networks worldwide.
Any dataset gathered from end users is subject to important challenges regarding data archival, distribution, processing and privacy. We will address this issues in the course of this project.
- David R. Choffnes, Fabián E. Bustamante. BitTorrent Dataset Description v1, covering data released as part of the SIGCOMM 2008 publication.
- Kai Chen, David R. Choffnes, Rahul Potharaju, Yan Chen, Fabián E. Bustamante, Dan Pei, Yao Zhao. Where the Sidewalk Ends: Extending the Internet AS Graph Using Traceroutes From P2P Users. In Proc. of CoNEXT, December 2009.
Other Public Dataset