Edgescope – Leveraging the View of a Distributed Internet Telescope

Sound Internet research requires comprehensive network measurements. While our measurement techniques are continually improving, however, the visible portion of the Internet, and thus the representativeness of our measurement, is conversely shrinking. Although some distributed research platforms offer many vantage points in the network, much of the Internet’s growth occurs in domains beyond their reach, such as behind NAT boxes and firewalls or in regions of the Internet not exposed by public BGP feeds. Providing access to universal network information would seem to require a widespread deployment of measurement agents in the edge of the network, introducing regular test traffic at sufficient scale to drive general experimentation. While a straightforward implementation of this vision at a global scale appears economically and socially infeasible, we posit that much of this traffic is already being generated and can be opportunistically measured.

The goal of EdgeScope is to leverage the network view of end systems located at the edge of the network. To expose this view, we exploit the fact that peers in large-scale peer-to-peer systems (P2P) have a unique and valuable perspective on network conditions, one to which today’s researchers, network operators and users have limited or no access. Through extensions to a popular BitTorrent client, we have collected diagnostic information that not only validates our design goals but also provides a rich source of data regarding edge system network views. Combined, these datasets cover hundreds of thousands of users located in hundreds of counties and thousands of networks worldwide.

Any dataset gathered from end users is subject to important challenges regarding data archival, distribution, processing and privacy. We will address this issues in the course of this project.

People

Publications

Other Public Dataset