ECI 2017 - Internet-Scale Experimentation


Fabián E. Bustamante

Location and Time

Lectures:July 24-28, 2017, 9AM-12PM
Fac. de Ciencias Exactas, UBA

Course overview

How would you ...

  1. Evaluate the effectiveness of a feature you added to your startup’ new app?
  2. Understand the tense relationship between Netflix and ISP?
  3. Characterize the impact of population growth on urban spaces?
  4. Understand what determines the quality of experience of Internet users?
  5. Measure the consequences of network censorship on user experience?

The answers to this seemingly disparate set of questions share a common requirement – carrying experimentation at Internet-scale.

Internet-scale Experimentation is a graduate-level seminar exploring the challenges of large-scale networked system experimentation and measurements.

Over the last few decades, networked systems have become an integrated part of everyday life and a critical piece of our economic, educational, health and defense systems. This fact is normally brought up as evidence of the success and broader impact of our field of work.

The other, typically avoided, side of the story is the complications that this translates into for experimentalists. Today it is virtually impossible to run a randomized controlled experiment at even fractions of the scale of many of our systems. Despite this, as we explore new ideas in these uncharted territories we are reasonably asked to provide better evidence of the effects of interventions. In this seminar we will discuss ongoing projects on networked systems experimentation and their applications, in wired and wireless settings, that address some of these challenges.

The class consists of two major components: reading and reviewing papers and doing a research project on your own. For the research part of the course, you will have the chance to work (and expand) some existing platforms and datasets as you formulate and try to answer these and other interesting questions of Internet scale.


  • Introduction to the course and platforms
  • Strategies, approaches and ethical considerations
  • Network infrastructure, routing and traffic
  • Networked systems and their users; moving up the stack
  • On dangerous grounds – privacy, security and censorship


There are two options for a final exam: a final project or a take-home written exam based on a subset of the papers discussed in class. You can choose either (or both!), but should let me know soon.

Projects can be done in teams of up to two students. The written exam is individual, however, and should be your own work. It is OK to meet with colleagues, study together and discuss readings but what you turn in must be your own work.

The written final exam will include five questions from the reading (only papers marked with a [Q]. The questions will test your understanding of the motivating problem, the proposed solution/analysis and its evaluation (something between Keshav's second and third reading pass).

For the "project" option, you or your team should contact me to discuss the ideas you would like to explore using RIPE Atlas. You can find some examples in the Reading and Projects tab.

Day 1: Introduction to the instructor, the course and the platforms we will rely on
Slides: [A] | [B] | [C]
Internet architecture, background
  • Internet Measurement: Infrastructure, Traffic and Applications, Chap. 2-4, M. Crovella and B. Krishnamurthy. J. Wiley et Sons, 2006
  • Lessons Learned from using the RIPE Atlas Platform for Measurement Research, V. Bajpai, S. J. Eravuchira, J. Schonwalder, ACM SIGCOMM CCR, 45(3), July 2015
Meta - How to read a research paper
  • How to read a research paper, S. Keshav, ACM SIGCOMM CCR, 37(3), July 2007
  • On Power-Law Relationships of the Internet Topology, M. Faloutsos, P. Faloutsos, and C. Faloutsos, SIGCOMM 1999
    [paper] [Q]
Day 2: Strategies, Approaches and Ethical Considerations Strategies
Slides: [A] | [B] | [C] | [D]
  • Strategies for Sound Internet Measurement, V. Paxson, IMC 2004
  • What are our standards for validation of measurement-based networking research?, B. Krishnamurthy, W. Willinger, Hotmetrics 2008
    [paper] [Q]
  • Error and attack tolerance of complex networks, R. Albert, H. Jeong and A-L. Barabasi, Nature 406, 27 July 2000
    [paper] [Q]
  • Need, Want, or Can Afford Broadband Markets and the Behavior of Users, Z. Bischof, F. Bustamante, R. Stanojevic, IMC 2014
    [paper] [Q]
Ethical considerations
  • Ethical considerations in network measurement papers, C. Partridge, M. Allman, CACM, 59(10), October 2016
  • Experimental evidence of massive-scale emotional contagion through social networks, A. Kramer, J Guillory and J. Hancock, PNAS, 111(24), June 2014
Day 3: Network Infrastructure, Routing and Traffic
Slides: [A] | [B]
  • Internet Inter-domain Traffic, C. Labovitz, S. Iekel-Johnson, D. McPherson, J. Oberheide, and F. Jahanian. SIGCOMM 2010
    [paper] [Q]
  • Inter-Domain Traffic Estimation for the Outsider, M. Sanchez, F. Bustamante, B. Krishnamurthy, W. Willinger, G. Smaragdakis, J. Erman, IMC 2014
    [paper] [Q]
Day 4: Networked systems and their users
Slides: [A] | [B] | [C]
  • Mapping the Expansion of Google’s Serving Infrastructure, M. Calder, X. Fan, Z. Hu, E. Katz-Bassett, J. Heidemann, R. Govindan, IMC 2013
    [paper] [Q]
  • Anatomy of a Personalized Livestreaming System, B. Wang, X. Zhang, G. Wang, H. Zheng, B. Zhao, IMC 2016
    [paper] [Q]
  • EYEORG: A Platform for Crowdsourcing Web Quality of Experience Measurements, M. Varvello, J. Blackburn, D. Naylor, K. Papagiannaki, CoNEXT 2016
Day 5: On dangerous grounds – privacy, security and censorship
Slides: [A] | [B]
  • A look at the consequence of Internet censorship through an ISP lens, S. Khattak et al., IMC 2014
    [paper] [Q]
  • Alibi Routing, D. Levin, Y. Lee, L. Valenta, Z. Li, V. Lai, C. Lumezanu, N. Spring, B. Bhattacharjee, SIGCOMM 2015
    [paper] [Q]

Reading and Projects

On reading

You may find the following documents useful:

Some project ideas

  1. Using RIPE Atlas TraceMON to discover network topology, J. Den Hertog, AFRINIC Blog, July 2017
  2. Using RIPE Atlas to Measure Latency to Reunion Island, R. Noordally, RIPE NCC Blog, April 2017
  3. Using RIPE Atlas to validate international routing detours, A. Shah, Jan 2017
  4. Routing Detours: Can We Avoid Nation-State Surveillance?, A. Edmundson, RIPE NCC Blog, November 2016
  5. Using RIPE Atlas to Measure Cloud Connectivity, J. Read, RIPE NCC Blog, September 2016