CS 445: Internet-Scale Experimentation

ANNOUNCEMENTS

Remember to check this (and Canvas) regularly!

ADMINISTRATIVE INFORMATION

STAFF

Professor

Fabián E. Bustamante Seely Mudd #3905 +1 847 491-2745 fabianb@cs…

TA

TBD

LOCATION AND TIME

**Lectures: **TuTh 9:30-10:50AM | Frances Searle Building 1-421

**Professor Office Hours: **TBD

CATALOG DESCRIPTION

Internet-scale Experimentation is a graduate-level seminar exploring the challenges of large-scale networked system experimentation and measurements.

This course satisfies the Systems requirement and the project requirement.

COURSE PREREQUISITES

CS 340 “Intro to Computer Networking” or CS 345 “Distributed Systems”.

If you have taken similar courses somewhere else or have not taken any of these courses, but would like to register for this seminar, please contact me.

In compliance with Section 504 of the 1973 Rehabilitation Act and the Americans with Disabilities Act, Northwestern University is committed to providing equal access to all programming. Students with disabilities seeking accommodations are encouraged to contact the office of Services for Students with Disabilities (SSD) at +1 847 467-5530 or ssd@northwestern.edu. SSD is located in the basement of Scott Hall. Additionally, I am available to discuss disability-related needs during office hours or by appointment.


How would you …

    - Evaluate the effectiveness of a feature you added to your startup’ new app?- Understand the tense relationship between Netflix and ISPs in 2014?- Characterize the impact of Covid19 on the Internet?- Understand what determines the quality of experience of Internet users?- Measure the consequences of network censorship on user experience?

The answers to this seemingly disparate set of questions share a common requirement – carrying experimentation at Internet-scale.

Internet-scale Experimentation is a graduate-level seminar exploring the challenges of large-scale networked system experimentation and measurements.

The other, typically avoided, side of the story is the complications that this translates into for experimentalists. Today it is virtually impossible to run a randomized controlled experiment at even fractions of the scale of many of our systems. Despite this, as we explore new ideas in these uncharted territories we are reasonably asked to provide better evidence of the effects of interventions.

The other, typically avoided, side of the story is the complications that this translates into for experimentalists. Today it is virtually impossible to run a randomized controlled experiment at even fractions of the scale of many of our systems. Despite this, as we explore new ideas in these uncharted territories we are reasonably asked to provide better evidence of the effects of interventions.

In this seminar, we will discuss ongoing projects on networked systems experimentation and their applications, in wired and wireless settings, that address some of these challenges. Throughout the quarter you will be reading, reviewing, and presenting papers and doing a research project on your own. For the research part of the course, you will have the chance to work (and expand) some existing platforms and datasets as you formulate and try to answer these and other interesting questions of Internet scale.

TOPICS

  • Introductory notes: Internet architecture, practical issues and good practices for Internet-scale experimentation- Experimental platforms: Experimental design, context of experiments- Experimental design- End-to-end and up-the-stack- Network infrastructure- Traffic- Applications and distributed services: DNS, Web, P2P, VoD, OSN, ..- Botnets and other maladies- Security and ethical issues

COURSE ORGANIZATION

The course is organized as a series of paper discussions and a single term-long project.

Most class meetings will be centered around paper presentations and discussion. You should read each paper before coming to class and be prepared to discuss it.

What you get out of school depends on what you put on it, that is particularly true of seminars. You should read all [su_label type=“default” class=""]background[/su_label], [su_label type=“success” class=""]question[/su_label] and [su_label type=“important” class=""]review[/su_label] papers before coming to class (see tags in the Calendar tab) and be prepared to discuss them. As a way of reading enforcer, before each class, you must answer (in Canvas) a short question of all required readings you are not assigned as a reviewer.

The class will run as mini-conference with you as the Program Committee member. We will use a subset of the papers included in the schedule as our submissions. Each of you will write reviews for 4-6 of them. We will discuss the papers in a PC meeting at the end of the quarter to decide what paper “should be accepted” for publication.

While there is no textbook for the course, a great book in Internet measurement is M. Crovella and B. Krishnamurthy, Internet Measurement: Infrastructure, Traffic and Applications, Wiley 2006.

COMMUNICATION CHANNELS

There are a number of communication channels set up for this class:

  • We will use the course web site (here and Canvas) to post announcements related to the course. You should check this regularly for schedule changes, clarifications and corrections to assignments, and other course-related announcements.
  • We will use the Piazza site (see the menu option in Canvas) for general discussions, and help you identify potential projects and groups. 
  • We will use Canvas or email for class discussion.- There is always email for questions that would be inappropriate to post on the newsgroup/discussion-board. When using email to contact the staff please start your subject line with “ISE: helpful-comment” to ensure a prompt response.

GRADING

I use a criterion-referenced method to assign your grade; in other words, your grade will be based on how well you do relative to predetermined performance levels, instead of in comparison with the rest of the class. Thus, if a test has 100 possible points, anyone with a score of 90 or greater will get an A (90-92: A-, etc.), those with scores of 80 or greater will get a B, those with scores of 70 or greater will get a C, and so on. Notice that this means that if everyone works hard and gets >90, everyone gets an A.

Total scores (between 0 and 100) will be determined, roughly, as follows:

  • Paper discussion participation (and questions) 15%- Paper review 20%- PC meeting participation 5%- Paper presentation 20%- Project 40%