ANNOUNCEMENTS
Remember to check this (and Canvas) regularly!
STAFF
Instructor
Fabián E. Bustamante
Seely Mudd #3509
fabianb@cs…
TAs
Kedar Thiagarajan
Seely Mudd
kedarthiagarajan2028@..
Caleb Wang
Seely Mudd
caleb.wang@..
LOCATION AND TIME
Lectures: Tuesdays and Thursdays 11:00-12:20PM | Tech L4 (map)
Professor Office Hours: By appointment
TA Office Hours:
- TBD
TA/Resitation Sessions: Thursdays 4:00-5:00 pm | Frances Searle 1441
CATALOG DESCRIPTION
Basic principles behind distributed systems (collections of independent components that appear to users as a single coherent system) and main paradigms used to organize them.
COURSE PREREQUISITES

In compliance with Section 504 of the 1973 Rehabilitation Act and the Americans with
Disabilities Act, Northwestern University is committed to providing equal access to all
programming. Students with disabilities seeking accommodations are encouraged to contact the office of Services for Students with Disabilities (SSD) at +1 847 467-5530 or ssd@northwestern.edu. SSD is located in the basement of Scott Hall. Additionally, I am available to discuss disability-related needs during office hours or by appointment.
Distributed systems are collections of networked computers that coordinate their actions through message exchanges. Most computing systems you interact with every day are indeed distributed (e.g. email, the Web, Google, Skype, Facebook, …) for a variety of reasons such as fault tolerance, performance, and the geographical nature of the requirements.
In this course, we will discuss some of the basic principles behind distributed systems as well as common approaches and techniques used to build them. We illustrate these ideas through case studies of widely used or seminal systems.
SOME OF THE TOPICS COVERED
- Networking and Communication
- Physical and Logical Clocks
- Coordination in Distributed Systems
- Distributed storage and file systems
- Name services
- Global state and transactions
- Replication and consistency
- Consensus
- Fault tolerance
- Security and privacy
COMMUNICATION CHANNELS
There are a number of communication channels set up for this class:
- We will use the course website and associated Canvas site to post announcements related to the course. You should check this regularly for schedule changes, clarifications and corrections to assignments, and other course-related announcements.
- We will use Piazza for class discussion. The TAs and I will check Piazza frequently and answer unresolved questions, but you’re also encouraged to collaborate with each other and answer each other’s questions.
- There is always email for questions that would be inappropriate to post on the newsgroup/discussion board. When using email to contact the staff please start your subject line with “CS345:
” to ensure a prompt response (do replace “helpful-comment” by an actual helpful comment, of course 🙂 ).
COURSE ORGANIZATION
The course is organized as a series of lecture and paper discussions, projects, homework assignments, and a final exam.
- Lectures and discussions – A set of lectures on the core of the material.
- Readings – Paper reading in preparation for (not substitution of) the lecture.
- Homework assignments – A set of assignments meant as reader enforcers.
- Projects – Four programming projects to give you a better understanding of the subject matter and experience with the Go programming language.
- A final exam.
GRADING
I use a criterion-referenced method to assign your grade; in other words, your grade will be based on how well you do relative to predetermined performance levels, instead of in comparison with the rest of the class. Thus, if a test has 100 possible points, anyone with a score of 90 or greater will get an A (90-92: A-), those with scores of 80 or greater will get a B (80-82: B-), those with scores of 70 or greater will get a C, and so on. Notice that this means that if everyone works hard and gets >93, everyone gets an A.
Total scores (between 0 and 100) will be determined, roughly, as follows:
- Pre-class reading/participation 15%
- Homework assignments 20%
- Projects 45%
- Final Exam 20%
POLICIES
Late Policy:
You are allotted 7 late days for use at your discretion throughout the quarter. These are designed to give you flexibility when life happens—no questions asked.
How it works: There is a 30-minute grace period after each deadline. Submissions within this window are not considered late. After the grace period, any submission up to 24 hours late uses 1 late day. Every additional 24-hour period past the deadline (not counting the grace period) consumes another late day. (e.g., 15′ late, no penalty; 10 hours late, 1 late day used).
You have 7 late days total across all assignments. Use them wisely—once they’re gone, they’re gone. If you exhaust your late days, additional late work will receive zero credit unless prior arrangements have been made or exceptional circumstances apply.
Late days cannot be used for group assignments unless all members have enough remaining days.
Extensions: Life happens. If you’re dealing with something serious (illness, emergencies, etc.), reach out before the deadline—we can work something out.
Cheating vs. Collaboration:
Collaboration is a really good thing and we encourage it. On the other hand, cheating is considered a very serious offense. When in doubt, remember that it’s OK to meet with colleagues, study for exams together, and discuss assignments with them. However, what you turn in must be your own (or for group projects, your group’s own) work. Copying code, solution sets, etc. from other people or any other sources is strictly prohibited.
For projects, we do code walkthrough by randomly selected groups with the staff. The idea is simple if we pick your group you and your teammates will meet with the TA and/or instructor and walk them through your code, answering any questions they may have.
To get full credit you must be able to carry the walkthrough, showing you understand your code.
Note that our random sampling is with replacement, i.e., you may do it multiple times.
The following is our intended calendar with topics, slides (as they become available) and reference material. Papers, except when tagged as [ref], may be part of homework assignment/finals questions. All papers are available in Canvas (“Reading” folder in the “Files” section); some links in the Calendar point to those files.
Week | Date | Topics and Reading | Due |
1 | 04/01 | Northwestern Monday: Classes scheduled to meet on Mondays meet on this day. | |
04/03 | Introduction
Reading:
|
||
2 | 04/08 | Networking
Reading:
|
|
04/10 | Communication and Organization
Reading:
|
||
3 | 04/15 | Physical and Logical Clocks
Reading:
|
|
04/17 | Global State
Reading:
|
||
4 | 04/22 | Coordination
Reading:
|
|
04/24 | Failure and Failure Detection
Reading:
|
||
5 | 04/29 | Consistency and Replication
Reading:
|
|
05/01 | Eventual Consistency
Reading:
|
||
6 | 05/06 | Overlay Networks
Reading:
|
|
05/08 | Scaling Out Key-Value Stores
Reading:
|
||
7 | 05/13 | Consensus Problem and The Impossibility of Consensus
Reading:
|
|
05/15 | Consensus
Reading:
|
||
8 | 05/20 | Byzantine Fault Tolerance
Reading:
|
|
05/22 | Distributed File Systems
Reading:
|
||
9 | 05/27 | Content Distribution Networks
Reading:
|
|
05/29 | Transactions | ||
10 | 06/03 | Distributed Transactions
Reading:
|
|
06/05 | New Distributed Systems
Reading:
|
ASSIGNMENTS
There are several pre-class reading assignments and eight homework assignments. These are mostly meant as reading enforcers. There are four projects (the first one split in two parts), and a final exam.
We will post all assignments in the Canvas’ site for the course.
PRE-CLASS READING
Before most classes, you will be ask to read a designated research paper—at minimum, the abstract, introduction, and conclusion—to get a high-level sense of the work. You’ll submit two things: (1) a one-sentence summary that captures the paper’s core problem and contribution, and (2) a question or point of confusion you’d like discussed in class. This could be about a technical detail, an unclear concept, or a broader implication of the work. Responses are due before class and serve to guide discussion and surface common sticking points.
HOMEWORK ASSIGNMENTS
Homework assignments dive deeper into the material, often revisiting a paper you’ve already read or building on ideas discussed in lecture. These assignments ask you to critically engage with the work—identifying strengths and weaknesses, exploring interesting technical details, or reasoning through hypothetical scenarios that test your understanding. They are meant to sharpen your analytical thinking and prepare you to evaluate and design systems like the ones we study.
PROJECTS
There will be four projects, including a MapReduce library and a replicated state machine protocol. Projects are to be done in teams of 2-3 students (1 is not allowed).
All projects will be done in Go, a language that was originally created within Google, but is now a fully open-source project. Go is garbage-collected and has built-in coroutines (called goroutines) and channels, making it highly suited to building distributed systems. Its standard library is already pretty comprehensive. For example, take a look at the net and rpc packages.
MATERIALS
Papers
- A set of research papers (see the calendar tab).
- S. Keshav, How to Read a Paper, ACM SIGCOMM CCR, 37(3), 2007 (A very useful reference)
Go useful links
- A Tour of Go
- Effective Go
- Go by example
- Network Programming with Go
- Links to Go forums, mailing list, Slack, and FAQ
Reading List
- Google’s Introduction to Distributed System Design
- J. Dean and S. Ghemawat, MapReduce: Simplified Data Processing on Large Clusters, Proc. of OSDI, 2004
- K. Seemakhupt et al., A Cloud-Scale Characterization of Remote Procedure Calls, Proc. of SOSP, 2023
- K. Seemakhupt et al., A Cloud-Scale Characterization of Remote Procedure Calls,Proc. of SOSP, 2023
- L. Lamport, Time, Clocks, and the Ordering of Events in a Distributed System, Communications of the ACM, July 1978, pages 558-564
- M. Chandy and L. Lamport, Distributed Snapshots: Determining Global States of Distributed Systems, ACM Trans. Comput. Syst., 3(1):63-75, 1985
- P. Hunt et al., ZooKeeper: Wait-free coordination for Internet-scale systems, Proc. of USENIX ATC, 2010
- J. Leners et al., Detecting failures in distributed systems with the FALCON spy network, Proc. of SOSP, 2011
- D. Scales et al., The Design of a Practical System for Fault-Tolerant Virtual Machines, ACM SIGOPS OSR, December 2010
- D. Terry et al., Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System, Proc. of SOSP, 1995
- G. DeCandia et al., Dynamo: Amazon’s Highly Available Key-value Store, Proc. of SOSP, 2007
- M. Fischer, N. Lynch, M. Paterson, Impossibility of Distributed Consensus with One Faulty Process, Journal of the ACM, 32(2), April 1985
- D. Ongaro and J. Ousterhout, Practical Byzantine Fault Tolerance, Proc. of OSDI, 1999
- S. Ghemawat, H. Gobioff, and S.-T. Leung, The Google File System, Proc. of SOSP, 2003
- F. Chen et al., End-User Mapping: Next Generation Request Routing for Content Delivery, Proc. of SIGCOMM, 2015
- J. Corbett et al., Spanner: Google’s Globally-Distributed Database, Proc. of OSDI, 2012
- J. Ferreira Loff et al., Antipode: Enforcing Cross-Service Causal Consistency in Distributed Applications, Proc. of SOSP, 2023