Content Delivery | Aqualab - Northwestern University

namehelp

Tue, 09 Oct 2012 00:00:00 +0000

Overview

namehelp improved web performance by obtaining more accurate redirections to nearby content delivery network servers. In short: it helped whenever you, or your ISP, used a remote DNS service.

The Problem

DNS is the Internet’s phone book, and it is invisible to almost everyone. Every site you visit, every message you send, begins with a lookup; a complex page may need dozens, and a browser may perform hundreds in a day. Most people never think about it, because their ISP provides the service transparently.

Over the preceding years, third-party public DNS services from Google, OpenDNS and Norton had become popular, and for good reasons — better security, better privacy, faster resolution. Through a large-scale study spanning over 10,000 hosts in nearly 100 countries, we found that while public DNS may indeed resolve names faster, web performance could suffer badly.

The cause is a hidden interaction with another invisible service. More than 70% of the top 1,000 websites rely on CDNs, which replicate content across thousands of servers and steer each user to a nearby copy — using the location of the user’s resolver as a proxy for the user. When the resolver is a public service somewhere else entirely, that proxy fails: users were being sent to replicas three times farther away than necessary.

CDNs and public resolvers were working on a fix, but the proposed standard had not achieved broad adoption, leaving users choosing between good web performance and the security and privacy they had switched for.

What namehelp Did

namehelp let users have both. It ran personalized benchmarks in the background, from the user’s own machine, to determine the optimal DNS configuration, and when it detected less than optimal web performance it corrected it by interacting with DNS and CDNs so content came from the nearest available replica.

Beyond benchmarking, which existing tools such as namebench and DNS Benchmark already did, namehelp added:

automatic DNS server configuration based on the sites a user actually visits
caching so frequently-visited sites resolved instantly
faster failover when a DNS server went down

It installed as a local system service — users pointed their machine at 127.0.0.1 — with browser extensions for Chrome and Firefox providing a dashboard.

The dashboard’s DNS view. In this example namehelp resolved in 9.51 ms against the network default’s 148.28 ms.

Where the gain came from: the CDN-hosted domains for which namehelp found a nearer replica.

Reception

The first release went out on 9 October 2012 for Windows, macOS and Debian/Ubuntu, with RHEL and CentOS packages following. Within a month it had 10,000 users. Coverage in McCormick News led to ACM TechNews, Computerworld and Network World, and from there to Lifehacker — which brought 2,000 new users in a single day — and on to outlets in France, Poland, Russia, Turkey, Brazil and Vietnam. Several reported the tool as speeding up web performance by 40%.

Source was released on 26 October 2012. A Firefox extension followed in 2013, a 2.0 beta for macOS in January 2018, and Namehelp Mobile brought DNS performance comparison to Android.

Privacy

The privacy policy was deliberately short: namehelp never collected personally identifiable information or the sites a user visited. Anonymous reporting — usage statistics, performance-improvement results, benchmark results for a generic set of popular sites, and results of other network measurements — could be switched off entirely.

Sinopsis y ¿por qué usar namehelp? (español)

La página original de namehelp se publicó en inglés y español. Se conserva aquí la versión en español.

Sinopsis. namehelp mejora el rendimiento web obteniendo redirecciones más apropiadas hacia servidores de contenido más cercanos. Básicamente, namehelp mejora tu rendimiento web si vos o tu proveedor de internet utiliza un servicio de DNS remoto.

¿Por qué usar namehelp? El Servicio de Nombres de Dominio (DNS) es una especie de guía telefónica para el Internet y es esencial para su funcionamiento. La mayoría de nosotros no nos percatamos de su existencia porque nuestro proveedor generalmente lo ofrece de manera transparente.

Los últimos años han visto el surgimiento de servicios de DNS público ofrecidos por compañías como Google, OpenDNS y Norton DNS, que presentan ventajas de seguridad, privacidad y velocidad de resolución. Aun así, a través de un estudio de gran escala que incluyó a más de 10,000 usuarios en más de 100 países, nuestro laboratorio descubrió que tu rendimiento web puede sufrir seriamente, aún cuando el servicio de DNS público efectivamente reduce el tiempo de resolución.

El problema se debe a la interacción del servicio DNS con las redes de entrega de contenido (redes CDN). Aproximadamente el 70% de los 1,000 sitios más populares utilizan redes CDN, que replican el contenido en miles de servidores y redireccionan a los usuarios al más cercano. La utilización de servicios de DNS público puede ocasionar que los usuarios sean redireccionados a servidores ¡mucho más lejos de lo necesario!

namehelp te permite obtener lo mejor de ambos mundos — utilizar un servicio de DNS público sin reducir tu rendimiento web. Ejecuta pruebas desde tu computadora para determinar tu configuración óptima de DNS y, si determina que estás recibiendo un rendimiento sub-óptimo, lo corrige automáticamente interactuando con DNS y CDNs para asegurar que el contenido sea entregado desde la copia más cercana.

Availability

namehelp is no longer distributed. It was released for Windows, macOS, and Debian- and RHEL-based Linux, with source available from October 2012.

Papers

Content Delivery and the Natural Evolution of DNS — IMC 2012
namehelp: Intelligent, Client-Side DNS Resolution — SIGCOMM 2012 poster, also in CCR

Coverage

Network World, Computerworld, ACM TechNews and McCormick News (2012); Lifehacker and ghacks.net (2012); MakeUseOf (2013); and coverage in France (Korben), Poland (Spider’s Web), Russia (Софт@Mail.Ru), Turkey (chip.com.tr), Brazil (IDG Now!, Dicas para Computador) and Vietnam (PC World VN, Mía Lùi).

People

Lead & PI

Fabián E. Bustamante (Northwestern University)

Students

With thanks to Andrés Bustamante for the logo, Mario Sánchez for the Spanish translation, and Richard Campbell and Keith Petersen for early feedback.

namehelp built on namebench, BeautifulSoup, dnspython, Flask, Jinja2, Werkzeug, graphy, httplib2, ntplib, PyASN1, pydhcplib and Python-RSA.

Client-Side DNS and Content Delivery — the project namehelp was built for

Client-Side DNS and Content Delivery

Fri, 01 Jun 2012 00:00:00 +0000

Project Overview

Content delivery networks decide which replica should serve you based on where your DNS query appears to come from. For most of the Web’s history that was a reasonable proxy: your resolver belonged to your ISP, so it sat near you, and a CDN steering toward the resolver was steering toward you.

Public resolvers broke that assumption. When users moved to third-party DNS services for speed, reliability, or filtering, their queries began arriving at CDNs from somewhere else entirely — and the CDN, doing exactly what it was designed to do, sent them to a replica near the resolver rather than near the user. Users adopting a service to make browsing faster could end up fetching content from much further away.

This project measured that interaction at scale and then did something about it.

Measuring the Interaction

Content Delivery and the Natural Evolution of DNS measured the end-to-end cost of remote DNS on CDN performance, and gave the first evaluation of the industry’s answer to it: the EDNS client-subnet extension, which attaches a truncated client address to the query so a CDN can steer on the client rather than on the resolver.

The extension works where it is deployed. The difficulty is that it requires both the resolver and the CDN to participate, and adoption was thin — so the performance cost kept falling, unevenly and invisibly, on precisely the content CDNs work hardest to place well. That gap is what made a client-side answer worth building.

namehelp

namehelp is the response, built on what the paper calls Direct Resolution: a proxying DNS server on the user’s own machine that keeps a third-party resolver for general queries — so users keep the reliability and speed they chose it for — while resolving CDN-hosted names directly, so the query carries the client’s own network position rather than the resolver’s. It recovers most of the performance that remote resolution gives away, and it needs neither the CDN nor the resolver to agree to anything.

Running at the client is the point. It needs no cooperation from the CDN, no change at the resolver, and no support from the ISP, which is what made it deployable by an individual user rather than by agreement among providers.

Software

namehelp — a proxying DNS server that keeps the benefits of third-party DNS while avoiding the performance penalty for CDN-hosted content. Ten thousand users within a month of release.

Papers

Content Delivery and the Natural Evolution of DNS — IMC 2012
namehelp: Intelligent, Client-Side DNS Resolution — SIGCOMM 2012 (demo)
Behind the Curtain – Cellular DNS and Content Replica Selection — IMC 2014. Carries the same question into cellular networks, and also belongs to the mobile thread.

Coverage

namehelp was covered by Network World in 2012, and picked up by outlets in France, Turkey, Brazil, Vietnam, Poland, and Russia.

People

Lead & PI

Fabián E. Bustamante (Northwestern University)

Students

Ongoing Work

The questions namehelp raised — where resolution should happen, and what it costs the user when it happens somewhere else — are still open, and two efforts currently under submission continue them: Onoma, and a study of hidden DNS resolvers. Both have been presented as posters.

Decentralization, Privacy and Performance for DNS — SIGCOMM 2021
The Aleph — decoding geographic information from DNS records
AquaLab Research Group

SideStep and DraFTP

Sat, 18 Aug 2007 00:00:00 +0000

In Brief

Many studies have shown that the default Internet path can be suboptimal, leading to poor throughput between hosts. SideStep was the first scalable and efficient system to provide free, open access to alternative paths — sometimes doubling or tripling performance. It reused information from content delivery networks to locate quality overlay paths and stream data over them, and was packaged as a Java library and stream API easy to drop into an existing application.

To demonstrate the effect in something people actually used, we released DraFTP, an open-source FTP suite built on SideStep.

Motivation and Approach

The Detour study showed Internet paths are generally suboptimal in both latency and loss. RON — the Reliable Overlay Network project — then showed that diverting a transfer through an intermediate host often beats the direct path. Part of the reason is that ISPs carry each other’s traffic under business arrangements, and those arrangements outrank optimal path selection; a neighbouring ISP may simply have a better route than yours.

RON was never widely deployed because it required active probing with overhead proportional to the square of the number of nodes — unacceptable for Internet-scale applications with millions of users. The opposite approach, picking detour points at random, has negligible overhead and rarely helps.

SideStep sat between those extremes: it found high-quality detour points with high probability at a constant amount of measurement overhead per node. It did so by treating CDN redirection dynamics as hints about good candidates. Drafting Behind Akamai had already shown those redirections are driven primarily by network conditions; SideStep showed they are cheap to obtain, cheap to validate, and accurate enough to drive a large-scale detouring service.

In a measurement study across hundreds of computers worldwide, SideStep improved performance more than half the time. The rest of the time there was no better path to find, so performance was unchanged — it could not make throughput worse, because candidate paths were evaluated by splitting the stream across the current and candidate route.

Why the Names

We called it SideStep because it sidesteps the poor default path. The FTP suite is DraFTP because, like SideStep, it exploits drafting: letting CDNs absorb the cost of measurement, much as cyclists in a peloton travel fast by letting the riders in front break the wind.

On Using Someone Else’s Measurements

A recurring question was whether CDNs would try to stop this. They could try, but it seemed unlikely. Akamai already published measurement information publicly. SideStep used ordinary DNS lookups to gather hints and downloaded no content, and those lookups could be as infrequent as once an hour — a negligible load beside the hundreds of millions of people fetching CDN content daily. The requests were also indistinguishable from those of any normal user of that content, making them impractical to filter.

For the avoidance of doubt: this was not AkaRouting, Akamai’s paid detouring service. SideStep was free and used none of its paths.

Availability

SideStep was distributed as a background service with a stream API, alongside DraFTP, to users who registered on the lab site. It is no longer available.

Papers

On the Effectiveness of Measurement Reuse for Performance-Based Detouring — INFOCOM 2009
Drafting Behind Akamai (Travelocity-Based Detouring) — SIGCOMM 2006

People

Lead & PI

Fabián E. Bustamante (Northwestern University)

Students

David R. Choffnes
Ao-Jan Su

Collaborators

Aleksandar Kuzmanovic (Northwestern University)

Reusing CDN Measurements — the project SideStep was built for

Ono

Sun, 01 Jul 2007 00:00:00 +0000

Motivation and Approach

The goal was simple: improve download speeds for your BitTorrent client. In most peer-to-peer applications the choice of which peer to download from is essentially arbitrary. When most peers offer good performance the random choice works fine — but if the peers you draw are on the other side of the world, your downloads suffer.

Ono avoided that by proactively finding peers close to you in a networking sense, which generally means better response time and better transfer rates. It identified those peers by reusing the network measurements content delivery networks already perform, rather than probing paths itself. A CDN constantly decides which replica should serve which client; two clients steered to the same replica are, in all likelihood, near each other. Ono read that signal instead of generating its own — needing only infrequent DNS lookups, never active path measurement.

Results

Ono’s coverage map: users in nearly every country, shaded by concentration.

From results collected across participating users — well over 100,000 of them — Ono located peers along paths with two orders of magnitude lower latency and 30% lower loss rates than those BitTorrent picked at random.

Those higher-quality paths translated into throughput:

In constrained settings, where peers are limited by available bandwidth, Ono delivered a 31% average improvement in download rate.
Where bandwidth was plentiful, it increased download rates by 207% on average, and improved median rates by 883%.

The result mattered because it contradicted the prevailing assumption of the period. Reducing cross-ISP traffic was understood to be something ISPs wanted and users paid for. Ono showed the two interests could be served at once.

Why the Name

Ono is the Hawaiian word for the King Mackerel, and also means “good to eat” — we hoped users would find the plugin as tasty as the fish, which is likewise known as the wahoo, a sound we hoped they would make on seeing their download speeds. The name also ties the lab’s aquatic theme to Akamai, the CDN whose measurements Ono relied on, and whose name is Hawaiian for “clever”.

Why Not the Alternatives

Vuze already shipped a network coordinate system. As Ledlie et al. had shown, those coordinates were badly inaccurate, and our own independent measurements agreed: only 10% of them carried less than 10% error, while more than 60% were off by 100% or more.

Coarse heuristics had a different problem. Class C subnets and AS numbers scale well, but say little: there may be nobody downloading your file on your subnet, and your AS may span an entire country. Neither reflects current network conditions. Reusing a CDN’s live view avoided both failure modes.

Vuze itself was chosen because it was the most used client of its day, was Java-based so it ran anywhere, and supported plugins — so Ono required no changes to a user’s existing installation and could update itself. Ono carried a 100% Java DNS implementation, and ran wherever Vuze did: tested extensively on Windows and Linux, more lightly on macOS.

Beyond the plugin, we implemented a tracker that used Ono to return nearby peers, and a reduced version of the Ono service in Java for integration elsewhere.

Availability

Ono shipped through the official Vuze plugin channel and reached version 1.8.8 by late 2008, with translations contributed by users in several languages. It was released under the GPL, with source included in the distributed jar.

It is no longer available. Both download hosts of that era (aqua-lab.org and merlot.cs.northwestern.edu) stopped resolving long ago, and the Vuze plugin ecosystem it depended on is gone.

Historical release notes and support issues (2007–2008)

Kept as a record of the deployment.

17 Oct 2008 — GUI updated in Ono 1.8.8, with an updated German translation.
16 Oct 2008 — Vuze 4.0 broke the plugin’s GUI. Ono continued to bias peer selection correctly; only the display of its information was affected.
4 Aug 2008 — Added a map of Ono coverage.
12 May 2008 — Added a link to the paper demonstrating Ono’s effectiveness.
1 May 2008 — Site updated with deployment performance results and developer resources.
9 Jan 2008 — Backend revamped for efficiency; GUI work underway; several translations added.
18 Aug 2007 — Latest versions published through the Vuze plugin list; call for translators.
12 Jul 2007 — Version 1.5.3 released with bug fixes.

Two support issues came up often enough to document:

Vuze took longer to shut down after installing Ono. Expected — Ono was reporting performance statistics for analysis on close. The wait was configurable, and statistics collection could be disabled entirely.
Screen flicker and raised CPU on Windows Vista. A rare interaction, resolved by reducing the number of simultaneous pings Ono performed.

Papers

Taming the Torrent: A Practical Approach to Reducing Cross-ISP Traffic in Peer-to-Peer Systems — SIGCOMM 2008
Drafting Behind Akamai (Travelocity-Based Detouring) — SIGCOMM 2006
Taming the Torrent — USENIX ;login: 2010

Coverage

Ono was covered by Slashdot, El País, and the Chronicle of Higher Education in 2008.

People

Lead & PI

Fabián E. Bustamante (Northwestern University)

Students

David R. Choffnes

Contributors

Xian Yi Teng
Nikola Borisov
Shane Ponting

Reusing CDN Measurements — the project Ono was built for

Reusing CDN Measurements

Fri, 01 Sep 2006 00:00:00 +0000

Project Overview

Large content delivery networks spend enormous effort measuring the Internet. To decide which replica should serve a given client, a CDN continuously evaluates paths, load, and latency, and encodes the result in the redirection it hands back. Those redirections are public. Anybody who asks a CDN where to fetch an object gets a small piece of that network view for free.

This project started from the observation that the redirections are a signal in their own right, not just a mechanism for fetching content. If two clients are sent to the same replica at the same time, that says something about where they sit in the network relative to one another. If a client’s redirection changes, something about the path has changed. A CDN’s infrastructure, in other words, doubles as a measurement service with a global vantage point, updated continuously, at no cost to the person reading it — provided the signal can be interpreted correctly.

The work that followed ran that idea through three problems: finding better paths than the ones routing gives you, positioning hosts relative to each other, and choosing peers in a peer-to-peer system.

Detouring

Drafting Behind Akamai showed that CDN redirections can be used to find one-hop detours that beat the default path, without running any measurements of your own. A client watches which replicas a CDN steers it toward, infers path conditions from the pattern, and routes through a peer that the CDN’s own view suggests is well positioned.

SideStep took the same insight further, examining how effective reused measurements are as a substitute for dedicated probing, and packaging the result as a service. We also released DraFTP, an FTP suite built on SideStep, to show the improvement in an application people already used.

Positioning

Relative Network Positioning via CDN Redirections turned the signal toward a different question. Rather than asking which detour is fastest, it asked where hosts sit with respect to one another — using agreement between two clients' redirections as evidence of proximity. This gives a positioning system that needs no landmark infrastructure and no active probing, because the CDN has already done that work.

Peer Selection: Ono

Ono applied the idea where it mattered most commercially. BitTorrent clients pick peers largely at random, which generates enormous volumes of traffic that cross ISP boundaries — expensive for providers, and slower than necessary for users. The prevailing assumption was that these two interests were opposed, and that reducing cross-ISP traffic meant accepting worse performance.

Ono showed otherwise. By using CDN redirections to identify peers that were close in the network, it biased peer selection toward nearby peers, and both numbers improved: less traffic crossed ISP boundaries, and downloads got faster. This was the argument in Taming the Torrent, and it landed in the middle of an active dispute about ISPs throttling peer-to-peer traffic.

Ono shipped as a plugin for the Vuze/Azureus BitTorrent client, which put it in the hands of ordinary users rather than a testbed.

Software

Ono — a Vuze/Azureus plugin implementing CDN-based peer selection. Over 100,000 users; cut latency on selected paths by two orders of magnitude and raised download rates by up to 207%.
SideStep and DraFTP — detouring by reusing CDN measurements, plus an open-source FTP suite built on it. Improved performance more than half the time, and never made it worse.

Papers

Drafting Behind Akamai (Travelocity-Based Detouring) — SIGCOMM 2006
Relative Network Positioning via CDN Redirections — ICDCS 2008
Taming the Torrent: A Practical Approach to Reducing Cross-ISP Traffic in Peer-to-Peer Systems — SIGCOMM 2008
On the Effectiveness of Measurement Reuse for Performance-Based Detouring — INFOCOM 2009
Taming the Torrent — USENIX ;login: 2010

Coverage

Ono was covered by Slashdot, El País, and the Chronicle of Higher Education in 2008.

People

This was a joint project with Aleksandar Kuzmanovic’s group at Northwestern.

Faculty

Fabián E. Bustamante (Northwestern University)
Aleksandar Kuzmanovic (Northwestern University)

Students

David R. Choffnes
Ao-Jan Su (advised by Aleksandar Kuzmanovic)

AquaLab Research Group

Content Delivery | Aqualab - Northwestern University

namehelp

Overview

The Problem

What namehelp Did

Reception

Privacy

Availability

Papers

Coverage

People

Related Links

Client-Side DNS and Content Delivery

Project Overview

Measuring the Interaction

namehelp

Software

Papers

Coverage

People

Ongoing Work

Related Links

SideStep and DraFTP

In Brief

Motivation and Approach

Why the Names

On Using Someone Else’s Measurements

Availability

Papers

People

Related Links

Ono

Motivation and Approach

Results

Why the Name

Why Not the Alternatives

Availability

Papers

Coverage

People

Related Links

Reusing CDN Measurements

Project Overview

Detouring

Positioning

Peer Selection: Ono

Software

Papers

Coverage

People

Related Links