Mobile Network Performance and Content Delivery - Part 1

Mar 20, 2014: Web performance is determined by more than just SpeedTest's reported “Network Performance”. The video you watch, the DNS service you choose, and the server it's delivered from have a much larger impact on your perceived performance.

Led by John Rula

This is Part 1 of a blog series which looks at the current state of network performance and content distribution on cellular networks. This work attempts to challenge conventional wisdom on the state of mobile content distribution, specifically that little can be done to improve outside of the cellular network. Part 1 looks the performance on mobile devices of Domain Name Services (DNS), an intergral often overlooked service required for the internet to function.

The Domain Name Service

The Domain Name Service is a globally distributed system that translates a human readable hostname (i.e. facebook.com) to its routable IP addres (e.g. 173.252.110.27). Nearly all requests form a device must first resolve a domain name in order to download content, and the importance of domain name services is only increasing. According to a report by Arbor Networks last year, the number of DNS requests sent across mobile operators has quadrupled between 2010 and 2012. Aside from an increase in mobile network use, mobile websites have begun to incorporate an increasing number of unique domain names in their sites for items such as ad networks, analytics providers and social networking plugins.

Public DNS services like GoogleDNS or OpenDNS provide an alternative to ones Local DNS service provided by their internet service provider. These services popularity has grown recently due to claims of higher reliability, better performance, and a more secure infrastructure. A low performing DNS service can kill your performance and any DNS failures can render your device practically useless. Current cellular operator policy prohibits users from selecting their own DNS service without first rooting their device.

Performance of Public DNS on Cellular Networks

We investigated the performance of public DNS services GoogleDNS and OpenDNS against the cellular operators provided Local DNS. Figure 1 shows the median, 75th and 90th percentile times taken to perform a DNS resolution for each of the public DNS services with the provided Local resolver. We see the Local DNS service performing better around 80% of the time, with a long tail which peforms much worse than the public DNS services (almost twice the time at the 90th percentile). These results seem to follow those done on the wired internet, where the public DNS services are typically located farther away from clients, and therefore have a longer latency between them.

Resolution Time (ms)
(smaller is better)
Percentile
Figure 1. Time taken for DNS resolution across the three services. Median, 75th and 90th percentile times shown.

Another factor affecting network performance including DNS performance is the radio that your phone is currently using. Most phones have several different radios which it can use at any time (e.g. LTE, EVDO (3G), or CDMA 1x (2G)). Each of these radios have different performance capabilities and cause different levels of network performance. Figure 2 below shows the DNS resolution time to each users Local resolver for each radio technology used.

Resolution Time (ms)
(smaller is better)
Radio Technology
Figure 2. Average time taken for DNS resolution under each radio type.

While 4G technologies like LTE clearly display the best performance, we are not always connected with 4G radios. We looked at the percentage of time users are connected to each radio type. Figure 3 shows the proportion of time CDMA users are connected to each radio type: LTE, EVDO Rev A, EHRPD and CDMA 1x. While 75% of the time is spent in LTE for our users, we suprisingly see that nearly 10% of time is spent connected to CDMA 1x, which has an average DNS resolution time of 1.25 seconds! It seems that regardless the number of signal bars you have, the type of signal is even more important for your network performance.

Figure 3. Percentage of time our users spent in each radio state.

We have released a tool called Namehelp Mobile which measures the performance of your ISP provided DNS service against several public DNS options. In addition, it also measures the role that your DNS service has with Content Delivery Networks (CDNs) in mobile networks, and how your ISP provided DNS service can actually deliver worse performance in many cases. Read more about this in the second part of this series.

Comparing broadband services on users' terms

Feb 12, 2013: Paying for higher download speeds may not improve your web performance; it also depends on the pages you visit.

Led by Zachary Bischof

Recent studies on broadband services (such as those published by SamKnows in collaboration with the FCC in the US and Ofcom in the UK) focus primarily on comparing the performance of ISPs in traditional networking terms, such as throughput, latency and packet loss.

While increased throughput rates or lower latency should improve user perceived performance, it is unclear how strongly these two are related. For example, for some applications, such as web browsing, subscribing to a faster Internet service will not always result in a better user experience. A range of other factors, such as latency over the last-mile and to the DNS server, as well as the agreements the ISP has with content providers, will also have a significant impact on performance.

In this post, we compare users' download throughput rates with the time it took to load a web page. For this experiment, we include results from bing.com, craigslist.org, and paypal.com, three sites in the top 20 most popular sites (according to Alexa.com). In the figures below, we look at how page loading time is affected by download throughput rate for two ISPs, Verizon and Comcast, across these websites.

First, we look at Bing, a search engine optimized to decrease loading time. Here we see that for users in Verizon's network, page-loading times are fairly consistent for all users (all are less than 500 ms). In other words, increasing download throughput rate does not result in significantly faster page-loading times. Since this website has been optimized to reduce response time, we expect latency to be a bottleneck in improving performance, not download throughput. For users in Comcast's network, on the other hand, page-loading times are higher for many users and show a much wider range, with 33% of users seeing page load times over 500 ms. Again, increasing throughput does not result in faster page-loading times.

www.bing.com
Verizon
Comcast
Download throughput rate (Mbps)
Loading time (ms)
Download throughput rate (Mbps)
Loading time (ms)

Next, we look at Craigslist, a classified ad site as an example of a minimalistic design (few objects to retrieve). Here we see that for Verizon, increasing the download throughput rate appears to result in a decrease in page loading time, but only to a point. Users with a download throughput rate above approximately 5 Mbps see similar page loading times. When looking at Comcast, we again see that increases in download throughput do not necessarily result in faster page-loading times and see a wider variation in page-loading time (as compared to Verizon).


www.craigslist.org
Verizon
Comcast
Download throughput rate (Mbps)
Loading time (ms)
Download throughput rate (Mbps)
Loading time (ms)

Last, we look at page-loading times for PayPal. Compared to the Craigslist example, Paypal has a larger number of objects to be retrieved and therefore makes significantly more requests (about 34 for PayPal and 8 for Craigslist). For users in Verizon's network, we again see that increasing download throughput leads to faster page-loading times. However, similar to Craigslist, beyond a point, further increases in throughput rates do not result in faster page-load times. For Comcast, we again see a much wider range in page-loading times. This time, the range in page loading times is about twice as high as the Craigslist example. This is likely due to the fact that for most Comcast users, download throughput is not a bottleneck in improving page-loading time. Instead, other factors, such as latency and DNS performance may be negatively impacting performance.

www.paypal.com
Verizon
Comcast
Download throughput rate (Mbps)
Loading time (ms)
Download throughput rate (Mbps)
Loading time (ms)

In future posts we'll be further investigating how different metrics (throughput, packet loss, latency, and DNS performance) affect the performance of network applications such as web browsing, video streaming, and P2P, as well as comparing the performance of services offered by ISPs. Be sure to check back for more posts.

Web Performance as Collateral Damage from Remote DNS

Oct 24, 2012: Using remote DNS such as Google DNS can result in poor web performance. Get the best of both worlds—the benefits of public DNS and the best web performance by installing namehelp today!

In a study to be presented at the Internet Measurement Conference in Boston this November, we show how remote DNS services (such as Google DNS or OpenDNS) can actually hurt your overall web performance by inadvertently directing you to download content from far-away servers. Internet Service Providers (ISPs) have traditionally provided DNS service via servers located in the ISP's own network—near to the users. Recently we have seen a growing trend of users switching to remote DNS services due to their speed, reliability and security features.

Using remote DNS can actually reduce your web performance due to a poor interaction with Content Delivery Networks (CDNs), which are commonly used by websites to (ironically) improve their performance. This is because CDNs try to direct users to a nearby server for downloading content based on the location of your DNS server. If your DNS server is far away from you (as is the case with remote DNS), you may not be directed to the nearest server, resulting in slower downloads. Our study finds that using remote DNS can double the time it takes to fetch content for 40% percent of users!

Relative end-to-end latency % difference
(smaller is better)
CDF of measurement locations
Relative benefit of namehelp compared to Google DNS.

We have released a tool called namehelp that you can install today to address this issue and improve your web performance. It achieves this by directly contacting CDNs to determine a server near to you. The figure above shows that, for locations where using Google DNS adversely affects web performance, namehelp significantly improves performance compared to simply using Google DNS. For the average user, this translates to a 38% performance improvement!

Essentially, namehelp gives you the best of both worlds—it lets you use remote DNS services while maintaining the best possible web performance. Download and install it today!