The objective was not however just to see if there was a problem or not, rather to understand if there were any trends. One of the key questions we wanted to answer was  whether having third-party DNS providers provided any significant advantage. From our data, when looking at the average DNS availability over a period of time, the improvement is not as large as we thought it might be. However, we saw that when the problems did occur, the availability for sites that hosted their own DNS dropped down significantly more than those with 3rd party DNS providers. The result is not entirely surprising, since most 3rd party providers are built with lots of geographically dispersed sites adding topological and geographic redundancy, while sites hosting their DNS might have as few as 2 or 3 name servers served through one or two providers. Be as it may, it was good to quantify that effect and we will monitor this over the next three quarters to see if things get better or worse.

Table 1: Summary of DNS availability

The detailed analysis is available as part of VeriSign’s State of DNS Availability report. Our work is also mentioned in a recent press release by VeriSign.

Here’s how we did it. We measured the latency to each resolver from approximately 3000 points around the globe with a minimum of 50 points per country. This means that these results are based on the best-case resolution time, assuming you tried to retrieve a DNS record that was in the public resolver’s cache. The results shown are the average over a 24-hour period.

Using this data, we determined which DNS provider’s resolvers were fastest (on average) as seen by each of the vantage points. We picked a winner per country based on the provider that was most frequently the fastest in that country. The map below is colored accordingly. As you can see, if you are in the US, chances are that your fastest public resolver is an OpenDNS resolver. But if you’re in India, it’s probably Google.

Fastest DNS providers per country, UltraDNS (yellow) applies to Hong Kong.

Global Results:

United States Results:

Most regions have a clear frontrunner — OpenDNS in North America, Central America, and Africa, Google in South America and the Asia-Pacific region, and Dyn in eastern Europe. In western Europe, however, there is a bit more contention for the top spot, with each of the top three providers making a strong showing. At a country level, Level3 DNS resolvers did not make it to the top anywhere while Ultra’s DNS resolvers made it to the top in Hong Kong (though you can’t see it on the map).

Now that our initial curiosity about the performance of different public DNS resolvers has been satisfied, our next task is to understand how the performance of public resolvers compares to the default resolver provided by your local ISP. That’s definitely more involved and will be the subject of an upcoming article.

The Battle of wikileaks.org

We began monitoring the wikileaks.org domain after we learned about two rounds of DDoS attacks that managed to take the site offline on November 28 and November 30.

Two days later, according to a statement on their website, WikiLeaks’ DNS provider EveryDNS.net terminated service for the wikileaks.org domain at “10PM EST, December 2, 2010″, rendering the site unreachable. (The statement has since been removed, but the full text can be found here.) Indeed, we saw the domain become unavailable immediately after 10PM EST (otherwise known as 3AM UTC, December 3rd):

In response, WikiLeaks staged a retreat, giving up wikileaks.org in favor of wikileaks.ch, a domain under the authority of the famously neutral Swiss. Though the reason isn’t clear, it is more likely a practical than political one; at the time, EveryDNS seemed to have overlooked the fact that they were serving DNS records for wikileaks.ch as well. But it didn’t take long for them to notice the oversight — EveryDNS terminated service for wikileaks.ch shortly thereafter. We began monitoring the domain later that day, just in time to catch wikileaks.ch transitioning to a new set of DNS servers:

However, problems remained. As you can see in the chart above, it took three full days for wikileaks.ch to reach full availability globally. It remains globally available as of the time of writing, using no fewer than 14 different name servers. A number of other European WikiLeaks domains are also globally available, including wikileaks.de and wikileaks.fi.

But that’s not the end of the story. We found some cases of bogus IP addresses for wikileaks.ch coming out of China, which was also affecting neighboring Vietnam:

DNS manipulation is one of the techniques commonly used by China to block undesirable domains. This backs up reports that the site is being blocked in China.

Unexpected Payback

EveryDNS is not the only company refusing to do business with WikiLeaks. They were evicted from Amazon’s cloud after only a day. A number of financial institutions, including PayPal, MasterCard and Visa, then refused to process transactions for WikiLeaks.

This perceived injustice drew the attention of the loose band of troublemakers collectively referred to as Anonymous. Yesterday, in support of WikiLeaks, Anonymous staged a massive, retaliatory DDoS attack against Visa, MasterCard, PayPal, Amazon, and others under the moniker Operation Payback.

The attack against MasterCard was perhaps the most successful, and we managed to catch the tail end of it, just as their site was recovering. The map below shows DNS availability of www.mastercard.com in different countries on a scale of 0-100%. The majority of the impact seemed to be in Europe and the Americas where we saw significant drops in DNS availability.

By contrast, Amazon (which we were already monitoring) appears to have been unaffected. Below is the availability we observed for www.amazon.com and www.mastercard.com in the Netherlands, the country where the first arrest was made in connection to the attacks.

The War Is Not Over

As these incidents show, DNS is a critical but vulnerable part of any company’s Internet infrastructure, as well as an effective tool to monitor other types of problems and attacks. Watch this space for more updates as the war escalates.

When we started ThousandEyes in August 2009, we saw the need to understand the various elements outside an organizational network that can affect data delivery.
During our study on website availability, we found DNS to be a major source of availability degradation. Moreover, we observed that a lot of these problems were regional in nature and went undetected.

We immediately saw a clear need for a product that understood the DNS in detail and provided the capability to ensure its correct global operation and hence improve overall uptime of a site. We also wanted to build something that did not just indicate where the problems were but also helped in solving these problems by providing planning capabilities. The end result, inSight is a product that is packed with in-depth network analytics around availability, performance and security of DNS along with what-if failure scenario analysis and site planning.

While, inSight and the focus on DNS is our first product, we have other products currently being conceived and developed that go beyond DNS and are excited about the future.