Crisis or Known Flaw?
A Cotse HelpDesk Editorial
By John Holstein, Cotse Help Desk Coordinator
With Contributing Author: Steve Whitacre
Due to a recent flame email I received, I think it prudent to explain a little bit about DNS servers before you read the actual article.
When a request is made, via web browser or command line "PING" to contact a web server via domain name, the request goes to a DNS server whereby it resolves to a known IP address. This address is returned to the browser/command and allows it to complete the task by sending to the appropriate IP. This being the case, if the DNS servers send incorrect data (DNS Poisoning) to the browser or command, the browser doesn't know the difference and it attempts to contact the number given.
I would like to take this opportunity to redirect you to a very enlightening site describing how DNS Poisoning works and the effects thereof: Sans.Org
Let me start out by saying, by no means am I am a "Network Engineer". However, I am well versed in the mechanics of most network architectures. I don't spend my days in a fantasy-world dreaming of building the new "12 Lane Information Super Highway." I spend my days looking at problems and solving them, real world style.
Everyone has a theory about this or that, submerging oneself in their work only leads the intelligent worker to develop more efficient ideas. Over the past few weeks, I have noticed an ever increasing change in relation to D omain N ame S erver (DNS) lookups regarding web address translation into I nternet P rotocol (IP) addresses.
I have noticed on several occasions, DNS servers were resolving IP addresses to erroneous hosts. For instance; I would enter in www.mydomain.com into a web browser, whereby the DNS server would select www.notmydomain.com. Was this a case of "Hi-Jacking" traffic or was it simply an error within the translation? I seriously doubt these intermittent problems were caused by "Hi-Jacking" but there lies enough information to speculate that someone may have been "practicing" for a major hit against DNS servers.
I changed DNS servers for my surfing needs on numerous occasions and found that several of these databases would present corrupted data approximately 1 out of every 1000 or so attempts. Granted, numerically speaking, that isn't a very high percentage (0.1%). However, in the "point and click" world of today, 1000 "clicks" really isn't a lot.
Take into consideration when you visit www.cnn.com or any other top-name news or web search site. When you visit theses sites, they have contained within the HTML presented to you, various web advertisement firms, partner/associate agencies and rendered graphics from other sites. When you go to www.cnn.com, the simple "point and click" turns into a plethora of web activity from other sites. Increasing the first "point and click" from just going to CNN to retrieving various documents from other domains such as IDG.Net and various other sites.
This in itself, decreases the number of clicks it takes to get to the 1000 click mark. If in fact, there is only one additional out-of-domain rendered HTML document or graphic contained on the original "point and click" site, this doubles the percentage of failed requests during DNS fault times.
So you ask, what's my point? The point is, if someone is "practicing" "DNS Hi-Jacking" and their results from the practice run is 0.2% for a very brief period of time, what can we expect if they "Hi-Jack" services for a two hour period? Maybe an unreachable host such as www.microsoft.com & www.yahoo.com were in the past couple of weeks?
So you might say that this is theoretical and impossible. Maybe. But again, I seriously doubt "impossible" would be the word to describe much of anything these days. The ability to "Hi-Jack" traffic is nothing new. We have known about this for years yet the media "experts" have decided to bring this into the lime-light and generate some traffic on their web-sites.
Recently, in my article "Microsoft Having Problems?" I discussed the issue of Microsoft placing all of the 4 known DNS servers for www.microsoft.com on one subnet. Granted, this is a very amateurish way of structuring their DNS system yet Microsoft did not admit to this being a problem even though we now find this week that MS has changed their DNS Architecture to a more standardized, multi-subnet structure. Even though this change has taken place, you will see by visiting that link that the first 4 DNS Servers are still contained on the 207.46.138.* subnet:
Domain servers in listed order:
Contributing Author Steve Whitacre;
Concerning a recent DNS problem with Earthlink:
".....Pinging the servers timed out 90% of the time, with an occasional "destination net unreachable" thrown in there for good measure. When doing a tracert from 209.178.xxx.xxx to their primary DNS server at 207.217.xxx.xxx - I was shuttled into the 207.217.AAA subnet where my packets were passed back and forth between .94 .62 and .30 on that subnet. Trying to tracert to their secondary DNS server at 207.217.bbb.bb also plopped me into the 207.217.cc subnet where I was unable to escape the loop...."
Could the MS and Earthlink DNS problems be related?
"...Network problems will always occur as any networking geek will tell you, but the fact that two of the largest internet entities are having similar problems within days of each other leads me to wonder if perhaps it isn't a coincidence. Could we be looking at the next wave of DDoS attacks? Similar to what happened in 2000 only this time instead of directing the attacks at the webservers, the attackers are targeting the routers in front of the DNS servers, resulting in massive DOS outages....."
Earthlink Tech Support / System Admin's later reported to Steve that the errors in DNS were caused by "...a server issue as opposed to a router issue." The Tech wouldn't go into any additional details. Regardless, it was a *problem* with DNS. We don't expect them to tell the truth. We have come to live with that.
Continuing with Microsoft
Let's assume that a "Hi-Jacking" of Microsoft DNS did not take place. We assume this by the inability to reach any website during the down time. This is the first sign of a "Hi-Jacking", packets being redirected toward another unknown site. We also know this by deductive reasoning because the information contained in the DNS servers or the lack-thereof, did not point to another website, the "normal" hi-jacking method. The normal method would be to point requests for a popular website, such as www.yahoo.com toward a site containing "Anarchy", "Cracking" or "Political Agenda" web sites. Taking the average user, that wouldn't normally visit such sites, on an adventurous journey into the dark corners of the net.
Furthering our assumption, let's speculate what did happen. Could there have been a break-in to a network appliance in-line with the DNS servers whereby the "Net-Burglar" vandalized internal settings. This in itself, without proper log files would point the blame at a "Lowly Tech".
To continue our speculation, could the down-time have been a result of a D istributed D enial of S ervice (DDoS) attack? I am sure that it *could-have-been*, but there's no reason to believe that MS wouldn't have divulged such information as it points the blame elsewhere. Unlike the "Net-Burglar" theory, whereas the appliance was also broken into by a third party, some of the blame could still be placed on Microsoft if proper logging and security measures were not in place.
It's all a matter of perspective. If enough information is presented, a large majority of the public will believe it. For the ones of us that don't, where we provide other theories and suggestions that are lateral in thought and destructive to a large corporation, we are labeled as "kooks" and washed away like the sands of time.
You must remember that Big-Business is just that. They are in it for the money and not for being a malevolent entity to succor your wounds. When it comes down to it, it's the bottom line that matters to them. They will protect their assets at all costs. In the case of corporations such as Microsoft that have more assets than God, you can bet your paycheck, they will be covering their ASSESts whenever possible.
Additionally: At the time of publication, I messaged this URL to a friend in IRC without telling him what the subject matter was. He immediately replies "...can't get to it, my DNS is FUBAR'd." Nice.
Problems? Questions? Bugs? Email me.
Return to the Help Desk