Thoughts on Comcast and Time Warner Cable

When I wrote a review of Susan Crawford’s Captive Audience two months ago, I didn’t expect to be revisiting it so quickly. Then came the news that Comcast was planning to buy Time Warner Cable, gaining a few million more customers and several regional sports networks. With the acquisition of NBC, Comcast is clearly making a play to be in the content business. There’s not much growth potential left in being a service provider, so it makes sense that Comcast would want to hedge their bets. That’s why I suspect they’re more interested in acquiring regional sports nets (live sports being one of the main reasons people don’t cut the cord) than the few million subscribers they’d pick up if the deal is approved.

It’s not like Comcast and Time Warner were really competing, despite how “competitive” the FCC and Comcast claimed the industry to be a few years ago. The cable companies largely have agreed not to step on each others’ toes. In most places, customers have exactly one choice for cable TV provider. Individual consumers will see no difference in the competitive landscape, so it’s easy to dismiss this as a non-issue (as I initially did). Where this may get interesting is when it comes time for networks to renegotiate carriage agreements. Comcast would have greater leverage to low-ball content providers, potentially squeezing a few out of business. As long as other modes of TV exist (e.g. satellite, AT&T U-Verse), I expect Comcast will remain somewhat constrained in their ability to harm content providers, but they’ll continue to be able to prevent competition from sprouting up.

Of course, it’s not guaranteed that this buyout will occur. Despite the relative ease with which the FCC and the Department of Justice approved Comcast’s purchase of NBC, the landscape has changed somewhat. Denying AT&T’s purchase of T-Mobile was a surprisingly pro-consumer decision, and it’s possible that this deal is doomed as well. I don’t follow Washington closely enough to say what’s likely. All I know is that I can’t wait for Metronet to extend their fiber offering to my neighborhood. I’ve been told it may happen as early as next month.

Book review: Captive Audience

I recently learned Of Susan Crawford’s book Captive Audience when she was a guest on the “This Week in Law” podcast. In Captive Audience, Crawford examines the merger of Comcast and NBCUniversal. Crawford makes no attempt to hide her feelings on the nation’s largest cable provider getting (further) into the content business. The book is more of an advocacy journalism work than a dispassionate academic report. Comcast’s supporters may object to Crawford’s arguments, but her characterizations are refreshingly fair. She is quick to point out that the players are acting, not like evil madmen, but rational business actors pursuing their self-interests. Her main concern is that these interests do not line up with what she believes to be the public’s best interests.

Crawford does not blame Comcast CEO Brian Roberts for this disconnect, though his company has worked tirelessly to keep the status quo. The root of the problem is that the Internet industry is both unregulated and uncompetitive. Crawford rejects the notion that DSL, cellular, and satellite services are competitors to cable companies. DSL is too slow and satellite too high-latency for modern Internet applications and cellular, while convenient, is limited by lower bandwidth and small screen sizes.

The state of regulation for cable providers is like that of the early days of the rail road and electrical industries, which is to say non-existent. Cable providers lack the common carrier requirements imposed on the phone companies. As a result, Comcast and others are free to turn the Internet into a walled garden of curated channels, much like the current state of cable television. As dire of a picture as Crawford paints, it’s hard to see it as a likely threat. Plausible, certainly, but I don’t see it on the horizon.

Nevertheless, America clearly has an Internet problem. Our speeds and prices are worse than most of the developed world. In an age where high speed Internet access is increasingly important to social, academic, and economic activities, one third of Americans don’t subscribe to high speed Internet service. A strong correlation between non-subscribership and low socioeconomic status. If Internet connectivity is necessary for prosperity, expensive Internet prevents upward mobility.

Absent competitive pressure, the public interest can only be enforced by regulation. Interestingly, it was the Nixon administration that first sought to prevent monopolies in the cable industry. In recent years, Republicans and Democrats have proven equally unwilling to impose regulation on the industry. Municipal and private sector fiber installations seem to be the only near-term hope for keeping Comcast in check.

In short, I found Captive Audience to be an informative and compelling read. Crawford takes the reader through the history of monopolies in the United States and of the cable industry. She examines the technical and political reasons that Comcast became and remains a monopoly. In closing, Crawford looks at the effect that the Comcast/NBC merger had on AT&T’s failed attempt to purchase T-Mobile. I highly recommend this book to anyone interested in Internet policy.

Comcast DNS problems

As of this evening, it looks like Comcast is having problems with a few of their DNS servers.  The servers in Chicago and Detroit are offline, rendering the Internet nigh-unusable for many of their customers in parts of the midwest.  I haven’t seen any official announcement, other than the red lights on the DNS status page, but I’ve heard a lot of complaints from people using their mobile phones to connect to the Internet.  If you’re one of the affected customers, you can change your DNS settings to use any combination of the following servers:

  • OpenDNS: 208.67.222.222 and 208.67.220.220
  • Google: 8.8.8.8 and 8.8.4.4
  • Level3: 4.2.2.1 and 4.2.2.2

It’s interesting to me how different this is than it would have been a few years ago.  With the popularity of smart phones, many people have access that doesn’t rely on their ISP.  I’ve already provided support to several people on Twitter who are once again able to waste a Sunday evening surfing the web.  I wonder how much of a credit I’ll get on my bill.

Update, 10:29 PM: Engadget has a brief article up now.  Comcast has been providing updates via Twitter, but there’s currently no ETA for repair.

Update, 6:48 AM: Based on Twitter timestamps, it looks like Comcast engineers got things fixed around 2AM EST.  All of their servers are reporting OK.  I haven’t seen any explanation of what happened, but that will probably come later today.

Comcast’s bot alert service is a good idea terribly implemented

Brian Krebs reported yesterday that Comcast will be implementing its bot detection feature nationwide.  Comcast will apparently put an overlay on websites when visited from an IP that exhibits signs of bot activity.  I don’t claim to be a security expert, but I think I’ve been in the business long enough to say “that’s really stupid.”

While I agree with Comcast’s efforts to fight bot infestations, they are going about it in exactly the wrong way.  Running man-in-the-middle code is unacceptable, regardless of the intent.  If the code is inserted into anything other than HTTP traffic, it will almost certainly break things, and I imagine that certain kinds of HTTP applications will break, too (specifically automated retrieval/parsing of sites).   Additionally, it opens up another attack vector if Comcast itself suffers a breach.

Perhaps the worst part of this plan, though, is the impact it has on user education.  For most users, nuance is not appropriate.  Despite repeated warnings about the illegitimacy of “Your computer is infected!” pop-ups, people still click on them.  Now suddenly there’s the Comcast nag with a link to download anti-malware tools.  Comcast seems to assume that users can handle the nuance.  My own experience suggests otherwise.

Unlike the authors of some of the comments on the post, I’m not concerned that Comcast can determine when a host (well, a customer’s connection, which may have several hosts behind the router) is operating as part of a botnet.  While they could be inspecting the contents of the packets, it’s more likely that they’re just using the routing information and other already-visible data.  There are some hosts and traffic patterns that are generally indicative of bot activity, but not conclusively so.  That’s how the network security group at my employer works, in fact: they determine that a host is displaying suspicious behavior, and notify the local admins to investigate.  Sometimes, it’s a false alarm, which is another cause for concern. If users get the Comcast “you’re a bot!” warning, act on it, and it turns out to be false, will they take it seriously again?

I don’t have an answer for Comcast.  They’re trying to do a great thing by combating botnets (not altruistically, of course, but helping their network helps their customers too, so who’s to complain?), but the current method of informing affected users is a really bad idea.

Google DNS: A rare miss?

I’ve been a big fan of Google’s services for many years. GMail, Google Calendar, Google Talk, Google Voice, and Google Docs are all a regular part of my day.  (Admittedly, I haven’t quite figured out how I’ll use Google Wave, but I’m sure there’s a use for it somewhere.) So when I heard about Google offering a DNS service, I was very interested.  DNS (the Domain Name Service) is a vital part of the Internet.  It is what allows people to visit Funnel Fiasco without having to remember that the IP address is 72.52.153.36. Or to visit www.facebook.com without typing in 69.63.181.11.

A while ago, I switched from using my ISP’s DNS service to OpenDNS.  OpenDNS gives users the option to filter domains by content, which is a somewhat useful tool for parents and businesses.  Unfortunately, OpenDNS, like most ISP DNS services, returns a search page when a domain isn’t found. Sure, that might be handy for web browsing, but other services expect to be told a domain doesn’t exist when it doesn’t exist.  Google said that they’ll return appropriate responses for non-existent domains.

Before I made the switch, I decided to investigate which DNS service gave me the fastest responses.  I tested 8 DNS servers from 4 different services (Google, Comcast, OpenDNS, and Level3) at different times over the past few days.  The final result surprised me.  Google’s service was slower than both Level3 and OpenDNS, and slower than one of the two Comcast servers I tested.  Box plots are below, although it seems some of the calculation is off (for example, a DNS resolve time < 0 ms is not reasonable).

DNS resolve times for google.com

DNS resolve times for google.com

DNS resolve times for funnelfiasco.com

DNS resolve times for funnelfiasco.com

DNS resolve times for facebook.com

DNS resolve times for facebook.com

Average hostname resolve times in milliseconds

Google #1
(8.8.8.8)
Google #2
(8.8.4.4)
Level3 #1
(4.2.2.1)
Level3 #2
(4.2.2.2)
OpenDNS #1
(208.67.222.222)
OpenDNS #2
(208.67.222.220)
Comcast #1
(68.87.72.130)
Comcast #2
(68.87.77.130)
Google.com 51 42 25 25 25 26 24 62
Funnelfiasco.com 41 40 46 26 37 26 41 87
Facebook.com 39 37 26 25 25 29 25 63

So what’s the conclusion?  Well, it looks like the Level3 servers (4.2.2.1 and 4.2.2.2) are the fastest.  Tests by intMain.net support my own conclusions. Google’s DNS service might be faster for some people, but not for everyone.  If Google adds more servers, that might change.  In the meantime, it looks like I have some resolv.conf edits to make.

(P.S. Box plots created thanks to software from Vertex42.com)

Comcast DNS rumors

Rumors are flying around the intertubes about Comcast intercepting DNS traffic and returning replies from their own servers.  The Domain Name Service (DNS) is the Internet service that allows us to use names like “www.google.com” instead of remembering that if you want to do a search, you have to go to 74.125.67.100.  Most people use the DNS servers provided by their Internet Service Provider, but there are a number of reasons you might want to use a third-party service.  Regardless, an ISP intercepting DNS traffic and forcing people to use their own servers is considered shady by many people.

As a Comcast customer who happens to use Open DNS, I naturally took an interest in these rumors.  It took only a few seconds to check to see that I was not, in face, a victim of Comcast’s alleged abuses.  If the discussion on /. and Fark.com is any indication, nobody else is either.  It has been suggested that only Earthlink customers who get service indirectly from Comcast are affected.  The evidence for that is scant, too.  Basically, it seems like a load of crap at the moment.