December 23, 2014

IPv6 and DOCSIS 2.0 Performance Degredation

Note: This post is exceedingly technical in nature. For those experiencing similar problems, skip directly to the fix.

The Problem #

After coming home during break from university, one of my first objectives was to replace a dying wireless router purchased in December of 2013 with a better one. Like many routers in the house prior, it began suffering connection speed degradation issues, to the point where the connection was next to unusable for any more than one client streaming video or downloading files at a time. As a result, I immediately replaced the router with a recently purchased AirPort Extreme Base Station.

While speed and bandwidth metric sites such as speedtest and speedofme showed improvement in download speed (6Mbps down versus 30Mbps), eerie performance problems cropped up. On some websites, performance was not better, but was dramatically worse. Loading a Google search result page would give text results instantly, but images coming through Google’s CDN loaded in minutes, rather than milliseconds. I originally placed the blame on the Comcast leased RCA DHG535-2 Internet/VoIP modem, attributing it to some unforeseen hardware failure. In part, this assertion was correct, but it was indeed fixable to a certain extent, without a trip to Comcast.

Claire #

While this problem was happening, I kept my normal browsing habits the same, and noticed something odd while browsing. Many websites I visit use CloudFlare, so I found it pertinent at one time or another to install Claire, CloudFlare’s diagnostic extension that shows information specific to CloudFlare sites. Interestingly, after switching routers, Claire’s IPv6 indicator was suddenly lit up on many websites I visited. Curious, I pinged Google in Windows and noticed an IPv6 address responding, rather than an IPv4 address. This was somewhat unexpected, but not entirely surprising. The new router supports IPv6, and the previous one did not. The surprise was that Comcast’s new Dual-Stack IPv6 layer was enabled, and IPv6 results were loading as expected.

Finding The Cause #

Curious, I decided to try a basic ping to Google, using ping and ping6 respectively. The results were, to say the least, astonishingly revealing.

➜  ~  ping6 -c 10 google.com
PING6(56=40+8+8 bytes) 2601:1:8d00:1631:b982:9724:a4cc:8c4c --> 2607:f8b0:400f:802::2000
16 bytes from 2607:f8b0:400f:802::2000, icmp_seq=0 hlim=55 time=1034.667 ms
16 bytes from 2607:f8b0:400f:802::2000, icmp_seq=1 hlim=55 time=87.667 ms
16 bytes from 2607:f8b0:400f:802::2000, icmp_seq=3 hlim=55 time=1607.411 ms
16 bytes from 2607:f8b0:400f:802::2000, icmp_seq=4 hlim=55 time=1492.180 ms
16 bytes from 2607:f8b0:400f:802::2000, icmp_seq=5 hlim=55 time=2277.740 ms
16 bytes from 2607:f8b0:400f:802::2000, icmp_seq=6 hlim=55 time=1935.792 ms
16 bytes from 2607:f8b0:400f:802::2000, icmp_seq=7 hlim=55 time=1918.191 ms

--- google.com ping6 statistics ---
10 packets transmitted, 7 packets received, 30.0% packet loss
round-trip min/avg/max/std-dev = 87.667/1479.093/2277.740/675.006 ms

➜  ~  ping -c 10 google.com
PING google.com (74.125.30.139): 56 data bytes
64 bytes from 74.125.30.139: icmp_seq=0 ttl=43 time=31.978 ms
64 bytes from 74.125.30.139: icmp_seq=1 ttl=43 time=30.941 ms
64 bytes from 74.125.30.139: icmp_seq=2 ttl=43 time=32.790 ms
64 bytes from 74.125.30.139: icmp_seq=3 ttl=43 time=31.751 ms
64 bytes from 74.125.30.139: icmp_seq=4 ttl=43 time=34.881 ms
64 bytes from 74.125.30.139: icmp_seq=5 ttl=43 time=34.945 ms
64 bytes from 74.125.30.139: icmp_seq=6 ttl=43 time=32.180 ms
64 bytes from 74.125.30.139: icmp_seq=7 ttl=43 time=32.503 ms
64 bytes from 74.125.30.139: icmp_seq=8 ttl=43 time=32.410 ms
64 bytes from 74.125.30.139: icmp_seq=9 ttl=43 time=32.136 ms

--- google.com ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 30.941/32.652/34.945/1.225 ms

Not only was the IPv6 average response time poor, but ping6 encountered 30% packet loss in only 10 ICMP requests. Even more surprisingly, the IPv4 network results appeared unaffected, with no packet loss and acceptable response times.

The Fix #

Based on these two result sets, I decided to turn off the AirPort Extreme Base Station’s support for IPv6. Rather than using native mode and connection sharing, I switched to link-local only. The performance problems vanished, and of course, IPv6 is now disabled. Whether or not the problem is directly correlated to IPv6 and the modem, it is certainly worth investigating f you are encountering similar performance problems.

The Root Cause…? #

Throughout all of this, it becomes difficult to place the blame on one specific weak point. The router, the AEBS, is most likely not to be the root cause, as it is both on the latest firmware (7.7.3) and is fairly new. The broadband modem, the RCA DHG535-2, however, is a DOCSIS 2.0 modem with support for IPv6 only in software. A buggy firmware release may be causing issues with housing the 128-bit addresses for IPv6 (versus 32 bit for IPv4), though one would hope that this was tested prior to being rolled out to the general public. Lastly, the fact that Comcast’s IPv6 rollout is dual-stack puts a greater amount of stress on the network. It could be that the IPv6 infrastructure in my town is not fully ready for primetime, or has yet to be deployed at scale yet.

Regardless, disabling IPv6 did indeed solve my problem, at least in the short term. DOCSIS 3.0 and IPv6 are both the emergent standards, but this at least bought time during the holiday season–the last time anyone wants to be caught dealing with Comcast to activate a new modem.

Kudos