WideScale IpV6 Issues...
The Market Ticker - Commentary on The Capital Markets
Logging in or registering will improve your experience here
Main Navigation
Display list of topics
Sarah's Resources You Should See
Sarah's Blog Buy Sarah's Pictures
Full-Text Search & Archives
Legal Disclaimer

The content on this site is provided without any warranty, express or implied. All opinions expressed on this site are those of the author and may contain errors or omissions.

NO MATERIAL HERE CONSTITUTES "INVESTMENT ADVICE" NOR IS IT A RECOMMENDATION TO BUY OR SELL ANY FINANCIAL INSTRUMENT, INCLUDING BUT NOT LIMITED TO STOCKS, OPTIONS, BONDS OR FUTURES.

The author may have a position in any company or security mentioned herein. Actions you undertake as a consequence of any analysis, opinion or advertisement on this site are your sole responsibility.

Market charts, when present, used with permission of TD Ameritrade/ThinkOrSwim Inc. Neither TD Ameritrade or ThinkOrSwim have reviewed, approved or disapproved any content herein.

The Market Ticker content may be sent unmodified to lawmakers via print or electronic means or excerpted online for non-commercial purposes provided full attribution is given and the original article source is linked to. Please contact Karl Denninger for reprint permission in other media, to republish full articles, or for any commercial use (which includes any site where advertising is displayed.)

Submissions or tips on matters of economic or political interest may be sent "over the transom" to The Editor at any time. To be considered for publication your submission must include full and correct contact information and be related to an economic or political matter of the day. All submissions become the property of The Market Ticker.

Considering sending spam? Read this first.

2018-12-03 09:43 by Karl Denninger
in Technology , 183 references Ignore this thread
WideScale IpV6 Issues...
[Comments enabled]

Someone -- or more like a few someones -- have screwed the pooch.

IPv6, which is the "new" generation of Internet protocol, is an undeniable good thing.  Among other things it almost-certainly resolves any issues about address exhaustion, since it's a 128 bit space, with 64 bits being "local" and the other 64 bits (by convention, but not necessity) being "global."

This literally collapses the routing table for the Internet to "one entry per internet provider" in terms of address space, which is an undeniable good thing.

However, this presumes it all works as designed. And it's not.

About a month ago there began an intermittent issue where connections over IPv6, but not IPv4, to the same place would often wind up extremely slow or time out entirely.  My first-blush belief was that I had uncovered a bug somewhere in the routing stack of my gateway or local gear, and I spent quite a bit of time chasing that premise.  I got nowhere.

The issue was persistent with both Windows 10 and Unix clients -- and indeed, also with Android phones.  That's three operating systems of varying vintages and patch levels.  Hmmmm.....

Having more or less eliminated that I thought perhaps my ISP at home was responsible -- Cox.

But then, just today, I ran into the exact same connection lockup on ToS's "Trader TV" streaming video while on XFinity in Michigan.  Different provider, different brand cable modem, different brand and model of WiFi gateway.

Uhhhhhh.....

Now I'm starting to think there's something else afoot -- maybe some intentional pollution in the ICMP space, along with inadequate (or no!) filtering in the provider space and inter-provider space to control malicious nonsense.

See, IPv6 requires a whole host of ICMP messages that flow between points in the normal course of operation.  Filter them all out at your gateway and bad things happen --- like terrible performance, or worse, no addressing at all.  But one has to wonder whether the ISP folks have appropriately filtered their networks at the edges to prevent malicious injection of these frames from hackers.

If not you could quite-easily "target" exchange points and routers inside an ISP infrastructure and severely constrict the pipes on an intermittent and damn hard to isolate basis.  

Which, incidentally, matches exactly the behavior I've been seeing.

I can't prove this is what's going on because I have no means to see "inside" a provider's network and the frames in question don't appear to be getting all the way to my end on either end.  But the lockups that it produces, specifically on ToS' "Trader TV", are nasty -- you not only lose the video but if you try to close and re-open the stream you lose the entire application streaming data feed too and are forced to go to the OS, kill the process and restart it.

The latter behavior may be a Windows 10 thing, as when I run into this on my Unix machines it tends to produce an aborted connection eventually, and my software retries that and recovers.  Slowly.

In any event on IPv4 it never happens, but then again IPv4 doesn't use ICMP for the sort of control functionality that IPv6 does.  One therefore has to wonder..... is there a little global game going on here and there that amounts to moderately low-level harassment in the ISP infrastructure -- but which has as its root a lack of appropriate edge-level -- and interchange level -- filtering to prevent it?

Years ago ports 138 and 139 were abused mightily to hack into people's Windows machines, since SMB and Netbios run on them and the original protocol -- which, incidentally, even modern Windows machines will answer to unless turned off -- were notoriously insecure.  Microsoft, for its part, dumped a deuce in the upper tank on this in that turning off V1 will also turn off the "network browse" functionality, which they never reimplemented "cleanly" on V2 and V3 (which are both more-secure.)  Thus many home users and more than a few business ones have it on because it's nice to be able to "see" resources like file storage in a "browser" format.

But in turn nearly all consumer ISPs block those ports from end users because if they're open it can be trivially easy to break into user's computers.

One has to wonder -- is something similar in the IPv6 space going on now, but instead of stealing things the outcome is basically harassment and severe degradation of performance?

Hmmmm....

Go to responses (registration required to post)
 

 
Comments.......
User: Not logged on
Login Register Top Blog Top Blog Topics FAQ
User Info WideScale IpV6 Issues... in forum [Market-Ticker]
Nazbuster
Posts: 14
Incept: 2008-10-20

Danville, CA
Report This As A Bad Post Add To Your Ignored User List
Much of your description is at the edge of my understanding of the tech, but Im impressed at the depth of your knowledge and ability to put it into clear form in your narrative.

Im trying to track another weird network behavior for a friend and perhaps you have some insight:

She accessed the patent database with both her iPhone and Lenovo laptop with no problems while using an XFinity WiFi. While traveling, she uses an ATT Hotspot device (essentially a phone without calling). Through this device she can use her phone successfully to access patents but the laptop is able only to search for a patent but gets blocked from seeing the document.

I sense its some sort of ip blocking or port issue but this problem is beyond my pay grade. Any suggestions on where to look?

----------
Supporter of efforts to get money out of politics.
Tickerguy
Posts: 155156
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
That's likely some sort of port-blocking problem BUT it shouldn't be. Is there a particular software package on the laptop or is this being done via a standard browser?

----------
Winding it down.
Dcsleeper
Posts: 468
Incept: 2012-10-11

Northern VA
Report This As A Bad Post Add To Your Ignored User List
WTH? ICMP is blacked in MANY places, firewalls etc. How can this possibly work?
Tickerguy
Posts: 155156
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
ICMP on IP4 and IP6 are VERY different. ICMP on IP4 is not part of router discovery, nor path characteristic discovery.

Have a look-see here: https://www.iana.org/assignments/icmpv6-....

Some of those your network had BETTER handle internally, and others, if you actually get (or take) addresses you better handle too, including, in the "take" case, TO YOUR UPSTREAM.

The problem is that IF you're injudicious with where you accept these or where you allow them to be emitted someone with nasty intentions can do some very, very ugly things and while YOU think your network looks just fine the people trying to pass traffic over it will find their experience rather different than you think it is.

Whatever this cock-up is it's pretty recent in that up until about a month ago IPv6 was working just like v4. But a recent svn resync against freebsd.org's servers LITERALLY FAILED on my box back home over V6 after running at DIAL UP Speed, but when I shut it off and went at it over v4, it was fine. Hmmmm.... That's what started me poking around; my first (of course) assumption was that my LOCAL stack was fubar'd somehow, but I was able to disprove that. Then I figured "maybe Cox is, again, improperly spelled" but NOW that I've seen it on an Xfinity connection 1,100 miles away to OTHER end points.... uh, no, that ain't likely since they have nothing in common at all!

----------
Winding it down.
Roundabout
Posts: 151
Incept: 2009-10-16

South Side of the Sky
Report This As A Bad Post Add To Your Ignored User List
Karl, I also had issues in the past week similar to what you describe. My router (an Asus RT-N66W)was getting slower and slower, until finally when I rebooted it, it refused to connect to my modem and couldn't get a WAN IP address assigned to it. This modem and router have worked just fine together for several years now, so it wasn't a wiring or configuration issue. I first called Asus, they had no suggestions on what the issue could be, and passed the problem off to my ISP (Spectrum) and claimed the problem was with Spectrum's DHCP servers. I was hesitant to call Spectrum, as a few years back, they changed the speed on my modem to a faster 60 Mbps (I was paying for 30 Mbps) and forgot to charge me for the upgrade. However, I was pretty much forced to call them to try and determine where the issue was. I was only getting about 1 - 2 Mbps downloads at this point. Of course, they first had me plug the LAN wire directly to my PC.

Then they "refreshed" the modem (it was my own modem, that I purchased, an Arris SB6183) and for a moment I thought the problem was solved, as I saw - to my surprise - over 100 Mbps on a speed test. I hung up, and plugged the wire back into my router, and the problem was still there. I even reset the router to factory using the reset button on the back - same problem. I didn't have another router to test out, and it was pouring rain that night, so I decided to call Spectrum back to see if I could figure this out. Big mistake, as the guy that answered began to check into my account and told me I wasn't supposed to be getting 100 Mbps and while I was on the phone, I saw the modem reboot. I logged into the modem page after the call, to see all but one of the 16 channels disabled! My maximum speed slowed down (with a direct modem to PC connection) to 15 - 25 Mbps due to this change.

Meanwhile, while not helping me at all, he also informed me that they had just increase the monthly cost of service by $5.00 to $59.99 for my "30 Mbps" service. Just great, now I'm paying more for even less speed, and my network issue still isn't fixed. I decided, since I couldn't change it now, to go the next day down to the Spectrum store and change my service to the 100 Mbps level, even though it was going to cost me even more by changing it (they now charge $65.99 for this speed). However, they loan you a "free" modem at this price (how generous of them). I picked up the modem, and also bought a new router on my way home, determined to isolate the problem to the modem or the router.
First, I plugged in the modem and activated it online. Plugged it into my old router...same problem. No WAN IP. By the way, the message on the router was "Your ISP's DHCP is not working properly" with no indication on how to fix it. An internet search for this message yielded zero useful information, I finally gave up and hooked up the brand new router, and everything worked finally. Refusing to believe my Asus router was defective, I played with pretty much every setting, trying anything to get it to pick up a WAN IP. I finally enabled IPV6 and set the connection type on the IPV6 page to "native", and what do you know? It worked! Immediately, in fact, as good as it ever did. So what happened that suddenly it required that IPV6 had to be enabled in order to get a WAN IP? I have no idea, but I suspect that something changed at Spectrum, and they aren't talking about it. No one, not one person that I spoke to there had any idea what to suggest, and instead blamed my router saying it was defective.

At least I finally got everything working again, after tearing my hair out for two days, and no help from Spectrum or Asus. Sadly, I'm now paying $11/month more for the same service I was getting before this issue cropped up. I'm not happy about that, but there's not much I can do about it. Maybe you have some insight about what caused this problem, it sounds very much like it's connected to the problems you've had recently. Sorry about the long post.
Tickerguy
Posts: 155156
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
No, it's not.

There are two ways, basically, to get IPv6 addressing - DHCP (called "DHCP6") and the native, built-into IPv6 addressing system which uses prefix and neighbor advertisements.

Which is "better"? It's not a simple answer. Native addressing is arguably better if you can do it, in that it's how IPv6 was designed. However, it requires that the provider (and your gear) know how to do that properly AND that it's properly configured both at your end AND THEIRS, and the important part is that on THEIRS they have to make DAMN SURE that mistakes -- or ****ERY -- do NOT propagate back up into THEIR network.

What I SUSPECT is happening is that there's some ****ery going on, AND the providers are not properly insulated against it in their configurations. It requires some noodling on the part of the provider to make sure that native autoconfig works BUT if someone intentionally or through stupidity starts tossing bad advertisements back up the pipe you DO NOT honor them.

DHCP6 is much "easier" as it works almost-identically to how IPv4 DHCP works -- it's "one way" and so ****ery, whether by accident or intent, is MUCH harder.

BTW "native" (SLAAC) will usually hand you a /64. This is sort of ****ed up if you want to have more than one "division" on your network. Most ISPs that do DHCP6 will honor a request for up to a /56, which makes life much easier -- but not all.

----------
Winding it down.

Roundabout
Posts: 151
Incept: 2009-10-16

South Side of the Sky
Report This As A Bad Post Add To Your Ignored User List
Thanks for that, Karl. Unfortunately, I don't know a huge amount about DHCP or TCP/IP, so I don't really completely understand how this all works. It was purely a "punt" on my part to enable IPv6 on my router. This was after I tried rebooting the modem and the router multiple times - perhaps it still might have picked up a WAN IP if I had continued to try. I don't understand why it would suddenly one day be required to have IPv6 enabled in order to get the IP address, what changed recently? Did Spectrum change something and not tell anyone? Who knows? Certainly no one I spoke to there even hinted at this, and continued to say my router was defective. The standard line was "since everything works fine when you bypass the router and connect straight to your PC, the problem is with your router".

I have no idea what the average consumer would do in a case like this, most likely give up and hire someone to come out and fix the issue. I'm not smart enough to figure out too much when it comes to networking, other than basic configuration. I'm just glad it works now and hopefully will continue without issue for a long time.
Tickerguy
Posts: 155156
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
Probably.

My guess is that for whatever reason they shut down their DHCP6 server. That's sort of a bitch since most ISPs, as noted, will hand out either a /56 or at least a /60, allowing you to subnet in your house. While not a LOT of people want or need that it's quite useful if you have (for example) a guest WiFi password on a VLAN that CANNOT see anything inside (e.g. your nice Plex server for media, for example) but can get out and browse.

The Netgear WiFi router I picked up recently for a remote comes with IPv6 shut off but if you turn it on it has a choice "auto" -- it tries both SLAAC and DHCP, and then saves whichever works and displays it. That's not terrible. Xfinity is using DHCP (at least at this point it is, and around here.)

SLAAC is configuration-requirement free (it's "auto-assignment", basically) but DHCP can be "not", depending on the router. Get it wrong and really bizarre things happen.

IPv6 is supposed to make all of the BeeEss with the old v4 address space just "go away" since it's (allegedly) all auto-configured, including router discovery. And it is -- when it works, and assuming nobody is screwing around maliciously. In the LAN/small network world you might be able to assume that in the local environment but assuming "no ****ery" on the Internet is, well, naive -- at best. The real question is whether the ISPs are putting in appropriate edge filters so their customers, either through unintentional screwups or worse, intentional "hose you" packet emission, can't do exactly that.

----------
Winding it down.
Roundabout
Posts: 151
Incept: 2009-10-16

South Side of the Sky
Report This As A Bad Post Add To Your Ignored User List
Thanks again for your explanation, at least now I sort of understand what may have happened here. It sure would be nice, if they're going to shut down their DHCP6 server, to let customers know when they call in - you almost feel like they do this kind of thing on purpose to drive you crazy. Or maybe, to try and push you to use their equipment ($$$), since they don't/won't support customer owned equipment.

Perhaps my Asus router is too old - about 3 yrs old - and it lacks the features of your newer Netgear (automatic selection of SLAAC/DHCP) and such. It's been a pretty solid performer though, so I didn't want to get rid of it unnecessarily. Now that everything is working properly, I'll keep it a few more years if I can.
Tickerguy
Posts: 155156
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
It may not be able to do autodetect but SLAAC IPv6 config is required by IPv6 stacks, so it certainly can get an address. The issue is that they changed the way they handed them out without telling people -- which is VERY uncool.

----------
Winding it down.
Ckaminski
Posts: 5006
Incept: 2011-04-08

Report This As A Bad Post Add To Your Ignored User List
Who in the world needs 64bits for a single network?

Tickerguy
Posts: 155156
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
Nobody, but it makes easy auto-configuration on the host end, since the network interface has a MAC address which is six octets, or 48 bits. It thus becomes easy for an algorithm to use that to mashup a unique host address given a prefix.

Further, far fewer than 64 bits is required to uniquely identify all the reasonably-possible network providers, which means if you "arbitrarily" call that space 48 bits (for example) in the global routing table then the provider has 16 more bits (65k) for their own internal use. This means whatever internal segregation they need they can have, and they can also hand out prefixes to customers that are shorter than a /64 (most cable companies, for example, will give you somewhere between a /56 and /60 if you configure your DHCP client to ask for it, so you can have a reasonable number of subnets easily fit within the SLAAC autoconfig algorithm on your premises.

The nice thing about having "more than you'll ever need" is that it makes it easy to do things like let even a residential customer have enough space to work within a given wide autoconfig paradigm that uses, for example, the MAC address of the device (which is wildly sparse for a given person's location) yet not "crowd" anything else.

It also makes IPv6 quite hack resistant since determining WHICH addresses are valid and in use, if not listed in DNS somewhere, damn near impossible to figure out by poking all of them. This materially reduces the plausible attack surface for the black hat dudes, and thus is good from that point of view too.

----------
Winding it down.
Bodhi
Posts: 691
Incept: 2008-02-23

Georgia
Online
Report This As A Bad Post Add To Your Ignored User List
I am assuming that IPv6 will be the end of dynamic public IP addresses. That will be much more convenient when running servers. Windstream won't sell me a static address. I can then carry my VoIP phone with me on the road and connect back to my phone system without worrying if the public IP address has changed because the router hiccuped. I can figure it out by pinging my NOIP URL, but a static address will be one less headache.
Tickerguy
Posts: 155156
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
It's not static. You still need DDNS of some sort (because the address you are handed can and will change) but publicly-visible is now trivial, since no NAT is required whatsoever as every device is globally unique.

This would provoke instant horror among the security-conscious except that the address space is so vast that actually attempting to probe it is a waste of time. That doesn't change the requirement to have good security on anything that is actually NAMED (and thus easily visible) but for anything not it means that inbound hacking is basically a non-issue.

----------
Winding it down.
Bodhi
Posts: 691
Incept: 2008-02-23

Georgia
Online
Report This As A Bad Post Add To Your Ignored User List
Thanks for the info.
Tickerguy
Posts: 155156
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
No problem.

I like IPv6 plenty, but it comes with complexity that people seem to be not handling all that well, particularly when it comes to maturation of "don't **** up the control traffic" stuff.

With IPv4 one of the "games" was to spoof RST packets and abort connections, along with other similar ****ery. IPv6 has enough normal operational stuff embedded in control traffic that you can REALLY **** up someone's world (e.g. by trashing routing in the short term at least within a provider network) IF proper precautions are not taken.

BGP4 used to get screwed with on a somewhat-regular basis until everyone started doing things to prevent it. This looks like the same sort of ****ery, frankly.....

----------
Winding it down.
Mtdm
Posts: 507
Incept: 2009-07-23

NH
Report This As A Bad Post Add To Your Ignored User List
Cheap network gear barely gets IP4 right, zero chance of them getting 6 right. Especially when all the wrong ways have yet to be fully enumerated.

And even for enterprise gear ... lots of LECs and other network folks do NOT refresh every three or five years or whatever, they like to keep stuff around for decades... which means that, best case, while you may have IP6 support it's not really current and doesn't behave quite like you expect, to the point where even "experts" have a hard time getting configurations correct. What's that you say? You don't think they patch all the routers out there on the interwebs, do you?
Login Register Top Blog Top Blog Topics FAQ