Guest Wireless Weirdness

Started by deanwebb, December 13, 2016, 08:53:52 PM

Previous topic - Next topic

deanwebb

Clients in location Alpha cannot use guest wireless... well, they can reach sites by IP address, but not URL. Unless they have a cookie that caches the IP of the URL. And some IP addresses work for some clients there, but not others.

At site Beta, we set up an AP to connect to the WLC in Alpha and use the same CAPWAP tunnel to the WLC in the DMZ at location HQ... And clients at location HQ also can't use guest wireless!

Everything in site Beta works just fine.

:zomgwtfbbq:

What is going on? The clients in Alpha and Beta are in the same subnet and have the same inbound/outbound firewall rules and use the same DMZ WLC...
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

SimonV

What sort of authentication do you have on these guest networks? Is this before or after auth?

It's probably the firewall though.

wintermute000


Nerm

Just to clarify that I understand the question correctly. You have two sites and guest users at one site work fine but guest users at another are reaching sites by IP but not resolving DNS?

Otanx

Assuming site A and B have different subnets is the DNS server allowing lookups from the site A space?

-Otanx

deanwebb

Quote from: Nerm on December 14, 2016, 09:12:10 AM
Just to clarify that I understand the question correctly. You have two sites and guest users at one site work fine but guest users at another are reaching sites by IP but not resolving DNS?

More like, whoever is connected to the lucky test AP we have, regardless of what WLC it's associated with, gets full Internet. Connect to some other AP associated with that WLC? Most of your IP addresses will work, although some failed. No DNS works, unless it's a cached entry or a cookie has an IP address in it.

Vendors that somehow manage to set up a VPN through the guest wireless can get full Internet, but it's because their Internet is now going through their vendor's network.

The subnet itself is a massive /19 that's available all over the Americas. It's the *regional* guest wireless solution. It's for HQ, as well as sites A-Z.

I'm very curious how the troubleshooting in Brazil went this morning, but I'm taking a sick day because my eye is starting to twitch bad after too much stress at work. Twitching should be manageable by tomorrow, so I'll see if they have a solution by then.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

icecream-guy

probably simple issue like an incongruous netmask typo in a ACL, e.g.  255.252.255.0 or something like that .
:professorcat:

My Moral Fibers have been cut.

deanwebb

Quote from: ristau5741 on December 14, 2016, 11:30:33 AM
probably simple issue like an incongruous netmask typo in a ACL, e.g.  255.252.255.0 or something like that .

That thought did occur to me... I joked at one point that only odd-numbered IP addresses were being allowed out.

My biggest beef was that nobody collected enough data to determine if it was all, most, or some of the people at a site having issues and nobody followed up to check proactively at other sites to see if they had the same problem or if all was well with them. By the time it got escalated to me late in the day, people were already on their way home, so we were limited in our testing.

And when our testing failed to reproduce the error, we were all like... :glitch:

We didn't even have the guy that did the maintenance on the guest wireless connection over the weekend so we could ask him...

:whatudo:

Because the same maintenance was done in two regional guest wireless configs, and they're both working just fine.

:wha?:

I know, it's a mess. I feel like either EVERYTHING should be broken or nothing should be broken... instead, it seems to be random, and only in the Americas, not EMEA or APAC.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Nerm

So what config/whatever differences are there between the "working" test AP and existing setup AP's?

deanwebb

Quote from: Nerm on December 14, 2016, 03:09:29 PM
So what config/whatever differences are there between the "working" test AP and existing setup AP's?
I don't think there were any... pretty much, here's your IP, welcome to the wireless network. Corporate SSIDs worked just fine through the same APs.

I'll know more about this when I get back to work tomorrow.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

The guys managing the perimeter security put the IPS on the Internet connection into Layer 2 fallback mode and that cleared things up.

Before that, they guys in India set up a test AP and had excellent results.

Now I want to know why it is that the IPS only affected *some* of the traffic and not *all* of it, discriminating, it seems, based upon usage of DNS and AP placement...
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.