Vendor TAC problems again

[Dieselboy]

I have an problem and raised a case for support since I have not been able to resolve it. I will admit I've not searched for bugs yet, as it's time consuming and not straight forward and I will do it once I've made a cuppa tea .

The issue is that we have a single AP in a remote office, which is flexconnect (HREAP, basically the AP connects to a controller here in Australia where the management is done. But the AP allows AP to local network traffic)

We have a printer which is wifi (there's no infrastructure cabling yet!) and the printer is a wifi client connected to the SSID on this AP as are the users and their laptops etc. Users cannot print. Cannot ping the printer from a laptop on the same wifi, but can from everywhere else. Can manage and ping the printer from the Australia office. So ran some more tests and in short, wifi to wifi communication is not working.

I thought I was clear and concise in the initial case notes but was happy for them to get me to provide additional stuff:

TITLE: Wifi users cannot connect to wifi printer - 2504WLC / 7.6.130.0
As per the title, our Sri Lanka users cannot connect to a wifi printer on their wifi network, from their devices which are also using the wifi as a primary method to access the network.

- Broadcast forwarding is enabled
- peer to peer blocking is disabled

Aside from the above two things,
I'm not sure what else could be blocking this traffic.

In Sri Lanka the AP is a 1602 and is flexconnect mode.
The WLC is located in Australia.

I'm not sure if this is a fault or whether I have missed something with the configuration.

Thanks

So the support team asked me for the show run and gave me 10 questions to answer which I thought some of them were pretty daft to be honest but I can understand that they need to cover bases and possibly have people raise requests who miss things off which are important. Here's my response to the questions.

1)is this a new installation? – new printer only
2)If not, is there any change in the network? no
3)Are other clients getting connection to Wi-Fi? Wifi is working fine for everyone, except we need wifi-wifi which is not working as described
4)Is printer is connected to Wi-Fi and users are not getting its access? Yes but only if the users are wifi
5) Is printer itself is not getting connected to Wi-Fi? Not true.
6)If printer is connected to Wi-Fi  and users are not getting its access, is printer getting the ip address? Printer is working fine on the wifi, except we need wifi-wifi which is not working, as described. Printer has an IP address and I can print to the printer from Australia using my laptop which has a wired connection.
7)Is printer and users connecting to same WLAN? Yes.

So we set up a webex and they start looking through the controller, even though I had sent in the configuration file I didn't mind letting them look of course. Until it started going on for quite some time, holding up my work and we were clearly not getting anywhere with them accessing the controller. Things were being checked which were not really related to the issue at hand.

I had an idea that what if wifi user on SSID-1 tries to ping wifi user on SSID-2 on the same AP and the wifi is on the same broadcast domain. I tested this and this in fact works. (we have one SSID which is on both 802.11abgn radios and another which is purely on 802.11an - since I found some users getting interference on 2.4GHz and they could not manually switch their laptops over to 5GHz. So I created a 2nd SSID and gave it 5GHz capabilities only.

I brought the webex to a close by having a short discussion which went something like, "we can probably accept that this is not a config issue. I try and find a defect". They said okay, they will do some research and get back to me.

They got back to me and the following few emails were exchanged...

From: Bharti
Sent: 11 February 2016 14:43
To: Tony
Subject: Wifi users cannot connect to wifi printer - 2504WLC / 7.6.130.0

Hi tony,

Thanks for your time on call. As discussed over call, I have done some researches and wanted to ask the following:
>Is mac filter enabled or disabled on the ssid to which clients are connecting?

If it is enabled, try disabling mac filter on the ssid and test it.

From: Tony
Sent: Thursday, February 11, 2016 12:24 PM
To: Bharti
Subject: Wifi users cannot connect to wifi printer - 2504WLC / 7.6.130.0

Hi Bharti,
No, mac filtering is not enabled – however if it were enabled I cannot understand why this would cause our problem; unless it is a known defect.

Again, the business impact is still medium/high as all SL users cannot print. There is no wired network there.

Thanks
tony

From: Bharti
Sent: 11 February 2016 15:21
To: Tony
Subject: Wifi users cannot connect to wifi printer - 2504WLC / 7.6.130.0

Hi Tony,

Thanks for the update. We are suspecting that there may be some firewalls or antivirus on the clients which drops the pings. so, if there are any firewalls or antivirus, kindly disable it and then test it.
Please let me know about the results.

From: Tony
Sent: 11 February 2016 15:25
To: Bharti
Subject: Wifi users cannot connect to wifi printer - 2504WLC / 7.6.130.0

This is not the case. The printer does not have a firewall.

Is there someone at [vendor's name removed] that can help me with our issue? Our issue is the following:

Wifi clients on an SSID cannot connect to other wifi clients on the same SSID which are on a flexconnect AP.


Thanks,

I can accept there could be procedures for troubleshooting but this one hurt my head. I'd say I get these types of exchanges every 3 to 6 months and I don't raise that many support cases.

Just thought I'd post for a chuckle

[Dieselboy]

Re-queued the case, just had a 5 minute call with a helpful support person in GMT+2. Already miles ahead

Going through the release notes of the latest 8.0 code, found this:

CSCut14210
FlexConnect arp-cache enabled—AP is not responding on behalf of client
[Symptom:
When clients are in the same VLAN, wireless-to-wireless communication fails.
With flexconnect arp-cache enabled, the AP should be responding to arp requests on behalf of the wireless clients. ARP never completes because the AP is not responding on behalf of the client.

Conditions:
WLAN is Flex local switched. Flexconnect arp-cache is enabled globally on the WLC.

Feature introduced in 8.0, so earlier releases should not be affected.

I am having some trouble comprehending the text. I cannot tell if this is just on / enabled in previous releases or not. Seems like 8.0 code gives you the option of enabling or disabling it. I'm guessing that the arp-cache is enabled in previous releases and for me right now it is not working. It says feature introduced in 8.0 - but I think the "feature" is reference to the ability to be able to turn this on/off.

This post meant to be general chit-chat, but I probably should have posted it in the wireless section :-s

[wintermute000]

moved

And yes, TAC quality (and code quality...) has gone downhill in the last few years, no question.

[deanwebb]

That first guy didn't just have zero clues. He had negative five clues, which means he's going to go down multiple wrong paths in desperate efforts to execute CYA maneuvers.

At least he didn't say that you needed DNS turned on to execute pings. *That* guy works for BT.

[Dieselboy]

That first guy didn't just have zero clues. He had negative five clues, which means he's going to go down multiple wrong paths in desperate efforts to execute CYA maneuvers.

At least he didn't say that you needed DNS turned on to execute pings. *That* guy works for BT.

Sometimes I do question if I'm being hard on these people, but if I am contacting vendor technical support then I expect them to have a certain level of understanding at least. It's like taking your car to a mechanic and they they don't know how to get the wheel off.

A few times I've allowed them to explain something to me which I know is complete BS, because for one I don't like to be rude and interrupt and two do they think they can get away with BS? I'd never dream of going to a customer site, and giving a load of BS to the CEO about a problem they've called me in for. Also sometimes it's entertaining to see how far the hole goes which they dig for themselves. The deeper it gets, the easier it is to fill it up.

I've lost count the number of times I have explained something to them, for them to say "okay let me see if I understand this, you mean that...." And what sometimes follows is so bizarre and off-track that I have a few times asked them how they got to that based on what I just told them.

Oh and regarding BT - I once took on an entire support team at an ISP who re-sold their (BT) DSL service in London including some of BT. Basically, I set up the DSL but RADIUS was assigning the wrong IP address to the router. This meant that there was no internet connection at all. So I reported it to the ISP who done all their "checks" and said yes it's all good to go - of course it was not. Eventually as this went on for 2 weeks with our customer not having a working internet connection and top level managers got involved from all involved including the company I worked for. The CEO of our customer was of course furious, who made calls to the ISP who just blamed me for not knowing how to configure the router. It was fixed in the end when BT sorted it out. But there was no way I was backing down. I'm just thankful my colleague backed me up, my manager at the time (great guy) and the director who was involved didn't make any rash decisions.

Reading what I've just wrote above, I think I'm now *that guy* I used to encounter rarely when I was younger.

[deanwebb]

We should all strive to be that really knowledgeable wizard of a dude.