ASA SourceFIRE dropping packets

[Dieselboy]

I have a TAC case open where the ASA is dropping legitimate packets. Not done a packet capture yet to see why as I ran out of time. This was the advice from the tac.

However when TAC looked at the logs on the module by using grep there were some logs with source and destination IP beginning 72.x.x.x We have nothing beginning with that IP range and doing a location lookup it is in the USA.
Tac engineer stated it was spoofed packets to which I stated this was not possible, since the destination IP would not have been routed towards my network. Even if the ISP had some bad routes and sending this traffic towards me, then my internet router in front of the ASAs would have routed them back to the ISP and not forwarded them to the ASA. I do have users in the usa in the same location so I mentioned could this be SSL VPN traffic (no comment)
He also said I had asymmetric routing issues which could be causing this problem too (dropped packets) I also advised this is not possible since we only have one way in and out as the network is pretty simple: access switch > core nexus via vPC > Primary ASA > internet router > ISP

I did get the impression that the tac engineer got the hump with me. When I was explaining the above to him by drawing a diagram he kept blowing into the phone microphone which was pretty off-putting. I guess that he didn't like me explaining to him why he was incorrect, but he didn't know the ins and outs of my network so I was trying to explain. He has yet to get back to me so I guess I will re-queue the case on Monday.

[deanwebb]

Spoofed packets:
There's a difference between wondering aloud about spoofed packets and saying, "Look, right there! Evidence of packet spoofing! ZOMGWTFBBQ H4X!" Did this TAC guy guess or point to specific packets in the logfiles?

Asymmetric routing:
Betcha he saw this in a case once upon a time and he's seeing if he can play that card again.

Microphone breathing:
Maybe he has nasal congestion, left his nose plugs in after a swim, or lacks social awareness. Might be a combination of those factors. I once worked with a mic-blower and it drove me up the wall that he didn't use mute when he wasn't talking.

But, yeah, gotta get that capture.

If SF is active, then you could be dealing with false positives. Any count increments on signatures being triggered?

I don't consider the SF module to be part of the firewall proper, so I still feel comfortable with:

[Dieselboy]

So after hearing nothing from the tac engineer I called tac to re-queue the case. I explained I wasn't getting any support in all honesty and I would like to re-queue for a different engineer.
The girl who answered my call stated that re-queuing was no problem but first she had to leave him a voicemail to advise him. I said "no problem".

She came back to me about a minute later and advised she had some good news, the tac engineer was available now. I mentioned that I had already explained to her the issues with the current tac engineer and that I did not want to work with him any further; since I was not happy that he was unable to provide me with support.

She said "oh, he's on the line".


May be after hearing comments about his attitude to his work he will self-reflect and improve...

After a brief discussion I agreed to do a webex.

To make matters worse, I was not able to reproduce the two issues I wanted assistance with:
- SSL VPN client traffic dropping
- traffic to Cisco.com dropping

All that I had done was remove the trust rule for the SSL VPN traffic I put in the access-control-policy (to permit the traffic), and re-apply the policy to the SFR module.
Journey continues.

[deanwebb]

Aw, man... she should have told you up front that she had a warm transfer.

[Dieselboy]

I always try and ensure that any words I say about somebody, that I should not be ashamed / concerned / worried about that person hearing them for this very reason. But at the same time, if I were him I may be reluctant to help me.

Typical response from helpdesk persons - not listening to me

Why transfer me when I said I didn't want to work with him anyway?
The icing on the cake was that I didn't see the issue during the webex. I heard his heavy breathing again on the mic. I did advise him that I could hear him blowing into the mic and it's not that great. He moved the mic away and asked me if it was okay, but a few minutes later it's back again.

I don't think we're going to be facebook friends

[deanwebb]

Well... is the dropping time-sensitive? Can you have it throw a syslog when those packets get dropped? Might be a pattern there.

[Dieselboy]

it does or is at least supposed to log a syslog.

I found yesterday that there is a bug whereby the management centre does not apply everything to the SFR module even though it reports as successful. This could be the reason for my weird experiences, so I'm in the process of updating. Weird as in: not working, then add a access control policy rule to bypass the traffic. Then remove the rule and re-apply so as to break it again to show tac, however it was not broken. Even from different internal source IPs (ie not simply maintaining the connection through the firewall to continue working).

I've also noticed that some things noticeably slow down when activating the SFR module inline (ie not monitor only). Internet speed tests are good but seems like some new connections take a while. Loading the cisco tac support cases can be hit and miss. I do have IPS enabled though so maybe that is slowing things down. access control policy has 1 rule for bypassing some clientless VPN traffic and 3 rules for inspecting HTTP, FTP and SMTP.
I did notice a section in the config guide that rules can slow things down and there's recommendations there to increase performance. I'll read through that again today.

[deanwebb]

IPS will always slow down line speed. It's the nature of the beast.

Best way to improve performance, of course, is to turn off all security and use only 64-bit packets. Packets/sec will SCREAM!

[Dieselboy]

Since updating to the latest release, things seem really good and pages are loading. Before the update, some content within the pages were just not loading. I could not even get to open a new TAC case on the cisco.com website.

Going to propose we bin the firewalls and all security appliances and software, based on your recommendations. So to maximise throughput I'm going to suggest that I create a DHCP pool for our ISP allocated IPv4 block on a layer 3 switch and assign public routable IP addresses to all of our users computers to avoid NAT.  I've taken your recommendations to be god-like since you are a CCNP-Security professional.





- not really

[deanwebb]

As long as you have no internet connection, you will be secure from Internet-based threats.

And it won't be the firewall.