Weird ASA 9.7 issue (VTI Tunnel)

[Dieselboy]

Can anyone verify this for me?

I have a ASA5515X with configure VTI IPSEC tunnel to a remote 2901 router. I can see through packet captures and drop log messages that about 20% TCP traffic coming in on the VTI tunnel (ie inbound from the remote site) is having its response traffic (ACK) dropped by the ASA on the inside interface due to "deny tcp no connection" as though the TCP session is not being monitored / set up.

As a test, I put in a couple of service policy rules to implement "TCP state bypass" for the traffic between the subnets over this VTI tunnel. This resolved the TCP dropping issue and no more issues with connectivity.

So now I want to reproduce the problem, take enough logs and captures to log this with Cisco. However I'm unable to reproduce the issue now. I've tried clearing the connections but the only deny TCP messages I get in the log buffer are due to me clearing them and devices trying to re-use those connections.

My theory is that this new VTI tunnel feature in ASA code exists a bug where the ASA is not tracking TCP states from traffic coming in on the tunnel.

I'm also undecided whether to re-install my TCP state bypass rules, or leave this with TCP tracking enabled.

I also had to refresh and gain a better understanding on how the service-policy rules are matched:

Feature Matching Within a Service Policy
A packet matches class maps in a policy map for a given interface according to the following rules:

1. A packet can match only one class map in the policy map for each feature type.

2. When the packet matches a class map for a feature type, the ASA does not attempt to match it to any subsequent class maps for that feature type.

3. If the packet matches a subsequent class map for a different feature type, however, then the ASA also applies the actions for the subsequent class map, if supported. See Incompatibility of Certain Feature Actions for more information about unsupported combinations.

[Otanx]

Not sure if this is what you are hitting or not, but I have seen something similar before. Do you know if there are any problems with communications, or is it just log noise? We had an ASA that was doing something similar. It wasn't causing any actual problems, and we think what was happening that it was closing the connection on the firewall after the first FIN/ACK, and not waiting for the second FIN/ACK. So when the second FIN comes into the firewall it give the no connection error. The server eventually just times out the connection anyway so no real problems except some extra resources being used. It wasn't causing any issues we could find so we just ignored it. This would have been on 8.2 or 8.4 code not 9.x. If you are sending logs to a log server look for logs just before the no connection log using the same ephemeral port. If you see a ASA-6-302014 log message for the connection it means the ASA thinks the session is closed (the log will tell you why), but one endpoint tried to continue sending data.

-Otanx

[Dieselboy]

Thanks for the reply, I'll watch out for that one.

My issue is different and it seems as though TCP traffic coming in from the remote site (IPSEC encrypted) and reaching VTI tunnel; is not logging the connection state. So when the destination replies with SYN ACK, the ASA drops the packet on the inside interface with "deny tcp no connection" and this breaks the client communication to the internal web server.
I am only having this issue with the VTI tunnel source traffic.

Traffic sourced from inside or another normal interface with a destination to the remote site (over the VTI tunnel) doesn't have any issues. So it's only breaking some traffic that is sourced from our remote office with a destination of here in Australia across the VPN tunnel and specifically the VTI tunnel.

I've asked Cisco to lab it up, because now I've put the workaround in and then removed the workaround I cannot reproduce the issue. And I'm a bit too worried that if I leave the workaround off, that the issue will come back on its own when I'm not able to fix it quickly.