Excessive TCP DUP ACK

[LynK]

Hey guys,


I was wondering if I could get some assistance regarding some output on a wireshark between our Web server VIP and the clients hitting our public site. I am seeing excessive TCP DUP ACKs, and I have been doing some research. Some say it is a common speed/duplex issue... but outside of that, I have not seen any real relevant information.

The infrastructure looks like this:

Remote customer -> internet router -> ASA 5520 -> DMZ -> F5 LB (one arm) -> chosen web server in the pool

[Mario]

Hi Lynk,
In my experience that happens when there are losses between the clients and the server. Do a long extended ping and check results (if server allows pings). Try differente packet sizes.

ping 8.8.8.8 -t -l (packet size in Bytes)

Yes, it may be because of duplex mismatch but its not the only cause. Among some may be:
- Bad cabling
- link saturation
- bad wireless signal/interference

You should do tests on various segments on the path from clients to servers (hopefully you have access to all)

A good software to test the path is WinMTR (www.winmtr.net) but as I said before devices in the path must allow pings through and replies (the ASA would probably be a problem)

--
Mario

[Seittit]

I'm very intrigued with the fact that every duplicate ACK is coming from the same TCP sequence (1).

Within which part of the network was the capture taken, on the server or before the load balancer?

[SimonV]

I had something similar once, turned out to be a hardware problem upstream on the Telco's network

[killabee]

I've mostly heard of it being caused by packet loss.  The question is...where is the packet loss happening? The speed/duplex would definitely attribute to that, but so could a ton of other things considering traffic is coming from a remote customer over the Internet.

You'd need to do an end-to-end capture to get better visibility on what's happneing.  Luckily the Analyze and Statistics options in Wireshark are good in pointing out traffic flaws systematically so you don't have to stare-and-compare.

[Seittit]

Looking at the delta timers between TCP acks tells me that it's not likely a packet loss issue here. The client is aggressively ACK'ing TCP sequence 1 towards the server, so aggressive that it's not likely the client at all. I would start my investigation with the load balancer, see if you can run a capture on both sides of the appliance to see what traffic looks like on both sides of the VIP. You may be running into a bug on the load balancer appliance, I've seen some real interesting results from bugs on our A10 appliances so I can't dismiss it at this point.

[Mario]

Well seen Seittit, I didn't notice there was an attachment.

But if the capture was done as said between the clients and the VIP I wouldn't go for a problem in the appliance since this is the an ACK from the client to the server. Unless packets from server to client were filtered out.

[LynK]

Sorry,

I was just getting back from vacation. The capture was taken on the outside interface (going out) of our ASA. The source address is on the left, and the pulic ip ending in .21 is our public IP, which then goes to our load-balancers vip. We are utilizing F5 viprion load-balancers.... if you are using the same load-balancers.