Cisco Nexus 3k (two) with vPC port channels to fabric interconnect icmp problem

Started by Dieselboy, February 20, 2020, 11:31:52 PM

Previous topic - Next topic

Dieselboy

I had a problem reaching a VMs on a VLAN from the nexus switch. Switch 1 had an IP while switch 2 did not, and switch 2 did not have any svi in this VLAN (18) either. Basic diagram is like this:

Nexus core (po1 is VPC peer link) -> port channel to fabric interconnect -> multiple links to Cisco chassis (VM Host).
When I couldnt ping, I could always ARP from the VM on the nexus'.

Nexus switch 1 has IP 192.168.18.1 and nexus switch 2 did not have any SVI for this VLAN (it was on my to do list if I proceed with this set up).

Symptom:
Sometimes I can ping the VM and sometimes I cant. Furthermore, the VM has 2 x IP addresses in this VLAN and the nexus could ping one of them and not the other. A host in the remote site could ping one IP and not the other. Weird thing was it was not the same IPs as the nexus switch, it was in fact the opposite. So the host could ping what the nexus could not and vice versa. I then found other VMs on this subnet couldnt be pinged.

I found that if I shut the link down from the fabric to the nexus 2 (the one without the IP and no VLAN 18 svi) then the issue went away for the concerned VM but I still had icmp issues to others.

So then I was going to add an SVI and IP address to switch 2 so that I could run some ping tests from there. As soon as I added the IP on the SVI, all the problems went away and I can ping everything.
Layer 2, the packet could be coming in from the fabric to switch 2, and switch 2 would need to switch it over to switch 1 via the peer-link channel to reach the nexus switch 1 IP.  At this point, I'm still confused why adding the IP on switch 2, which is not used as any gateway or anything; resolved the issues. I need to go and refresh on VPC and see if the root cause was related to that. But the tests show this is a layer 2 issue, although ARP was working, ping was not.

Does anything above stand out to anyone here? Maybe I missed it.

Otanx

Was your vlan defined on switch 2? Adding the SVI will add the layer 2 definition, or adding an access switch port will do it as well. However, just having a trunk port allowing the vlan will not create the definition.

-Otanx

Dieselboy

It was defined as I could see mac addresses within it, on both switches. There are no access ports in this vlan. Just the 3x port-channel trunk ports .One is the peer link and the other two are the resilient channels to each fabric switch.