ASA active / standby with OSPF to two separate L3 switches

[AnthonyC]

Did you put in the vPC peer gateway command? (as otherwise it would cause problem with HSRP)

[killabee]

@Dieselboy,
Why do you think the N3K is immune to the issue of peering over a vPC? I ask because AFAIK only the N6K doesn't have that issue (though I can't find the CiscoLive preso that mentions that).

My experience with Nexus is that although some stuff works for the N5K, it may not work for the N7K, and what works on one version or line card doesn't work on the other, so as a principle I always refer to the official config guides for the software version and for the specific Nexus I'm working on for verification on what works and what doesn't, and be very attentive to detail.

What's odd is the config guides for the N3K don't say anything about routing peering in combination with a vPC...but the N5K and N7K guides explicitly state what works and what doesn't (e.g. running a secondary L2 trunk between the switches for SVI L3 peering on the peer devices is supported on the N5K, but not on the N7K), so unless you can find documentation that states that your setup is supported on the N3K (or your TAC buddy says differently), I wouldn't trust this to work even if you prune VLAN 5.

In any case, the "recommended" way of doing routing in your case is L2 vPC from Nexus to your ASAs, static routes on the Nexus pointing to the ASA, then redistributing those into your IGP.  Look here starting on page 104:
http://www.cisco.com/c/dam/en/us/td/docs/solutions/CVD/Aug2014/CVD-DataCenterDesignGuide-AUG14.pdf

The specific config on that doc applies to the N5K, but unless you find documentation that excludes the traditional issues from the N3K, don't assume anything :-).

You forego dynamic routing from the ASA directly, but if you just follow that guideline you'll save yourself the headaches I went through in trying to figure out where the OSPF packet went, why adjacencies weren't forming, and learning all the discrepancies among the Nexus family and software versions.

[Dieselboy]

Did you put in the vPC peer gateway command? (as otherwise it would cause problem with HSRP)

Yes peer-gateway is enabled but it would not cause a problem with HSRP without it enabled. Not in this scenario.

@Dieselboy,
Why do you think the N3K is immune to the issue of peering over a vPC? I ask because AFAIK only the N6K doesn't have that issue (though I can't find the CiscoLive preso that mentions that).

My experience with Nexus is that although some stuff works for the N5K, it may not work for the N7K, and what works on one version or line card doesn't work on the other, so as a principle I always refer to the official config guides for the software version and for the specific Nexus I'm working on for verification on what works and what doesn't, and be very attentive to detail.

What's odd is the config guides for the N3K don't say anything about routing peering in combination with a vPC...but the N5K and N7K guides explicitly state what works and what doesn't (e.g. running a secondary L2 trunk between the switches for SVI L3 peering on the peer devices is supported on the N5K, but not on the N7K), so unless you can find documentation that states that your setup is supported on the N3K (or your TAC buddy says differently), I wouldn't trust this to work even if you prune VLAN 5.

In any case, the "recommended" way of doing routing in your case is L2 vPC from Nexus to your ASAs, static routes on the Nexus pointing to the ASA, then redistributing those into your IGP.  Look here starting on page 104:
http://www.cisco.com/c/dam/en/us/td/docs/solutions/CVD/Aug2014/CVD-DataCenterDesignGuide-AUG14.pdf

The specific config on that doc applies to the N5K, but unless you find documentation that excludes the traditional issues from the N3K, don't assume anything :-).

You forego dynamic routing from the ASA directly, but if you just follow that guideline you'll save yourself the headaches I went through in trying to figure out where the OSPF packet went, why adjacencies weren't forming, and learning all the discrepancies among the Nexus family and software versions.

I definitely don't think n3k is immune to vPC issues, actually quite the opposite. I think you have misunderstood my posts or not read them fully. " I wouldn't trust this to work even if you prune VLAN 5." - it definitely will work if you prune VLAN 5 since pruning vlan 5 from the vPC trunk is removing it from the vPC configuration.
I've still yet to read that design guide! it's on my list.

I had TAC look at the secondary issue regarding not being able to ping the 2nd ASA. The guy believed he found a forwarding table issue (like a CEF problem, but the Nexus equivalent). As a test, we failed over HSRP by adjusting the priorities and everything began working fine. We then failed it back and everything remained working fine and has been ever since. On my list of things to do is to bring the switches up to a later code as they've been up for 500+ days. I've not yet had a chance to look through the release notes to justify it though, ie for security fixes. However Cisco advised the above is not normal and an upgrade is likely in order. I also think the above is what caused our OSPF not to work.

In any case, static routing is working fine and is less hassle.

[killabee]

Hmmm, now I'm confused because I sure did read your post where you made this comment:

"I have 2 x ASA5515X's which I've configured OSPF to two separate layer 3 switches. The switches are Nexus and to get around the vPC issue (which I don't think affects the 3000 series nexus anyway)..."

So what vPC issue are you referring to that you don't think affects the N3K?

[wintermute000]

Nexus is bleeding edge bug fun time. 500+ days, upgrayed that shiz!!!!!!!!

Not surprised its looking like a bug as I've found cisco design guides specifically outlining your topology and the solution of running the vlan across a non-vPC.

A colleague of mine is dealing with 2 separate N7k issues where the sup randomly reboots. LOLOLOL (and you know 7k = its going to be a pretty darned important switch at a pretty darned important site/DC)

https://tools.cisco.com/bugsearch/bug/CSCui72592

[Dieselboy]

Hmmm, now I'm confused because I sure did read your post where you made this comment:

"I have 2 x ASA5515X's which I've configured OSPF to two separate layer 3 switches. The switches are Nexus and to get around the vPC issue (which I don't think affects the 3000 series nexus anyway)..."

So what vPC issue are you referring to that you don't think affects the N3K?

This part: "(which I don't think affects the 3000 series nexus anyway).." was referring to the differences between n5k and n7k where they treat vPC slightly differently, that's all. It's still documented to not peer OSPF across a vPC though.

Nexus is bleeding edge bug fun time. 500+ days, upgrayed that shiz!!!!!!!!

I don't really want to. It's all working fine (now), it's completely internal with no customer access secured by a firewall. My biggest concern is upgrading it and introducing bugs / issues. Even if I do read the release notes Cisco's "internal bug / private bug" system can make the release notes somewhat un-useful. Really annoys me