VXLAN integration into current L2 DC infrastructure

[LynK]

@Winter

1) Yes I a completely aware the SVIs need to be consistent

That document is amazing. I really need to start leaning on more cisco live documentation. I rarely use it...

I have a question for you though. For those of you that have gone the DCI route already. When you are stretching your infrastructure do you use 1 HA pair of firewalls at each DC (A/A or A/P, it doesn't matter) or separate clusters for each DC? I personally think separate clusters are the way to go... however most of my experience is theoretical at best.

It discusses all the options in the document, I am just curious what path do you go and why?

[LynK]

Adding another comment to this thread. When looking into VXLAN limiations, we were considering how awesome it would be to run FCoE over VXLAN, but of course there are limiations

According to document:https://www.cisco.com/c/en/us/td/docs/switches/datacenter/pf/configuration/guide/b-pf-configuration/Forwarding-Configurations.html

"FCoE
FCoE over the VXLAN fabric is not supported. However, FCoE and VXLAN can co-exist. FCoE and VXLAN services are provided on separate ports

To enable FCoE, use separate links from the fabric to MDS and connect to the target device. Refer Cisco NX-OS FCoE Configuration Guide for Nexus 7000 Series and MDS 9000 and Cisco Nexus 5600 Series NX-OS Fibre Channel over Ethernet Configuration Guide for details."

Does this mean it can only be done as L2 on the leaf switches? That would suck.

[wintermute000]

Separate clusters, a very big non fan of stretched HA. Google Ivan Pepjlnaks rants on the topic

[deanwebb]

Separate clusters, a very big non fan of stretched HA. Google Ivan Pepjlnaks rants on the topic

Stretched HA is an abomination.

[Dieselboy]

So I have a question if that's ok. I can see the use case in a multi tenant env. providing hosting for customers. What about a DR scenario? Where one site would be active and DR site would be standby. In a DR situation, the VMs would be spun up at the remote site and their network addresses would not need to change. I'm thinking that this would be best for active/active DR.

[wintermute000]

Think about routing (always think about routing). You have to get the traffic IN and OUT somewhere. Which site? asymmetric or symmetric? hairpin or not? stateful devices? failover? symmetry/asymmetry in prod vs failover? What about partial failures?

There are duct tape solutions of varying levels of elegance (/32 host routes, LISP, etc.)

Also even if you have diverse dark fibre you are basically designing one inter-dependent stretched failure domain which will split brain if DCI is completely lost. Active/passive is preferable if you can afford the spare capacity.

Ivan Pep and packetpushers have ranted on this topic many a time

Unfortunately thanks to vmotion and legacy server stacks we'll never get away from stretched L2, it lets every other IT department push their complexity down to the 'invisible' network plumbing stack. Remember this: doing a complex L2 stretch solution is effectively pushing your technical debt away from legacy must be L2 adjacent clustered apps / can't change IP apps + servers to the common network. If everything was L3+DNS aware (brave new stateless micro-services container model anyone? How do you think GOOG + FB do it?) then you could have a 'simple' L3 network