VXLAN integration into current L2 DC infrastructure

[LynK]

Hey guys,

Do any of you know any best practices for this? Sure VXLAN is great, and is the future but how does one go about implementing it into existing infrastructure? Do you just take the core L2 DC switches and interconnect them into the leafs temporarily? Or am I trying to overthink things here. Lets say for example, company X does not have any nexus infrastructure, but 6509E's.

They are slated to stand up a new data center with VXLAN, but first need to incorporate it with the legacy DC. VLANs need to be exchanged/used at both DCs. L2/L3 fiber options are both available. How would you design this?

[icecream-guy]

Hey guys,

Do any of you know any best practices for this? Sure VXLAN is great, and is the future but how does one go about implementing it into existing infrastructure? Do you just take the core L2 DC switches and interconnect them into the leafs temporarily? Or am I trying to overthink things here. Lets say for example, company X does not have any nexus infrastructure, but 6509E's.

They are slated to stand up a new data center with VXLAN, but first need to incorporate it with the legacy DC. VLANs need to be exchanged/used at both DCs. L2/L3 fiber options are both available. How would you design this?

underlay or overlay ?

I should be nice and not tell you to google "VXLAN Design guide", so I won't.

I assume by your post that you are looking for practical experience with this technology.
I have no practical experience with this technology. But I'm looking over a design and deployment guide now.

Thanks

[LynK]

@ristau

You should know better that I would have done my research and I have scoured through many different design guides, but nothing talks about parallel integrations.

[deanwebb]

I went ahead and tried some google-fu and got smacked down since all the articles mentioned Nexus stuff, which you said this guy has none of.

[icecream-guy]

@ristau

You should know better that I would have done my research and I have scoured through many different design guides, but nothing talks about parallel integrations.

good thing I didn't say anything.

doesn;t seem to be alot of requirements

Code Select Expand


Physical Switch prerequisites

For successful VXLAN operation, these requirements must be met:


    DHCP should be available on VXLAN transport VLANs.

    Note: Fixed IP also works.

    VXLAN port (UDP 8472) is opened on firewalls (if applicable).
    Port 80 is opened from vShield Manager to the Hosts and used to download the vib / agent.
    VMware recommends 5-tuple hash distribution for Link Aggregation Control Protocol (LACP).

    Note: 5-tuple hash distribution adds better load balancing as the hash includes the inner frame entropy represented in the UDP source port.

    MTU size requirement is 1600.
    VMware recommends that you enable IGMP snooping on your L2 switches, to which VXLAN participating hosts are attached.
    If IGMP snooping is enabled at L2, then IGMP querier must be enabled on the router or L3 switch, with connectivity to the multicast enabled networks.
    If VXLAN traffic is traversing routers, multicast routing must be enabled.



here
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2050697

[LynK]

@ristau

You should know better that I would have done my research and I have scoured through many different design guides, but nothing talks about parallel integrations.

good thing I didn't say anything.

doesn;t seem to be alot of requirements

Code Select Expand


Physical Switch prerequisites

For successful VXLAN operation, these requirements must be met:


    DHCP should be available on VXLAN transport VLANs.

    Note: Fixed IP also works.

    VXLAN port (UDP 8472) is opened on firewalls (if applicable).
    Port 80 is opened from vShield Manager to the Hosts and used to download the vib / agent.
    VMware recommends 5-tuple hash distribution for Link Aggregation Control Protocol (LACP).

    Note: 5-tuple hash distribution adds better load balancing as the hash includes the inner frame entropy represented in the UDP source port.

    MTU size requirement is 1600.
    VMware recommends that you enable IGMP snooping on your L2 switches, to which VXLAN participating hosts are attached.
    If IGMP snooping is enabled at L2, then IGMP querier must be enabled on the router or L3 switch, with connectivity to the multicast enabled networks.
    If VXLAN traffic is traversing routers, multicast routing must be enabled.



here
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2050697

I'm referring to upgrading 1 DC at a time where for example there are 6509-E cores. Would it be a simple as a L2 Port channel to one of the leaf switches through the PTP fiber links?

[NetworkGroover]

Hrmm.. not sure about integration in Brownfield.... you need an encap and decap point... outside of those points it's just routed normally.. so.. like in a two-tier spine leaf design you'd have a pair of edge leaves that take care of that for you, connected to routers via trunks (Since VXLAN uses VLANS tied to VNIs).  The routers then route the traffic normally... in a DCI situation you'd have a pair of VXLAN-capable devices that do the encap/decap at each end.

[NetworkGroover]

I don't know how Cisco says you should do things... but I'd rather have VXLAN capable devices downstream that take care of encap/decap and just let the core route as normal....

But wait.. L2 cores?  If you have L2 cores then why do you need VXLAN?

A sanitized diagram may help here explain what you are trying to accomplish.

[wintermute000]

Bridge the real l2 domain into a layer 2 only vxlan evpn. Then move the l3 gateway via turning on anycast gateway and irb and disabling the old layer three.

[Dieselboy]

Watching this 😊 need to design a DR site, but haven't done my research yet.

[burnyd]

Bridge the real l2 domain into a layer 2 only vxlan evpn. Then move the l3 gateway via turning on anycast gateway and irb and disabling the old layer three.

I can get behind that.

[burnyd]

You can bridge your L2 domain into a new vxlan based fabric.  I would recommend evpn.  If you are going to stick with the same infrastructure physically and everything is vmware then nsx is a great solution.

[NetworkGroover]

You can bridge your L2 domain into a new vxlan based fabric.  I would recommend evpn.  If you are going to stick with the same infrastructure physically and everything is vmware then nsx is a great solution.

Heh - and if you go the NSX route, you've got an all-round SME in burnyd

[LynK]

Hey guys,

digging this thread back up after a lot of research/thoughts processing. But before the outcomes. let me clarify everything.

1) Core in old DC is running l2 & l3.
2) Everything below core is L2
3) New DC is going to be VXLAN EVPN

Now to connect the two DC we have a few options.

1) Stand up 2 VTEPs (in old DC) that span across the DCI link to the new DC spines (or leafs). Remove all L3 off of cisco core, and onto VTEPs. Run vPC between VTEP and core, and make the old DC core L2.

2) Keep the old DC running L2/L3. Use separate networks in each DC for now, and run L3 between the two (between transit leaf nodes). Once new VXLAN infrastructure is ordered for old DC, we then migrate everything over and it is done.

Honestly. Both are good options, but both would require some work. Option 1 requiring Migration off of the DCI line to new DC spines, and then back onto old DC spines, then running link leaf to leaf for DCI. Option 2 requires pretty much the same work, but does not require purchasing additional gear to run at old DC (and migrating everything off of old DC core onto VTEPs, which then run as cores essentially).

Thoughts?

[wintermute000]

2.) is by far the better option. L2 span is bad, no matter what whizbang technology you are employing.

With 1.) you do realise that in a VXLAN EVPN, anycast GW must be consistent across the entire fabric i.e. you can't have a SVI that exists only on one switch - the closest analogue would be the pre-VXLAN routing solution of using a handoff to separate L3 router on a stick. So you're merging L2 domains across sites, moreover, its the same fabric (same overlay) = stretched failure domain. I would not suggest doing this unless its a disciplined short term migration measure (you know how duct tape temporary fixes that work end up panning out)

Check out multi-site EVPN. This is the future. Each DC remains a separate VLXAN EVPN fabric, and you tie the two together with a third multi-site EVPN for the best of all worlds (separate overlays and failure domains, but with seamless L2/L3, and its all BGP based still so happy days)

https://www.ciscolive.com/online/connect/sessionDetail.ww?SESSION_ID=95611&tclass=popup