Messages

The "load balancer" part is done from the end device, it's either the VDS on vmware ar the Linux bound on Proxmox or the intelligence inside our dual controller network storages or the multiplexor card on our windows server. That part is working correctly and is doing what it is supposed to do.

What i don't understand is why when there is a change in the topology due to a failure, my switch keeps blocking the ports and preventing my hardware to failover correctly...

My real question is : Am I missing something ? Is there a specific config that i don't know for this kind of hardware that i have to setup on the switch side ? Or is it just a STP tuning problem ?

I'm not sure but I think that when you create a aggregate of two nics, it creates a VIP that contains the ip that you want to attribute to the two ports and this VIP also has its own MAC. So when a failover is happening the switches see the VIP MAC moving from a port to another and blocks it because the switch thinks there is a loop. (from what i'm seeing this guess looks wrong and you are correct)

From what i'm seeing in VMWare it's not the case, the MAC on the VIP is the same as one of the two hardware interfaces used for the failover. but i don't know for all our other hardware.

In any case, I don't get why i'm getting these kinds of errors :
2023 Nov 21 07:09:35.204 ciscoXX %L2FM-2-L2FM_MAC_MOVE_PORT_DOWN: Loops detected in the network for mac 0a1b.2c3d.4e5f a
mong ports Eth1/XX and PoXX vlan X - Port Eth1/42 Disabled on loop detection
or
2023 Nov  7 02:21:44.357 ciscoXX %L2FM-4-L2FM_MAC_MOVE2: Mac 0a1b.2c3d.4e5f in vlan XX has moved from Eth1/XX to Eth1/XY


Maybe i have to tune my STP timeouts in order to avoid this behaviour :

• Hello Time  2  sec 
• Max Age 20 sec
• Forward Delay 15 sec

[we either have dual-controller or dual network interfaces (via VDS or linux bound set up in active/standby configuration) on all our hardware]

Hi,

I'm opening this thread because i think i'm missing some knowledge regarding the spanning tree protocol on redundant/dual-controller hardware.

To explain my problem here are some details:
I'm working in an infrastructure where we are located in two datacenters, we have on each site a symmetrical configuration.
Our entry point in the infrastructure is a fortigate that is used as a gateway for all our VLANS (public and private) as well as our firewall, from there we are connected to two Cisco Nexus 9000 in a vPC configuration and from that we also have some catalysts connected also in vPC.

I have set all the STP weights at the switch level instead of on the interface level because it was recommended to me and it is easier to manage.
I have tried STP, rSTP, and MST with exactly the same results
Before and after the implementation of vPC i had exactly the same problems.

On this infrastructure are connected some end devices (hypervisors, NAS and some network storages (dell equallogic)).

All the connections in this infrastructure are setup to have a redundant path on another network hardware to avoid downtime if something goes down so we either have dual-controller or dual network interfaces (via VDS or linux bound set up in active/standby configuration) on all our hardware

I think i'm missing some knowledge with this kind of configuration because no matter what i tried i can't seem to be able to avoid network loops when we have for example a router update.

I studied the logs and err_dis_loop are occuring on our switches interfaces seemingly at random on our servers and storages (because each time it's happening on a different hardware). I understand that due to this configuration, with network loops everywhere that this is to be expected even if i tried to upgrade to a vPC infrastructure in order to reduce the problem.

I suspect that during the STP convergence, the dual controller is also switching its active interface in order to find a path that is working and the switch in not understanding what is happening because it sees the same MAC address on two interfaces so it blocks the ports (or at least one).

tldr: Is there some specific configuration that i have to set up in order to avoid blocked ports during a convergence in a infrastructure with dual controller / VDS / linux bound interfaces/hardware ?

Thank you in advance for your help and i will be happy to give you more information if you need them.

Yes, we do have latency sensitive applications and connections between servers.
But that problem doesn't scare me too much because we intend on migrating these servers at the same time (or ASAP).
And our latency between DC1 and DC2 is around 5ms.
I went and tried to create a lab with this configuration in mind but honestly i'm pretty lost. 
I'm not used at all with working with these technologies.
I found a video on youtube of someone creating a lab like this one : https://www.youtube.com/watch?v=69oa55LsoAc but he only works with one VLAN and i hope it will be the same with multiple VLANs.
I really think that this method if feasible and is adapted for our needs. The only thing i'm not sure about is my ability to set up something this critical.

We have too many servers in order to do that and it will take too long and require too many configurations in order to replaces every linkages for the IPs between the servers.
We are talking about 150 VPS, at least 8000 web servers and i'm not even talking about the DNS servers, the mailboxes ...
I think i'm going to go with VXLAN over IPsec
First by getting all my VLANs into a VXLAN
https://docs.fortinet.com/document/fortigate/6.2.0/new-features/392860/vlan-inside-vxlan
Than by making that VXLAN transit over a site to site IPSEC tunnel
https://docs.fortinet.com/document/fortigate/6.2.0/cookbook/821119/vxlan-over-ipsec-tunnel
I must find a way to do that networking-wise with a solution that will be totally transparent for the machines as well as for the customers.

Hi,
I'm contacting you because i'm facing a pretty complex problem and i'm a bit lost.
I'm working in a company that rents a bay in a datacenter and we are planning for some time to go big data with a second site to host our machines.
Our problem is that we would like to have our public IPs available on the first site as well as the second site.
I am wondering what technologies we can use to set this up.
Our main point of worrying is that we would like as much as possible to be able to keep on site A and B the same VLANs numbers and IPs (so that if we decide to take an hypervisor with all its machines and set it up on site B it will work without any modification in the configurations.
We are already planning on installing a site-to-site IPSec tunnel between the two datacenters in order to do live migrations (we have about 5ms of latency so in theory we can even have our storages on site A and our VMs on site B)
Some of the public IPs are provided by the DC itself but we also have a rank that we purchased at the RIPE. (we are owning it but we delegated the management to the DC1 so they are routing our IPs to us).
DC2 will also rent us some public IPs.

I'm joining a schema of our infrastructure so i can illustrate what i'm saying.
Based on this picture, in the first time we would like to empty Cluster 1 and ISCSI storage 1, move them to DC2, bring them back up with the same IPs and then be able to do a VMotion from site A to B with the VM keeping the same public IP.
We have also some nodes in Proxmox, my concern with the privates IPs between the 2 sites is concerning the Clustering of all these nodes (i know i probably can use NAT in order to "masquerade" the IP but i rather keep this setup as simple as humanely possible).
I'm having a hard time fining the documentation on somethin as specific as this. I asked around me already to friends that knows a lot more than me and they proposed me several ideas to achieve this:
- the first one told me that via the IPSEC tunnel (and some pretty specific routes) we can achieve this.
- another one told me that EBGP was the way to do it.
- and i found this thread : https://arstechnica.com/civis/viewtopic.php?f=10&t=1324975 that recommends more VRRPe and OTV.

Honestly, i'm a bit overwhelmed with the quantity of informations and i would like some advice on what you this is the best and easiest way to achieve this.

For our routers we are rocking Fortigates if that can help you.

Thank you in advance for your help and don't hesitate to ask me more informations if you need.