Flex Links Query

[RoDDy]

Hi Guys,

Quick question. I was looking at flex links as an alternative to STP, but remind me again.....this only monitors line protocol right? So my service could be down but as long as the line protocol is saying UP then it wont fail over to the standby right?

-Roddy

[Dieselboy]

Hi Roddy,

Sorry this post has been missed. I've never used flex links but from memory I think you're correct.
I found a quick link and had a skim through there as a refresh: http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/12-2SX/configuration/guide/book/flexlink.html

Can you give some examples where your "service could be down, but the line protocol could be up"?

I think something like a fibre link, which has one direction broken, eg. the Tx is broke could cause this?

But if this is true, then you could use UDLD on the links. I use UDLD a lot, basically UDLD forms a relationship across links and sends keepalives from A to B and expects B to return the keepalive. At the same time, keepalives are sent from B to A and expects A to return them to B. If the keepalives stop being returned, the link is put into error disabled state.

The good thing about UDLD is that you can enable it during normal business. If you have a link and need to enable UDLD, you must enable udld on both sides for the relationship to be formed. If you only enable UDLD on one side, nothing happens until the relationship is formed the first time.


To summarise; I think with UDLD it can detect and shut bad links in a situation which causes line protocol to be up but the link is bad. Along with flexlink, backup link(s) could be made active in the event UDLD has torn broken link(s) down.

[RoDDy]

Hi Roddy,

Sorry this post has been missed. I've never used flex links but from memory I think you're correct.
I found a quick link and had a skim through there as a refresh: http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/12-2SX/configuration/guide/book/flexlink.html

Can you give some examples where your "service could be down, but the line protocol could be up"?

I think something like a fibre link, which has one direction broken, eg. the Tx is broke could cause this?

But if this is true, then you could use UDLD on the links. I use UDLD a lot, basically UDLD forms a relationship across links and sends keepalives from A to B and expects B to return the keepalive. At the same time, keepalives are sent from B to A and expects A to return them to B. If the keepalives stop being returned, the link is put into error disabled state.

The good thing about UDLD is that you can enable it during normal business. If you have a link and need to enable UDLD, you must enable udld on both sides for the relationship to be formed. If you only enable UDLD on one side, nothing happens until the relationship is formed the first time.


To summarise; I think with UDLD it can detect and shut bad links in a situation which causes line protocol to be up but the link is bad. Along with flexlink, backup link(s) could be made active in the event UDLD has torn broken link(s) down.

Hi Dieselboy, firstly thanks for the response and assistance on this. I should have updated/added more info to this post TBH.

Basically we have a primary fiber link into our office and added a secondary link for redundancy which is via microwave (wireless transmission). Following this (a long while after), we noticed a number of MAC flaps in our network so it seems we were getting forwarding loops but weren't sure of the source as it is a fairly massive network. I traced the problem down to the office link and determined that something weird was happening with the spanning tree (RPVSTP) that was implemented.

This post was somewhat laziness on my part but after further investigation I found that the microwave/wireless link was flapping, so the transmission medium between the switches at opposite ends was going down but the link to the transmission equipment/line protocol was still showing as up as the interface/equipment didn't actually go down....just the wireless link between the 2 antennas which could possibly be due to interference on that frequency even though this shouldn't be the case as its regulated (this should answer your question).

So what was happening was the BPDUs weren't being received from the root bridge on the port that was being blocked when the microwave link went down, which changed the blocked port to a forwarding state, and when the microwave connection came back up.....forwarding loop. I was looking for an alternative but i simply added the command "spanning-tree guard loop" so that when the port stops receiving BPDUs it goes to a loop inconsistent state and continues to block the port without ever transitioning it to forwarding state. In the meantime i have the TX/microwave guys working on getting the flapping resolved on their equipment.

The logs will now show the below:
Sep 13 19:05:58.786 BGI: %SPANTREE-2-LOOPGUARD_BLOCK: Loop guard blocking port GigabitEthernet0/23 on VLAN0431.
Sep 13 19:16:55.072 BGI: %SPANTREE-2-LOOPGUARD_UNBLOCK: Loop guard unblocking port GigabitEthernet0/23 on VLAN0431.

Instead of:
Sep 10 07:25:13.573 BGI: %SW_MATM-4-MACFLAP_NOTIF: Host a80c.0d64.fe93 in vlan 431 is flapping between port Gi0/23 and port Po1

NO MORE LOOPS!!!

Hope I explained everything in enough detail now.

[NetworkGroover]

Hi Roddy,

Sorry this post has been missed. I've never used flex links but from memory I think you're correct.
I found a quick link and had a skim through there as a refresh: http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/12-2SX/configuration/guide/book/flexlink.html

Can you give some examples where your "service could be down, but the line protocol could be up"?

I think something like a fibre link, which has one direction broken, eg. the Tx is broke could cause this?

But if this is true, then you could use UDLD on the links. I use UDLD a lot, basically UDLD forms a relationship across links and sends keepalives from A to B and expects B to return the keepalive. At the same time, keepalives are sent from B to A and expects A to return them to B. If the keepalives stop being returned, the link is put into error disabled state.

The good thing about UDLD is that you can enable it during normal business. If you have a link and need to enable UDLD, you must enable udld on both sides for the relationship to be formed. If you only enable UDLD on one side, nothing happens until the relationship is formed the first time.


To summarise; I think with UDLD it can detect and shut bad links in a situation which causes line protocol to be up but the link is bad. Along with flexlink, backup link(s) could be made active in the event UDLD has torn broken link(s) down.

Hi Dieselboy, firstly thanks for the response and assistance on this. I should have updated/added more info to this post TBH.

Basically we have a primary fiber link into our office and added a secondary link for redundancy which is via microwave (wireless transmission). Following this (a long while after), we noticed a number of MAC flaps in our network so it seems we were getting forwarding loops but weren't sure of the source as it is a fairly massive network. I traced the problem down to the office link and determined that something weird was happening with the spanning tree (RPVSTP) that was implemented.

This post was somewhat laziness on my part but after further investigation I found that the microwave/wireless link was flapping, so the transmission medium between the switches at opposite ends was going down but the link to the transmission equipment/line protocol was still showing as up as the interface/equipment didn't actually go down....just the wireless link between the 2 antennas which could possibly be due to interference on that frequency even though this shouldn't be the case as its regulated (this should answer your question).

So what was happening was the BPDUs weren't being received from the root bridge on the port that was being blocked when the microwave link went down, which changed the blocked port to a forwarding state, and when the microwave connection came back up.....forwarding loop. I was looking for an alternative but i simply added the command "spanning-tree guard loop" so that when the port stops receiving BPDUs it goes to a loop inconsistent state and continues to block the port without ever transitioning it to forwarding state. In the meantime i have the TX/microwave guys working on getting the flapping resolved on their equipment.

The logs will now show the below:
Sep 13 19:05:58.786 BGI: %SPANTREE-2-LOOPGUARD_BLOCK: Loop guard blocking port GigabitEthernet0/23 on VLAN0431.
Sep 13 19:16:55.072 BGI: %SPANTREE-2-LOOPGUARD_UNBLOCK: Loop guard unblocking port GigabitEthernet0/23 on VLAN0431.

Instead of:
Sep 10 07:25:13.573 BGI: %SW_MATM-4-MACFLAP_NOTIF: Host a80c.0d64.fe93 in vlan 431 is flapping between port Gi0/23 and port Po1

NO MORE LOOPS!!!

Hope I explained everything in enough detail now.

Yep - that's the exact use case for Loop Guard.

http://aspiringnetworker.blogspot.com/2013/07/optimizing-and-protecting-spanning-tree.html
http://aspiringnetworker.blogspot.com/2013/07/optimizing-and-protecting-spanning-tree_13.html

Wow can't believe that was back in 2013....

[icecream-guy]

never heard of loop guard, thanks for the links I'll review that subject.

[NetworkGroover]

never heard of loop guard, thanks for the links I'll review that subject.

Really?  Are you trolling me?

[icecream-guy]

never heard of loop guard, thanks for the links I'll review that subject.

Really?  Are you trolling me?

no I've heard all the other spanning-tree guard commands, root guard, bpdu guard, port fast, even BPDU filtering, just not that one

[NetworkGroover]

Ah ok - sorry then.  Yeah it's a good one to know - very important tool - seen it bite Brocade in the arse cause they didn't have it.  They were installed at access and the fiber between it and the upstream Cisco distro switch degraded... the link stayed up but there was enough of an issue that BPDUs stopped being received so the link went from blocked to forwarding and created a loop.  Don't ask me how as I wasn't involved in all of the troubleshooting, but I know shortly thereafter they created a loop guard-like feature.

[RoDDy]

Well it certainly came in handy for me here lol. Glad my post could enlighten someone.

I find that a lot of people tend to cringe at the sound of STP and the various aspects of it partially due to the fact that STP was revamped and had so many changes/additions made, that it takes some reading and going over to actually know the topic inside and out.

[Dieselboy]

I find that a lot of people tend to cringe at the sound of STP and the various aspects of it partially due to the fact that STP was revamped and had so many changes/additions made, that it takes some reading and going over to actually know the topic inside and out.

Definitely. I've scared a colleague a couple of times in my last job because I implemented MST and implemented it pretty well in my opinion. This was years ago, I think he made me strip it all out (which I done with a smile to keep him happy, not because "you wouldn't like him when he's angry" but the complete opposite but I digress). The reason for MST - I was given a mixture of Cisco switches and HP switches and was told "make all this work". I think I used a separate MST instance for the SAN traffic so it could forward directly between the two switches used for servers and storage. In contrast to the voice and data traffic which those ports were spanning-tree blocked because voice and data would switch up to the firewall/router. Without MST, the HP switches only supported RSTP on VLAN 1.

Roddy about your link-flap-issue, I had a customer in London with two 100MB or 1GB P2P links from their office to their datacentre in Docklands. There was a strange issue at one point that resolved by itself and the provider couldn't shed any light on it. So I turned on UDLD on both of those links. All would be well for ages and then all of a sudden, one circuit at both ends would be error-disabled (because of udld). Every time this happened I would raise it with the provider and they wouldn't know what happened. I think we saw mac flaps as well at the time.

With loop-guard, how long does it wait in "loop-inconsistent" before forwarding normally? I can't find the details on that. It appears it would never forward is this true?

With loop guard enabled, the blocking port on switch C transitions into STP loop-inconsistent state when the max_age timer expires. A port in STP loop-inconsistent state does not pass user traffic, so a loop is not created. (The loop-inconsistent state is effectively equal to blocking state.)

Taken from this link below: also comparing UDLD to Loop-Guard:
http://www.cisco.com/c/en/us/support/docs/lan-switching/spanning-tree-protocol/10596-84.html#loop_guard_vs_uld

[NetworkGroover]

With loop-guard, how long does it wait in "loop-inconsistent" before forwarding normally? I can't find the details on that. It appears it would never forward is this true?

I talk about this at least indirectly in the second link I posted from my blog.  In my testing it stayed in loop-inconsistent until the erroneous condition was removed.  I'm sure I'm just arguing semantics here, but technically the port doesn't "forward normally" - it returns to Alternate Role, Discarding State.

[Dieselboy]

Ah okay - I was trying to think what would happen if the port is connected to something else like a router or end host.

So by implementing loop-guard, you're basically saying that this port should ALWAYS receive BPDUs and if the BPDUs stop then something is very wrong, block the link.

[SimonV]

So by implementing loop-guard, you're basically saying that this port should ALWAYS receive BPDUs and if the BPDUs stop then something is very wrong, block the link.

If my memory serves me well, loopguard is only implemented on STP-blocking links, where you have the risk of them erroneously going into forwarding mode.

[wintermute000]

dingdingding

its similar to UDLD in that its basically for catching unidirectional fibre links - have never seen loop guard triggered by faulty copper or RJ45 ports


I can't believe flex links are still a thing. I only remember reading about it prepping for the CCIE v4 written.

[SimonV]

dingdingding

its similar to UDLD in that its basically for catching unidirectional fibre links - have never seen loop guard triggered by faulty copper or RJ45 ports

I labbed loopguard going into effect by adding a bpdu-filter on the other end, so that might be one obscure scenario where you'd have a loop otherwise