weird switching event last night

[icecream-guy]

So I'm working on building the V1 TSHOOT topology in my home lab, needed another l3 switch, so I used my production 3550.  seems that broke access to the internet.

My 3550 is the network CORE, where all the good stuff happens. the SVI for vlan 90 (internet access VLAN & and others) internet VLAN is 192.168.9.1/24.  there is also a WAN switch in my utility room where the firewall (192.168.9.2) and DSL modem connect.  My CORE is VTP Server, WAN switch is VTP Client,  Seems that after I was done setting some stuff up building some of the L2 TSHOOT topology, creating a new VLAN 200 and configuring some port-channels between both the CORE and LAB 3550's the internet broke. In troubleshooting, I couldn't ping 192.168.9.2 from my CORE @ 192.168.9.1 nor the other direction from WAN to CORE.  Spanning-tree looked good,  mac address tables look fine, (WAN switch is Layer 2 only) incomplete for the firewall in ARP on the CORE. More troubleshooting, I could ping all my other VLAN SVI's from the WAN switch via the management IP 192.168.1.9, still couldn't ping the VLAN 90 SVI at 192.168.9.1, weird. shut/no shut the connecting ports, no luck, rebooted CORE and WAN switch, no luck. after about 30 mins I decided to switch over to my backup link to see if that worked.  (I have a backup link between CORE and WAN for redundancy JIC, (ports configured exactly the same just a copper link rather than fiber). When I unplugged the fiber link, everything started working, I could ping my firewall from CORE, and all the active SVI from the WAN.  odd... both sets of copper and fiber ports have exact same basic trunk all configuration. plugged the fiber back in,  no work,  changed the spanning tree port cost to use the backup link as the primary path, both ports active, no good, unplugged fiber, all works again.  scratching my head here...  so it's not a L2 issues, it works over the copper backup link, not a layer 3 issues where I can ping over the backup link, something has to be with the fiber port /connection, not L1, have link lights.... looking over the configurations for the fiber ports, basic trunk all, show vlan br, see VLAN 200 on my WAN Switch, I'm like wait that shouldn't be there, (back to the VTP server/Client relationship) it was auto created there when creating the VLAN 200 on CORE. So my next thought was to get rid of that VLAN on WAN and remove the VLAN from the trunks, once I did that, (had to change WAN VTP to transparent to remove the VLAN on WAN, also did it on CORE) connected up the primary fiber links and everything works now.  double weird.  VLAN 200 had nothing to do with any of this, (it's on a 10.x network). So I tried to put everything back the way is was, recreated the VLAN 200 on WAN, retrunked VLAN 200 to the WAN switch, still all worked like it should have been in the first place. Didn't change the VTP configurations back, didn't see how any of that would have caused a problem. not sure of the cause, chalked it up to a switch oddity and went to bed.  it was a fun troubleshooting session.

[Reggle]

Do you have UDLD running over the fiber links?
Are the fiber links using SFPs in so called dual-media ports, where there is an SFP slot and a native copper port are both physically present but have the same logical interface name on the switch?
Do you have DTP running over the links?
Is spanning-tree the same on both sides and what is the root bridge?

[icecream-guy]

Do you have UDLD running over the fiber links?
Are the fiber links using SFPs in so called dual-media ports, where there is an SFP slot and a native copper port are both physically present but have the same logical interface name on the switch?
Do you have DTP running over the links?
Is spanning-tree the same on both sides and what is the root bridge?

haver to check on some of this  I think udld may have been turned off on fiber link, I'll check.  the physical optics are the old school GBIC SC types.  WAN is a  2950 switches, CORE is a 3550  so the copper is (fa0/24 being uplink on CORE, and fa0/8 uplink on WAN)  & (gi0/2 uplink on CORE and gi0/1 uplink on WAN) 

I do remember some DTP i'll have to check, i think I turned that off. but i'll check again.

spanning-tree is RTSP, CORE is root


now I wish I had some logs...

[icecream-guy]

Do you have UDLD running over the fiber links?
Are the fiber links using SFPs in so called dual-media ports, where there is an SFP slot and a native copper port are both physically present but have the same logical interface name on the switch?
Do you have DTP running over the links?
Is spanning-tree the same on both sides and what is the root bridge?

have to check on some of this  I think udld may have been turned off on fiber link, I'll check.  the physical optics are the old school GBIC SC types.  WAN is a  2950 switches, CORE is a 3550  so the copper is (fa0/24 being uplink on CORE, and fa0/8 uplink on WAN)  & (gi0/2 uplink on CORE and gi0/1 uplink on WAN) 

I do remember some DTP i'll have to check, i think I turned that off. but i'll check again.

spanning-tree is RTSP, CORE is root


now I wish I had some logs...

both links are hard set to trunk dot1q on the CORE side, dot1q by default on the WAN side since it's a 2950.
the DTP were configured for the cross links  between the DSW and ASW (if you are familiar with the TSHOOT V1 topology odd references point there), production CORE and other 2950's (ASW1 & ASW2 were off at the time). I removed that when I set up the trunks, hard set trunk,  I don't like DTP.

udld disabled on both CORE and WAN with the 'udld port disable' interface configuration.

looking at configs, do have "spanning-tree etherchannel guard misconfig" but not running etherchannel between CORE and WAN

I think I'm getting a clue, my CORE is spanning-tree vlan 1-1000 priority 28672, my other 3550 DSW1 is not running spanning-tree
not sure why spanning-tree won't run in DSW1

dinner time...

pizza.... The L3 link on Po12 between DSW1 and CORE (DSW2) breaks the spanning-tree. I need to complete the rest of the L2 cross connects and all spanning-tree should work.

[paulaugust]

Your working for wire t1sop technology any thank you for sharing information regarding weird switching event last night