Switches incorrectly loading SNMP group on reboot

[config t]

As the subject says. When the switches reboot they create a duplicate SNMP group that is missing the read and write views. I'm having to go in and delete the identical group on around 10 switches per day. Seems to only affect 9300 and 3850's.

Haven't been able to find anything other than a couple vague messages on Cisco support that trailed off on dead ends.

[deanwebb]

Would those 10 be in a line that goes from your facility to the Internet, possibly through an Internet access method you're currently not aware of?

That's my first reaction, somebody touched them in an evil way and that's the evidence you're seeing.

[config t]

Possible. It's not consistent across the couple hundred switches, and only happens during a rare power outage, when NETOPS reboots for maintenance reasons, and more often on one of the enclaves that has a lot of reboots due to operational stuff and things.

Code Select Expand

show archive log config alltells me it always happens during or after the startup sequence. My best guess is a bug, because I can see the string in there and it's correct.

Right now it's only producing a minor effect on discovery, but when we start doing no shit NAC infrastructure actions it's potentially going to be an issue. For that reason I am going to ask them to open a TAC.

One of the guys mentioned we can add strings to the call home sequence that runs after startup so I may have them add the following and see if it helps:

Code Select Expand

no snmp-server group <group> v3 priv
no snmp-server group <group> v3 auth
snmp-server group <group> v3 auth read <view> write <view>
That will purge both the existing and duplicate group and then reconfigure it.

[deanwebb]

Found this from 2009:

"I changed the view name to all lower case as well as the group name, and now the settings stick after a reboot. Weird...

Thanks for your help!"

https://community.cisco.com/t5/network-management/snmp-v3-broken-after-reboot/td-p/1346995

[config t]

Nice    my group and view names are all caps and I had (very) briefly considered at least lower-casing the views since that's the part that always breaks.

I am now deeply suspicious..

*Edit
NETOPS team is going to love me if this is the issue.. their Solarwinds SNMP groups are always breaking and wouldn't ya know it.. those views are all caps as well.

[deanwebb]

Cisco can be very very picky about SNMP group names.

[Otanx]

That would be a crazy bug. I try to always do all caps for stuff because it helps stand out in the config as an admin assigned name instead of a key word, or config. At my old gig we did use lowercase for our SNMP stuff, but I don't remember why we did it. I wonder if we had a similar issue, but I don't remember it.

Thanks,
-Otanx

[config t]

Haven't had a chance to test it yet. NETOPS is done with upgrades for now so not a lot of switch reboots to replicate the issue.

We are thinking about implementing the call home post-boot script (or whatever it's called).. and just calling it a day.

[deanwebb]

We're hitting a similar issue in one of my agency customers and it looks like getting SNMP group and view names to lowercase is the way to resolve it.

[config t]

I love it when a thread from the forum(s) helps me fix a problem

[deanwebb]

Indeed. I just have to be careful not to ask the question here before I look for an answer, as one of the top search results will be my post asking the question here.