hulc LED Process bug

[LynK]

hey guys,

I have reached out to TAC referring this issue, but curious to see if anyone else has come across this as well. We are running a 3 switch 2960x stack which consists of:

Code Select Expand


*    1 52    WS-C2960X-48FPD-L  15.0(2)EX5            C2960X-UNIVERSALK9-M
     2 52    WS-C2960X-48FPS-L  15.0(2)EX5            C2960X-UNIVERSALK9-M
     3 52    WS-C2960X-48FPD-L  15.0(2)EX5            C2960X-UNIVERSALK9-M

We are seeing high CPU usage relating to this process:

Code Select Expand


CPU utilization for five seconds: 40%/1%; one minute: 40%; five minutes: 40%
PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
163   110518151    10468050      10557 22.55% 22.50% 22.25%   0 Hulc LED Process
169        9068        6569       1380  0.59%  0.09%  0.16%   1 SSH Process


I shut down the ports that are not in use (which will be) and the process goes down significantly.

Thoughts?

[deanwebb]

Did a search on "hulc LED process" and there were LOTS of hits asking about the high CPU on that process, going back a few years. Cisco claims it's fixed in 12.2(55), but people still see it. Shutting down ports, especially SFPs, is a workaround to getting the CPU utilization down, but opening up the ports sends it right back up again.

What IOS are the switches running?

[RTFM]

Bug "CSCtg86211"   Don't know how helpful it would be for your situation.

https://tools.cisco.com/bugsearch/

[Seittit]

well known bug with switches running 15.x code.

we fight it every few months, Cisco's recommendation is to shut down all ports not in use and reload when CPU peaks over 75%.

[LynK]

@seittit

thanks man.... i thought as much. Everything I was reading seemed to have applied to S-series switches.

I plan on turning off the extra ports when fully finished migrating, i went in and downed about 30-40 ports, and instantly saw a 10% decrease in CPU.

@dean

I did read that article that you read, I turned off the extra SFPs, and it went down about 5-6%

[sgtcasey]

We've been seeing this with 2960S and 3750X stacks for a couple years now.  This issue is one of the reasons we've decided where I work to not stack 3750X's more than 5.  Even at 5 sometimes they're just plain sluggish on the command line.