NTP Server Crash

Started by deanwebb, June 03, 2016, 08:50:37 AM

Previous topic - Next topic
Print
Go Down Pages1
User actions

deanwebb

June 03, 2016, 08:50:37 AM
Almost all my NAC devices were flashing with a warning... NTP server issue, error message said...
:zomgwtfbbq:

Tested connection to NTP server... Connection was up, but the offset was over 8000 seconds. That is bad because it is not good.

Changed all the NAC devices pointing to that one NTP server to use a different one, problem all cleared up.

Now I'm wondering what caused that NTP server to lose its damn mind like that...  :think:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Reggle

#1
June 05, 2016, 03:08:59 AM
You can let a Cisco NTP server on a router or switch crash if you change IP addresses on the device. And feeding an NTP server bogus packets might make it go nuts I've heard.
If it doesn't use ECC RAM, I blame bit flips...

deanwebb

#2
June 05, 2016, 07:54:26 AM
I'm going with bit flips.

We did a NAC client push when the issue was ongoing... all the clients had an... interesting experience when they tried to talk sense to the main NAC appliance. We're working with the vendor to resolve. (Most likely, remove and reinstall. Well, it's on Windows, after all...)
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Dieselboy

#3
June 07, 2016, 03:41:28 AM
I read years ago that bit flips are to do with a cosmic particle from outer space hitting your ram.

deanwebb

#4
June 07, 2016, 11:54:01 AM
Turns out, it was a WAN router blocking traffic from the NTP, so it kinda sorta wandered off in time... once we fixed that ACL, things went just fine.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.
Print
Go Up Pages1
User actions