ISE 2.3

Started by ZiPPy, September 21, 2017, 06:36:03 PM

Previous topic - Next topic

ZiPPy

Anybody running ISE 2.3 in production?  We are currently running ISE 2.1 patch 2.  We've hit a couple bugs, that require patch 5.  I was thinking to just jump to version 2.3 as some people said they had issues with 2.2.  Typically the even number version are more stable, but with ISE it loos like the odd numbers are more stable. 

Basically, our logs have exceeded 3GBs on our policy nodes(a little over 4GBs) and all our policy nodes crashed.  TAC is saying it's a bug.

Cheers,

icecream-guy

Quote from: ZiPPy on September 21, 2017, 06:36:03 PM
Basically, our logs have exceeded 3GBs on our policy nodes(a little over 4GBs) and all our policy nodes crashed.  TAC is saying it's a bug.

Did Cisco also recommend turning of logging?   :smug:  I'd try that.
:professorcat:

My Moral Fibers have been cut.

deanwebb

Do you know which logs are filling up the space? For example, if it's RADIUS logs, you might need to either turn off logging or to distribute the load better. If it's logging for a policy or set of policies, you'll want to look at those policies and see how often they're firing and maybe scale them back a bit to give the system time to take out the trash.

Even with circular logging enabled, logs can fill up their allotted space so fast that the cleanup jobs can't run in time to keep the space from being exceeded. At that point, either reduce the number of logs or spread the load. It's as true for a mail server as it is for anything else that logs stuff.

I've seen RADIUS logging fill up a 60GB space in a matter of 90 seconds on a device that was particularly overtaxed. Not an ISE box, but a CounterACT VCT that was getting all the RADIUS traffic for half the company because of two other servers not being available, and we only defined 3 servers on each WLC. After that, we went to a full 6 RADIUS servers, as the WLC interface would allow.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

ZiPPy

My bad.  I didn't mean to say logs, I meant to say the database.  Specifically the CPMNS database, which TAC can't seem to tell me what the stands for and I didn't find anything online.  Allegedly this is a bug, so my question is why did it all of a sudden start now?  Is it because it did exceed the 3GB mark and it wasn't truncating the logs.  If it even works that way.

deanwebb

Quote from: ZiPPy on September 22, 2017, 12:02:41 PM
My bad.  I didn't mean to say logs, I meant to say the database.  Specifically the CPMNS database, which TAC can't seem to tell me what the stands for and I didn't find anything online.  Allegedly this is a bug, so my question is why did it all of a sudden start now?  Is it because it did exceed the 3GB mark and it wasn't truncating the logs.  If it even works that way.

The *database* is hosed?  :twitch:

That is bad because it is not good.

Interesting that CPMNS is an acronym without explanation... also interesting is how there seem to have been database filling up errors since very early versions of ISE. You need to get your SE in and tell him:

:disappoint:

Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.