***CISCO ASA STOPS PASSING TRAFFIC AFTER 213 DAYS 12 HOURS

Started by LynK, March 31, 2017, 09:31:02 AM

Previous topic - Next topic
Print
Go Down Pages1 2
User actions

deanwebb

#1
March 31, 2017, 09:51:43 AM
Seems like this one is making the rounds, and it's a big one, so it's justified.

We're having some issues with unresponsive firewalls and are *downgrading* code to try and resolve the issues.

Ristau has another thread on this, but I'm not going to combine them. This is a big enough issue to warrant two threads.

For those of you just joining us, it may be wise to script a reboot of your ASA firewalls every 212 days.

:facepalm4:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

SofaKing

#2
March 31, 2017, 10:35:48 AM
Yikes - glad we went with PA as well  :P

We are still waiting on our replacement Cisco 43xx routers for the clock signal bug.
Networking -  You can talk about us but you can't talk without us!

Dieselboy

#3
March 31, 2017, 08:31:53 PM
This is BS. Network engineers aren't supposed to reboot stuff.

deanwebb

#4
April 01, 2017, 07:40:35 AM
Quote from: Dieselboy on March 31, 2017, 08:31:53 PM
This is BS. Network engineers aren't supposed to reboot stuff.

Absolutely. That's what sysadmins are for!
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

LynK

#5
April 03, 2017, 07:54:51 AM
Honestly. It is an inconvenience, but the ASA fail-over is nice, and it is very seamless. It is just another thing to throw on the "ASA is crap" list.
Sys Admin: "You have a stuck route"
            Me: "You have an incorrect Default Gateway"

icecream-guy

#6
April 04, 2017, 06:11:49 AM
THIS TIME IT _IS_ THE FIREWALL.

:rofl:
:professorcat:

My Moral Fibers have been cut.

Dieselboy

#7
April 04, 2017, 07:22:08 AM
I received an automated email from Cisco today about this, sent to our group email which the director / company owner received. The email says a fix is out but I've not looked yet. Was busy scouring packet captures for a packet loss issue.  :blank:

icecream-guy

#8
April 04, 2017, 07:45:16 AM
Quote from: Dieselboy on April 04, 2017, 07:22:08 AM
I received an automated email from Cisco today about this, sent to our group email which the director / company owner received. The email says a fix is out but I've not looked yet. Was busy scouring packet captures for a packet loss issue.  :blank:

yes there are fixes,  if you are running ASA 9.7.1 or greater use the arp limiter command
arp rate-limit <value>
this will restart the ARP rate limiter and extend another 5120 hours.

if running less that 9.7.1 reload the device before 5120 hours  (213 days)

:professorcat:

My Moral Fibers have been cut.

Dieselboy

#9
April 04, 2017, 06:48:12 PM
Quote from: ristau5741 on April 04, 2017, 07:45:16 AM
Quote from: Dieselboy on April 04, 2017, 07:22:08 AM
I received an automated email from Cisco today about this, sent to our group email which the director / company owner received. The email says a fix is out but I've not looked yet. Was busy scouring packet captures for a packet loss issue.  :blank:

yes there are fixes,  if you are running ASA 9.7.1 or greater use the arp limiter command
arp rate-limit <value>
this will restart the ARP rate limiter and extend another 5120 hours.

if running less that 9.7.1 reload the device before 5120 hours  (213 days)


Okay so Cisco's email said this:
QuoteWe estimate your device is currently at 33 days of uptime, and will therefore experience this issue in 180 days if no action is taken. Fixed versions of software are now available on the Software Download Center on cisco.com

They gave the bug ID (note number of cases  :XD:)

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvd78303

And then note the "fixed" software versions, for my 9.7.1 it says 9.7.1-4

and cisco.com downloads has 9.7.1-4 ready for the DL: asa971-4-smp-k8.bin
(PS you have to drill down in the downloads to find this release under "inter rim"  :squint:

deanwebb

#10
April 05, 2017, 07:01:14 AM
"Interim" where I work means "wait and see".
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

EOS

#11
April 05, 2017, 07:24:26 AM
Anyone going with the 9.7.1-4?   

We're undecided on which way to go..   and We have a while before the bug is triggered on our ASA's

Dieselboy

#12
April 06, 2017, 11:37:20 PM
I was going to, but haven't got round to it yet. When I went to the first release of 9.7.1 there was a memory page issue that was known to Cisco but an internal-only bug ID. So after reading the release notes and not finding anything alarming then setting up a VTI tunnel and running a few pings I was happy and went home. The next day I had a window before the remote site users arrived in the office, so I set up some static routes to send traffic over the VTI which was set up the night previous. Shortly after doing that, I lost my SSH session to the ASA-X. Luckily for me, I have an active/standby pair. Unluckily for me, when the standby ASA took over, it was of course running the same image as the primary and immediately crashed due to the same issue.

This isn't how it is supposed to be. It kind of devalues the release notes.

What about if I upgrade the primary only, and leave the backup unit as-is? I think the ASA will moan there is a version mismatch but they're both the same major version number. Even going from different version numbers I've managed to do upgrades without breaking anything. I can reboot the backup unit every 6 months and if there's a catastrophic problem with the latest release then the backup will take over. As the versions arent the same, the catastrophic problem shouldnt be there.

deanwebb

#13
April 07, 2017, 03:05:38 PM
We're looking at 9.5.2 as a good, stable release without a lot of baggage. We'll go to 9.7.5 probably after 9.7.7 is released.

:yeahright:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

EOS

#14
April 08, 2017, 07:31:02 AM
We're in the 9.5 range too...  Decision on which version should be decided next week.. 
Print
Go Up Pages1 2
User actions