Firewall Refresh Issues

Started by icecream-guy, December 17, 2018, 06:23:20 AM

Previous topic - Next topic

icecream-guy

Did a firewall hardware refresh last Sat. replaced a 5550 with a 5555-X.  this is a HA pair

So I move the connections,  primary firewall is up and working in about 2 minutes. I can ping next hops from both remote and firewall.
Standby firewall is having issues, can't ping management nor outside interfaces from remote
On Secondary, interfaces are up/up, ARP table is populated.
From the firewall, can't ping my outside next hop, management next hop, I _can_ ping my inside next hop.

This is weird,  interfaces up, arp table populated,  2/3 interfaces not passing traffic.
Look at the switch, all looks good there, MAC table is populated, can't ping firewall management interface.

What is going on ?  Frustrated.  so I shut/no shut interface on secondary,  no difference.
what else can I do, interface are up, it should be working, let me reload the firewall, maybe that will help,
firewall reloads, same issue, can't ping up/up interfaces, reload doesn't help. So I'm thinking getting time to call TAC, maybe something not right with the hardware.

I get one more idea,  to see if the issue is standby related, if the issue follows, the new primary should work, and the secondary should not.
So I I fail over the firewall to see if the issue follows. Secondary firewall is now active, and I am able to ping everything both from firewall and remote,
so let me check the secondary, see if I can ping everything again,  why yes, I can now reach all interfaces/next hops from both firewall and remote.
so everything is good, but I need to fail back to make sure the original configuration will continue to work, so I fail the firewall back, primary is now active.
everything is still working.  all good, maintenance is done. successful refresh.  :batdance:
:professorcat:

My Moral Fibers have been cut.

Dieselboy

#1
I had something like this with my 5515-x and I had put it down to something weird with the nexus 3k I have.

What code are you running?


icecream-guy

Quote from: Dieselboy on December 17, 2018, 09:53:50 PM
I had something like this with my 5515-x and I had put it down to something weird with the nexus 3k I have.

What code are you running?

9.7.1.4
:professorcat:

My Moral Fibers have been cut.

deanwebb

Try breaking HA, test, and then rebuild HA.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

icecream-guy

Quote from: deanwebb on December 18, 2018, 10:24:26 AM
Try breaking HA, test, and then rebuild HA.

That makes me so nervous,  I can't tell you how many times I've "ooopsed" and let the default config replicate to the firewall with the running config
Now I #copy run save_my_ass.saved.backup.config on both.
:professorcat:

My Moral Fibers have been cut.

deanwebb

Quote from: ristau5741 on December 19, 2018, 07:04:40 AM
Quote from: deanwebb on December 18, 2018, 10:24:26 AM
Try breaking HA, test, and then rebuild HA.

That makes me so nervous,  I can't tell you how many times I've "ooopsed" and let the default config replicate to the firewall with the running config
Now I #copy run save_my_ass.saved.backup.config on both.

No one lives forever, soldier!

But, yeah, do it carefully. VERY carefully.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Dieselboy

Quote from: ristau5741 on December 19, 2018, 07:04:40 AM
Quote from: deanwebb on December 18, 2018, 10:24:26 AM
Try breaking HA, test, and then rebuild HA.

That makes me so nervous,  I can't tell you how many times I've "ooopsed" and let the default config replicate to the firewall with the running config
Now I #copy run save_my_ass.saved.backup.config on both.

This has happened to me a few times as well. Recently, I re-joined a standby ASA back into a cluster after trying the FTD on the standby member then going back to ASA code. With this config copy in mind and how it's gone the wrong way for me a few times in the past (ie copy the default config and overwrite the real config on the other ASA) I made sure I had "failover lan unit primary" on the primary ASA and "failover lan unit secondary" on the standby ASA before enabling failover with "failover" command. I'm not sure if this is where I had gone wrong in the past, but this time around the config replicated the correct way and I found that there was no longer any need for the butt-clenching  :mrgreen: