It's still not the firewall...

Started by deanwebb, December 01, 2016, 11:31:45 AM

Previous topic - Next topic

deanwebb

:mssql:

Yes, it's the database guys again.

:oracle:

Their transactions are slow, so they pulled me into the major incident call to ask if it's the firewall.

Dudes... if it's slow, it's not the firewall. If it's 100% shut down, then, OK, is firewall...

I do this...  :matrix: and prove it's not the firewall and suggest that maybe it's a server or DB issue...

And then they do this...  :zomgwtfbbq: because it's NEVER their stuff...

So, I push back a little more firmly...

:notthefirewall:

And they're engaging in some crack database troubleshooting...

:itcrowd:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

Nerm


deanwebb

And guess what worked after hours of agonizing and asking if it would impact the ability to determine the root cause and when was it last patched and a bunch of stuff like that?

They turned it off and on again.

Now it's all working just fine.

PROTIP: If production is already at a halt because of an issue, rebooting the server isn't going to make it halt more, and will likely resolve things hours faster.

***Turn it off and on again.***
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

Happened again yesterday... funny thing was that, this time, the application could connect to the remote site from some IP addresses, but not others. I tried hitting it from the Interwebs and got blocked.

My verdict: there's an ACL on that remote site, and it needs to permit the IP that our guys are using for their connection.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

NetworkGroover

Quote from: deanwebb on December 01, 2016, 11:31:45 AM

Dudes... if it's slow, it's not the firewall. If it's 100% shut down, then, OK, is firewall...


Clearly you need to lubricate the firewall pipes because the DB packets are catching friction.  How do you still have a job?  ???
Engineer by day, DJ by night, family first always

deanwebb

Quote from: AspiringNetworker on December 05, 2016, 11:26:39 AM
Quote from: deanwebb on December 01, 2016, 11:31:45 AM

Dudes... if it's slow, it's not the firewall. If it's 100% shut down, then, OK, is firewall...


Clearly you need to lubricate the firewall pipes because the DB packets are catching friction.  How do you still have a job?  ???
I have a job because I'm the only one that empties out the token bucket when it fills up. Been 16 years since they last had token-ring on this network and we're STILL getting tokens that pop out empty network jacks. It's unbelievable.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

NetworkGroover

Quote from: deanwebb on December 05, 2016, 02:07:44 PM
Quote from: AspiringNetworker on December 05, 2016, 11:26:39 AM
Quote from: deanwebb on December 01, 2016, 11:31:45 AM

Dudes... if it's slow, it's not the firewall. If it's 100% shut down, then, OK, is firewall...


Clearly you need to lubricate the firewall pipes because the DB packets are catching friction.  How do you still have a job?  ???
I have a job because I'm the only one that empties out the token bucket when it fills up. Been 16 years since they last had token-ring on this network and we're STILL getting tokens that pop out empty network jacks. It's unbelievable.

Do they look like Super Mario coins?  :lol:
Engineer by day, DJ by night, family first always

deanwebb

Yes. Yes they do. Tokens were actually the inspiration for Super Mario.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

icecream-guy

Quote from: deanwebb on December 05, 2016, 02:07:44 PM
Quote from: AspiringNetworker on December 05, 2016, 11:26:39 AM
Quote from: deanwebb on December 01, 2016, 11:31:45 AM

Dudes... if it's slow, it's not the firewall. If it's 100% shut down, then, OK, is firewall...


Clearly you need to lubricate the firewall pipes because the DB packets are catching friction.  How do you still have a job?  ???
I have a job because I'm the only one that empties out the token bucket when it fills up. Been 16 years since they last had token-ring on this network and we're STILL getting tokens that pop out empty network jacks. It's unbelievable.

When what the last time you cleaned out he QoS token buckets? they may be overflowing too..They mostly pop out the empty switch and router ports.
:professorcat:

My Moral Fibers have been cut.

deanwebb

Quote from: ristau5741 on December 06, 2016, 06:28:23 AM
Quote from: deanwebb on December 05, 2016, 02:07:44 PM
Quote from: AspiringNetworker on December 05, 2016, 11:26:39 AM
Quote from: deanwebb on December 01, 2016, 11:31:45 AM

Dudes... if it's slow, it's not the firewall. If it's 100% shut down, then, OK, is firewall...


Clearly you need to lubricate the firewall pipes because the DB packets are catching friction.  How do you still have a job?  ???
I have a job because I'm the only one that empties out the token bucket when it fills up. Been 16 years since they last had token-ring on this network and we're STILL getting tokens that pop out empty network jacks. It's unbelievable.

When what the last time you cleaned out he QoS token buckets? they may be overflowing too..They mostly pop out the empty switch and router ports.


Not cleaning those out is why we had to build a new datacenter with automatic chutes for collection and disposal. The old datacenter is currently unusable space until the hazmat teams can clean out all the tokens and still-smoldering gear that didn't have good ventilation because of all the tokens that piled up in there.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

wintermute000

just get bigger pipes, then you won't have to clean them because the tokens won't clog them up as much.

deanwebb

I know. I really like these new OC-28800 lines with emergency shutoff taps...

Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.