OUTAGE REPORTS

Started by deanwebb, May 10, 2015, 08:42:12 AM

Previous topic - Next topic

deanwebb

Just upgraded to 2.0.15. Outage lasted for 0.0000000000000000000001 seconds, bet you didn't even notice it.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

icecream-guy

Quote from: deanwebb on February 01, 2018, 10:51:08 AM
Just upgraded to 2.0.15. Outage lasted for 0.0000000000000000000001 seconds, bet you didn't even notice it.

you are not meeting the 0.99999999999999999999999 up time requirement.
:professorcat:

My Moral Fibers have been cut.

deanwebb

Quote from: ristau5741 on February 02, 2018, 06:38:28 AM
Quote from: deanwebb on February 01, 2018, 10:51:08 AM
Just upgraded to 2.0.15. Outage lasted for 0.0000000000000000000001 seconds, bet you didn't even notice it.

you are not meeting the 0.99999999999999999999999 up time requirement.

:rage:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

Upgrading hosting to a virtual private server, since I was starting to max resources available on current hosting plan. Moves will take place behind the scenes during the day today, and should not be noticeable.

Basically, 118 times in the last month, we hit a max resources condition and could either not serve up a page in time or had to kill a process to get a page up. Going with the VPS option will get me dedicated resources and that will get us closer to our "23 nines" uptime goal.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

We've had a lot of Internal Error 500 errors today. I've done some forum maintenance tasks and have opened up a ticket with Dreamhost. I'll keep you posted on developments.

I went ahead and also mentioned that I've had very slow loading times since going to VPS, so maybe they can fix that, as well.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

Hmm... the 500s aren't happening now, and I'm wondering if the site response is now faster...

If so, well, I guess I should do maintenance tasks more often than once every 3 years... :whistle:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

And now the site is like lightning fast! This is great! The tech support said,

"Checking your account I found the VPS and MySQL server were split between data centers. This split can cause latency issues. I have consolidated your servers into the same data center. You should now see a performance improvement for your sites."

Indeed, I do! Awesome!
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

icecream-guy

Quote from: deanwebb on July 25, 2018, 07:28:02 PM
And now the site is like lightning fast! This is great! The tech support said,

"Checking your account I found the VPS and MySQL server were split between data centers. This split can cause latency issues. I have consolidated your servers into the same data center. You should now see a performance improvement for your sites."

Indeed, I do! Awesome!

gave up redundancy for speed, eh?

site does seem faster now.
:professorcat:

My Moral Fibers have been cut.

deanwebb

They have backups, we don't need HA!

:steamtroll:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

Upgraded to 2.0.17... not a lot of downtime, but I did want to acknowledge the change happened.

Small company change control FTW!

:problem?:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

Outage yesterday: tech support from Dreamhost said to me:

Thank you for contacting DreamHost support. I'm very sorry to hear about
the issues with your site. It looks as if your VPS was hung (not
responding) so I went ahead and restarted it and this fixed the hung
state. Currently your sites are now loading but if you run into issues
like this in the future you may want to try restarting your VPS from the
panel (just make one request as multiple requests can cause issues). If
that doesn't fix the issue let us know and we can take a look for you.

So, next time, I'll know what to do. :D
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

icecream-guy

Quote from: deanwebb on April 29, 2020, 09:40:01 AM
Outage yesterday: tech support from Dreamhost said to me:

Thank you for contacting DreamHost support. I'm very sorry to hear about
the issues with your site. It looks as if your VPS was hung (not
responding) so I went ahead and restarted it and this fixed the hung
state. Currently your sites are now loading but if you run into issues
like this in the future you may want to try restarting your VPS from the
panel (just make one request as multiple requests can cause issues). If
that doesn't fix the issue let us know and we can take a look for you.

So, next time, I'll know what to do. :D

sounds like a SOP to me.
:professorcat:

My Moral Fibers have been cut.

deanwebb

Quote from: ristau5741 on April 29, 2020, 02:56:08 PM
Quote from: deanwebb on April 29, 2020, 09:40:01 AM
Outage yesterday: tech support from Dreamhost said to me:

Thank you for contacting DreamHost support. I'm very sorry to hear about
the issues with your site. It looks as if your VPS was hung (not
responding) so I went ahead and restarted it and this fixed the hung
state. Currently your sites are now loading but if you run into issues
like this in the future you may want to try restarting your VPS from the
panel (just make one request as multiple requests can cause issues). If
that doesn't fix the issue let us know and we can take a look for you.

So, next time, I'll know what to do. :D

sounds like a SOP to me.


Now that I know.

:oracle:
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

On 16 November, there will be an upgrade of the SQL backend during the Pacific Time night - 1AM to 7AM US Pacific time. So, that may mean a bit of inconvenience for our APAC members. Sorry about that. Should only be for a few minutes and then everything gets restored OK.
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.

deanwebb

Planned outage:

On Thursday, August 11 we will be performing proactive maintenance on your MySQL server from 10:00 AM to 6:00 PM PDT. During this maintenance window we anticipate approximately 30 minutes of intermittent connectivity disruptions to that MySQL service. All other DreamHost services will remain unaffected.

There may be issues with the forums on the 11th, off and on. Please bear with. :)
Take a baseball bat and trash all the routers, shout out "IT'S A NETWORK PROBLEM NOW, SUCKERS!" and then peel out of the parking lot in your Ferrari.
"The world could perish if people only worked on things that were easy to handle." -- Vladimir Savchenko
Вопросы есть? Вопросов нет! | BCEB: Belkin Certified Expert Baffler | "Plan B is Plan A with an element of panic." -- John Clarke
Accounting is architecture, remember that!
Air gaps are high-latency Internet connections.