Jump to content
MDDHosting Forums

Major Outage - 09/21/18+ - Client Discussion


KevinD872

Recommended Posts

Hi Mike,

Just wanted to say I came to MDD back in 2010 & brought all my clients with me as I've been super happy with MDD's support. It's always a shock to the system when I have to deal with other hosts. I'm not super happy with the situation as it is a nail biter, but mistakes happen as we all know being in the tech business. Thank you so much for keeping everybody in the loop. & for changing the error message, it made me & everyone else look totally stupid.

 

I do have a question - would you recommend going forward we make sure we use the Cloudflare option in cpanel? Do you recommend cname or nameserver. I want to make a change to my system as I have frequent backups mandated but need to add a further layer of protection for my clients (some uncomfortable conversations about how I am going to protect them in the future - & here I am preaching to the choir...).

Thanks!

Link to comment
Share on other sites

 

Understood. That's fine. Go ahead over write (or erase everything first then restore, which ever will bring it back to the same state when the backup was taken.)

I wish I could do it directly from the forums but we'll need it in a ticket and we'll get to it as quickly as we can.

Link to comment
Share on other sites

The old backup server is so inconsistent every ETA we calculated varied wildly so I wasn't comfortable sharing that information and know it's quite likely inaccurate.

 

Kick it to the curb! ;)

 

Hopefully, this is the reason you needed to upgrade equipment and your processes to make sure this NEVER happens again.

Link to comment
Share on other sites

Latest ETA Calculations:

 

s1 Sunday, September 23, 2018 at 11:59:00 PM

p1 Monday, September 24, 2018 at 4:00:00 AM

r1 Monday, September 24, 2018 at 4:00:00 PM

s2 Monday, September 24, 2018 at 4:00:00 PM

p2 Monday, September 24, 2018 at 3:00:00 PM

r2 Tuesday, September 25, 2018 at 2:00:00 AM

s3 Tuesday, September 25, 2018 at 6:00:00 AM

r3 Tuesday, September 25, 2018 at 2:00:00 PM

s4 Tuesday, September 25, 2018 at 5:00:00 PM

r4 Tuesday, September 25, 2018 at 6:00:00 PM

s5 Tuesday, September 25, 2018 at 10:00:00 PM

s0 Tuesday, September 25, 2018 at 10:00:00 PM

 

Thank you! For now clients have been understanding but patience is wearing thin so this is super helpful.

Link to comment
Share on other sites

I have a request to everyone participating in this forum thread: Please keep in mind that if Mike is responding to you, he can't be addressing server issues or answering tickets, despite his valiant attempts to multi-task with little sleep. Some of the questions on here, like, "Are you going to give me a refund?", or "What do you recommend for my backups going forward?" IMO should wait. Even though you are justifiably anxious and participating here helps, your questions are not more urgent than the team getting to tickets (mine and others), helping people figure out how to restore their sites, and continuing to brainstorm with his team on how to improve ETAs.

 

Seeing my server with an ETA of Thursday (even if that is a worst case) certainly does challenge my resolve to be empathetic and calm about all this - but I will stick with it even if it is with clammy hands and shortened breaths, since I know that Mike and team are doing the best they can and are being honorably transparent, and everyone makes mistakes, including me. (I'm just lucky that the weight of the online business world isn't on my shoulders when I have to clean up from mine.)

 

Thanks for hearing me out.

Link to comment
Share on other sites

I have a request to everyone participating in this forum thread: Please keep in mind that if Mike is responding to you, he can't be addressing server issues or answering tickets, despite his valiant attempts to multi-task with little sleep. Some of the questions on here, like, "Are you going to give me a refund?", or "What do you recommend for my backups going forward?" IMO should wait. Even though you are justifiably anxious and participating here helps, your questions are not more urgent than the team getting to tickets (mine and others), helping people figure out how to restore their sites, and continuing to brainstorm with his team on how to improve ETAs.

 

Seeing my server with an ETA of Thursday (even if that is a worst case) certainly does resolve to be empathetic and calm about all this - but I will stick with it even if it is with clammy hands and shortened breaths, since I know that Mike and team are doing the best they can and are being honorably transparent, and everyone makes mistakes, including me. (I'm just lucky that the weight of the online business world isn't on my shoulders when I have to clean up from mine.)

 

Thanks for hearing me out.

I appreciate what you've had to say.

 

That said ETA's were just updated and the latest 100% restoration point is presently calculated as Tuesday at 10PM. It's two posts before yours :).

Link to comment
Share on other sites

I appreciate what you've had to say.

 

That said ETA's were just updated and the latest 100% restoration point is presently calculated as Tuesday at 10PM. It's two posts before yours :).

 

Great to hear, Mike ... I was typing as you posted. Still folks, hold off on the non-urgent, if only for my sanity. ;-)

Link to comment
Share on other sites

Thanks for the estimates. (I don’t know your time zone: maybe the same as my MDT). I know you’re swamped with tickets, so I won’t open one. I’m on s1 and I see ftp is already available and one of my add-ons (oddly enough) is online but my main domain and the other add-on are getting IP errors. An oddity, TBS. I’ll wait it out…

 

Good luck, folks.

Link to comment
Share on other sites

I have a request to everyone participating in this forum thread: Please keep in mind that if Mike is responding to you, he can't be addressing server issues or answering tickets, despite his valiant attempts to multi-task with little sleep. Some of the questions on here, like, "Are you going to give me a refund?", or "What do you recommend for my backups going forward?" IMO should wait. Even though you are justifiably anxious and participating here helps, your questions are not more urgent than the team getting to tickets (mine and others), helping people figure out how to restore their sites, and continuing to brainstorm with his team on how to improve ETAs.

 

While I'm sure they have been bombarded with support tickets, honestly I think this thread has been pretty quiet compared to what it could be. The few complaints we have seen are completely understandable but the fact that most posts have been pretty supportive of the efforts to fix this mess speaks volumes for how great MDDHosting has been for many of us clients over the years; Especially when it would seem that human nature typically results in more people complaining than praising in online forums.

  • Upvote 1
Link to comment
Share on other sites

Sorry about that - we always work by server time which is local time for us. EDT. I updated the post with that and will try to remember to include it for subsequent posts.

 

I'm getting exceptionally sleepy but want to keep clients informed for as long as I can so I should be up for a bit. If I forget to add the time zone feel free to chime in and point out it's EDT or remind me to add it if I forget.

Link to comment
Share on other sites

Today is my 10th anniversary using MDD's services. By far they have been the best webhost I have ever been using and that is why I have total confidence in Mike and his team. This was a human error, we all make those. When the situation is back to normal, it is good to analyze lessons learned from this and reduce the risk of this happening ever again.

 

Keep up the good work!

Link to comment
Share on other sites

I can not provide a new update for the ETA of S1 jsut yet but can only advise we are slowing the backups down a little bit. We actually had so many restore threads going that the restores were conflicting and trying to restore the same account. Once I have a new estimate on the rate that it is going I will update the thread.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...