Jump to content


Photo

Major Outage - 09/21/18+ - Client Discussion


  • Please log in to reply
419 replies to this topic

#21 fjpoblam

fjpoblam

    Member

  • Members
  • PipPip
  • 63 posts

Posted 22 September 2018 - 06:21 PM

Please understand, no gripe, just tinfoil hat, okay? 5:12PM MDT and my Wordpress blog (plus two addon sites) are still down. I have full backups for the addons, no problem. For the Wordpress database, blog entries dating back 15 years, I’m nervous that (*if* they’re restored) I shouldn’t also “backup” a blind copy to my “secret” Wordpress.com site. Else I can’t imagine how to maintain an off-host backup.

Ah, well, thanks so much for keeping us up to date pards. I’m sure you know it’s painful for us and I know it’s painful as well for you. I’m sorry my email of sympathy and gratitude generated a ticket. Unintended. Live long and prosper.
  • 0

#22 Arpeggio

Arpeggio

    Member

  • Members
  • PipPip
  • 27 posts

Posted 22 September 2018 - 06:29 PM

Was a little reassuring for me to read of other's sites getting back online, so I'll add that my critical site is back up now (where customers download auxiliary content for things they have already purchased). My other site (where customers can buy stuff) isn't as much an issue for me in comparison because it's not part of content people have already purchased, so I don't mind waiting on that one. Best Regards all.


  • 0

#23 Leah2

Leah2

    Member

  • Members
  • PipPip
  • 29 posts
  • Gender:Not Telling

Posted 22 September 2018 - 06:30 PM

Please understand, no gripe, just tinfoil hat, okay? 5:12PM MDT and my Wordpress blog (plus two addon sites) are still down. I have full backups for the addons, no problem. For the Wordpress database, blog entries dating back 15 years, I’m nervous that (*if* they’re restored) I shouldn’t also “backup” a blind copy to my “secret” Wordpress.com site. Else I can’t imagine how to maintain an off-host backup.

Ah, well, thanks so much for keeping us up to date pards. I’m sure you know it’s painful for us and I know it’s painful as well for you. I’m sorry my email of sympathy and gratitude generated a ticket. Unintended. Live long and prosper.

 

Hi, I find the program backup buddy by ithemes to be really helpful (no affiliation). I send my backups to my dropbox. Hope that helps.


  • 0

Electronic Logic Concepts

 

“What is Your Digital Strategy? Websites Built With SEO First Practices”

 

www.ELC-SEO.com


#24 Leah2

Leah2

    Member

  • Members
  • PipPip
  • 29 posts
  • Gender:Not Telling

Posted 22 September 2018 - 06:32 PM

Please understand, no gripe, just tinfoil hat, okay? 5:12PM MDT and my Wordpress blog (plus two addon sites) are still down. I have full backups for the addons, no problem. For the Wordpress database, blog entries dating back 15 years, I’m nervous that (*if* they’re restored) I shouldn’t also “backup” a blind copy to my “secret” Wordpress.com site. Else I can’t imagine how to maintain an off-host backup.

Ah, well, thanks so much for keeping us up to date pards. I’m sure you know it’s painful for us and I know it’s painful as well for you. I’m sorry my email of sympathy and gratitude generated a ticket. Unintended. Live long and prosper.

 

Hi, I find the program backup buddy by ithemes to be really helpful (no affiliation). I send my backups to my dropbox. Hope that helps.


  • 0

Electronic Logic Concepts

 

“What is Your Digital Strategy? Websites Built With SEO First Practices”

 

www.ELC-SEO.com


#25 skunkbad

skunkbad

    Member

  • Members
  • PipPip
  • 26 posts

Posted 22 September 2018 - 06:42 PM

Do you have plans to compensate customers for "pain and suffering"? Waiting up to 96 hours for a website to come back online is a little much to ask. As a developer, I have my customers calling me and asking about this. We could switch hosts, but wondering if you would give a refund, or how you plan to handle that situation.


  • 0

#26 fjpoblam

fjpoblam

    Member

  • Members
  • PipPip
  • 63 posts

Posted 22 September 2018 - 06:48 PM

@Leah2: Thanks!
  • 0

#27 interworld

interworld

    Newbie

  • Members
  • Pip
  • 5 posts

Posted 22 September 2018 - 08:25 PM

Mike... I haven't bothered to open a support ticket in the spirit of that it would only serve to help slow down the restoration process for everyone. That being said, would opening up a ticket help to get a client's site up any sooner (the squeaky wheel gets the grease theory)? My sites can wait.


  • 1

#28 hippy

hippy

    Newbie

  • Members
  • Pip
  • 5 posts

Posted 22 September 2018 - 08:34 PM

While i am not happy about my sites being down and losing out on sales and quotes because of it. The transparency from you, of what has happened and the updates to the status of progress being made to restore sites has kept me from blowing my top over it.

 

I been with yall for several years now and to this date, you have been the best hosting company i have ever had in my 17 years of hosting companies used. I am completely confident you will continue to be after this disaster has been fixed. 

 

One of the few complaints i have over this though is the landing page on my site now. Would be nice if it didnt make me look like a dumbass who didnt know how to configure a website. It was a SHTF event and overall i am satisfied with the way it has been handled.


  • 1

#29 glennk

glennk

    Newbie

  • Members
  • Pip
  • 5 posts

Posted 22 September 2018 - 09:23 PM

Would it be possible to speak to the percentage of sites that are complete vs those that are left? And what servers are being worked on?


  • 1

#30 npad69

npad69

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 22 September 2018 - 09:29 PM

I have about 4 accounts being hosted at MDD but I'm only worrying about 2 of them. The first one is a government website I am maintaining and good thing the downtime happened on a weekend since the account is being used primarily as their internal system to move things around in their workplace so I hope it gets restored completely before 8am monday (+800). The second one is used primarily to process hundreds of thousands of profile data, this is probably the one my client is going to lose a significant amount of money with since they had to pay a team of data encoders their daily wage and couldn't do anything due to the downtime. I just hope the data backup are as latest as possible since my client would have to pay the team again to redo the stuff they have already done.

 

I know this isn't helping the MDD team restore accounts faster and I'm aware they are doing the best and as honestly as they can, Sorry I just needed to vent out some stuff to ease out my worries. I know we'll just have to wait this out and hope for the best. Well at least nobody died right?


  • 0

#31 Brad

Brad

    Member

  • Members
  • PipPip
  • 29 posts

Posted 23 September 2018 - 12:27 AM

I'm finding it so hard to be chill about this down time.  After 24 hours I was still relatively relaxed about the whole ordeal but now this is leaving me with a bad feeling.  The worst is that the error page guests are seeing make us webmasters look bad.  MDD should be investing some time into maintaining it's clients reputations in the face of potential traffic being lost.  Which is a small ask.  Instead of a giant:

"SORRY!"

"If you are the owner of this website, please contact your hosting provider"

error page it would be great if it instead briefly explained the situation and excused us as webmasters from the RFO.  -_-  MDD should take a bit of a hit for this rather than us.


  • 0

#32 elatum

elatum

    Newbie

  • Members
  • Pip
  • 1 posts

Posted 23 September 2018 - 02:04 AM

Thank you for the update.


  • 0

#33 Mooball

Mooball

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 23 September 2018 - 02:38 AM

My site is great. Redirects to some random other staging site of another domain. That domain when I type it in works perfectly and doesn't redirect to mine... Pretty embarrassing.


  • 0

#34 cziv

cziv

    Member

  • Members
  • PipPip
  • 52 posts
  • Gender:Male

Posted 23 September 2018 - 03:08 AM

Can we have please some update with the % of accounts restored ?

 

What worries me more is the emails lost. If this continues for an other 24 hours, and tomorrow (Monday), i will be in akward situation since all my email is based in one domain here.

 

 


  • 0

#35 bobptz

bobptz

    Newbie

  • Members
  • Pip
  • 9 posts

Posted 23 September 2018 - 03:51 AM

Is it possible (and easy) to get a notification email when our account is restored?

 

After we reveive this email we can make thorow examinations on our sites and report any problems to you.

 

Otherwise, if I see some pages working and some errors elswhere, I may assume that the restoration or the final setup is not finished yet by you.


  • 0

#36 BuckoA51

BuckoA51

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 23 September 2018 - 04:57 AM

I  had hoped to wake up and find my site up again today but no luck... My websites are my only source of income, I'm losing serious money while this outage continues :(


  • 0

#37 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,889 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 23 September 2018 - 04:57 AM

Is it possible (and easy) to get a notification email when our account is restored?

 

After we reveive this email we can make thorow examinations on our sites and report any problems to you.

 

Otherwise, if I see some pages working and some errors elswhere, I may assume that the restoration or the final setup is not finished yet by you.

We are trying to notify everybody but we are moving thousands of accounts and silly enough when one is finished restoring it simply disappears from the list.

 

We have installed a set of solid state drives into the backup server and are in the process of doing a direct in-server copy of the S1 Server backups to these solid state drives.

 

Once this copy is completed these drives will be pulled out and moved into another server for restoration.  This should allow us to cut the restoration time of the S1 server by up to 75%.  Once the S1 server is done copying to these SSDs - we will be doing the same with the next server.

 

Meanwhile we are continuing to do account-level restorations to bring as many people online as quickly as we can.

 

Sorry for the lack of updates - I did manage to get a few hours of sleep.


  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#38 cziv

cziv

    Member

  • Members
  • PipPip
  • 52 posts
  • Gender:Male

Posted 23 September 2018 - 05:29 AM

This is a simple question. After one day and a half of outage, how many % of the accounts or data has been restored ?


  • 0

#39 Decio

Decio

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 23 September 2018 - 05:51 AM

Can we have please some update with the % of accounts restored ?

 

What worries me more is the emails lost. If this continues for an other 24 hours, and tomorrow (Monday), i will be in akward situation since all my email is based in one domain here.

 

 

Yes, a rough % would be nice.

 

About emails, just an idea: Redirect emails to a trial google apps account, for the time being. It you´re using cloudflare it should be very easy, if not, could make this extra step of configuring cloudflare for your domain.


  • 0

#40 Laimonas

Laimonas

    Newbie

  • Members
  • Pip
  • 14 posts
  • Gender:Male
  • Location:European Union
  • Interests:wine

Posted 23 September 2018 - 07:13 AM

Sorry guys, I can't understand one thing : why can't MDD place info: site i down due to MDD Hosting failure. All people think my site was hacked or my webmaster made something wrong and my phone is ringing non stop.
  • 0




3 user(s) are reading this topic

0 members, 3 guests, 0 anonymous users