Jump to content


Photo

[Completed] Fresco Server - Restoration Completed - File System Check Competed


  • Please log in to reply
218 replies to this topic

#21 panchulo

panchulo

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 20 June 2011 - 11:52 AM

May the Force be with you Mike....
  • 0

#22 ender

ender

    Newbie

  • Members
  • Pip
  • 8 posts

Posted 20 June 2011 - 12:06 PM

still no any updates?
  • 0

#23 SpecialOccasions

SpecialOccasions

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 20 June 2011 - 12:21 PM

MDD have been fantastic over the past year for me....but today this is extreme! Its hurting business :( :(


Yeah, MDD Hosting has been pretty rock solid for me so far, been with them about 6 months. They've been responsive, and even when hit with DDoS attacks, they have managed to be back online within 30 minutes in most cases.

This outage, however, has been really awful, relatively speaking. I've been fielding calls and emails all morning from grouchy clients that couldn't care less what a DDoS attack is they just know their site is down, and I'd better fix it.

I'm not going to play pile on not going to make a stressful night/day even worse but if we could at least get an update, that would be appreciated.


  • 0

#24 Software

Software

    Newbie

  • Members
  • Pip
  • 16 posts

Posted 20 June 2011 - 12:28 PM

MDD have been fantastic over the past year for me....but today this is extreme! Its hurting business :( :(




To say the least! No phone support doesn't help either. Any update would be nice right now.
  • 0

#25 panchulo

panchulo

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 20 June 2011 - 12:42 PM

any news ?
  • 0

#26 panchulo

panchulo

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 20 June 2011 - 12:51 PM

this is bad... seriously
  • 0

#27 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 20 June 2011 - 12:51 PM

The file system check left the server in an inoperable condition and, as such, we're going to be performing a full restoration of the server back to around 1:30 AM GMT-5. We'll update this thread as soon as the restoration ETA is known which should be within 30 minutes.
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#28 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 20 June 2011 - 01:16 PM

The restore is in progress and the ETA provided by the system is right at 7 hours. This can increase or decrease as it runs so keep in mind it's an estimated time of completion.

Posted Image
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#29 Software

Software

    Newbie

  • Members
  • Pip
  • 16 posts

Posted 20 June 2011 - 01:17 PM

My web guy said yesterday when he was working on it he got MySQL errors a few times.
  • 0

#30 Software

Software

    Newbie

  • Members
  • Pip
  • 16 posts

Posted 20 June 2011 - 01:19 PM

The restore is in progress and the ETA provided by the system is right at 7 hours. This can increase or decrease as it runs so keep in mind it's an estimated time of completion.

<removed image from quote due to redundancy>



Are websites going to be restored during this time or not until this is all done? I can not afford to lose a day of business. Honestly, I just can't.

Edited by MikeDVB, 20 June 2011 - 01:23 PM.

  • 0

#31 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 20 June 2011 - 01:22 PM

Are websites going to be restored during this time or not until this is all done? I can not afford to lose a day of business. Honestly, I just can't.

It's a restoration of the entire server and all data (a fresh server install does not have 470 GB of data, more like 3 GB). Once this restoration is done everything should come back online without issue.
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#32 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 20 June 2011 - 01:25 PM

The ETA seems to have stabilized around 5 hours 30 minutes. We'll update this thread periodically as the restoration runs.
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#33 benwiggy

benwiggy

    Newbie

  • Members
  • Pip
  • 5 posts

Posted 20 June 2011 - 01:30 PM

MDD have been fantastic over the past year for me....but today this is extreme! Its hurting business :( :(


MDD have been brilliant for me also - and several hours of downtime isn't the end of the world as a result of such a big DDOS attack. What's really important is no loss of data and prompt restoration.

For those who complain of a lack of updates I know Mike won't take a break until everything is up and running again - and I would rather he was working on restoring services than constantly posting updates.

As someone who has experience with half a dozen hosts - I am happy in the knowledge that MDD continues to deliver the best service around - and when attacks like this happen (which they inevitably do) - they are promptly resolved.
  • 0

#34 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 20 June 2011 - 01:31 PM

Current restoration status: 21.5 GB / 470 GB.
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#35 Leah2

Leah2

    Member

  • Members
  • PipPip
  • 29 posts
  • Gender:Not Telling

Posted 20 June 2011 - 01:35 PM

I was wondering what steps can I as a customer take in the future to ensure I don't lose an entire day. Mainly I am concerned about email. If we cannot get or send email our customers move on (yes... they are that impatient) & take their project with them. Not having our website up also hurts our business immensely. :(

I saw you mentioned name servers earlier in the thread. Would purchasing private name servers through you (or using a service like CloudFlare) keep our email going during a DDoS attack like today's and the unfortunate crashing of the server? If yes, how much is it?

BTW - thanks for keeping everyone updated!
  • 0

Electronic Logic Concepts

 

“What is Your Digital Strategy? Websites Built With SEO First Practices”

 

www.ELC-SEO.com


#36 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 20 June 2011 - 01:38 PM

I was wondering what steps can I as a customer take in the future to ensure I don't lose an entire day. Mainly I am concerned about email. If we cannot get or send email our customers move on (yes... they are that impatient) & take their project with them. Not having our website up also hurts our business immensely. :(

Not having your website online hurts us immensely as well, as we have more than just a few offline unfortunately. If email is absolutely critical to your business I'd personally suggest using something like the free edition of Google Apps, although, I'm not sure about POP3/SMTP with that - I do believe you may have to have the paid version for that.

I saw you mentioned name servers earlier in the thread. Would purchasing private name servers through you (or using a service like CloudFlare) keep our email going during a DDoS attack like today's and the unfortunate crashing of the server? If yes, how much is it?

No, your email is hosted on-server. The nameserver message was specific to the specific client and really doesn't even belong in this thread as it's not exactly related.

BTW - thanks for keeping everyone updated!

Sure thing, I wish we didn't have to do this at all but that's unfortunately not something we have control over at this point.

Current Progress: 25.7 GB / 470 GB
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#37 Leah2

Leah2

    Member

  • Members
  • PipPip
  • 29 posts
  • Gender:Not Telling

Posted 20 June 2011 - 01:47 PM

Not having your website online hurts us immensely as well, as we have more than just a few offline unfortunately. If email is absolutely critical to your business I'd personally suggest using something like the free edition of Google Apps, although, I'm not sure about POP3/SMTP with that - I do believe you may have to have the paid version for that.

No, your email is hosted on-server. The nameserver message was specific to the specific client and really doesn't even belong in this thread as it's not exactly related.

Sure thing, I wish we didn't have to do this at all but that's unfortunately not something we have control over at this point.

Current Progress: 25.7 GB / 470 GB


Thanks for the answer. It seemed too easy :)
  • 0

Electronic Logic Concepts

 

“What is Your Digital Strategy? Websites Built With SEO First Practices”

 

www.ELC-SEO.com


#38 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 20 June 2011 - 01:47 PM

Current Progress: 40 GB / 470 GB
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#39 Themeltingclock

Themeltingclock

    Newbie

  • Members
  • Pip
  • 13 posts

Posted 20 June 2011 - 01:55 PM

Current Progress: 40 GB / 470 GB

Ouch. I was hoping to come back from lunch and find that our site was up and running. Looks like an entire day is shot at this point.
  • 0

#40 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 20 June 2011 - 02:02 PM

Ouch. I was hoping to come back from lunch and find that our site was up and running. Looks like an entire day is shot at this point.

It could be worse - we could be like a lot of providers out there and not have backups at all... Thankfully we're not like most. We're still investigating the original cause of the data corruption but I am not sure when we'll have those details. We are doing this restore to fresh, yet identical, hardware just in case it was hardware failure that caused the issue.

Either way, I do apologize deeply to anybody who is experiencing this issue.

Current progress: 53.4 GB / 470 GB
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users