43 replies to this topic

[MikeDVB]

80 of 309 GBs restored.

[nbaintel]

Can you tell us what time the backups were made last night?

[derekd]

Well, this stinks...I was about to heap praise on this company for its reliability to some friends. Doh.

Any idea what caused a relatively recently deployed, low occupancy server to suddenly 'blow up'?

[MikeDVB]

Can you tell us what time the backups were made last night?

Around 1:30 am gmt-5 (EST).

Well, this stinks...I was about to heap praise on this company for its reliability to some friends. Doh.

Any idea what caused a relatively recently deployed, low occupancy server to suddenly 'blow up'?

We believe it was a hardware issue with the raid controller and we swapped it out before beginning the restoration.

[MikeDVB]

135 of 309 GBs restored.

[MikeDVB]

160 of 309 GBs restored. ETA 58 minutes.

[nestito]

Well, this stinks...I was about to heap praise on this company for its reliability to some friends. Doh.

Any idea what caused a relatively recently deployed, low occupancy server to suddenly 'blow up'?

I noticed that I couldn't reach any of my websites tonight so I decided to first check Server Status and Twitter to see if there was something going on globally. Sure nuff, there was and MDD was handling it.

MDD is reliable. This sort of thing can happen to any host. What sets a good host apart from from other in a time like this is there customer service and the staff is doing a good job. They are doing good by keeping us updated on what's going on with Gemini. Let them do their thing and fix the issue :-)

[MikeDVB]

Indeed, we're at 200 of 309 GB restored so it shouldn't be much longer. We don't anticipate any issues booting up the server once the restoration is completed but we're keeping extra staff on hand just in case.

[MikeDVB]

225 of 309 GB Restored. ETA 32 Minutes.

[MikeDVB]

245 of 309 GB restored.

[MikeDVB]

280 of 309 GBs restored, the system says around 11 Minute ETA.

[MikeDVB]

The server is booting up now.

[MikeDVB]

The server has fully booted up and all services are fully operational at this time. If you have any issues at all do please open a support ticket.

[rowanhippy]

Thanks very much Mike and the team for keeping us fully updated with what was going on and rectifying the problem. Its very much appreciated.

[MikeDVB]

Thanks very much Mike and the team for keeping us fully updated with what was going on and rectifying the problem. Its very much appreciated.

Absolutely - we identified the cause of the failure as a raid controller and we swapped the controller out prior to restoring the data. We're going to have LSI (the manufacturer) analyze the card to see if they can identify the cause as all the rest of the hardware checked out OK.

[nbaintel]

While it's good that you were able to recover files from last night.

Many customers, including myself, lost several hours of work which is quite depressing.

The up time for May is now down to 99.45% which is below your guaranteed 99.9%.

I believe you should compensate those who are effected.

[MikeDVB]

Losing hours of work is better than losing all work (i.e. all data). I'm not saying that things worked out how we wanted them to or that we're happy about it - but the current resolution is better than many providers would have been able to offer.

I do apologize for the trouble, however, we have no control over hardware failure unfortunately. It will inevitably happen no matter what provider you are with or what kind of systems they are running. We're operating an expensive backup infrastructure just for this specific type of instance and it's paid off in this situation.

If you have any specific questions, do please open a ticket.

[Al Stewart]

I just signed up with MDDhosting and this thread has dropped my jaw! I left enotch.com because they have been down 3 days and not one word was sent explaining the outage or the ETA of the fix. Even their own web sites went away! Wow! Talk about keeping customers informed!

[Charles]

Mike has went above and beyond to keep everyone informed of what was going on along with the status of the restore. Not many hosts would provide updates as often as Mike has.

There are even some hosts that claim to have backups but when it comes right down to it, they have backups that are over 2 weeks old or none at all!!!

Mike thanks for keeping everyone updated and for having backups like you do.

[MikeDVB]

We are very sorry for anybody who has lost any data - we wish we could bring it back and we did our absolute best to keep it before going to the last resort which is a full server restoration.