Jump to content


Highest Reputation Content


#7399 Major Outage - 09/21/18+ - Client Discussion

Posted by Bookproc on 26 September 2018 - 12:11 PM

I do NOT want to see "Service is down and will be fixed." I DO want to see a company being transparent and providing information along the way, including ETAs, if and when they are feasible, and explanations of changes in ETAs. In short, MDD provided all the information I wanted, perhaps more than I needed--but more is always better than less for me--and much much more than I could have expected in the circumstances. This was all deeply reassuring.

 

I have been with hosting providers that did the "It's down, it will be fixed" route, and *never again* is all I can say, because the corollary, though not expressed in precisely these terms, is generally "Now shut up and leave us alone, that's all we're going to tell you, and you're just going to have to wait it out without having a clue what's going on or how long it will take. Sucks to be you." I'm stuck with that kind of service with the cable company because their only competitor is just as bad, if not worse. Fortunately, MDD doesn't take that route, and I would never stay with a hosting provider that did

 

Different strokes for different folks.


  • 5


#7395 Major Outage - 09/21/18+ - Client Discussion

Posted by AMC4x4 on 26 September 2018 - 11:45 AM

Just a note in response to "newbie" ... I, too, want to see MDDHosting grow and prosper.  I've used their services many years and referred many people.  But a couple of things:

 

1.  This is 2018, not the mid-2000s when those generalized and generic terms were drafted. 

 

2.  This series of devastating technical errors is more akin to what happened a decade ago. This is 2018.

 

3.  This is not so much about tech stuff, this is about company reputation ... how a person operates their business in 2018.  Perception about a company's reputation is paramount, and Michael's lengthy emails, now reposted on industry websites, won't help MDDHosting's reputation.  When I read them, I thought, "oh, no! Just say the service is down and it will be fixed."

 

4.  Just this week, MDDHosting has disappeared from some rating services as a "top" site host.

 

5.  Why not use your name?

 

Why so hostile? My name is Aaron Clow and I've been an MDD customer since maybe 2013 or so? I have two accounts with them - a corporate account for the company I work for and another account for my personal projects. If you care to look, many of us are "Newbie's" because MDD has had so few issues over the years that we haven't had to post here. "Newbie" is a classification, not my name. Also, when we sign up for a forum account, it doesn't put your name in as your username. You pick one. So I picked one. 

 

If you prefer a hosting company that hides their activities behind a simple "service is down," and that's all the information you require when something untoward happens, I'm sure there are plenty of cheap places you can go for that kind of service. That's not how I do business. I'm forthright with people I speak to, and the reason I'm still with MDD is precisely because they do business this way. Look at how many people are still here and who say they aren't leaving. Why do you think that is?

 

So what's your name? Do people call you "Dehenderson?"

 

Cheers 


  • 3


#7384 Major Outage - 09/21/18+ - Client Discussion

Posted by Dehenderson on 26 September 2018 - 09:58 AM

Removed by Request


  • 3


#7521 Minor Test of Storage Snapshots Post-Disaster

Posted by MikeDVB on 23 October 2018 - 04:51 PM

As many of you already know we experienced a major disaster at the end of September where we were forced to restore data from a backup server due to a misconfiguration of snapshots on our storage platform.  Since this disaster we resolved the snapshot issue and today we were able to use the snapshots to help a client and we couldn't be happier with the experience.

 

We had a client today that had an employee submit a cancellation for services that shouldn't have been canceled.  The result was 43 accounts terminated that shouldn't have been.

 

In the past normally what we'd have done is turned to our backup server, which takes backups once per day, and restored the latest point we could.  This could have been as far as 24 hours prior to the termination of services.

 

What we did as it would give more recent data to the client as well as give us a good test of working with snapshots - was to mount a snapshot of the server just prior to the termination of the services.  We booted the server up, generated backups for the accounts which was exceptionally fast due to the SSD storage, and then restored those backups to the live server.

 

In the event of an actual disaster we'd simply mount the snapshots and boot them up without doing any backups or restorations and services would immediately come back online as they were when the snapshot was taken.

 

All in all we're very happy with this as we were able to provide more recent data to the client than otherwise would have been possible much faster than normally would have been possible.

 

The process was extremely simple and straightforward.  While we don't ever plan on needing snapshots for disaster recovery it is good to know that if we did - they are available and work very well for that purpose.


  • 2


#7383 Major Outage - 09/21/18+ - Client Discussion

Posted by sf2099 on 26 September 2018 - 08:58 AM

For whatever is worth to people considering leaving MDD, I would like to say that I will be staying with MDD.

 

Although I don't share the fanboysim from few individuals here, I know that Mike and his team have been solid since becoming their customer in Jan 2013.

 

Yes, no doubt about it.  This was a big, self-inflicted f***up.  BUT, Mike and his team stepped up and owned it and I believe that deserves some credit.

 

And I like his plans on minimizing the risk of these types of catastrophic events from happening again in the future.  I would like to hear more about backup testings in future.

 

As a small business owner myself, I do like supporting other small businesses.  So as long as Mike delivers on his promises, he has my support.

 

Lastly, what happened here does not absolve myself from being more diligent either.  Here is a screenshot of what Mike told me back in 2013 before I became his customer.  A good advice, still in 2018!

 

 

TSNbakq.jpg


  • 2


#7155 Major Outage - 09/21/18+ - Client Discussion

Posted by djMot on 24 September 2018 - 11:09 AM

 
The websites of two major Internet radio stations are waiting on S2 to come back up.  They were in the process of being redesigned anyway, and the only real lament I have is not having any backups since the redesign began.  But that's on me, and I'm fine with starting over if I have to.  The streams of these two stations are handled through a different streaming media host, so they are still up and broadcasting, it's just that the primary portal where the world finds the stations are 404.  I've posted on Facebook about it, and that's all I can do.
 
I've been with MDD for over 6 years now I think it must be and I do not plan on leaving.  I think this is all just one big "The World According To Garp" story.  The Cessna has just crashed into the bedroom, so I'm buying this house now for sure; i.e., what are the chances this will EVER happen again?
 
Personally, my biggest concern is how much I've relied on the email accounts attached to my various web properties.  I hope I'm not losing mail on the 18-some-odd email accounts attached to all my properties.  A few of those emails are receiving a total of about 1k spam emails per day.  Maybe the undeliverable bounces back to the spammers will get me off their lists.  HA!  Keeping optimistic here.

  • 2


#1673 New Backup Node Online! More Restoration Points

Posted by TotalZen on 07 December 2010 - 09:18 PM

Here's a php script that will perform automated backups of your database(s) and email them to you.

What I did was set up a separate Gmail account for it, and then filter all the emails/attachments to the Trash folder so the account doesn't max out. The trash can automatically deletes emails after 20 days, so the whole thing is on auto-pilot with zero maintenance.

Simply run the script nightly as a cron job...

/usr/bin/php / /home/...(your path goes here).../cron_backup.php

<?php

$datestamp = date("Y-m-d");      // Current date to append to filename of backup file in format of YYYY-MM-DD

/* CONFIGURE THE FOLLOWING SEVEN VARIABLES TO MATCH YOUR SETUP */

$dbuser = "database_user";            // Database username
$dbpwd = "database_pass";            // Database password
$dbname = "database_name";            // Database name. Use --all-databases if you have more than one
$filename= "dump_filename-$datestamp.gz";   // The name (and optionally path) of the dump file
$to = "email_address";      // Email address to send dump file to
$from = "mysql_backup@domain.com";      // Email address message will show as coming from.
$subject = "domain.com MySql Backup For: $datestamp";      // Subject of email

/* CONFIGURATION END */

$command = "mysqldump --opt --quick --skip-extended-insert -u $dbuser --password=$dbpwd $dbname | gzip > $filename";
$result = passthru($command);

$attachmentname = array_pop(explode("/", $filename));   // If a path was included, strip it out for the attachment name

$message = "Compressed database backup file $attachmentname attached.";
$mime_boundary = "<<<:" . md5(time());
$data = chunk_split(base64_encode(implode("", file($filename))));

$headers = "From: $from\r\n";
$headers .= "MIME-Version: 1.0\r\n";
$headers .= "Content-type: multipart/mixed;\r\n";
$headers .= " boundary=\"".$mime_boundary."\"\r\n";

$content = "This is a multi-part message in MIME format.\r\n\r\n";
$content.= "--".$mime_boundary."\r\n";
$content.= "Content-Type: text/plain; charset=\"iso-8859-1\"\r\n";
$content.= "Content-Transfer-Encoding: 7bit\r\n\r\n";
$content.= $message."\r\n";
$content.= "--".$mime_boundary."\r\n";
$content.= "Content-Disposition: attachment;\r\n";
$content.= "Content-Type: application/x-gzip; name=\"$attachmentname\"\r\n";
$content.= "Content-Transfer-Encoding: base64\r\n\r\n";
$content.= $data."\r\n";
$content.= "--" . $mime_boundary . "\r\n";

mail($to, $subject, $content, $headers,"-f $from");

unlink($filename);   // Delete the backup file from the server (optional)

?>

  • 2


#7372 Major Outage - 09/21/18+ - Client Discussion

Posted by SarisIsop on 26 September 2018 - 03:51 AM

All up and running on S5

 

Have lost a few posts on our forum, but it's no big deal and we are just pleased to be back on-line.

 

I would like to say a big thank you to Michael and all the staff and helpers at MDD for getting us out of this. No it shouldn't have happened, but it did and you all rolled up your sleeves on got on with.

 

Michael should be commended for transparency and honesty to his clients, it takes guts to take it on the chin but he has proven that he will always put his customers first.

 

I also want to shout out to the person that made this happen. Don't dwell on it, it's the past and you can not change that. Dust yourself off, hold your head up high and carry on your journey into the future.

 

We are all to blame and I for one will be taking regular cPanel backups, had I have done that I would have been back on-line within a couple of hours instead of waiting till today. Lesson learnt.

 

I will not be leaving MDD. I'm only a small fish but they have always looked after me, sometimes far beyond what I pay them for. I also will not be requesting any refund/compensation as I have lost no money over this and I feel you will be losing enough.

 

I know all of you at MDD have still a lot of work ahead of you and yes you will undoubtedly be on the big side of losing, but this episode has made you stronger and I still have full faith in you. 

 

Sorry to those of you that have been posting your anger especially Avatar for telling me I will just have to "Be Annoyed" I do understand as I once ran my own business, but I just wanted the original thread to be updates and not questions or complaints. Anyway Michael did eventually split the thread and lock it, something I would recommend doing in future.

 

Good luck to everyone whichever way you have decided to go, and thank you MDD.


  • 4