505MediaGroup Posted September 24, 2018 Report Share Posted September 24, 2018 P1 is back, thank you MDD! I know you're working ****** this and I appreciate you guys and all the great service over the years. I wanted to share this just so that everyone knows that in the world of hosting and IT every provider has issues. So, for my P1 domains I went into GoDaddy to update the new IP in DNS and guess what? There DNS servers are down. That's right, even the mighty GoDaddy has issues!!http://thearagonagency.com/2018-09-24_0827.png​http://thearagonagency.com/2018-09-24_0827_001.png If you've been doing this for any amount of time you that this happens. I've been at it since 2000, for 18 years and I've been with many hosting providers. No other has even come close to MDD hosting in service and support. I know when they come back from this they will be even better. I'm sticking around and I hope you do too. Quote Link to comment Share on other sites More sharing options...
cziv Posted September 24, 2018 Report Share Posted September 24, 2018 GUYS DO NOT POST SPECIFIC PROBLEMS HERE. OPEN A TICKET !!!!! 3 Quote Link to comment Share on other sites More sharing options...
KevinD872 Posted September 24, 2018 Author Report Share Posted September 24, 2018 Ok, in light of Interworld's comment, I guess there is/was a page like that, but I am not able to see it now for the reasons he mentioned. And maybe I never saw it before because of the root directory exception?If I'm following you correctly, it sounds like you were expecting 404 error pages to be similarly worded to the default page. I believe they only changed the default page. There are 3 or 4 other pages that could be changed to be less scary sounding. MDD: If this wouldn't take any time from performing the backups, would it be possible to replace all of the default pages (or also the 404 page at a minimum) to something closer to the default page you guys put in place of the original? Quote Link to comment Share on other sites More sharing options...
Stéphane Bergeron Posted September 24, 2018 Report Share Posted September 24, 2018 I'm with giantjack. Customers are pressuring us to leave MDDHosting and we have 6 client sites still down and all on S2 with no good news to give them. That this will go on all day still is placing us in a very uncomfortable position and the restoration process is indeed painfully slow. At the very least I would like more frequent updates on what is happening like what servers are being restored, etc. Frequent updates is really the least you can do at this point as most of us here are obviously defending you with our own angry clients because of your amazing track record in the past. But with some clients, that can only go so far and we need information to manage expectations. Especially since I had a crazy week planned already with no time to spare and I now have to manage this and field angry client calls when I should really be concentrating on other things... 1 Quote Link to comment Share on other sites More sharing options...
spocklikesspores Posted September 24, 2018 Report Share Posted September 24, 2018 My friend that has website + email with mdd has a backup of the website. I/he could open a ticket about a special restore effort for that; but what he really cares about is the email, not the website. I can't see how restoring the website will restore his email config at mdd. So what backup is it that people are talking about that would speed the return of email? Quote Link to comment Share on other sites More sharing options...
KevinJones Posted September 24, 2018 Report Share Posted September 24, 2018 KevinJones... for me there was a bit of a lag between cPanel and FTP working and the site itself coming up. Also the old 404 page is a good sign as it means your site is coming back up. Keep clearing things and if your site is bookmarked, make sure the http is typed in the browser. This was the scenario for me last night.Interworld, how long of a lag was there for you? It has been an hour and a half, and I still have no HTTP access. I am wondering in case there might be some other issue, but I don't want to jump the gun and open a ticket if it is not necessary. Again, I don't believe browser caching is an issue here (cleared cache, tried curl) Quote Link to comment Share on other sites More sharing options...
cziv Posted September 24, 2018 Report Share Posted September 24, 2018 I'm with giantjack. Customers are pressuring us to leave MDDHosting and we have 6 client sites still down and all on S2 with no good news to give them. That this will go on all day still is placing us in a very uncomfortable position and the restoration process is indeed painfully slow. At the very least I would like more frequent updates on what is happening like what servers are being restored, etc. Frequent updates is really the least you can do at this point as most of us here are obviously defending you with our own angry clients because of your amazing track record in the past. But with some clients, that can only go so far and we need information to manage expectations. Especially since I had a crazy week planned already with no time to spare and I now have to manage this and field angry client calls when I should really be concentrating on other things... Same here. 8 sites total, 2 mine, S2 and S3. This is definitely not good. Very slow process. 4 days. Quote Link to comment Share on other sites More sharing options...
SarisIsop Posted September 24, 2018 Report Share Posted September 24, 2018 At the very least I would like more frequent updates on what is happening like what servers are being restored, etc. Updates unfortunately are getting lost amongst all the other posts in this thread. Eric just posted this: Current ETA based on the progress up till now(Eastern Time):s1 Completedp1 Completedr1 Monday, September 24, 2018 at 11:00:00 PMp2 Monday, September 24, 2018 at 3:00:00 PMs2 Tuesday, September 25, 2018 at 1:00:00 AMr2 Tuesday, September 25, 2018 at 9:00:00 AMs3 Tuesday, September 25, 2018 at 4:00:00 PMr3 Wednesday, September 26, 2018 at 12:00:00 AMs4 Wednesday, September 26, 2018 at 4:00:00 AMr4 Wednesday, September 26, 2018 at 6:00:00 AMs5 Wednesday, September 26, 2018 at 12:00:00 PMs0 Wednesday, September 26, 2018 at 12:00:00 PM Quote Link to comment Share on other sites More sharing options...
KevinD872 Posted September 24, 2018 Author Report Share Posted September 24, 2018 (edited) My friend that has website + email with mdd has a backup of the website. I/he could open a ticket about a special restore effort for that; but what he really cares about is the email, not the website. I can't see how restoring the website will restore his email config at mdd. So what backup is it that people are talking about that would speed the return of email? Someone from MDD can confirm, but I believe what they were saying is that you can open a ticket to have a new blank cPanel account created for you and once you do that, you could then configure your email settings. I believe they also said that they would not overwrite anything (i.e. your email stuff) unless you specifically tell them to do so in a ticket. Here is the problem with this: If you use IMAP email, it would likely cause some problems if all of those emails (or any IMAP sub-folders) are suddenly missing. My entire family uses IMAP email through one of my domains and I have two others for business-related things. As much as I would like to restore my email, it is very complicated for me. I also use lots of filters. Edit: The backup they were referring to in this scenario is that if YOU have a local backup of your entire website, you can have them create the cPanel account and then you could upload your own backup (via FTP, etc) to restore your website quicker yourself. Edited September 24, 2018 by KevinD872 Quote Link to comment Share on other sites More sharing options...
webeye Posted September 24, 2018 Report Share Posted September 24, 2018 Current ETA based on the progress up till now(Eastern Time):s1 Completedp1 Completedr1 Monday, September 24, 2018 at 11:00:00 PMp2 Monday, September 24, 2018 at 3:00:00 PMs2 Tuesday, September 25, 2018 at 1:00:00 AMr2 Tuesday, September 25, 2018 at 9:00:00 AMs3 Tuesday, September 25, 2018 at 4:00:00 PMr3 Wednesday, September 26, 2018 at 12:00:00 AMs4 Wednesday, September 26, 2018 at 4:00:00 AMr4 Wednesday, September 26, 2018 at 6:00:00 AMs5 Wednesday, September 26, 2018 at 12:00:00 PMs0 Wednesday, September 26, 2018 at 12:00:00 PM This seems to be a sliding scale. So that one does not need to sift through pages and pages of posts to find your latest recovery time objective, can the current RTO be stickied? A total of 4 to 5 days down - and that is still not certain - for me. Google will start to drop the sites from its index, email not getting through and most smtp servers stop trying after 24 - 48 hours. I've opened a hosting account with another provider to at least get emails up and a few main web pages, but dns propagation takes 24 - 48 hours. In hindsight, I should have done this earlier. Quote Link to comment Share on other sites More sharing options...
LazyDodo Posted September 24, 2018 Report Share Posted September 24, 2018 For people that end up on the defaultwebpage.cgi page on P1, I figured out what was/is wrong (at-least for my site),the IP address of the server changed, and since my DNS records were set with a TTL of 4 hours some users will still have the wrong IP in their local dns cache (or their upstream dns servers) for a couple of more hours, and they end up on the the defaultwebpage.cgi page. Quote Link to comment Share on other sites More sharing options...
spocklikesspores Posted September 24, 2018 Report Share Posted September 24, 2018 Thanks Kevin that makes sense. We do have a backup of the website, but I correctly didn't think that was going to help with email. Your reply makes it clear that the half step for email would be made via a new cpanel. I will have to open a ticket with mdd to go over the risks/integration. They've pushed the restore times back and that has me worried. Quote Link to comment Share on other sites More sharing options...
richardmichael Posted September 24, 2018 Report Share Posted September 24, 2018 I've opened a hosting account with another provider to at least get emails up and a few main web pages, but dns propagation takes 24 - 48 hours. In hindsight, I should have done this earlier.Hey There, Wherever you host your DNS, lower the TTL as low as possible. Additionally, if you haven't already you should sign up to Cloudflare (it is free). Because your visitors will be going through Cloudflare from here on out, there won't be any secondary propagation for them later. They will keep visiting via Cloudflare, while you inform Cloudflare of where your website has moved to. If I was in your shoes, that is exactly what I would do. Quote Link to comment Share on other sites More sharing options...
richardmichael Posted September 24, 2018 Report Share Posted September 24, 2018 Double post. Quote Link to comment Share on other sites More sharing options...
cziv Posted September 24, 2018 Report Share Posted September 24, 2018 This seems to be a sliding scale. So that one does not need to sift through pages and pages of posts to find your latest recovery time objective, can the current RTO be stickied? A total of 4 to 5 days down - and that is still not certain - for me. Google will start to drop the sites from its index, email not getting through and most smtp servers stop trying after 24 - 48 hours. I've opened a hosting account with another provider to at least get emails up and a few main web pages, but dns propagation takes 24 - 48 hours. In hindsight, I should have done this earlier. Since used to work on the IT, it seems to me that a lot of bad decisions were made about how to restore the servers. A lot of time was wasted. Apparently they had no plan about it and they were trying things and at some point they changed their strategy. Anyway the google problem is with me also, started yesterday to get 404 increase notifications from google webmaster tool. And also a huge ammount of email is simply lost. Gone. If i knew that the restore was gone be more and more delayed, i would have switched to something else with a week old backup. But didn't expected that it would take a week to restore everything. And i'm pretty sure after the restore, there would be still some problems to resolve. Quote Link to comment Share on other sites More sharing options...
starki Posted September 24, 2018 Report Share Posted September 24, 2018 Is there a reason why the restoration sequence was changed? S2 was moved to after P2 in contrast to the previous annoucement. It's not a major difference in time, but neverless that's a detail a bit irritating in a sensible situation like this where people are losing money and/or customers. Quote Link to comment Share on other sites More sharing options...
KevinJones Posted September 24, 2018 Report Share Posted September 24, 2018 FWIW, http://p1.temporary-access.com gives 404 error. Quote Link to comment Share on other sites More sharing options...
djMot Posted September 24, 2018 Report Share Posted September 24, 2018 The websites of two major Internet radio stations are waiting on S2 to come back up. They were in the process of being redesigned anyway, and the only real lament I have is not having any backups since the redesign began. But that's on me, and I'm fine with starting over if I have to. The streams of these two stations are handled through a different streaming media host, so they are still up and broadcasting, it's just that the primary portal where the world finds the stations are 404. I've posted on Facebook about it, and that's all I can do. I've been with MDD for over 6 years now I think it must be and I do not plan on leaving. I think this is all just one big "The World According To Garp" story. The Cessna has just crashed into the bedroom, so I'm buying this house now for sure; i.e., what are the chances this will EVER happen again? Personally, my biggest concern is how much I've relied on the email accounts attached to my various web properties. I hope I'm not losing mail on the 18-some-odd email accounts attached to all my properties. A few of those emails are receiving a total of about 1k spam emails per day. Maybe the undeliverable bounces back to the spammers will get me off their lists. HA! Keeping optimistic here. 2 Quote Link to comment Share on other sites More sharing options...
interworld Posted September 24, 2018 Report Share Posted September 24, 2018 Interworld, how long of a lag was there for you? It has been an hour and a half, and I still have no HTTP access. I am wondering in case there might be some other issue, but I don't want to jump the gun and open a ticket if it is not necessary. Again, I don't believe browser caching is an issue here (cleared cache, tried curl) Not sure if the lag time wasn't caused by me using my bookmark... it didn't have the https:// in it. When I typed my url in by hand, my site came right up. Could have been a coincidence. Quote Link to comment Share on other sites More sharing options...
Stéphane Bergeron Posted September 24, 2018 Report Share Posted September 24, 2018 Updates unfortunately are getting lost amongst all the other posts in this thread. Eric just posted this: Current ETA based on the progress up till now(Eastern Time):s1 Completedp1 Completedr1 Monday, September 24, 2018 at 11:00:00 PMp2 Monday, September 24, 2018 at 3:00:00 PMs2 Tuesday, September 25, 2018 at 1:00:00 AMr2 Tuesday, September 25, 2018 at 9:00:00 AMs3 Tuesday, September 25, 2018 at 4:00:00 PMr3 Wednesday, September 26, 2018 at 12:00:00 AMs4 Wednesday, September 26, 2018 at 4:00:00 AMr4 Wednesday, September 26, 2018 at 6:00:00 AMs5 Wednesday, September 26, 2018 at 12:00:00 PMs0 Wednesday, September 26, 2018 at 12:00:00 PM I see those, what I'd like to know is what is being done right now and the approximate progress % and frequent updates of the same. Our clients are requesting more info and I have none to give them. Quote Link to comment Share on other sites More sharing options...
Michael D. Posted September 24, 2018 Report Share Posted September 24, 2018 Is there a reason why the restoration sequence was changed? S2 was moved to after P2 in contrast to the previous annoucement. It's not a major difference in time, but neverless that's a detail a bit irritating in a sensible situation like this where people are losing money and/or customers.We are simply doing things as quickly as we can. We had a short lapse in the ability to move the disks from Slow Backup to Fast Backup and in this lapse we went ahead and copied the smallest server which is S2. We did this in lieu of just sitting around doing nothing. If P2 was the smallest server it still would have been copied at that point. This is one of the reasons the ETAs are only estimates - because we are doing our best to predict how long it will take to copy and restore each servers' backups and this changes based upon the actual data being copied. For example a server with 1TB of data usage and 150,000,000 files is going to take a LOT longer to copy than a server with 4TB of data and 25,000,000 files. If you have issues with your site or account after the restoration you will need to open a support ticket. While I wish we could keep up with individual issues here on these forums it's not feasible. We have extra staff working on the helpdesk and we're doing our absolute best to keep up considering the ticket load. If your site is restored and you are seeing a cPanel error page there is a good chance your account is not on the same IP as it was and you're using third party DNS. If you log into your cPanel you can see your new site IP in the status bar or under 'Server Information'. You'll want to update this at your DNS. Originally we planned on trying to make sure everybody was put back on their original IPs but the work to do that would double the restoration time or more. 1 Quote Link to comment Share on other sites More sharing options...
Michael D. Posted September 24, 2018 Report Share Posted September 24, 2018 I see those, what I'd like to know is what is being done right now and the approximate progress % and frequent updates of the same. Our clients are requesting more info and I have none to give them.We're not keeping up with support tickets as it is - if we were to start providing this level of detail on the restore it would mean pulling somebody off of tickets and slowing down our rate of support by a fair margin. I will see if we can't provide some sort of periodic update but the ETA's are for full restoration of each server. If it's feasible to provide additional information we will do so. Quote Link to comment Share on other sites More sharing options...
tannersj Posted September 24, 2018 Report Share Posted September 24, 2018 The websites of two major Internet radio stations are waiting on S2 to come back up. They were in the process of being redesigned anyway, and the only real lament I have is not having any backups since the redesign began. But that's on me, and I'm fine with starting over if I have to. The streams of these two stations are handled through a different streaming media host, so they are still up and broadcasting, it's just that the primary portal where the world finds the stations are 404. I've posted on Facebook about it, and that's all I can do. I've been with MDD for over 6 years now I think it must be and I do not plan on leaving. I think this is all just one big "The World According To Garp" story. The Cessna has just crashed into the bedroom, so I'm buying this house now for sure; i.e., what are the chances this will EVER happen again? Personally, my biggest concern is how much I've relied on the email accounts attached to my various web properties. I hope I'm not losing mail on the 18-some-odd email accounts attached to all my properties. A few of those emails are receiving a total of about 1k spam emails per day. Maybe the undeliverable bounces back to the spammers will get me off their lists. HA! Keeping optimistic here. All I was concerned about was a few of my emails. They created the temp account, I re-created my emails which took about 5 mins and they're working great! Quote Link to comment Share on other sites More sharing options...
KevinJones Posted September 24, 2018 Report Share Posted September 24, 2018 Wanted to mention, after waiting 2.5 hours after P1 was restored and I had cpanel/ftp, my HTTP still wasn't working. I finally filed a ticket and found out I had to point my dns to a different ip. So while it is good to be patient even after your account is partially restored, it may be well to file a ticket if it is taking a long time to access your webpage, as there may be some other issue at play. Quote Link to comment Share on other sites More sharing options...
Plippers Posted September 24, 2018 Report Share Posted September 24, 2018 Mike / Tim. Maybe you guys could mention the IP changes after each transfer completes (sticky the new IPs somewhere). Seems that's the issue catching the transfered people out at the moment. Especially for those using CloudFlare or other external DNS. Cheers! 1 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.