Upgraded Hardware and Data Center Transfers

[Michael D.]

We are always striving to give our customers the absolute best service possible and as a result we're always looking for ways to improve the quality of our services. We are going to be transferring to newer and more powerful equipment in a new data center facility.

 

We've created a knowledgebase section, https://www.mddhosting.com/support/knowledgebase/21/Datacenter-Transfer, with common questions and answers. If you have any questions that you do not see answered in the knowledgebase please feel free to respond to this email with your questions. We want you to be comfortable with this upgrade and as excited as we are.

 

We're very good at transferring accounts and the move should be fairly transparent and you should not notice any discernable downtime. All Shared, Reseller, and Semi-Dedicated transfers have been scheduled from November 18th to November 25th and we will be moving accounts in stages. VPS Transfers have been scheduled from December 18th to December 25th. All transfers will be conducted during off-peak times from 9 PM EST to 7 AM EST.

 

If you are using external DNS or private nameservers, you will need to get with us so that we can be sure to update you directly with the new IP addresses to avoid any downtime. All clients will receive updated account information including new IP addresses and DNS IP addresses once the transfer is completed. If you aren't sure whether you are using external dns or private nameservers, then chances are that you're not however you are welcome to respond to this email and we can check for you!

 

We are moving all accounts to servers running Dual Quad Core Xeon 5620 Processors which have Hyper-Threading offering a total of 16 Logical cores per server with a minimum of 12 GB of system ram. This transfer is going to be a very nice performance upgrade for all accounts and we're really anxious to get everybody moved over.

 

As you likely know all of our current services are hosted out of the Softlayer Dallas facility and Softlayer has always been and always will be an excellent facility with top notch support, networking, and redundancy. We've purchased several servers outright and unfortunately Softlayer does not offer the ability for us to place our own hardware in their facility.

 

We're moving all of our services to the HandyNetworks facility located in Denver, Colorado and this will allow us to do many things that we've previously been unable to do at Softlayer. Rest assured that quality of service was the top priority in our minds when we were investigating new facilities and that we've spent several months communicating with several facilities before finally choosing HandyNetworks.

 

There are many exciting upgrades and changes coming over the next year and we're really looking forward to continuing to upgrade the quality of service and support that you receive.

 

If you have any questions at all about the new hardware, the new facility, the transfer process, or anything else at all - please do feel free to open a support ticket or to respond to this thread and we'll be more than happy to go over everything with you.

 

Thank you,

[KevinD872]

I had a look at your FAQ about this but didn't really find my answers:

 

1) Are you moving EVERYTHING related to each site, including any custom data in folders above the main www folder?

 

2) I know you can't send us an email with an exact time of transfer, but are you going to at least (for example) send an email to all hosting customers on server X maybe a day before and after the transfer of server X to the new one?

 

3) How old will the backup/data be that will be transferred to the new server?

 

4) How much downtime can we expect during the move? I.E. no email, etc.

 

 

I've only been with MDD for a short time and while I have been impressed and happy so far, I do have some concerns as posted above. One of the many reasons that I ultimately left my old hosting company after about 9 years was a badly botched server move.

 

You can skip the rest of this rather lengthy post as the following just further explains my reasons for being concerned. If interested, read on....

 

Specifically to my questions above:

#1: They failed to move my customer data that was stored at a level above my WWW folder (for security reasons). The result was that I ended up having to use the only backup I had, which was almost a year old. I lost almost a year's worth of customer passwords that I had no way of recovering. It also broke almost every CGI script on my site because certain key paths had changed and other files didn't get copied over.

 

#2: They had sent us an email stating our servers would be moved on a date about a month away and that we would be notified again a day or two before the move. I planned to backup my customer and other important data when I got that email, but they never notified anyone, and in fact they moved the servers before the date they said they would. Many of their customers woke up to a huge mess on their hands and they were no help to anyone. I was told that they weren't responsible for copying "extra" data or fixing CGI script-related problems on my site, even though it all worked fine before the move.

 

#3: As I said earlier, they hadn't even backed up or moved my customer data above the WWW folder, so I was burned pretty badly anyway, but I also have a 'comments' section on my site that is updated several times daily. After my site was transferred, I found that the data they had put on the new server was over a week old.

 

#4: Because things were botched so badly on their end and they broke several functions (including the ability for me to EDIT any of the html pages or folders in my site), I pretty much had nothing for about a week. I had some intermittent email functionality after a couple days of nothing, but they also failed to transfer ALL email accounts/redirects/forwards.

 

 

Again, I don't expect these things to happen with MDD. But I am hoping that by sharing my bad experiences with my previous host, you will understand why I am concerned. While my site may not be anything nearly as critical as lots of other sites on the web, it's my site and has been around since 2001 and has excellent search engine rankings. It's important to me.

[Michael D.]

[quote name='KevinD872' timestamp='1288810509' post='1555']1) Are you moving EVERYTHING related to each site, including any custom data in folders [i]above[/i] the main www folder?[/quote]Everything in your account is being moved including any and all files you've uploaded no matter where they're located in your account.

[quote name='KevinD872' timestamp='1288810509' post='1555']2) I know you can't send us an email with an exact time of transfer, but are you going to at least (for example) send an email to all hosting customers on server X maybe a day before and after the transfer of server X to the new one?[/quote]We will be keeping everybody up to date as to what is going on. As we've said we can't let you know when your specific account is being copied we will be keeping the transfer status updates public.

[quote name='KevinD872' timestamp='1288810509' post='1555']3) How old will the backup/data be that will be transferred to the new server?[/quote]We're not copying backup data, we're copying your live data.

[quote name='KevinD872' timestamp='1288810509' post='1555']4) How much downtime can we expect during the move? I.E. no email, etc.[/quote]If you're using our DNS you shouldn't notice any issues and if you're using your own or external DNS the only real potential for issues is if you don't update your DNS records quickly when we provide you with the new information.

[quote name='KevinD872' timestamp='1288810509' post='1555']I've only been with MDD for a short time and while I have been impressed and happy so far, I do have some concerns as posted above. One of the many reasons that I ultimately left my old hosting company after about 9 years was a badly botched server move.[/quote]This move isn't anything particularly special - to be honest we probably could have conducted the entire move with only a very small percentage of our client base noticing but that's simply not how we operate. We're letting everybody know about the move as we believe in being open and honest about everything that we do.

[quote name='KevinD872' timestamp='1288810509' post='1555']Specifically to my questions above:
#1: They failed to move my customer data that was stored at a level [i]above[/i] my WWW folder (for security reasons). The result was that I ended up having to use the only backup I had, which was almost a year old. I lost almost a year's worth of customer passwords that I had no way of recovering. It also broke almost every CGI script on my site because certain key paths had changed and other files didn't get copied over.[/quote]As I explained above, the entire account will be copied exactly as it is. The old server will be left online for some time as will the old backup system should you notice anything didn't copy over correctly. The new servers will be configured exactly like the old ones so there won't be any changes in system paths or anything like that.

[quote name='KevinD872' timestamp='1288810509' post='1555']#2: They had sent us an email stating our servers would be moved on a date about a month away and that we would be notified again a day or two before the move. I planned to backup my customer and other important data when I got that email, but they never notified anyone, and in fact they moved the servers before the date they said they would. Many of their customers woke up to a huge mess on their hands and they were no help to anyone. I was told that they weren't responsible for copying "extra" data or fixing CGI script-related problems on my site, even though it all worked fine before the move.[/quote]You don't have to worry about that - it's not how we operate.

[quote name='KevinD872' timestamp='1288810509' post='1555']#3: As I said earlier, they hadn't even backed up or moved my customer data [i]above[/i] the WWW folder, so I was burned pretty badly anyway, but I also have a 'comments' section on my site that is updated several times daily. After my site was transferred, I found that the data they had put on the new server was over a week old.[/quote]From everything you've already said I think it goes without saying that they did a very bad job of transferring everybody and everything over. We've done thousands of transfers over the last few years so we're pretty good at it.

[quote name='KevinD872' timestamp='1288810509' post='1555']#4: Because things were botched so badly on their end and they broke several functions (including the ability for me to EDIT any of the html pages or folders in my site), I pretty much had nothing for about a week. I had [i]some[/i] intermittent email functionality after a couple days of nothing, but they also failed to transfer ALL email accounts/redirects/forwards.[/quote]Again, that's not going to happen.

[quote name='KevinD872' timestamp='1288810509' post='1555']Again, I don't expect these things to happen with MDD. But I am hoping that by sharing my bad experiences with my previous host, you will understand why I am concerned. While my site may not be anything nearly as critical as lots of other sites on the web, it's my site and has been around since 2001 and has excellent search engine rankings. It's important to [i]me[/i].
[/quote]I do understand your concerns and you share the concerns of many of our customers. Those that have been with us longer have likely been through a transfer or two with us as we've upgraded hardware in the past and can attest that we've always done our best to make it as seamless as possible.

If you have any other questions, let us know.

[KevinD872]

Thanks for the quick response and reassurance! I'm sure everything will go smoothly. For obvious reasons though, my first thought when I saw the email about a server move was "No. Please no!"

 

But I have no doubt that it will go well and be for the best in the long run. I think a large part of the problems I and other customers of my old hosting company had was due to the fact that they moved us from CPanel hosting to Plesk. It was quite a different animal and anything beyond very basic website configurations were destined to get messed up. Either they didn't consider that possibility or didn't care (or both).

 

Thanks again!

 

Kevin

[Michael D.]

Yes, going from one distinct system to another, such as cPanel to Plesk or vice versa would cause a lot of issues and wouldn't be simple by any stretch of the imagination.

 

Honestly, I wouldn't want to undertake such a transfer on a large scale - we do them an account at a time occasionally for new customers and even then they're a painful amount of work.

[Michael D.]

The new hardware is arriving!

 

Hard Disks on the far left, 4x300GB SAS15k RPM Seagate Cheetah Drives + 4 1.5TB Western Digital RE4 hard disks.

Next to those are two LSI MegaRAID SAS 9260-4i SAS/SATA Raid Controllers

The little white boxes are the CPU Coolers, The brown boxes are the battery backup units for the raid controllers that will allow us to enable write cache improving disk performance significantly and then on top of the stack is four Intel Xeon 5620 Quad Core Processors with HyperThreading

http://www.screen-shot.net/2010-11-04_1812.png

 

Our APC 30AMP 208V Power Distribution Unit:

http://www.screen-shot.net/2010-11-04_1856.png

 

Our HP ProCurve 2848 48-port gigabit switch:

http://www.screen-shot.net/2010-11-04_1857.png

 

This is only a small portion of the hardware we're bringing online but I thought I'd post some pictures Sorry for the quality, they were taken with a cellular camera.

[BrickMaster]

That's actually pretty good quality, it's actually better than some of the pics from my camera.

 

It looks great!

[Michael D.]

That's actually pretty good quality, it's actually better than some of the pics from my camera.

 

It looks great!

It's just stuff for the first two servers going online this month. We're going to be bringing 3 more online next month in the new facility. The chassis and mainboards should be in tomorrow and then the RAM tomorrow, Saturday, or Monday.

[Blind Bandit]

It's just stuff for the first two servers going online this month. We're going to be bringing 3 more online next month in the new facility. The chassis and mainboards should be in tomorrow and then the RAM tomorrow, Saturday, or Monday.

 

It looks like a computer geeks Christmas in those pics.

 

So, I guess one of the first severs will be a back up server? Seeing as it has 1.5Tb drives.

 

Will MDD being consolidating some severs? As your colocating 5 severs and the status page shows 6 and the pingdom report page shows 8 severs.

 

Also kudos to MDD. I think no one would have begrudged MDD for staying at Softlayer. It takes a lot of guts and hard work to move. I'm really excited to see the changes, and benefits the new infrastructure will bring.

[Michael D.]

It looks like a computer geeks Christmas in those pics.

Just looking at them makes me giggle a little, I'm not going to lie.

 

So, I guess one of the first severs will be a back up server? Seeing as it has 1.5Tb drives.

Actually the 1.5TB drives are going to be for the semi-dedicated server believe it or not. The average disk usage on a shared account is right around 0.5GB where as the average for semi-dedicated is right at 18gb. We're going to be consolidating the Cypress/Demeter/Gemini down to just Cypress but the huge difference is we'll be going from single quad core (no hyper threading) to Dual Quad core (with hyper threading). The Semi-dedicated server will still have *much* less accounts than our standard shared/reseller offering more headroom and resources while still being quite limited on the account capacity compared to shared/reseller.

 

Will MDD being consolidating some severs? As your colocating 5 severs and the status page shows 6 and the pingdom report page shows 8 severs.

A little, as I said above. Even though we'll be doing some combining - performance across the board will still improve.

 

Also kudos to MDD. I think no one would have begrudged MDD for staying at Softlayer. It takes a lot of guts and hard work to move. I'm really excited to see the changes, and benefits the new infrastructure will bring.

It's definitely a big change, and one that we look forward to!

[Michael D.]

More hardware images of the build:

 

http://www.screen-shot.net/2010-11-05_2256.png

http://www.screen-shot.net/2010-11-05_2257.png

http://www.screen-shot.net/2010-11-05_2257_001.png

http://www.screen-shot.net/2010-11-05_2257_002.png

http://www.screen-shot.net/2010-11-05_2258.png

http://www.screen-shot.net/2010-11-05_2258_001.png

http://www.screen-shot.net/2010-11-05_2258_002.png

http://www.screen-shot.net/2010-11-05_2259.png

 

Since the images are large I'm just linking to them instead of inserting them directly into the post.

[Michael D.]

I've shifted out all of the semi-dedicated server concerns to the semi-dedicated section in this thread: http://forums.mddhosting.com/topic/347-hardware-upgrades-dc-transfers-semi-dedicated-discussion/

 

The concerns were very specific to semi-dedicated accounts, servers, and customers and as such I felt that was the best place to discuss those specific concerns.

 

Do feel free to continue posting any questions you may have in this thread.

[Blind Bandit]

I've shifted out all of the semi-dedicated server concerns to the semi-dedicated section in this thread: http://forums.mddhos...ted-discussion/

 

The concerns were very specific to semi-dedicated accounts, servers, and customers and as such I felt that was the best place to discuss those specific concerns.

 

Do feel free to continue posting any questions you may have in this thread.

 

Mike I have a question about the network quality and speeds. I can tell right now all of my traffic will go through Comcast (oh joy). And I'm curious how the rest of network preforms. The mix looks rather solid. I'm just curious if you feel the network is as solid as Softlayer's?

 

I was also curious as how Dos/DDos and other attacks will be handled with the new provider. Do they have hardware devices in place (cisco gaurd, Riorey, Intrugaurd) ect. or is the individual provider to deal with?

 

How will backups be handled? Is MDD also co locating a back up server? Or using another method for back ups?

 

Thanks Mike.

[Michael D.]

Mike I have a question about the network quality and speeds. I can tell right now all of my traffic will go through Comcast (oh joy). And I'm curious how the rest of network preforms. The mix looks rather solid. I'm just curious if you feel the network is as solid as Softlayer's?

They are a smaller provider and as such they do have less peers/connections however their connections are quality and we expect the service to be exceptional. We simply wouldn't choose a facility if we didn't feel their networking was up to par with our standards.

 

I was also curious as how Dos/DDos and other attacks will be handled with the new provider. Do they have hardware devices in place (cisco gaurd, Riorey, Intrugaurd) ect. or is the individual provider to deal with?

They do have Riorey equipment that can be used as-needed although hopefully we won't have to use that much, if at all

 

How will backups be handled? Is MDD also co locating a back up server? Or using another method for back ups?

We're going to be colocating a backup server as well. We always suggest no matter who the provider is (even if it's us) that our customers keep a copy of their site and databases off-provider just in case. It's simply a good thing to do.

[Michael D.]

The servers are racked and online, we're still going to begin transfers as scheduled

[skunkbad]

I know you were scheduled to start the move today. So, how's it going so far?

[Michael D.]

I know you were scheduled to start the move today. So, how's it going so far?

It's going as expected - we throttled up the pipes to 1,000 megabit on both sides and it's running right at around 450 MBPS total for the two servers being transferred currently. The bottleneck is seeking all of the small files while still actively serving pages

 

Echo, Cypress, and Demeter will be done by morning as long as things keep going well and then Fresco is slated for tomorrow night.

[skunkbad]

It's going as expected - we throttled up the pipes to 1,000 megabit on both sides and it's running right at around 450 MBPS total for the two servers being transferred currently. The bottleneck is seeking all of the small files while still actively serving pages

 

Echo, Cypress, and Demeter will be done by morning as long as things keep going well and then Fresco is slated for tomorrow night.

 

Sounds good. Couple of questions:

 

1) I was on Cypress before it died a couple of weeks ago. Does my site revert back to the state it was in before it was moved over to Fresco, or does the backup come from the current state of the site on Fresco? I've got my own backups just in case, but would like to be ready to upload if needed.

 

2) If Cypress is done tomorrow morning, does that mean requests for my site will start being served from that server right away, or is there some DNS propagation that will take place first?

[Michael D.]

1) I was on Cypress before it died a couple of weeks ago. Does my site revert back to the state it was in before it was moved over to Fresco, or does the backup come from the current state of the site on Fresco? I've got my own backups just in case, but would like to be ready to upload if needed.

If you're on Fresco right now your account will not be moved until tomorrow unless we decide to go ahead and extend the transfer process this morning.

 

2) If Cypress is done tomorrow morning, does that mean requests for my site will start being served from that server right away, or is there some DNS propagation that will take place first?

Anybody who hasn't visited the site an hour or less before the transfer started will see it on the new server right away, anybody visiting the site while the transfer happens will see it on the old server for a couple of hours.

[mipra]

Looking forward for the Fresco change. I have submitted a ticket for both NS1 and NS2 so that I can update each IP since I use our own NS

 

Please update here when the Fresco has done transferring...

 

Cannot wait to feel the smell of the new hardwares....

 

Edit: Well....you sure spend one heck of a late sleep today seeing you being online as of this post

[Mike1000]

Could you also post the new Nameserver IP's for Demeter here as well please as I use personal nameservers on it:-)

 

Many thanks,

Michael

[Michael D.]

Sleep? What is sleep?

[skunkbad]

I just want to say good job and thank you. Even though I was working on my site, somehow it got transferred to the new datacenter without any loss of data (or at least I haven't noticed anything missing). I think it happened when I was at lunch or something...

[Michael D.]

I just want to say good job and thank you. Even though I was working on my site, somehow it got transferred to the new datacenter without any loss of data (or at least I haven't noticed anything missing). I think it happened when I was at lunch or something...

We're pretty good at transfers - we did have some issues with a specific reseller where cPanel didn't properly assign dedicated IPs but we've identified the cause of that issue and reported it to cPanel accordingly so they can patch it.

[Michael D.]

Could you also post the new Nameserver IP's for Demeter here as well please as I use personal nameservers on it:-)

 

Many thanks,

Michael

Sorry, I missed your post.

ns1: 173.248.187.10

ns2: 173.248.191.11