Completed Cypress Server Migration / Upgrade - Within 14 days

[Michael D.]

Hello,

 

We are aware of the issues that Cypress has been having from time to time, and have ordered a brand new server to replace it. Currently Cypress is one of our older servers which has the largest amount of disk space available to it which resulted in a lot of our larger clients being placed there. The downside to larger disks is that they are slower and less apt to be able to keep up, especially if a major strain is placed on them such as backing up an account that is several gigabytes in size during a peak time period. We have put provisions into our Terms of Service that specifically forbid conducting backups during peak periods which are outlined in the Terms of Service, but that doesn't stop the occasional customer from doing so anyways resulting in issues until we can get in and stop it, and even then, it takes some time for the server to stabilize afterwards.

 

Due to this issue, we've ordered a brand new and very robust server to replace the Cypress server, it uses a RAID10 array of 8 1TB disks to give us the capacity we need as well as CacheCade Raid Caching using RAID1 dual 240 GB Solid State Drives capable of 6 gigabits per second each to sit in front of that array ensuring that it remains fast and responsive. The new server will also have nearly three times the RAM of the current Cypress server and roughly 20% more overall CPU power.

We will send another email out letting everybody know at what time the transfer will be conducted - due to the issues that the Cypress server is currently facing, we will not be giving a large anoung of lead time once the new server is online in order to restore the quality of service back to what you expect from us.

 

If you have any questions at all, by all means, please feel free to reply to this thread. We are happy to answer any and all questions you have.

 

Thank you!

[Michael D.]

It looks like this will be happening towards the end of this next week. Likely will start moving customers this next weekend, but will update this thread before we get started as well as emailing everybody affected.

[Michael D.]

The new server is here and built, however, we believe we've received a system board that is defective. While the server powers on and operates, there are a few core features the system has that are not working as they should. Unfortunately we may need to RMA this server board and wait for a new one to arrive. On the positive side, the server boards tend to ship out very quick.

 

I will keep this thread updated.

[Michael D.]

The new hardware was swapped, I just need to configure the server (install the OS, install cPanel, firewall, Softaculous, and about 100 other things). I am going to go ahead and plan on transferring the weekend of the 14th and 15th of this month. This will give me plenty of time to set the new server up as well as will allow time for anybody who needs to prepare for the migration.

 

I will update this thread.

[Michael D.]

I'm working on setting up the new server now. Due to the transition likely taking a few days we're not keeping the name the same so that everything remains available at all times for all customers and there is no confusion as to which server you're working on, at least as far as the cPanel.

 

The new server name will be "jasmine". We're going to be conducting the transfers starting as early as tonight and hope to have them all finished by this upcoming Sunday at the latest.

 

If you have any questions, let us know.

[Michael D.]

It's going to take a little longer than I anticipated, so I don't think we'll get to transfers tonight. I will keep this thread updated as to the status as I work on this.

[billhector]

No. Are you serious? Let me get this straight.

 

You are now going to change the server name so that I have to change all my email preferences (along with all my clients, because I have a reseeler account)? Really? And this happens at sort of a random time, in the middle of the night?

 

Which I guess wouldn't make much of a difference if I could have created a subdomain name that was mapped and worked (for example, mail.domain.com). But I could never get that to work, for me or clients, so I just gave up.

 

Please tell me that the situation isn't as dire as I think.

 

BW

[Michael D.]

No. Are you serious? Let me get this straight.

Yes, we're upgrading from older hardware to newer hardware. We've warned of this impending migration quite early (over two weeks ago).

 

You are now going to change the server name so that I have to change all my email preferences (along with all my clients, because I have a reseeler account)? Really? And this happens at sort of a random time, in the middle of the night?

Or you can just use mail.themaindomain.com which will work 100% fine so long as you use our DNS. You will get an SSL warning that you can accept permanently and then you won't see it again until the SSL is updated (once per year).

 

Which I guess wouldn't make much of a difference if I could have created a subdomain name that was mapped and worked (for example, mail.domain.com). But I could never get that to work, for me or clients, so I just gave up.

mail.theirdomain.com works 100%, I'll be happy to demonstrate this for you by remote connecting into your computer and showing you. Just open a support ticket and I'll be happy to dedicate a half hour to this for you.

 

Please tell me that the situation isn't as dire as I think.

No, it's not nearly as dire as you make things out. Beyond that, would you rather stay on an older slower server that has issues regularly instead of doing a little work and being on a new server?

 

You are upset that you may have to change some email server settings, think about how much work we have to do to bring a new server online, configure it, then migrate hundreds of accounts over without messing anything up / missing anything / forgetting anything or making any mistakes.

 

Now... Let's take a look at the alternative - keeping the server name the same:

1. The server name will not be pointed to the new server until the transfer is completed which will take anywhere from 24 to 96 hours.
   
2. Once DNS is updated to point to the new server, it can take anywhere from 2 to 72 hours for DNS to propagate fully world-wide, resulting in a total waiting period of up to 168 hours.
   
3. During the 168 hours, there is a very good chance that email would be going to the new server, while you continued to check the old server, if you choose not to make any changes to your email client(s).
   
4. If you wanted to receive email before the transfer was done and DNS was updated, you would still need to modify your mail client settings, so not changing the server name would not save you from having to do this, unless you're willing to go days without new emails.
   
5. You would likely want to update your mail client settings back to the server name once the transfer was done and the DNS was updated, meaning you would end up updating your mail client settings not once, but twice.
   
6. You wouldn't 'lose' any email, but if you don't update your mail client settings you could definitely see a delay of up to 168 hours on email delivery if you chose not to make any changes at all.
   

We don't do things to make them inconvenient for you, we do them because they're the best way to handle a specific situation. If you have any alternative suggestions, we're all ears.

[billhector]

I was well aware that you where changing servers. That's not the issue. You kept us updated fine. What we did not know is that you were changing server names. That we did not know. At all.

 

I will open a ticket in the morning, EDT, when i can keep on top of the communication and you can help with our settings.

 

BTW: "mail.theirdomain.com" does not work 100% of the time. I know. It doesn't work for me, despite me setting up the dns how I thought it was supposed to be.

 

Thank you. We'll speak later.

[Michael D.]

I was well aware that you where changing servers. That's not the issue. You kept us updated fine. What we did not know is that you were changing server names. That we did not know. At all.

I could keep the name the same, but then you're going to either need to update your mail settings twice so you get emails, or not update them at all and go potentially for days without emails coming in.

 

I will open a ticket in the morning, EDT, when i can keep on top of the communication and you can help with our settings.

I'll see that you're updated once all of your accounts are copied so that you can have your users update their mail settings. It would be as simple as replacing "cypress" with "jasmine" in any mail settings.

 

Beyond that, once the transfer is done we will point the "cypress" name over to the new server for a few weeks just in case anybody hasn't updated their mail settings. It will likely stay this way until we bring another new server online which will likely be named "cypress" as I don't like to have a gap in the alphabet [silly, I know].

 

BTW: "mail.theirdomain.com" does not work 100% of the time. I know. It doesn't work for me, despite me setting up the dns how I thought it was supposed to be.

Open a ticket, and I'll troubleshoot this for you. If you are using our DNS it will work 100%, if not, you will need to cname it to your main domain like this:

 

mail. CNAME yourmaindomain.com.

 

For example, here is the DNS record for mail.mikedvb.com (my personal site, hosted here at MDD):

;; QUESTION SECTION:
;mail.mikedvb.com.			 IN	 A

;; ANSWER SECTION:
mail.mikedvb.com.	 14372 IN	 CNAME mikedvb.com.
mikedvb.com.	 14372 IN	 A	 173.248.187.4

You will notice that it asks for the A-record "mail.mikedvb.com" which returns the CNAME of "mail.mikedvb.com" to "mikedvb.com" and then the A-record of "mikedvb.com" is "173.248.187.4".

 

Thank you. We'll speak later.

Sure, but next time before freaking out/getting upset/venting your frustration in a public venue, you may want to get in touch with us via a support ticket. I'm always here in case you need something like this discussed as it's outside the scope of most basic tech support representatives.

 

As I said above, we don't do things to make your life harder, and are happy to help simplify the process/make your life easier.

[Michael D.]

I'm copying accounts over and doing my best to notify account holders as quickly as possible. If you have any questions, let me know. This process is going to take probably 48 hours or more, and I'll post updates periodically.

 

Once you are on the Jasmine server, you *will* see a Trial cPanel license - that will go away once the transfer is completed.

 

Edit: Just for some clarification, we're currently copying reseller accounts and their customers and updating the reseller account holders when we get started and when we finish with each reseller. I'll update this thread in more detail once we start moving shared customers.

[Michael D.]

The transfer process is still in progress. We are updating individual account holders as their accounts are copied over.

[Michael D.]

The transfer process is still in progress, it's taking longer than expected as we're notifying each and every account holder as we go along instead of sending one mass email at the start and one mass email at the end.

[die2mrw007]

Pretty awesome server config...great move

[Kevin]

Not that it is a huge deal to me but was wondering if you have an update on the progress of the transfers. Thanks

[Kevin]

Not that it is a huge deal to me but was wondering if you have an update on the progress of the transfers. Thanks

 

Looks like my account is being transferred over now so this message can be ignored

[Scott]

Not that it is a huge deal to me but was wondering if you have an update on the progress of the transfers. Thanks

 

Everything is on track, the transfers have been mostly happening in the evening/night during off-peak hours. I suspect everyone will be moved over before the week is over.

[Michael D.]

All accounts have been copied over - if you didn't receive a notice/update I apologize - if you wish to open a ticket you are welcome to do so and I will see why you didn't get a notification.

 

Either way, we are going to be powering down the Cypress server in the early AM EST on Friday, July 27th, 2012 for about 2 hours. This 2 hour shutdown will help us identify who is using external DNS and hasn't updated and anybody else who may still be connecting to the old server such as for their cPanel, email, etc.

 

After the server comes back online, we will update/move any data that needs moved a second time (such as due to not having updated DNS) and will work with anybody who needs help at that point. We expect to permanently power down the old server in the late evening on Friday, and will leave all data on the server intact until Tuesday evening - just in case.

[Michael D.]

All sites still resolving to the old server are now redirected (307 Temporary Redirect) to an information page detailing that the issue is temporary for and that DNS needs updated. The landing page with this error message is returned with a HTTP 503 code. Ultimately it goes 302 (Temporary Redirect) -> 307 (Temporary Redirect) -> 503 (Service Unavailable). The reason for this, is so that Google and other search engines do not update your search results to point to the error page, or to include any of the redirections.

 

We can disable the redirection on an account-by-account basis upon request to give you time to update your DNS. At this point, unless your site is entirely static, there is likely new data that needs to be copied over to the new server (such as new posts, user comments, etc). We urge you to get in contact with us as soon as possible should you need some more time before the server is taken offline.