
[Resolved] Cypress Outage - DDoS Attack Verified
#1
Posted 12 March 2011 - 05:15 PM
The DDoS attack on Cypress is now resolved.
Everything on the Cypress server should be returning to normal. We have null routed the affected IP addresses and moved all clients to a new shared IP. If you are still having issues, try clearing your DNS cache and then open a support ticket so we can look into your account specifically.
If you are the target of the attack, we will open a ticket with you and let you know. If you didn't see any such ticket, then the attack wasn't targeting you

The remainder of this post will remain unchanged to keep a full record of events intact.
====================
Update #1:
For those of you just joining us: We are seeing a 1,300 mbps DDoS attack aimed at our Cypress server. We are currently working with our datacenters networking team to null route affected IPs and migrate clients on those IP's to a new one. If you are using external DNS, such as CloudFlare, and you do NOT have a dedicated IP, please open a ticket so that we may get the new IP address to you when it is available.
The remainder of this post will remain how it was originally to keep a full record of events intact.
====================
We are aware of a current issue with the Cypress server and we are currently investigating the cause of this issue. More information will be made available as soon as we have it.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#2
Posted 12 March 2011 - 05:23 PM
We're working on identifying the attack, mitigating it, and we're going to do what we can to prevent it from happening again however the internet is a hostile environment and attacks are inevitable.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#3
Posted 12 March 2011 - 05:32 PM
It seems that today's issue and yesterday's issue are similar and likely related. It's a large influx of traffic into the server that causes the connection tracking in the server to fail resulting in the server becoming unresponsive. We're still working on identifying the source/destination and investigating ways to prevent this from happening again.
The server is online however it does take 10 to 15 minutes for it to "catch up" after being rebooted.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#4
Posted 12 March 2011 - 05:35 PM
Thanks for the update.http://www.screen-shot.net/2011-03-12_1725.png
It seems that today's issue and yesterday's issue are similar and likely related. It's a large influx of traffic into the server that causes the connection tracking in the server to fail resulting in the server becoming unresponsive. We're still working on identifying the source/destination and investigating ways to prevent this from happening again.
The server is online however it does take 10 to 15 minutes for it to "catch up" after being rebooted.
#5
Posted 12 March 2011 - 05:42 PM

#6
Posted 12 March 2011 - 06:02 PM
The IP that is going to be null-routed does have several customers on it and we will be migrating them individually to new IPs so that we can identify the target of the attack more specifically and then that one particular account will be null routed until the attack subsides.
If you have any questions we do ask that you try and ask them here unless they are account specific so that we can provide centralized disbursement of information.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#7
Posted 12 March 2011 - 06:12 PM
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#8
Posted 12 March 2011 - 06:16 PM
Is the new ip address temporary or permanent? I'll need to update my nameserver as I have my DNS hosted at cloudflare.We just got with the network operations team at our facility and they're definitely seeing an ingress 500 MBPS+ hitting the server. We're putting a null-route in place for the target at which point the server will come back online.
The IP that is going to be null-routed does have several customers on it and we will be migrating them individually to new IPs so that we can identify the target of the attack more specifically and then that one particular account will be null routed until the attack subsides.
If you have any questions we do ask that you try and ask them here unless they are account specific so that we can provide centralized disbursement of information.
Peace,
Brian
#9
Posted 12 March 2011 - 06:22 PM
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#10
Posted 12 March 2011 - 06:24 PM
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#11
Posted 12 March 2011 - 06:28 PM
We hope to have this resolved within the next 10 to 15 minutes.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#12
Posted 12 March 2011 - 06:28 PM
#13
Posted 12 March 2011 - 06:33 PM
How does the number of DDOS attacks Cypress experiences compare to the other servers? It seems like its always Cypress that's getting attacked all the time?
I'm not aware of the statistics on this off hand. We do post a public record of events such as this on these forums, so you should be able to look through the history and see when other servers got hit. Once this issue is resolved, we may be able to spend more time looking into the history of such things.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#14
Posted 12 March 2011 - 06:33 PM
How does the number of DDOS attacks Cypress experiences compare to the other servers? It seems like its always Cypress that's getting attacked all the time?
It's probably the same site that's being targeted repeatedly, I would guess.
#15
Posted 12 March 2011 - 06:38 PM
To my knowledge Cypress has never been hit with a DDoS we weren't able to quickly mitigate in the past and it has been quite a while since our last really damaging DDoS attack. We do leave all of the threads here on the forum so you're welcome to look into it.How does the number of DDOS attacks Cypress experiences compare to the other servers? It seems like its always Cypress that's getting attacked all the time?
No - if you become the target of a DDoS you get moved to a Dedicated IP and you stay there - if you ARE repeatedly hit to the point that it's causing issues for our other customers we would have no choice but to ask you to move to a service where you wouldn't affect others such as a dedicated server.It's probably the same site that's being targeted repeatedly, I would guess.
The site that is under attack has just been identified and has never been attacked before in it's 2+ years of being hosted with us. We will be getting in contact with the site operator meanwhile everything should normalize for everybody else within the next few minutes.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#16
Posted 12 March 2011 - 06:43 PM
No - if you become the target of a DDoS you get moved to a Dedicated IP and you stay there - if you ARE repeatedly hit to the point that it's causing issues for our other customers we would have no choice but to ask you to move to a service where you wouldn't affect others such as a dedicated server.
The site that is under attack has just been identified and has never been attacked before in it's 2+ years of being hosted with us.
I see, good to know.
#17
Posted 12 March 2011 - 06:44 PM
My URL is registered with 1and1 (I know they are far from good but so far I did never have issues with them).
Will I have to change the name server as well?
#18
Posted 12 March 2011 - 06:48 PM
My URL is registered with 1and1 (I know they are far from good but so far I did never have issues with them).
Will I have to change the name server as well?
If you're using our nameservers, you will not need to change anything. If you're using something other than our nameservers, such as CloudFlare or an external DNS server then yes, you will need to contact our support department for your new IP address.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#20
Posted 12 March 2011 - 06:57 PM
That was actually Scott that responded to youThanks Mike, I am indeed on your nameservers
.

Beyond that, everybody should be back online as we've moved the site under attack to it's own IP address and null routed that IP. If you are still having issues by all means do open a ticket so we can look specifically into your account.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
1 user(s) are reading this topic
0 members, 1 guests, 0 anonymous users