Jump to content
MDDHosting Forums

[Ended] DDoS attack on Demeter Server


Recommended Posts

Hello,

 

Unfortunately one customer on our Demeter server was hit by a fairly large DDoS attack today coming from around 12,000 individual systems and making upwards of 12,000 requests per second to the server.

 

There was a short bout of downtime (around 5 minutes) while we logged into the server and tweaked everything to handle the attack including hardening the TCP stack on the server to better handle the number of requests the server was seeing as well as optimizing LiteSpeed for this specific form of attack.

 

Every attack is different and as such different actions are necessary to mitigate the attack and attack mitigation isn't always 100% possible. We always do our best to keep our clients online however ultimately if the attack is too large we'll do what we have to, to keep the server online.

 

If you have any questions at all about this, by all means feel free to respond to this thread.

 

Thank you,

Link to comment
Share on other sites

Just in case anybody is curious:

 

http://www.screen-shot.net/2010-05-27_1556.pngThis is a 100% SYN flood attack meaning the packets are extremely small and it takes a lot to reach 80 megabit/second.

 

So far the server after being adjusted to handle the attack is performing well. We're going to keep our eyes on the server so that we can act quickly should the attack change or shift in any way.

Link to comment
Share on other sites

It looks as though Cisco Guard (our DDoS protection system) is doing a good job of filtering out the attack. It's definitely still going on however not much of the attack is reaching the server at this point.

 

http://www.screen-shot.net/2010-05-27_1616.png

 

I'll see if networking can't get me details from CiscoGuard about the attack such as packets/second etc...

Link to comment
Share on other sites

As usual .. you guys are on top of things.

 

Just curious though ... has the attack stopped?

No, it's not stopped - we've still been mitigating the attack and it looks like earlier today they switch to a different attack vector that is more difficult to mitigate however we were eventually able to stop it:

 

http://www.screen-shot.net/2010-05-28_1551.png

Link to comment
Share on other sites

Looks like it actually got worse there for a while. 140mb wow.

Yeah, whoever is doing it is pretty determined to get this one particular site offline. Just for the record - the specific site and site operator under attack have been made aware so if we've not told you that your site is under attack, you're not the target.

 

That's one of the major downsides of a DDoS is that it's a carpet-bomb type of attack. They want to take a single site offline so they hit that site with a DDoS that has the collateral damage of causing issues for everybody else on the server.

 

At any rate, the attack is still ongoing but hopefully they'll give up after a while of seeing that they're not actually taking the site offline.

Link to comment
Share on other sites

Well I know it is not my site then. Thanks for the updates and hopefully they do give up.

One of two things eventually happens.

  1. They give up after not getting the site offline.
  2. They increase the attack astronomically and take the site down.

 

If they give up great - if the attack does get so large that we can't mitigate it ultimately we would be forced to take that one particular site offline to keep the server up but we do our best not to have to do that.

 

If anything changes we'll post an update :)

Link to comment
Share on other sites

Great work! I hope things wont get worse.

 

But: Don't you think such topics should be private / visible for registered users (that's no protection but if you can't see such a topic you may not guess there is one. Your words may influence the "attacker" with his further actions. Just my thoughts =D

Link to comment
Share on other sites

I just think you shouldn't underestimate the person/group behind this. Only posting those screens may give them some informations how specific attacks are being handled. Im no professional these are just my thoughts.

And no the "attacker" wont become upset (they don't aim at mdd so why would they anyway) =D it's just:

If they give up great - if the attack does get so large that we can't mitigate it ultimately we would be forced to take that one particular site offline to keep the server up but we do our best not to have to do that.

Never show any public weakness! xD

 

Please don't end in a discussion here. Just my opinion and Michael has to decide what he wants to publish. Mdd is doing great!

Link to comment
Share on other sites

I just think you shouldn't underestimate the person/group behind this. Only posting those screens may give them some informations how specific attacks are being handled. Im no professional these are just my thoughts.

We've dealt with dozens of DDoS attacks over the last two years and have everything from a sad 1mbps attack up to a 15GBPS attack (which took over 100,000 compromised systems to achieve).

 

We're not under-estimating them but we're also not going to hide. We believe in being open with our client base when we're facing an issue as to exactly what we're facing, how, why, and what we're doing about it.

 

Never show any public weakness! xD
As far as I know I've not posted any weakness - we've dealt with enough attacks that we have a fair bit of knowledge on how to handle them and what we can or cannot handle.

 

Please don't end in a discussion here. Just my opinion and Michael has to decide what he wants to publish. Mdd is doing great!

There are certainly details about the attack that aren't being published as they simply aren't relevant to the client base on the server :)

Link to comment
Share on other sites

 Share

×
×
  • Create New...