Jump to content


Photo

[Ended] DDoS attack on Demeter Server


  • Please log in to reply
19 replies to this topic

#1 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,900 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 27 May 2010 - 02:39 PM

Hello,

Unfortunately one customer on our Demeter server was hit by a fairly large DDoS attack today coming from around 12,000 individual systems and making upwards of 12,000 requests per second to the server.

There was a short bout of downtime (around 5 minutes) while we logged into the server and tweaked everything to handle the attack including hardening the TCP stack on the server to better handle the number of requests the server was seeing as well as optimizing LiteSpeed for this specific form of attack.

Every attack is different and as such different actions are necessary to mitigate the attack and attack mitigation isn't always 100% possible. We always do our best to keep our clients online however ultimately if the attack is too large we'll do what we have to, to keep the server online.

If you have any questions at all about this, by all means feel free to respond to this thread.

Thank you,
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#2 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,900 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 27 May 2010 - 02:58 PM

Just in case anybody is curious:

Posted ImageThis is a 100% SYN flood attack meaning the packets are extremely small and it takes a lot to reach 80 megabit/second.

So far the server after being adjusted to handle the attack is performing well. We're going to keep our eyes on the server so that we can act quickly should the attack change or shift in any way.
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#3 bunnykins

bunnykins

    Member

  • Members
  • PipPip
  • 143 posts
  • Gender:Female

Posted 27 May 2010 - 03:18 PM

Thank you for letting me know. So far the site is loading fine. :)
  • 0
My name is Amy and I enjoy helping people with web sites.

#4 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,900 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 27 May 2010 - 03:18 PM

It looks as though Cisco Guard (our DDoS protection system) is doing a good job of filtering out the attack. It's definitely still going on however not much of the attack is reaching the server at this point.

Posted Image

I'll see if networking can't get me details from CiscoGuard about the attack such as packets/second etc...
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#5 bunnykins

bunnykins

    Member

  • Members
  • PipPip
  • 143 posts
  • Gender:Female

Posted 27 May 2010 - 03:22 PM

Ok good luck and keep us updated.
  • 0
My name is Amy and I enjoy helping people with web sites.

#6 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,900 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 27 May 2010 - 03:39 PM

Ok good luck and keep us updated.

Sure thing :)
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#7 bunnykins

bunnykins

    Member

  • Members
  • PipPip
  • 143 posts
  • Gender:Female

Posted 27 May 2010 - 04:03 PM

Thank you. Keep up the good work.
  • 0
My name is Amy and I enjoy helping people with web sites.

#8 Flyer

Flyer

    Newbie

  • Clients
  • Pip
  • 3 posts

Posted 28 May 2010 - 07:30 AM

As usual .. you guys are on top of things.

Just curious though ... has the attack stopped?
  • 1

#9 bunnykins

bunnykins

    Member

  • Members
  • PipPip
  • 143 posts
  • Gender:Female

Posted 28 May 2010 - 10:35 AM

Just curious though ... has the attack stopped?

Ditto
  • 0
My name is Amy and I enjoy helping people with web sites.

#10 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,900 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 28 May 2010 - 02:53 PM

As usual .. you guys are on top of things.

Just curious though ... has the attack stopped?

No, it's not stopped - we've still been mitigating the attack and it looks like earlier today they switch to a different attack vector that is more difficult to mitigate however we were eventually able to stop it:

Posted Image
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#11 bunnykins

bunnykins

    Member

  • Members
  • PipPip
  • 143 posts
  • Gender:Female

Posted 28 May 2010 - 03:08 PM

Looks like it actually got worse there for a while. 140mb wow.
  • 0
My name is Amy and I enjoy helping people with web sites.

#12 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,900 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 28 May 2010 - 03:55 PM

Looks like it actually got worse there for a while. 140mb wow.

Yeah, whoever is doing it is pretty determined to get this one particular site offline. Just for the record - the specific site and site operator under attack have been made aware so if we've not told you that your site is under attack, you're not the target.

That's one of the major downsides of a DDoS is that it's a carpet-bomb type of attack. They want to take a single site offline so they hit that site with a DDoS that has the collateral damage of causing issues for everybody else on the server.

At any rate, the attack is still ongoing but hopefully they'll give up after a while of seeing that they're not actually taking the site offline.
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#13 bunnykins

bunnykins

    Member

  • Members
  • PipPip
  • 143 posts
  • Gender:Female

Posted 28 May 2010 - 04:11 PM

Well I know it is not my site then. Thanks for the updates and hopefully they do give up.
  • 0
My name is Amy and I enjoy helping people with web sites.

#14 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,900 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 28 May 2010 - 04:16 PM

Well I know it is not my site then. Thanks for the updates and hopefully they do give up.

One of two things eventually happens.
  • They give up after not getting the site offline.
  • They increase the attack astronomically and take the site down.

If they give up great - if the attack does get so large that we can't mitigate it ultimately we would be forced to take that one particular site offline to keep the server up but we do our best not to have to do that.

If anything changes we'll post an update :)
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#15 viVa

viVa

    Newbie

  • Members
  • Pip
  • 2 posts

Posted 28 May 2010 - 04:41 PM

Great work! I hope things wont get worse.

But: Don't you think such topics should be private / visible for registered users (that's no protection but if you can't see such a topic you may not guess there is one. Your words may influence the "attacker" with his further actions. Just my thoughts =D
  • 0

#16 bunnykins

bunnykins

    Member

  • Members
  • PipPip
  • 143 posts
  • Gender:Female

Posted 28 May 2010 - 04:55 PM

As far as I know mdd has not said any thing yet that should cause the attacker to become upset and make things worse. Although I could be wrong.
  • 0
My name is Amy and I enjoy helping people with web sites.

#17 viVa

viVa

    Newbie

  • Members
  • Pip
  • 2 posts

Posted 28 May 2010 - 05:13 PM

I just think you shouldn't underestimate the person/group behind this. Only posting those screens may give them some informations how specific attacks are being handled. Im no professional these are just my thoughts.
And no the "attacker" wont become upset (they don't aim at mdd so why would they anyway) =D it's just:

If they give up great - if the attack does get so large that we can't mitigate it ultimately we would be forced to take that one particular site offline to keep the server up but we do our best not to have to do that.

Never show any public weakness! xD

Please don't end in a discussion here. Just my opinion and Michael has to decide what he wants to publish. Mdd is doing great!
  • 0

#18 bunnykins

bunnykins

    Member

  • Members
  • PipPip
  • 143 posts
  • Gender:Female

Posted 28 May 2010 - 05:25 PM

Yes mdd is great. I am glad to be a client. Any way what I meant by upset was more or less provoking them into a much worse attack.
  • 0
My name is Amy and I enjoy helping people with web sites.

#19 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,900 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 28 May 2010 - 05:25 PM

I just think you shouldn't underestimate the person/group behind this. Only posting those screens may give them some informations how specific attacks are being handled. Im no professional these are just my thoughts.

We've dealt with dozens of DDoS attacks over the last two years and have everything from a sad 1mbps attack up to a 15GBPS attack (which took over 100,000 compromised systems to achieve).

We're not under-estimating them but we're also not going to hide. We believe in being open with our client base when we're facing an issue as to exactly what we're facing, how, why, and what we're doing about it.

Never show any public weakness! xD

As far as I know I've not posted any weakness - we've dealt with enough attacks that we have a fair bit of knowledge on how to handle them and what we can or cannot handle.

Please don't end in a discussion here. Just my opinion and Michael has to decide what he wants to publish. Mdd is doing great!

There are certainly details about the attack that aren't being published as they simply aren't relevant to the client base on the server :)
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#20 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,900 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 29 May 2010 - 04:35 PM

It appears as though the attacks have ended at this point. We'll obviously update the thread if it recommences.
  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users