Jump to content


Photo

S3 Server - Service Disruptions


  • Please log in to reply
1 reply to this topic

#1 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 26 October 2019 - 07:27 PM

At 11:02 AM ET on Sat October 26, 2019 the S3 server exhausted all of its physical RAM resulting in an outage.  Before bringing the server back online we applied an additional 30GB of ram to the system bringing it up to a total of 120GB.

 

At 7:53 PM ET on Sat October 26, 2019 the S3 server once again experienced a brief outage.

 

We presently believe the issue to be with CloudLinux and it not properly applying RAM limits to all accounts.

 

A ticket has been opened with CloudLinux and we are watching the server closely.


  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/

#2 MikeDVB

MikeDVB

    Forum Administrator

  • Staff Administrator
  • PipPipPipPipPip
  • 2,893 posts
  • Gender:Male
  • Location:Central Indiana, USA

Posted 26 October 2019 - 09:40 PM

It looks like OOM Killer [out of memory killer] was going wild over a few accounts - and the amount it was running was causing system wide issues.  The system itself didn't exhaust RAM, but the user-level issues did cause some problems.

 

We adjusted some system settings as per CloudLinux's advice and we believe this issue should not recur.  We are going to continue watching the server closely regardless.

 

The changes recommended by CloudLinux have been applied to all servers.


  • 0
Michael Denney - MDDHosting LLC - Providing Hosting since 2007
Scalable shared hosting plans in the cloud! Check them out!
Highly Available Cloud Shared, Reseller, and VPS
http://www.mddhosting.com/




2 user(s) are reading this topic

0 members, 2 guests, 0 anonymous users