S1 Server - Random Lag/Inaccessibility + Short Outage
-
This topic is locked
#21
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 09:23 PM
35%
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#22
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 09:26 PM
File System Check is 43% Completed
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#23
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 09:28 PM
46%
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#24
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 09:31 PM
File System Check is 55% Completed
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#25
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 09:32 PM
70%
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#26
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 09:42 PM
It moved on to checking the directory structure and it appears this portion is going to take longer. It is currently at 71.8%
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#27
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 09:51 PM
The file system check has reached 73% Completed.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#28
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 09:59 PM
The file system check has reached 75%
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#29
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 10:09 PM
The file system check has reached 80%.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#30
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 10:19 PM
The File System Check has reached 87% completed.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#31
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 23 March 2016 - 10:33 PM
The file system check has completed and the server is booting up.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#32
-
- Staff Administrator
-
- 421 posts
MDDHosting Staff
- Gender:Male
Posted 23 March 2016 - 10:53 PM
S1 is now back online and handling requests "normally." The reboot was initially meant to correct a configuration issue, which we are now monitoring to see if the previous stability problems are resolved. I should also note that S1 may be slower for the next 10 or 20 minutes, as most servers of this size are when they come back online.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#33
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 24 March 2016 - 08:43 AM
So far this morning the performance of the S1 server has been very much improved. I'm hesitant to call it 'fixed' as I want to allow more time to pass before we come to that determination.
I'm highly optimistic, however, that the issues are resolved.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#34
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 24 March 2016 - 03:24 PM
Other than a MySQL restart issued by me at ~1:15 PM today the server has been rock solid and performing well.
We do not anticipate any further issues with the S1 server at this time.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#35
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 24 March 2016 - 11:46 PM
The R1Soft System Kernel caused a panic on S1.
We're rebooting and working to restore services as soon as possible.
As a result of this issue we're going to be disabling R1Soft on this server for at least a couple of days while we investigate/evaluate our options.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#36
-
- Staff Administrator
-
- 421 posts
MDDHosting Staff
- Gender:Male
Posted 24 March 2016 - 11:57 PM
The reboot is complete and sites are responding once again. We are still investigating the source of the kernel panic with R1soft and have no further details regarding that at this time.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
#37
-
- Staff Administrator
-
- 2,902 posts
Forum Administrator
- Gender:Male
- Location:Central Indiana, USA
Posted 25 March 2016 - 12:14 AM
Interestingly enough R1Soft managed to load up the storage subsystem enough that we identified a previously unknown performance bottleneck.
We made an adjustment during the outage and average storage latency is down to 1ms from 10ms after this reboot. We made the same adjustment live to the rest of the fleet.
Hopefully this will be the good that comes from the bad at this point.
█ Scalable shared hosting plans in the cloud! Check them out!
█ Highly Available Cloud Shared, Reseller, and VPS
█ http://www.mddhosting.com/
Also tagged with one or more of these keywords: Current
Current
MDDHosting Announcements →
Server and Network Announcements →
Jasmine Server - DDoS on 173.248.188.150Started by MikeDVB, 19 Feb 2013  Current
|
|
|
||
Current
MDDHosting Announcements →
Server and Network Announcements →
Echo Server - DDoS on IP 173.248.187.73Started by MikeDVB, 31 Jan 2013 Current
|
|
|
||
Current
MDDHosting Announcements →
Server and Network Announcements →
DDoS attack on on Echo serverStarted by Scott, 28 Nov 2012 Current, DDoS, Echo
|
|
|
1 user(s) are reading this topic
0 members, 1 guests, 0 anonymous users
