Michael D. Posted April 21, 2014 Report Share Posted April 21, 2014 While performing a normal R1Soft Agent update on the Kobold server, the server crashed. We're rebooting it now and will keep this thread updated. Quote Link to comment Share on other sites More sharing options...
Michael D. Posted April 21, 2014 Author Report Share Posted April 21, 2014 The server did not require a file system check and is coming back online now. Things should be stable within ~5 minutes. We will be re-trying the update and will keep this thread updated if any further issues are experienced. Quote Link to comment Share on other sites More sharing options...
Michael D. Posted April 21, 2014 Author Report Share Posted April 21, 2014 The server is back online but will be sluggish for ~10 minutes while it re-loads it's caches and catches up to the surge of requests just after the outage. To be clear - the update that was performed was an innocuous update [the type that is performed all the time, on a regular basis]. The reason for the crash is, as of yet, unknown but we're working with the software vendor [idera Software/R1Soft] to diagnose the cause. Also - we did update 13 servers with this update and only Kobold crashed. Quote Link to comment Share on other sites More sharing options...
Michael D. Posted April 21, 2014 Author Report Share Posted April 21, 2014 MySQL is rebuilding some data due to the unexpected crash - it will be a few minutes before it's back online and I will update this thread. Quote Link to comment Share on other sites More sharing options...
Leah2 Posted April 21, 2014 Report Share Posted April 21, 2014 Whew - I just about had a heartattack while working on a clients website & I got an error establishing the database connection. Glad to know that MDD is already aware of the issue! Quote Link to comment Share on other sites More sharing options...
Michael D. Posted April 21, 2014 Author Report Share Posted April 21, 2014 MySQL is back online. ANything relying upon MySQL is going to be a little sluggish while the MySQL server refills it's caches and processes the onslaught of inbound queries due to the outage. This shouldn't take more than ~5 minutes or so. Quote Link to comment Share on other sites More sharing options...
Michael D. Posted April 21, 2014 Author Report Share Posted April 21, 2014 Due to the delayed restart of MySQL the sluggishness will last a little longer than expected but I'm watching it now and it's starting to subside. I do not expect the same issue when performing the update again, however, we will be waiting until around 1 AM ET before we start the update. It should only take moments and should not affect service but due to this issue experienced today we're not going to risk taking the server down again during peak-time. Quote Link to comment Share on other sites More sharing options...
Michael D. Posted April 21, 2014 Author Report Share Posted April 21, 2014 MySQL and the server as a whole appears to be back to normal.I am now starting the server-wide database check and repair. Quote Link to comment Share on other sites More sharing options...
Leah2 Posted April 21, 2014 Report Share Posted April 21, 2014 All my sites are back online and working correctly. Thanks! Quote Link to comment Share on other sites More sharing options...
Michael D. Posted April 22, 2014 Author Report Share Posted April 22, 2014 The R1Soft update was applied successfully. One new 'side-effect' of this update is that users on the Kobold server should now be able to do their own database restorations via the R1Soft icon in their cPanel. It may take a day or two for this to work [i am not sure that older restoration points will be capable of this, but new ones moving forward should be]. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.