24 replies to this topic

[ericr]

S3's fsck is running.  I had some unexpected issues getting the server to boot back up.  it should be completed within 10 minutes.

I am rebooting s4 to start it's fsck.

[ericr]

S3 is already done and booting.

[ericr]

S4 is online.  I will ask that if you have issues at this point to please open a ticket so that we can investigate.

[Laimonas]

I would also appreciate info on S1 today's outage reasons for 2 hours.

[MikeDVB]

I would also appreciate info on S1 today's outage reasons for 2 hours.

The network disruption to our storage platform is the cause for the outage on S1 I believe.

 

We're currently working with our storage vendor as well as our facility to determine exactly how this happened so we can prevent it.  We are using a fully dual redundant network where any part of it can fail and everything remains online.  This failure was outside of our network border, however, it still caused connectivity issues and that's why we're still investigating.

 

Once I know why it happened I will post it here.  I do not, as of yet, have that diagnosis.