kevnich83

Honestly, as I stated before, we're all human. Every tech and engineer has done something very STUPID like this in their career. I have broken more things than I can forget, but it's simply the process of learning. I can guarantee that he/she will NEVER forget this and I sincerely hope they're able to forgive themselves, LEARN and move on. Granted, the fact that they were able to run one command that has that devastating of effects on a production system is a little hard to believe. I come from a system where you have a lab that's as identical of equipment as you can and run all of your commands there, get a script from it then only after verifying everything is fine in the lab, take that script and run it on production. But again, I don't know their environment. If I were MDD, I would be contacting the storage vendor and see if there's a way to disable or require a confirmation or escalation before even being able to do a drop command like that. Humans are humans and are the weakest link in the tech world. You have to program your hardware and software around that fact, sadly.

This would be excellent advice for everyone. However, I'm also going to add that if you're using WHM (I haven't seen the options for Cpanel), there's a backup configuration that can be configured to automatically send daily server backups to an Amazon S3 account, then configure lifecycle rules, versioning rules, etc on the S3 side so you have continuous backups of your hosting reseller or VPS account remotely. I have backups going back a solid 12 months. These are daily, weekly and monthly. I then have lifecycle rules that push all backups over 30 days to Amazon Glacier and expire them out after one year. If you think it takes hours to set this up, it took me a grand total of about 15 minutes. That 15 min was well spent knowing my backups are transferred offsite every morning at 2am. Always, always have your own backups and use some scripting to try and automate the process to off-site your own backups so you don't have to remember to download your own server backups. At the end of the day, we're all human.