Anyone who has worked as a system administrator or network engineer will understand where I’m coming from with that question. As business owners, online publishers, and web service providers, we often focus on the process of creating backups and designing disaster recovery plans. But far less attention is paid to regularly checking that those processes are working as intended.
Backups use a combination of network, server, and software. At every stage of a complex backup process, there’s room for something to go wrong. Even worse, server backups can fail silently in unanticipated ways. You might be under the confident impression that your backup scripts are running, only to find that when you really need them, the data is corrupt, out-of-date, or simply not there at all.
GitLab’s recent data loss incident is an important lesson to us all. Because of an error by a system administrator, a chunk of GitLab.com data was deleted. That’s regrettable, but with adequate backups, it needn’t have been catastrophic. Unfortunately, when GitLab tried to restore the deleted data, they found their backup process had failed silently — user data was irretrievably lost.
It’s not enough to create backups and devise automated processes. Anyone whose business depends on data should also regularly verify that comprehensive and up-to-date backups are created as expected.
What can businesses do to make sure their backups are reliable?
- Automate. Reliable backup processes must be automated. People forget, they procrastinate, and they make mistakes. A well thought-out automated backup process is less prone to failure than one that depends on human intervention.
- Move data offsite. If your backup drives are kept in the same office or data center as your servers, you’re vulnerable. Your businesses data should exist in more than one location — and the more locations it exists in, the safer it is. Cloud data storage plans make it easier and cheaper than ever before to spread the risk.
- Verify. Automation gets you part of the way to data security, but as I’ve already discussed, if you don’t regularly verify the status of backups, you can’t know that they’re viable. I recommend creating a consistent routine of test restores and monitoring, and making someone responsible for ensuring the validity of all backups.
Finally, if your organization doesn’t have the technical expertise to automate, move data offsite, and verify backups, outsource your backup processes to someone who does. Our FutureProtect Server Backup service offers comprehensive server backups with continuous data protection.