We’ve recently updated the way VaultPress stores your backup data, to make it more space-efficient and to allow for faster restore times.
Now seems like a great time to describe how VaultPress works.
Replicated, but not duplicated
Everything that VaultPress backs up is pushed to a replicated filesystem. We store multiple copies of everything to protect your data.
However, VaultPress also works hard to avoid backing up duplicate copies of unchanged data we already have on file. Taking more copies of your data than our replication system already produces would be a waste of storage space, and would add unwanted extra traffic to your site during each backup.
Before taking a fresh copy of anything from your site, we check to make sure it has changed and is worth backing up again.
Keeping track of your data
Because we avoid unnecessarily duplicating unchanged data, each backup snapshot can contain data taken at various times, depending on when each file or database table was last changed. Most pieces of data in our replicated filesystem are reused in multiple backup snapshots, too.
To track which files belong in each snapshot, VaultPress generates a Manifest file for each snapshot. Manifest files contain a list of every piece of data that belongs in each snapshot.
For example, if photo.jpg is changed once, we keep two copies of it and reference each version from multiple snapshot manifests.
Every day, VaultPress scans your site for any file or database table that is new or has changed, and generates a fresh backup copy as required.
Staying up to date
Some of our plans offer live backups, in which we keep an up-to-date backup of your site and generate hourly snapshots that include all of your latest updates.
If your VaultPress account has live backups enabled, the VaultPress plugin installed on your site quietly watches for changes to your data. Whenever you upload a file, create or edit a post, or make other changes to your site, the VaultPress plugin detects it and notifies VaultPress.com of the change.
When a file is changed on your site, VaultPress backs up a fresh copy of that file and includes it in the next snapshot manifest file.
When you make a change to a database table, however, VaultPress doesn’t take a full copy of that table. Some WordPress tables can get quite large, and re-copying a table for every update would be slow and bandwidth-hungry.
Instead, VaultPress creates a new file that describes what has changed, adds it to our replicated filesystem, and includes it in your next snapshot manifest file. As you make multiple changes to your database, multiple sequential SQL change files are stored to track each change.
When a table is modified, we store a record of the changes between daily full backups. Each manifest contains more change records than the last.
Building a restore file
When you tell VaultPress to restore your site or choose to download a backup snapshot, VaultPress generates a new restore file. Each restore file is a zipped archive containing all of your site’s files and SQL table dumps to regenerate your database.
To generate a restore file, VaultPress first reads your backup snapshot’s manifest file. It uses that file to find every piece of data relevant to that backup and combines them into one archive file ready for restore or download.
Each table in your database is assembled from one full table dump, and all of the change records generated up to the snapshot you requested.
About the VaultPress update
Before the VaultPress update, we were storing each SQL row in a separate file in our replicated filesystem. That slowed our restore times, because each table had to be regenerated from potentially millions of individual files.
Now we store table dumps and changes, building a restore typically involves far fewer files per table.
Based on our tests, the new backup system reduces the time required to build restore files by anywhere from 25% to 90%, depending on the contents of each backup. Sites with large database tables will be affected the most, but the speed improvements will vary from site to site.
Aside from faster restore times, you shouldn’t notice any further differences in our backup service.
If you run into any trouble, or have any further questions, please let us know at vaultpress.com/contact.