How VaultPress Works

We’ve recently updated the way VaultPress stores your backup data, to make it more space-efficient and to allow for faster restore times.

Now seems like a great time to describe how VaultPress works.

Replicated, but not duplicated

Everything that VaultPress backs up is pushed to a replicated filesystem. We store multiple copies of everything to protect your data.

However, VaultPress also works hard to avoid backing up duplicate copies of unchanged data we already have on file. Taking more copies of your data than our replication system already produces would be a waste of storage space, and would add unwanted extra traffic to your site during each backup.

Before taking a fresh copy of anything from your site, we check to make sure it has changed and is worth backing up again.

Keeping track of your data

Because we avoid unnecessarily duplicating unchanged data, each backup snapshot can contain data taken at various times, depending on when each file or database table was last changed. Most pieces of data in our replicated filesystem are reused in multiple backup snapshots, too.

To track which files belong in each snapshot, VaultPress generates a Manifest file for each snapshot. Manifest files contain a list of every piece of data that belongs in each snapshot.

snapshot_sharing

For example, if photo.jpg is changed once, we keep two copies of it and reference each version from multiple snapshot manifests.

Every day, VaultPress scans your site for any file or database table that is new or has changed, and generates a fresh backup copy as required.

Staying up to date

Some of our plans offer live backups, in which we keep an up-to-date backup of your site and generate hourly snapshots that include all of your latest updates.

If your VaultPress account has live backups enabled, the VaultPress plugin installed on your site quietly watches for changes to your data. Whenever you upload a file, create or edit a post, or make other changes to your site, the VaultPress plugin detects it and notifies VaultPress.com of the change.

When a file is changed on your site, VaultPress backs up a fresh copy of that file and includes it in the next snapshot manifest file.

When you make a change to a database table, however, VaultPress doesn’t take a full copy of that table. Some WordPress tables can get quite large, and re-copying a table for every update would be slow and bandwidth-hungry.

Instead, VaultPress creates a new file that describes what has changed, adds it to our replicated filesystem, and includes it in your next snapshot manifest file. As you make multiple changes to your database, multiple sequential SQL change files are stored to track each change.

table_updates

When a table is modified, we store a record of the changes between daily full backups. Each manifest contains more change records than the last.

Building a restore file

When you tell VaultPress to restore your site or choose to download a backup snapshot, VaultPress generates a new restore file. Each restore file is a zipped archive containing all of your site’s files and SQL table dumps to regenerate your database.

To generate a restore file, VaultPress first reads your backup snapshot’s manifest file. It uses that file to find every piece of data relevant to that backup and combines them into one archive file ready for restore or download.

Each table in your database is assembled from one full table dump, and all of the change records generated up to the snapshot you requested.

About the VaultPress update

Before the VaultPress update, we were storing each SQL row in a separate file in our replicated filesystem. That slowed our restore times, because each table had to be regenerated from potentially millions of individual files.

Now we store table dumps and changes, building a restore typically involves far fewer files per table.

Based on our tests, the new backup system reduces the time required to build restore files by anywhere from 25% to 90%, depending on the contents of each backup. Sites with large database tables will be affected the most, but the speed improvements will vary from site to site.

Aside from faster restore times, you shouldn’t notice any further differences in our backup service.

If you run into any trouble, or have any further questions, please let us know at vaultpress.com/contact.

This entry was posted in General. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s