I almost lost a bunch of data this week. Here’s how my backups saved me (just), and what I’ve changed.

Warning

What happened

Actually this started months ago. I accidentally deleted my entire Documents folder! I was being too quick on the keyboard and I literally just Cmd+A then Cmd+Deleted everything in there. Whoopsy.

I undid by pressing Cmd+Z a second later. But the damage was done. I store my Documents folder in iCloud Drive and I knew as soon as it happened that I’d likely triggered a cascade of network activity. My command to delete would have still been half-way to the cloud when I issued the command to undo (i.e. move those files out of my Trash back to Documents), and it would be a more robust service than iCloud that handled that without issue.

What I should have done at that point was stop everything and spend a few hours making sure everything was in place. I didn’t do that.

Discovering the issue

This week I finally found time to upgrade myself to Life Admin v2. I thought I’d take the opportunity to really clean up some old stuff, and in doing so I realised that a bunch of really old archive stuff was missing.1 The folders were there, but there was nothing in them.

So now we have a problem: I need to get a backup, but not one from yesterday. That would be easy. I need to get one from months ago, and I’m not even sure exactly when.

Backups worked, but…

Obviously I have backups. I wrote a whole series about it. But backups are complex, mine especially so. This is of my own making, but also hard to avoid given my situation. (I won’t recap here: read that series.)

Here are a few of the problems I encountered.

• Not knowing which backup, as in from which date, to restore.

• I use Arq and it allows you to search for a specific file across all of your backups. It’s really fast. This made things much easier.

• Its restore is also really fast, and this backup was local, i.e. a hard drive in the same building, not cloud. So I could quickly do test restores to identify which was the right one.

• Not knowing which backup, as in literally which one, to restore.

• Because the way I’d configured them – which I think is the way most people configure them – didn’t make it really obvious where things were.

• So when I came to use them, there was confusion. Not what you want in a time of crisis.

• More on this below.

• My backups are encrypted.

• Which they should be. And I use 1Password, and my vault is reasonably neat.

• But when I tried the password I thought it should be, it failed. Shit. I tried another. Failed! Oh, shit. This is bad.

• I must have mis-copy/pasted the first password. When I tried again, that worked.

• But to have any possibility of doubt at all here is a total failure.

‘Set and forget’

This whole thing got me wondering if we’re doing it all wrong. Backup services advertise themselves as ‘set and forget’: just install, then never think about it again!

That’s what got me into this problem.

The received wisdom is that you should occasionally test your backups. But you know what I think most people do? They think of a file that they deleted from their Desktop yesterday and they restore that. This is not a realistic test. It doesn’t stress you, or your system.

Conscious backups aka ‘restore points’

I think there’s room in life for two types of backups. The first is the one I’ve already got: the ‘set and forget’. 90% of the time this is going to do the job.

But the other type is what I’m calling a ‘restore point’. In the IT world, a restore point is a backup at a specific point in time. You’re saying that you can restore to this point in time.

Today is a restore point

I finished my migration to LAS v2 today, and with it a bunch of tidying up and moving around of data.

Today – now – my system is in a really nice state. I’ve just looked at everything and I’m deeply familiar with it. I’m sure it’s good. Today is a restore point.

So I’m configuring a new backup. Here’s what makes it different from my daily backups.

‘Restore point’ rules

1. They are manually triggered.

When I decide the time is right, I’ll run this backup. I’ll be adding something to my task system to prompt me to do this, perhaps quarterly.

1. They have no retention rules.

Normal backups are smart about the data they retain. Your computer might perform a backup every hour, but you don’t need to keep every one of those backups. That would be a waste of space.

A typical ‘retention cycle’ might be to keep:

• Every hour for the last 72 hours.
• Every day for the last 30 days.
• Every week for the last 52 weeks.
• Every month forever.

My ‘restore points’ have no retention cycle. Every one is saved forever.

1. They have no exclusion rules.

My daily backup excludes a bunch of files and folders. Anything that looks temporary, or like a cached version, or like something I’ve downloaded, or like something that I know I could get from the cloud, is excluded.

For my daily rules, this keeps them lean and fast. This is particularly important for me as I live on the road. But that isn’t relevant for ‘restore point’ backups. It is more important that they are complete, so they have no exclusions.

---

• They must be documented in your JDex.

Lesson 1: if something unexpected happens, don’t just move on. Deal with it completely.

---

100% human. 0% AI. Always.

Footnotes

Specifically, the file that started Johnny.Decimal: a carefully-designed PDF of ticket prices that was stuck to our office wall. I wanted a way to reference this file that wasn’t an ugly file path, so I came up with the idea of the numbers.

That was 15 years ago, and I have no use for this file today. I think about it perhaps once a year. But I’d be devastated if I lost it.

↩

┌

─ ◁/▶ ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────

┐

•

└

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

┘