Today I blogged about…

Gmail Backup and notmuch GUI

I am still working adding services to my NAS and one thing that was on my todo list for a while was a proper GMail backup.

You never know when the powers that be deny you access to your mails for some obscure reasons. One guard is using my own domain. When I a can’t use GMail anymore, I can just use my own mail server or a different provider. But it would suck losing all my old mail.

Backup

I don’t necessarily need a running IMAP server to access the backup. But the backup should be in a format that allows my to point an IMAP server to the backup and be ready to go. That means I want the mails in a Maildir directory.

For the backup I am using GMail’s IMAP server. To your mails by IMAP you need to enable IMAP, have 2fa enabled (as you should in general) and create an app password. You also want to hide some of the default tags from IMAP. Basically only the INBOX, your sent mail and all your custom tags should be exposed. This avoids downloading a bunch of duplicates.

To do the actual syncing, I decided to use the mbsync command from the isync package. More specifically I am using the theohbrothers/docker-isync Docker image.

The image will run one sync, then exit. I added a sleep after execution and then let Docker’s restart policy taking care of running the next sync.

compose.yml

services:
mbsync:
image: theohbrothers/docker-isync:latest
container_name: mbsync
restart: unless-stopped
user: "1000:100"
environment:
GMAIL_USER: ${GMAIL_USER}
GMAIL_PASS: ${GMAIL_PASS}
volumes:
- ./volumes/vmail:/mail
- ./config/mbsyncrc:/mbsyncrc:ro
command: <
sh -c 'while true; do
echo "[$(date)] Running sync...";
/sync;
echo "[$(date)] Sync finished. Sleeping...";
sleep 1800;
done'

The configuration for the synchronization is relatively straight if forward. I added a bunch of comments for your and my understanding:

config/mbsyncrc

### Remote Config ###
IMAPStore gmail-remote
Host imap.gmail.com
Port 993
UserCmd "echo $GMAIL_USER"   # from environment
PassCmd "echo $GMAIL_PASS"   # from environment
AuthMechs LOGIN
TLSType IMAPS
PipelineDepth 1              # Avoid quota errors
Timeout 60                   # More graceful timeouts

### Local Config ###
MaildirStore gmail-local
Path /mail/
Inbox /mail/INBOX
SubFolders Verbatim

### Sync Settings ###
Channel gmail
Far :gmail-remote:
Near :gmail-local:
Patterns *
SyncState *
CopyArrivalDate yes          # Don't mess up message timestamps when moving them between folders.
Sync Pull                    # Download changes only, don't sync local changes up to the server.
Create Near                  # Automatically create new folders in the local copy.
Remove Near                  # Automatically remove deleted folders from the local copy.
Expunge Near                 # Expunge deleted messages from the local copy.

With the appropriate user and app password in a .env file, this will create your backup (the first run can take a loooong time).

Introducing notmore.

Next I wanted a way to look at my backup. Of course I could set up an IMAP server and point a mail client to it, but that seems relatively involved. What I wanted was a quick way to search through the Maildir.

A quick web search revealed notmuch - a xapian based mail search engine (and more that I am not using). It is super fast and simple to use. But it’s CLI only. I wanted something with a web interface.

The general recommendation is netviel. A python/JavaScript interface to notmuch. Unfortunately it seems to be mostly abandoned with the last release 3 years ago and I had a really hard time getting it to work on modern Python versions within a Docker container.

So instead I decided to roll my own. I vibe engineered a PHP based search interface that wraps around the notmuch command line interface. It let’s you search using notmuch’s comprehensive syntax, displays the result threads and allows you to drill down into the individual mails. Attachments can also be downloaded through the interface. Simple, not more.

The tool is aptly named “notmore” and available on Github.

The repo also builds a ready to use Docker image. You can just point it at the Maildir created by the sync job above.

My full docker compose setup can be found in my github repository for the NAS docker setup.

Comments