A while ago I wrote a post about my backup solution and replacing Crashplan – a once great product I was a happy user of. It served pretty much all my backup needs in one product, but alas it was too good to last.

Eventually I settled on Duplicati on my home server backing up to Backblaze, and Urbackup to back up my various devices to the NAS. But since then a few things have changed:

• The upgrade to Ubuntu 18.04 broke the Urbackup installation on my server. I never really got around to fixing it, so my device backups have been manual. Fortunately the server hosts the important stuff, and I don’t keep much on my devices that aren’t saved elsewhere, but it’s still not ideal.
• If a broken server wasn’t enough, Urbackup discontinued support for MacOS earlier this year, which made the product useless to me.
• Perhaps somewhat mitigating this for Mac clients, the Samba project released version 4.8.0, which includes support for MacOS time machine (see “Time Machine Support with vfs_fruit”).
• Dropbox have started being dicks.

… Dropbox?

Er, yeah. Despite writing “I think that you should never use Dropbox for anything remotely private or sensitive”, words that I stand by today, I have not only been using Dropbox… but for private and sensitive things.

The main reason is that it’s the only sync tool that’s always worked well across Windows, Mac and Linux (I regularly use all 3). I’ve also had a free 6.8GB plan for years, from the days when you could do something as simple as install the app and get 500MB, or enable camera uploads and get a free 2GB. This has always been enough for my documents, plus camera uploads from my phone so long as I archive them off once a year or so.

I probably would have paid for it if the plans and pricing were more granular, but I have no need for 2TB, and £8 per month still seems a bit much to pay for the ~20GB of data I would like synchronised between devices. Historically it has cost a lot more – in the region of £15 (for 1TB) if I recall correctly.

On security, I feel the risk of exposure is small. With a strong unique password, and 2FA courtesy of a Yubikey, the greatest risks are a probably a rogue employee or large-scale hack of Dropbox via a software bug. I’m sure there would be a lot of people in deeper trouble than me if that happened.

So how have they started being dicks? Three ways:

1. Discontinuing support for filesystems other than ext4 on Linux.
2. Enforcing a limit of 3 devices on free users.
3. Trying to get more in your face, and milking the customer base, rather than just sticking to file sync and keeping out of the way.

Despite being the Rolls-Royce of file-syncing (reliability and functionality wise, definitely not security wise), these things have made the product a lot less appealing recently. But then I was never a paying user, so perhaps this is just good strategy! Perhaps I would have been though, if they’d just stuck to basics, supported linux properly and kept costs down, rather than trying to expand into the enterprise market…

So while this might seem a digression (because sync is not backup!), I bring up Dropbox because I discovered a new tool which suits the Duplicati setup well and is probably going to replace it for me: syncthing.

Syncthing

Syncthing is a decentralised file synchronisation tool. Your files never go to the cloud or anyone else’s hardware; they are copied directly between your devices. This creates some interesting problems for restricted networks and NAT traversal, so syncthing supports a concept of relays, which essentially provide a connection path for your devices. The data is passing through the relay, but it’s encrypted only for the receiving device with perfect forward secrecy. So even if a relay was compromised, and the attacker captured all the traffic, AND he knew the device IDs of both sides, he still wouldn’t be able to read the data. The docs have a short write-up on security here.

And there is some interesting data on its usage here – this is clearly a widely-used project.

So it works practically anywhere, syncs my files privately, appears to be a sustainable project, and unlike Dropbox it doesn’t rely on me storing my data in a way that any one else could read without access to one of my devices. What’s not to like?

A little friction never hurt anyone

Syncthing is by techies for techies. Because it’s a write-once run everywhere application (even the Android version installs the same go-based back-end and web UI), it doesn’t integrate seamlessly on any platform. Let’s start with installation:

• MacOS is probably the easiest courtesy of homebrew – you just brew install syncthing, followed by brew services start syncthing and you’re ready to go. It even opens the web UI for you.
• Android is just as easy courtesy of the app wrapper – install the app, grant permission to allow it to run in the background and you’re done.
• Debian/Ubuntu Linux comes in third, because the package doesn’t install a systemd unit. It’s simple enough to create, but not everyone is a sysadmin.
• Windows comes last, because pointing and clicking around in Task Scheduler sucks.

None of these are any trouble at all for a technical user, and the approach is perfectly reasonable. With a small team of volunteers, maintaining native ports for each device is impossible. But it has to be mentioned, because for your average Dropbox user, having to manage a system service is going to be a non-starter.

Next is integrating a new device into your “cluster”. It works as designed, but in an asynchronous/eventually-consistent way. So after adding a new device… nothing happens, until some background process kicks in on the other device and a pop-up appears in the UI prompting you to accept. Again, nothing wrong with this, it’s just not intuitive to someone unfamiliar with the asynchronous nature of the product, and the UI gives little guidance.

Other (minor!) complaints

• The Android app is very basic, and seems like it won’t be terribly nice to my battery, but we’ll have to see.
• There is no iOS app, mainly because it can’t run go code. Thanks to Apple’s love of homogeneity, the whole thing would have to be rewritten in Swift or Objective-C and it’s not surprising that no one has taken this on.
• The web UI uses HTTP basic auth, which precludes the use of plugin-based password managers, and is easy to brute-force. Something form-based would be preferable for devices that have to expose it to the network.
• A web UI listening on localhost is potentially a large attack surface, as code running in your web browser can call out to it. Firefox and Chrome have some protection against this, but I’d suggest setting an admin password on every device, not just servers. That way if some JS code in your browser tries to brute force it, you’ll at least notice your lap burning!

Overall though, in my early testing, syncthing appears to work brilliantly once it’s set up. Cudos to the developers, because this is a hard problem, and other than some window-dressing it’s difficult to see how anyone could do a better job.

Time Machine and Samba

The last piece of backup-related news since my last article is that Samba added support for time machine – the built-in MacOS backup utility that supports backing up your files to a remote volume. I haven’t gotten around to testing this yet, because the required version of Samba didn’t land in Ubuntu until 18.10 and I’m sticking with LTS releases. But if it works as advertised, I can’t see why you’d want to use anything other than time machine to backup a Mac.

This page provides some nice clear instructions if you’re so inclined – just skip to the configuration if you’re already on Ubuntu 19.04 or another distro that provides samba >= 4.8.0.

Summary

My backup solution has evolved a fair bit. I didn’t really consider file synchronisation as part of the strategy, but having an open-source private solution such as syncthing, which can sync to my home server, means those files are now being backed up in exactly the same way. Backblaze B2, Duplicati, and syncthing are all I need for now; and when Ubuntu 20.04 is released with a newer version of Samba, Time Machine will be icing on the cake.