How my data survived a corrupted hard drive
… and what I’ve learned about backups and self-hosting my files.
I haven’t posted anything since Monday because I moved my data from a self-hosted Nextcloud (the one at my home) to the Nextcloud-based service Storage Share by Hetzner (my cloud provider of choice). But why?
The reason for this and why I could not just move my data but had to do some intermediate steps, here:
On Tuesday, I wanted to copy some old photos I found on a Synology NAS (which is also located at home but not actually used) to my Nextcloud. From the Synology NAS, I could simply download them to my desktop via SFTP using Filezilla. But since Nextcloud only supports WebDAV (and I only learned about rclone later), I thought it would be best to copy the 1500 or so files directly to the server via SFTP, as that would probably be faster. Thought, done. But I couldn’t access the directory of Nextcloud directly with my SSH account and so I first uploaded the files to another directory and then copied them with
rsync to the right directory.
But this last step somehow didn’t work as it should and suddenly I got error messages telling me something about “I/O error” and “read-only disk” and somehow nothing wanted to work anymore. A reboot didn’t work either, because the device didn’t want to boot at all. Conclusion: The file system on the 2TB-HDD from Seagate was broken.
With some effort I got the device running again and tried to copy the files again, again it did not work. So don’t even try again but find another solution…
Since my self-hosted password manager Bitwarden (or bitwarden_rs) was also running on the same device, the first step was to restore the bitwarden_rs database using the backup and move it to one of my virtual servers (at Hetzner). Everything worked fine and the password manager could synchronize again.
When restoring the files, however, I seemed to have somehow broken the backup with restic (and resticker) (I got some weird errors), so I couldn’t simply restore my Nextcloud data as well. Too bad. But now comes my great luck. On Monday I switched the backup storage from Backblaze B2 to Scaleway, but left the data at B2, which means I still had a backup from Monday. Yay!
However, since my fear of another hardware or software failure was too great and I’ve heard good things about Hetzner’s Storage Share, I thought I’d better let pros manage the infrastructure for my Nextcloud in the future and move my files into the cloud.
For this purpose I temporarily created a powerful VPS with several large volumes attached, made a snapshot of the B2 bucket with the backup (180 GB) and downloaded and unpacked the ZIP file of the snapshot to the VPS using the b2-CLI. Using restic I restored the complete last file state and copied all relevant files with rclone (an awesome tool!) to my Storage Share. Since I haven’t uploaded many new files to my old Nextcloud since Monday, it was no big loss to restore the backup. I found rclone while searching for a way to copy files via CLI using WebDAV. rclone is the
rsync for a number of cloud providers and file transfer protocols.
I’m really grateful to myself for not yet having deleted the backup at B2. If I had done so, I would have irretrievably lost some photos I took from 2011 to 2012.
In the future I trust Hetzner to have built the hardware for the Storage Shares much more fail-safe than I had built my Nextcloud using an Odroid HC2 and a single HDD. Now I have to pay a few euros per month, but I can sleep well at night knowing that my files are not suddenly gone.
Why do I use Hetzner Storage Share instead of just Google Drive, OneDrive or Dropbox? I prefer to store my files with an European provider. Although there is a secret service in Germany that may read my data, the provider is not obligated to provide access to the NSA.
Tags: Backup, Nextcloud, rclone, restic