jlelse's Blog

Thoughts, stories and ideas

How I sync my Nextcloud with an S3 bucket using Drone CI and rclone

Published on in 👨‍💻 Dev
Short link: https://b.jlel.se/s/3fc
Share online  Translate 

A few days ago, an OVH data center went up in flames and was completely destroyed. This reminded me that it is important to store backups in another place on earth if possible, so better not in the same data center as the server that might burn down.

Lately, I’ve been using the Nextcloud instance I rent from Hetzner as my backup storage. However, that is in the same data center as some of my virtual servers, so not very secure. Instead, I now have backups set up with Scaleway S3 buckets. Using restic and rclone, my servers are backed up to storage buckets at Scaleway in Paris.

But in addition to that, I also set up a regular (daily) sync of my Nextcloud to a storage bucket. rclone makes it easy to copy files back and forth between different cloud storage providers and across different protocols.

On my Git server, I created a private repository that contains both the rclone configuration and a configuration for my self-hosted Drone CI instance. In Drone CI, a container is started with rclone, which then syncs the files from Nextcloud with the files in the Scaleway bucket.

kind: pipeline
type: docker
name: default

steps:
- name: sync
  image: rclone/rclone:1.54
  commands:
  - "rclone --config rclone.conf sync nextcloud: scaleway:mynextcloudbucket"

In the Drone settings it is then possible to set up a cronjob.

Drone CI settings with cronjob

And I have to say that works pretty well. The VPS has a fast network connection to the Nextcloud, but also to Scaleway and within minutes many gigabytes can be copied. If I tried to do that from home, it would take hours.

Tags:

Jan-Lukas Else
Interactions & Comments