WD My Passport Pro – RClone Backup to Cloud (AWS S3)

By 23/06/2019Uncategorized

I’ve setup my WDMPP to perform a regular cloud sync of my pictures into an Amazon S3 data store so that when it is on an internet connection, it will sit and run in the background and upload the pictures.

Note, I’m only backing up photos rather than video as I intend to run this on a 4G mifi hotspot and don’t want 4k video uploads to trash my data allowance.  I’ll run the risk of losing these in the event of a failure whilst mobile, but its something I can live with.

First all, you need to have rclone installed on your WDMPP which involves using the SSH terminal.  I’ll create a separate article at some point, but there is plenty of information about how to do this on the internet.

Create two files within the root of the harddrive

rclonescript.sh is the command which executes the backup script

rclone copy /media/sdb1/ AmazonS3:wdmpp.backup/ -v --log-file /media/sdb1/logs/rclone.log --copy-links --ignore-case --filter-from /media/sdb1/filestocopy.txt

Command Meaning
rclone copy Use the copy command in rclone
/media/sdb1/ Source root path to look for data
AmazonS3:wdmpp.backup/ Destination root path to send data. In this instance, I’m using AWS S3, but the same principle should work for other cloud services
-v Verbose mode
log-file /media/sdb1/logs/rclone.log rclone logs to this path (note, you’ll need to mkdir the logs directory)
–copy-links Follow Symlinks for copying – seems to be required
–ignore-case Because WDMPP backs up from a variety of devices, don’t be case sensitive when applying filters
filter-from /media/sdb1/filestocopy.txt This is the filtering definition rclone uses to identify the files to copy.

You will need to chmod +x this file to make it executable
chmod +x /media/sdb1/rclonescript.sh

/media/sdb1/filestocopy.txt is the filtering rules.

- /logs/
- /.USB/
- /.wdmc/
- /.wdcache/
- /.DS_Store/
- *.txt
+ *.jpg
+ *.png
+ *.heic
+ *.bmp
+ *.raw
- *

Include (-) / Exclude (+) File or Path Description
/logs/ Exclude the logs path where rclone writes its own log to
/.USB/ Exclude the system .USB path
/.wdmc/ Exclude the system .wdmc path
/.wdcache/ Exclude the system .wdcache path
/.DS_Store/ Exclude the system .wdcache path
*.txt Exclude any text files that exist (some of my camera devices create text logs which I’m not interested in copying).
+ *.jpg Copy any jpeg files with the extension jpg
+ *.png Copy any Portable Network Graphics files with the extension png
+ *.heic Copy any  High Efficiency Image File Format files with the extension heic (these come from my phone)
+ *.bmp Copy any bitmapped files with the extension bmp (Not expecting any of these, but heh)
+ *.raw Copy any RAW camera files (my camera uses the .raw extension
* Exclude anything else

You can obviously change your filters as you need to, for example including video files or whatever else you write to the disk. I had to put the excludes before the includes as I found otherwise it wouldn’t necessarily behave as expected.  This seems to work well for me.

Once you’ve tested that it works,  it can be added to cron
First, create the cron path

mkdir /var/spool/cron

Then create the crontab

crontab -e


8 * * * * /media/sdb1/rclonescript.sh >/dev/null 2>&1

In this crontab, it runs the script every 8th minute of each hour. If you’re not sure how to create a cron job, https://crontab-generator.org/ is a great website for building cron lines.

Author verdegemn

More posts by verdegemn

Leave a Reply