This was a really useful assistance on setting up and configuring VAAPI / Intel Skylake GPU on ESXi 6.7
I’ve setup my WDMPP to perform a regular cloud sync of my pictures into an Amazon S3 data store so that when it is on an internet connection, it will sit and run in the background and upload the pictures.
Note, I’m only backing up photos rather than video as I intend to run this on a 4G mifi hotspot and don’t want 4k video uploads to trash my data allowance. I’ll run the risk of losing these in the event of a failure whilst mobile, but its something I can live with.
First all, you need to have rclone installed on your WDMPP which involves using the SSH terminal. I’ll create a separate article at some point, but there is plenty of information about how to do this on the internet.
Create two files within the root of the harddrive
rclonescript.sh is the command which executes the backup script
rclone copy /media/sdb1/ AmazonS3:wdmpp.backup/ -v --log-file /media/sdb1/logs/rclone.log --copy-links --ignore-case --filter-from /media/sdb1/filestocopy.txt
|rclone copy||Use the copy command in rclone|
|/media/sdb1/||Source root path to look for data|
|AmazonS3:wdmpp.backup/||Destination root path to send data. In this instance, I’m using AWS S3, but the same principle should work for other cloud services|
|—log-file /media/sdb1/logs/rclone.log||rclone logs to this path (note, you’ll need to mkdir the logs directory)|
|–copy-links||Follow Symlinks for copying – seems to be required|
|–ignore-case||Because WDMPP backs up from a variety of devices, don’t be case sensitive when applying filters|
|—filter-from /media/sdb1/filestocopy.txt||This is the filtering definition rclone uses to identify the files to copy.|
You will need to chmod +x this file to make it executable
chmod +x /media/sdb1/rclonescript.sh
/media/sdb1/filestocopy.txt is the filtering rules.
|Include (-) / Exclude (+)||File or Path||Description|
|–||/logs/||Exclude the logs path where rclone writes its own log to|
|–||/.USB/||Exclude the system .USB path|
|–||/.wdmc/||Exclude the system .wdmc path|
|–||/.wdcache/||Exclude the system .wdcache path|
|–||/.DS_Store/||Exclude the system .wdcache path|
|–||*.txt||Exclude any text files that exist (some of my camera devices create text logs which I’m not interested in copying).|
|+||*.jpg||Copy any jpeg files with the extension jpg|
|+||*.png||Copy any Portable Network Graphics files with the extension png|
|+||*.heic||Copy any High Efficiency Image File Format files with the extension heic (these come from my phone)|
|+||*.bmp||Copy any bitmapped files with the extension bmp (Not expecting any of these, but heh)|
|+||*.raw||Copy any RAW camera files (my camera uses the .raw extension|
|–||*||Exclude anything else|
You can obviously change your filters as you need to, for example including video files or whatever else you write to the disk. I had to put the excludes before the includes as I found otherwise it wouldn’t necessarily behave as expected. This seems to work well for me.
Once you’ve tested that it works, it can be added to cron
First, create the cron path
Then create the crontab
8 * * * * /media/sdb1/rclonescript.sh >/dev/null 2>&1
In this crontab, it runs the script every 8th minute of each hour. If you’re not sure how to create a cron job, https://crontab-generator.org/ is a great website for building cron lines.