[ mslm dvlpmnt ]

Backups are crucial, especially for those who work professionally on desktop computers. Computers can unexpectedly fail; you might wake up one morning to find your computer won’t boot. While you can potentially replace the computer, the files you’ve worked on for years and stored on it aren’t as easily replaceable. Unless you have a reliable backup strategy in place.

In this article, we’ll not only introduce a specific backup tool with a simple graphical interface but also delve into the entire strategy, which includes:

• Behavioral changes in your work habits,
• Data sovereignty and protection,
• Simplicity (low-tech) and reliability in backups.

People who work professionally on computers typically use desktop machines running Windows, Mac, or Linux. This article describes the backup strategy for Mac and Linux machines (I haven’t supported Windows for years).

Required Purchases

We don’t want to simply toss our valuable and sometimes sensitive work files into the ‘cloud’. Clouds are simply ‘someone else’s computer’. We want our data to stay with us, and we don’t want to depend on the internet, especially considering the scenario of computer failures mentioned earlier.

You’ll need 2 equally sized USB hard drives. The required size depends on your needs. As a developer, 1 terabyte drives suffice for me. A video editor or graphic designer might need more.

Acquire them and format them with a file system that allows read and write access from all your systems, if not already done.

Keep one hard drive constantly plugged into your computer, and store the other in a shaded cupboard, as it’s only for mirroring the first hard drive.

Occasionally, or perhaps automatically at all times, you should be able to mirror the first hard drive. It should be a complete copy of the first hard drive. Because hard drives can fail too, having an identical second copy of your work files gives you enough time to buy a new, second USB hard drive in case of failure. After all, the likelihood of two hard drives failing simultaneously is very low.

Since you need to connect at least 2 USB drives to your computer to mirror them, you should either have 2 free USB ports or consider purchasing a USB hub with multiple ports.

Behavioral Changes

Let’s be honest, working on the local hard drive and then transferring new files to the USB drive at the end of the day… hardly anyone does that. Or they simply forget. That’s why you should leave your local system as the operating system and keep your work files on the first USB drive as much as possible – open, edit, and save them from there. This may not be feasible for certain tasks, like working with large multimedia files; you’d prefer to work on the faster local SSD drive for those. But ideally, you’d work on your files on the USB drive, so you always have the latest version when it’s time to mirror them.

Backup Software

Especially on MacOSX, I’ve explored various backup tools, including the built-in Time Machine. They’re either not free, not intuitive, not fast enough, or hide their proprietary operations behind a graphical interface, which I personally dislike. Ultimately, a backup is just a copying process, and I prefer to see it clearly.

That’s why I use the proven and fast open-source command-line tool rsync. And here, I’ll show you how to install it and use it to mirror the first USB drive onto the second.

Why rsync?

Rsync is a tool for efficiently copying files between locations. Imagine you have a folder full of photos on your laptop and want to back them up to your external hard drive. Rsync can help you do that quickly and reliably. Here’s what makes it special:

• Smart updates: Rsync only transfers the parts of files that have actually changed since the last copy. This saves time and bandwidth compared to copying everything again.
• Preserves details: It can keep important file characteristics like creation date and permissions intact during the copy.
• Versatility: It can copy files locally (between folders on your computer) or remotely (between your computer and another device on a network).
• Error handling: Rsync can detect and handle errors during transfer, like connection drops, and attempt to resume from where it left off.

Here’s an analogy: Think of rsync like a librarian carefully comparing two books (source and destination) and only replacing the changed pages (modified parts of files) instead of copying the entire book again.

Overall, rsync is a powerful tool for keeping your files backed up and synchronized across different locations, making it a favorite among tech users.

Installing rsync

All commands mentioned here are executed in the command line.

On Debian-based Linux systems like Ubuntu:

To verify if rsync already exists, you can use the command:

dpkg -L rsync

This command will list the files installed for the rsync package. If rsync is installed, you will see a list of files. If not, you’ll receive an error message.

If rsync is not installed, you can install it using the following command in your terminal:

sudo apt update && sudo apt install rsync

This command will update the package list and then install rsync using the apt package manager.

That’s it! rsync should now be installed on your Ubuntu system.

On MacOSX:

To install rsync on the Mac, you’ll need the package management tool Homebrew. On their homepage, the first thing you’ll see is how to install it. At the time of writing this article, you can do that with the command line:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

From there, you can use Homebrew to install rsync. Of course, you can also compile the rsync source code yourself if you’re more experienced and don’t want to install Homebrew, but that’s more complicated and error-prone.

You can also uninstall Homebrew if you want. You can do that with the command:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall.sh)"

This won’t affect the installation of rsync or other tools installed with Homebrew; they’ll remain on the system. They just won’t be as easily updated, so I’d recommend leaving it installed.

To install rsync on the Mac, use:

brew install rsync

Once installed, you can update it as follows:

brew update && brew upgrade rsync

Issue the command:

rsync --version

and check it against the latest version listed on the rsync website.

Mirroring with Rsync

When you connect USB drives to Linux, they’re usually mounted under:

/media/[your-local-username]/[name_of-usb_drive]

On Macs, it’s under:

/Volumes/[name_of-usb_drive]

The rsync command is structured as follows:

rsync [some options] [source_drive] [destination_drive]

So if your first USB drive is named BACKUP1 and your second is BACKUP2, and you’re logged into your Mac system as Joe, the simplest command would be:

rsync /Volumes/BACKUP1/ /Volumes/BACKUP2/

But we want to use some options. For example:

rsync -av --info=progress2 --chmod=+r /Volumes/BACKUP1/ /Volumes/BACKUP2/

What do these options do?

• -a: Preserves permissions, Preserves timestamps, Transfers symbolic links, Preserves device files, Preserves owner (if possible)
• -v: The -v parameter is a helpful tool for monitoring the progress and details of your rsync file transfers.
• –info=progress2: Works only from rsync version 3.0 onwards. It shows you, as the name suggests, the global progress of the entire copying action. If you have to settle for an older rsync version, you still have the –progress parameter, which is a bit less detailed.
• –chmod=+r: I’ve added this parameter for security reasons, as in some cases of different file systems, my files and directories were no longer readable. In most cases, you don’t need it. But I just use it as a standard.

Execute the command, and you’ll see rsync copying your files to the other drive at top speed. Only the new and modified ones. This is a clean, stable, and fast backup for me, as it should be.

Problem: Your shell removes the last slash in this command.

Solution: Put the paths in quotation marks

rsync -av --info=progress2 --chmod=+r "/Volumes/BACKUP1/" "/Volumes/BACKUP2/"

If you don’t want to type or copy this long command every time, simply create a file named mirror_drives.sh wherever you prefer and write or copy the above command into it. In the future, you can initiate the mirroring process with ./mirror_drives.sh or sh mirror_drives.sh.

For Advanced Users

The command line above is already sufficient enough. But let’s level up.

You can get a bit more progress statistics by using:

--info=progress2,stats2

And when you want the file sizes to be human readable:

-h

When you always backup new or modified files, old files that you’ve since deleted remain on the target medium and accumulate. If you want to mirror your source medium 1-to-1, meaning you also want deleted files to be removed from the target medium, you need to tell rsync with this parameter:

--delete

Now, perhaps you don’t really want to delete the files that have been removed from the source medium, but rather keep them somehow. Then you need to instruct rsync to preserve these files in another directory. This can be done with:

--backup --backup-dir="/directory/of/deleted/files/"

You might also want to exclude certain directories or file types:

--exclude="*.tmp" --exclude=".Trash"

The complete rsync command would then look like this:

rsync -avh --delete --backup --backup-dir="/Volumes/BACKUP2/_deleted-files" --info=progress2,stats2 --chmod=+r --exclude="*.tmp" --exclude=".Trash" "/Volumes/BACKUP1/" "/Volumes/BACKUP2/"

Automate Backups

Now imagine you want to run this command line every night at 3 AM:

1. Write it in a text file and save it with the .sh file extension. To ensure your script has the correct shebang, add this as the first line:

#!/bin/bash

…and then the command line above, with the path and filename adjusted to your circumstances, of course.

2. Make sure your script is executable:

chmod +x /path/to/your/script.sh

3. Open the cron editor and choose a text editor if you’re doing this for the first time (e.g., nano):

crontab -e

4. Then add a new line with the following format:

0 3 * * * /path/to/your/script.sh

This line runs the script every day at 3 AM. You can adjust the timing according to your preferences:

• The first number (0) represents the minute (0-59)
• The second number (3) represents the hour (0-23)
• The three asterisks stand for: day of month, month, and day of week.

If your script produces output, it will be sent by email to the owner of the cron job by default. If you want to redirect the output to a file, you can do it like this:

0 3 * * * /path/to/your/script.sh > /path/to/logfile.log 2>&1

This redirects both standard output and error messages to the specified log file.

If you want to create a system-wide cron job (which runs with root privileges), you can use these files instead:

sudo nano /etc/cron.daily/myscript

And then place your script there (without the cron timing specification) and make it executable:

sudo chmod +x /etc/cron.daily/myscript

5. Save the file and exit the editor (in nano with CTRL+O to save and CTRL+X to exit).
6. To verify that your cron job was successfully created, you can run the following command:

crontab -l

7. Done.