Home Backup tools How To Backup Files And Directories Using Rsync In Linux

How To Backup Files And Directories Using Rsync In Linux

By Karthick
Published: Last Updated on 718 Views

Looking for a reliable and robust solution to backup your data in Linux? Rsync got you covered! This guide walks you through the steps to backup files and directories using Rsync in Linux.

1. Introduction

If you never back up your data considering everything will work fine, then I would say you are wrong.

Just ask any Linux users out there, they will share the pain of not backing up their system/data. As a newbie, I did that mistake once and learned the importance of backup.

There are plenty of Backup tools exists for Linux and Unix platforms. Rsync (Remote Sync) is considered to be one of the most widely used and trusted backup tool.

2. Backup files and directories using Rsync in Linux

Using Rsync, we can,

  1. backup entire system,
  2. or backup individual files and directories.

In the first type, we copy the entire Linux file system to an external device or a network share. You can use this copy to bring the machine to its original state in case of any system failure.

The second type is backup or copy the selective files and directories only. For example, you can copy the important config files or the whole $HOME directory, and exclude everything else.

We already have shown you how to backup entire Linux system using Rsync. This guide focus on selective backup files and directories using Rsync in Linux.

2.1. Choosing storage medium

As the first step, decide where you are going to backup your data. It is very important!

You can backup your data to an external hard disk, a Cloud storage, or a Network-attached Storage (NAS). A simple and cheap option is to use an external hard disk.

If you have two separate drives in your machine, you can use any one of the drives to backup your data considering both will not fail at once.

Note:

If you are serious about Backup, don't put the backup in the same drive itself. Always use external drive, secondary drive, network share and cloud storage etc.

For demonstration purpose, I will be using a 1 TB external hard disk throughout this guide.

2.3. Backup files using Rsync

The general syntax of Rsync to copy data from one location to another is:

rsync [OPTION…] SRC… [DEST]

Here,

  • OPTION - the list of Rsync options
  • SRC - Source file/directory
  • DEST - Destination

Here is a simple Rsync command to backup or copy file from one location to another in the local system itself.

$ rsync src_file dest_file

Examples:

$ rsync -auv ostechnix.txt ostechnix-bak.txt

Here,

  • -a - Archive mode mode (maintaining file properties and sym-links)
  • -u - Skip files that are newer on the receiving end.
  • -v - Verbose mode

The above command saves the backup file ostechnix-bak.txt file in the current directory. You can also specify different destination directory like below:

$ rsync -auv ostechnix.txt Downloads/ostechnix-bak.txt

If you don't specify the target filename, Rsync will use the same name of source file to the target file.

Saving backup files in the same disk is not actually a backup. You must save the backup data on an external drive or network share or at least in a different partition.

For instance, the following command backup the file to my external drive:

$ rsync -auv ostechnix.txt /media/karthick/"My Passport"/

Rsync has many advanced options. Take a look at the following example:

$ rsync -azvhP ostechnix.txt /media/karthick/"My Passport"/

This command transfers file in archive (to preserve attributes) and compressed (zipped) mode with verbose and human-readable progress bar.

2.4. Backup directories using Rsync

Instead of backing up single file, you can put all files in a directory and backup that directory with Rsync. This comes in handy when mirroring and synchronizing data.

For the purpose of this guide, I will show you how to backup HOME directory using Rsync.

2.4.1. Backup HOME directory using Rsync

Rsync is an incremental file copying tool. Meaning - it only syncs new or modified data in the source directory (i.e. /home in our case) instead of syncing all the data again from scratch.

To backup my $HOME directory to my external drive, I used the following command:

$ rsync -avxP --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick /media/karthick/"My Passport"/

Let us break down the above command and see what each option does.

  • The flag -a is the combination of many options. This flag takes care of recursive copy and retaining the attributes like file/directory permission, owner, etc.
  • The flag -v will enable the verbose mode that will give more logs in the terminal when you run the rsync command.
  • The flag -x will restrict rsync to sync data within the file system boundaries. If you have mounted any file system inside your home directory that will be skipped during the sync operation.
  • The flag -P is the combination of two flags. It will display the progress of each sync and keep partially transferred files.
  • The flag --delete will delete all directories and files that are not available in the source but available in the destination. You must be very careful while using flag.
  • The flag --exclude-from=File accepts a file as an argument. This file contains the list of files and directories that will be ignored by rsync.

I have created a file named ignore_list.txt which contains the files and directories that need to be excluded.

In the above command, this part --exclude-from=/home/karthick/Work/Backup/ignore_list.txt defines the exclusion list.

Here is the contents of the ignore_list.txt file:

Excluded list
Excluded list

The list of items to be excluded from the above image is my personal choice and it may differ for you. Modify the parameter according to your environment.

The last two parts of the above Rsync command are the source and destination locations. In my case, the source is my $HOME directory and the destination is my external drive. Replace source and destination according to your environment.

Heads Up: If you want to know the mount point for your external drive, run the lsblk command.

$ lsblk

sdb             8:16   0 931.5G  0 disk  
└─sdb1          8:17   0 931.5G  0 part  /media/karthick/My Passport

Before running the rsync command for the first time, it is always recommended do a dry run. The dry run will simulate what changes will be made. Use the same command but with the -n flag.

$ rsync -avxP --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick /media/karthick/"My Passport"/

Sample output:

 sending incremental file list
 karthick/
 karthick/.bash_history
         20,527 100%    0.00kB/s    0:00:00 (xfr#1, ir-chk=1056/1060)
karthick/.shutter/
karthick/.shutter/drawingtool.xml
            228 100%    4.54kB/s    0:00:00 (xfr#2, ir-chk=1037/1060)
karthick/.shutter/profiles/
karthick/Pictures/
karthick/Pictures/Selection_001.png
         37,626 100%  349.94kB/s    0:00:00 (xfr#3, to-chk=47/3001)
karthick/Pictures/Selection_002.png
         25,815 100%  227.12kB/s    0:00:00 (xfr#4, to-chk=46/3001)
karthick/Pictures/Selection_003.png
         63,275 100%  532.69kB/s    0:00:00 (xfr#5, to-chk=45/3001)
karthick/Pictures/Selection_004.png
         77,965 100%  581.20kB/s    0:00:00 (xfr#6, to-chk=44/3001)
karthick/Pictures/Selection_005.png
         75,863 100%  532.99kB/s    0:00:00 (xfr#7, to-chk=43/3001)
karthick/Pictures/Selection_006.png
         11,628 100%   75.20kB/s    0:00:00 (xfr#8, to-chk=42/3001)
karthick/Pictures/Selection_007.png
         24,311 100%  150.26kB/s    0:00:00 (xfr#9, to-chk=41/3001)
karthick/Work/Backup/
karthick/Work/Backup/home_dir_backup.sh
            764 100%    4.34kB/s    0:00:00 (xfr#10, to-chk=20/3001)
karthick/Work/Backup/home_dir_backup.sh~
            796 100%    4.37kB/s    0:00:00 (xfr#11, to-chk=19/3001)
karthick/Work/Backup/ignore_list.txt
            182 100%    1.00kB/s    0:00:00 (xfr#12, to-chk=18/3001)
karthick/Work/Backup/ignore_list.txt~
            182 100%    1.00kB/s    0:00:00 (xfr#13, to-chk=17/3001)
...
...
...
sent 88,424 bytes  received 9,459 bytes  195,766.00 bytes/sectotal size is 3,028,935,519  speedup is 30,944.45 (DRY RUN)

If everything goes as you wanted,  remove the -n flag and run the command to backup your data.

$ rsync -avxP --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick /media/karthick/"My Passport"/

Depending upon the size, the first run will take some time.

sending incremental file list
karthick/
karthick/.bash_history
         20,527 100%    0.00kB/s    0:00:00 (xfr#1, ir-chk=1056/1060)
karthick/.shutter/
karthick/.shutter/drawingtool.xml
            228 100%    4.54kB/s    0:00:00 (xfr#2, ir-chk=1037/1060)
karthick/.shutter/profiles/
karthick/Pictures/
karthick/Pictures/Selection_001.png
         37,626 100%  349.94kB/s    0:00:00 (xfr#3, to-chk=47/3001)
karthick/Pictures/Selection_002.png
         25,815 100%  227.12kB/s    0:00:00 (xfr#4, to-chk=46/3001)
karthick/Pictures/Selection_003.png
         63,275 100%  532.69kB/s    0:00:00 (xfr#5, to-chk=45/3001)
karthick/Pictures/Selection_004.png
         77,965 100%  581.20kB/s    0:00:00 (xfr#6, to-chk=44/3001)
karthick/Pictures/Selection_005.png
         75,863 100%  532.99kB/s    0:00:00 (xfr#7, to-chk=43/3001)
karthick/Pictures/Selection_006.png
         11,628 100%   75.20kB/s    0:00:00 (xfr#8, to-chk=42/3001)
karthick/Pictures/Selection_007.png
         24,311 100%  150.26kB/s    0:00:00 (xfr#9, to-chk=41/3001)
karthick/Work/Backup/
karthick/Work/Backup/home_dir_backup.sh
            764 100%    4.34kB/s    0:00:00 (xfr#10, to-chk=20/3001)
karthick/Work/Backup/home_dir_backup.sh~
            796 100%    4.37kB/s    0:00:00 (xfr#11, to-chk=19/3001)
karthick/Work/Backup/ignore_list.txt
            182 100%    1.00kB/s    0:00:00 (xfr#12, to-chk=18/3001)
karthick/Work/Backup/ignore_list.txt~
            182 100%    1.00kB/s    0:00:00 (xfr#13, to-chk=17/3001)…
…
…
sent 3,029,863,865 bytes  received 51,335 bytes  53,626,817.70 bytes/sec
total size is 3,028,934,994  speedup is 1.00

2.4.2. Create backup script

Remembering a lengthy command might be hard. To avoid typing the long rsync command in the terminal, you can create a simple BASH script and call it whenever you want to initiate the backup.

Create a script named home_dir_backup.sh:

$ nano home_dir_backup.sh

Add the following lines:

#!/usr/bin/env bash

# PURPOSE : RSYNC COMMAND TO BACKUP MY HOME DIRECTORY TO EXTERNAL DEVICE..
# REPLACE SOURCE, DESTINATION AND EXCLUDE ACCORDING TO YOUR ENVIRONMENT.

rsync -avxP --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick /media/karthick/"My Passport"/

Add, remove or change the Rsync options as per your requirement. Also change the source and destination paths with your own. Save and close the file.

Make the script executable:

$ chmod +x <path-to-home_dir_backup.sh>

Now run the script at anytime to start the $HOME directory backup:

$ ./path-to-home_dir_backup.sh

Use a shorter script name to easily remember it.

You can also create a soft link for the script file in /usr/bin or any user-specific directories that is available in the $PATH variable.

2.4.3. Create alias

The another way to avoid typing long commands is to use aliases.

Edit your ~/.bashrc file and add the following line at the end:

alias backup="rsync -avxP --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick /
media/karthick/"My Passport"/"

Source the bashrc file to take effect the changes immediately:

$ source ~/.bashrc

Now simply run the following command to backup the $HOME directory:

$ backup

It is up to you to decide what do you want to use. Either you can create an alias or create a script and run rsync by triggering the script. In my case, I have created a script and then created an alias for that script.

3. Schedule backup

You can use cron scheduler to schedule backup jobs to run at a particular time on a particular day.

Edit the current user's cron jobs using command

$ crontab -e

Define your backup schedule. Say for example, to run backup every day at 3am, use the following line:

0 3 * * * <rsync-command-to-execute>

Depending upon your requirement you can schedule daily, weekly or monthly jobs.

If you're new to Cron Jobs, we have compiled a list of useful cron job examples to schedule tasks in Linux. For more details on cron usage, refer the following guide.

4. Conclusion

In this guide, we looked at how to backup files and directories using Rsync in Linux. As you can see, copying files and directories with Rsync is very easy!

We also have shown you how to take backup your home directory using the rsync command and some of the ways to run and schedule it.

This method can also be applied to any directory or file structure in Linux to backup the data. If you think you have a better backup solution, we would love to hear it from you via the comment section below.

You May Also Like

1 comment

Jalal Hajigholamali July 21, 2021 - 10:47 am

Hi,
I prefer rsync,
Very useful command

Reply

Leave a Comment

* By using this form you agree with the storage and handling of your data by this website.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More