Looking for a reliable and robust solution to backup your data in Linux? Rsync got you covered! This guide walks you through the steps to backup files and directories using Rsync in Linux.
Table of Contents
1. Introduction
If you never back up your data considering everything will work fine, then I would say you are wrong.
Just ask any Linux users out there, they will share the pain of not backing up their system/data. As a newbie, I did that mistake once and learned the importance of backup.
There are plenty of Backup tools exists for Linux and Unix platforms. Rsync (Remote Sync) is considered to be one of the most widely used and trusted backup tool.
2. Backup files and directories using Rsync in Linux
Using Rsync, we can,
- Backup entire system,
- Br backup individual files and directories.
In the first type, we copy the entire Linux file system to an external device or a network share. You can use this copy to bring the machine to its original state in case of any system failure.
The second type is backup or copy the selective files and directories only. For example, you can copy the important config files or the whole $HOME
directory, and exclude everything else.
We already have covered how to backup entire Linux system using Rsync.
This guide focus on selective backup files and directories using Rsync in Linux.
2.1. Choosing storage medium
As the first step, decide where you are going to backup your data. It is very important!
You can backup your data to an external hard disk, a Cloud storage, or a Network-attached Storage (NAS). A simple and cheap option is to use an external hard disk.
If you have two separate drives in your machine, you can use any one of the drives to backup your data considering both will not fail at once.
Note:
If you are serious about Backup, don't put the backup in the same drive itself. Always use external drive, secondary drive, network share and cloud storage etc.
For demonstration purpose, I will be using a 1 TB external hard disk throughout this guide.
2.3. Backup files using Rsync
The general syntax of Rsync to copy data from one location to another is:
rsync [OPTION…] SRC… [DEST]
Here,
OPTION
- the list of Rsync optionsSRC
- Source file/directoryDEST
- Destination
Here is a simple Rsync command to backup or copy file from one location to another in the local system itself.
$ rsync src_file dest_file
Examples:
$ rsync -auv ostechnix.txt ostechnix-bak.txt
Here,
-a
- Archive mode mode (maintaining file properties and sym-links)-u
- Skip files that are newer on the receiving end.-v
- Verbose mode
The above command saves the backup file ostechnix-bak.txt
file in the current directory. You can also specify different destination directory like below:
$ rsync -auv ostechnix.txt Downloads/ostechnix-bak.txt
If you don't specify the target filename, Rsync will use the same name of source file to the target file.
Saving backup files in the same disk is not actually a backup. You must save the backup data on an external drive or network share or at least in a different partition.
For instance, the following command backup the file to my external drive:
$ rsync -auv ostechnix.txt /media/karthick/"My Passport"/
Rsync has many advanced options. Take a look at the following example:
$ rsync -azvhP ostechnix.txt /media/karthick/"My Passport"/
This command transfers file in archive (to preserve attributes) and compressed (zipped) mode with verbose and human-readable progress bar.
2.4. Backup directories using Rsync
Instead of backing up single file, you can put all files in a directory and backup that directory with Rsync. This comes in handy when mirroring and synchronizing data.
For the purpose of this guide, I will show you how to backup HOME directory using Rsync.
2.4.1. Backup HOME directory using Rsync
Rsync is an incremental file copying tool. Meaning - it only syncs new or modified data in the source directory (i.e. /home
in our case) instead of syncing all the data again from scratch.
To backup my $HOME
directory to my external drive, I used the following command:
$ rsync -avxP --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick /media/karthick/"My Passport"/
Let us break down the above command and see what each option does.
- The flag
-a
is the combination of many options. This flag takes care of recursive copy and retaining the attributes like file/directory permission, owner, etc. - The flag
-v
will enable the verbose mode that will give more logs in the terminal when you run the rsync command. - The flag
-x
will restrict rsync to sync data within the file system boundaries. If you have mounted any file system inside your home directory that will be skipped during the sync operation. - The flag
-P
is the combination of two flags. It will display the progress of each sync and keep partially transferred files. - The flag
--delete
will delete all directories and files that are not available in the source but available in the destination. You must be very careful while using flag. - The flag
--exclude-from=File
accepts a file as an argument. This file contains the list of files and directories that will be ignored by rsync.
I have created a file named ignore_list.txt
which contains the files and directories that need to be excluded.
In the above command, this part --exclude-from=/home/karthick/Work/Backup/ignore_list.txt
defines the exclusion list.
Here is the contents of the ignore_list.txt
file:
The list of items to be excluded from the above image is my personal choice and it may differ for you. Modify the parameter according to your environment.
The last two parts of the above Rsync command are the source and destination locations. In my case, the source is my $HOME directory and the destination is my external drive. Replace source and destination according to your environment.
Heads Up: If you want to know the mount point for your external drive, run the lsblk
command.
$ lsblk sdb 8:16 0 931.5G 0 disk └─sdb1 8:17 0 931.5G 0 part /media/karthick/My Passport
Before running the rsync
command for the first time, it is always recommended do a dry run. The dry run will simulate what changes will be made. Use the same command but with the -n
flag.
$ rsync -avxP -n --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick /media/karthick/"My Passport"/
Sample output:
sending incremental file list karthick/ karthick/.bash_history 20,527 100% 0.00kB/s 0:00:00 (xfr#1, ir-chk=1056/1060) karthick/.shutter/ karthick/.shutter/drawingtool.xml 228 100% 4.54kB/s 0:00:00 (xfr#2, ir-chk=1037/1060) karthick/.shutter/profiles/ karthick/Pictures/ karthick/Pictures/Selection_001.png 37,626 100% 349.94kB/s 0:00:00 (xfr#3, to-chk=47/3001) karthick/Pictures/Selection_002.png 25,815 100% 227.12kB/s 0:00:00 (xfr#4, to-chk=46/3001) karthick/Pictures/Selection_003.png 63,275 100% 532.69kB/s 0:00:00 (xfr#5, to-chk=45/3001) karthick/Pictures/Selection_004.png 77,965 100% 581.20kB/s 0:00:00 (xfr#6, to-chk=44/3001) karthick/Pictures/Selection_005.png 75,863 100% 532.99kB/s 0:00:00 (xfr#7, to-chk=43/3001) karthick/Pictures/Selection_006.png 11,628 100% 75.20kB/s 0:00:00 (xfr#8, to-chk=42/3001) karthick/Pictures/Selection_007.png 24,311 100% 150.26kB/s 0:00:00 (xfr#9, to-chk=41/3001) karthick/Work/Backup/ karthick/Work/Backup/home_dir_backup.sh 764 100% 4.34kB/s 0:00:00 (xfr#10, to-chk=20/3001) karthick/Work/Backup/home_dir_backup.sh~ 796 100% 4.37kB/s 0:00:00 (xfr#11, to-chk=19/3001) karthick/Work/Backup/ignore_list.txt 182 100% 1.00kB/s 0:00:00 (xfr#12, to-chk=18/3001) karthick/Work/Backup/ignore_list.txt~ 182 100% 1.00kB/s 0:00:00 (xfr#13, to-chk=17/3001) ... ... ... sent 88,424 bytes received 9,459 bytes 195,766.00 bytes/sectotal size is 3,028,935,519 speedup is 30,944.45 (DRY RUN)
If everything goes as intended, remove the -n
flag and run the command to backup your data.
$ rsync -avxP --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick /media/karthick/"My Passport"/
Depending upon the size, the first run will take some time.
sending incremental file list karthick/ karthick/.bash_history 20,527 100% 0.00kB/s 0:00:00 (xfr#1, ir-chk=1056/1060) karthick/.shutter/ karthick/.shutter/drawingtool.xml 228 100% 4.54kB/s 0:00:00 (xfr#2, ir-chk=1037/1060) karthick/.shutter/profiles/ karthick/Pictures/ karthick/Pictures/Selection_001.png 37,626 100% 349.94kB/s 0:00:00 (xfr#3, to-chk=47/3001) karthick/Pictures/Selection_002.png 25,815 100% 227.12kB/s 0:00:00 (xfr#4, to-chk=46/3001) karthick/Pictures/Selection_003.png 63,275 100% 532.69kB/s 0:00:00 (xfr#5, to-chk=45/3001) karthick/Pictures/Selection_004.png 77,965 100% 581.20kB/s 0:00:00 (xfr#6, to-chk=44/3001) karthick/Pictures/Selection_005.png 75,863 100% 532.99kB/s 0:00:00 (xfr#7, to-chk=43/3001) karthick/Pictures/Selection_006.png 11,628 100% 75.20kB/s 0:00:00 (xfr#8, to-chk=42/3001) karthick/Pictures/Selection_007.png 24,311 100% 150.26kB/s 0:00:00 (xfr#9, to-chk=41/3001) karthick/Work/Backup/ karthick/Work/Backup/home_dir_backup.sh 764 100% 4.34kB/s 0:00:00 (xfr#10, to-chk=20/3001) karthick/Work/Backup/home_dir_backup.sh~ 796 100% 4.37kB/s 0:00:00 (xfr#11, to-chk=19/3001) karthick/Work/Backup/ignore_list.txt 182 100% 1.00kB/s 0:00:00 (xfr#12, to-chk=18/3001) karthick/Work/Backup/ignore_list.txt~ 182 100% 1.00kB/s 0:00:00 (xfr#13, to-chk=17/3001)… … … sent 3,029,863,865 bytes received 51,335 bytes 53,626,817.70 bytes/sec total size is 3,028,934,994 speedup is 1.00
2.4.2. Create backup script
Remembering a lengthy command might be hard. To avoid typing the long rsync
command in the terminal, you can create a simple BASH script and call it whenever you want to initiate the backup.
Create a script named home_dir_backup.sh
:
$ nano home_dir_backup.sh
Add the following lines:
#!/usr/bin/env bash # PURPOSE : RSYNC COMMAND TO BACKUP MY HOME DIRECTORY TO EXTERNAL DEVICE.. # REPLACE SOURCE, DESTINATION AND EXCLUDE ACCORDING TO YOUR ENVIRONMENT. rsync -avxP --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick /media/karthick/"My Passport"/
Add, remove or change the Rsync options as per your requirement. Also change the source and destination paths with your own. Save and close the file.
Make the script executable:
$ chmod +x <path-to-home_dir_backup.sh>
Now run the script at anytime to start the $HOME directory backup:
$ ./path-to-home_dir_backup.sh
Use a shorter script name to easily remember it.
You can also create a soft link for the script file in /usr/bin
or any user-specific directories that is available in the $PATH
variable.
2.4.3. Create alias
The another way to avoid typing long commands is to use aliases.
Edit your ~/.bashrc
file and add the following line at the end:
alias backup="rsync -avxP --delete --exclude-from=/home/karthick/Work/Backup/ignore_list.txt /home/karthick / media/karthick/"My Passport"/"
Source the bashrc
file to take effect the changes immediately:
$ source ~/.bashrc
Now simply run the following command to backup the $HOME directory:
$ backup
It is up to you to decide what do you want to use. Either you can create an alias or create a script and run rsync by triggering the script. In my case, I have created a script and then created an alias for that script.
3. Schedule backup
You can use cron
scheduler to schedule backup jobs to run at a particular time on a particular day.
Edit the current user's cron jobs using command
$ crontab -e
Define your backup schedule. Say for example, to run backup every day at 3am, use the following line:
0 3 * * * <rsync-command-to-execute>
Depending upon your requirement you can schedule daily, weekly or monthly jobs.
If you're new to Cron Jobs, we have compiled a list of useful cron job examples to schedule tasks in Linux. For more details on cron usage, refer the following guide.
4. Rsync graphical frontends
If you're a new to Linux, you may find it difficult to understand all Rsync options and flags. No worries! There are a few graphical frontend tools for Rsync available. One such tool is Grsync.
Grsync is a simple graphical user interface for Rsync. You don't have to memorize any Rsync commands and its options. You can do everything via an easy-to-use GUI.
For more details, refer the following guide:
5. Conclusion
In this guide, we looked at how to backup files and directories using Rsync in Linux. As you can see, copying files and directories with Rsync is very easy!
We also have shown you how to take backup your home directory using the rsync command and some of the ways to run and schedule it.
This method can also be applied to any directory or file structure in Linux to backup the data. If you think you have a better backup solution, we would love to hear it from you via the comment section below.
Related Read:
- Resume Partially Transferred Files Over SSH Using Rsync
- How To Backup And Restore Files Using Deja Dup In Linux
- How To Setup Backup Server Using Rsnapshot
- How To Synchronize Files With Unison On Linux
- How To Synchronize Local And Remote Directories In Linux
- How To Synchronize Files And Directories Using Zaloha.sh
- CYA – System Snapshot And Restore Utility For Linux
- Restic – A Fast, Secure And Efficient Backup Application
1 comment
Hi,
I prefer rsync,
Very useful command