Home Rsync Resume Partially Transferred Files Over SSH Using Rsync

Resume Partially Transferred Files Over SSH Using Rsync

Resume file copying over SSH with Rsync in Linux

By sk
Published: Last Updated on 20.8k views

Picture this scenario. You're copying some large files from one system to another over SSH using scp command. Suddenly the power is gone, so the network connection is dropped and the copy process is terminated in midway. What would you do? Just restart the copy process once the power is back online? Yes, It does work. However, the scp command doesn't have resume option, so it will simply start copying the files from the beginning and overwrite the existing files. This is bit annoying and time-consuming task.  This is where Rsync utility comes in help! We can easily resume partially transferred files over SSH using Rsync. It helps you to resume the interrupted copy or download process where you left it off.

What Is Rsync?

For those wondering, Rsync is a fast, versatile file copying utility that can be used to copy and transfer files or folders to and from remote and local systems.

It has a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied.

Rsync is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination.

Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Just like scp, rsync can also copy files over SSH. In case you wanted to download or transfer large size files and folders over SSH, I recommend you to use rsync utility as described below.

Install Rsync In Linux

Please that the rsync utility should be installed on both sides (i.e. remote and local systems) in order to resume partially transferred files. Rsync is available in the official repositories of most Linux distributions, so you can install it using the distribution's default package manager.

On Alpine Linux, we can install Rsync like below:

$ sudo apk add rsync

On Arch Linux and its variants, install Rsync using command:

$ sudo pacman -S rsync

On Debian, Ubuntu, Linux Mint:

$ sudo apt-get install rsync

On Fedora, RHEL, CentOS, AlmaLinux and Rocky Linux:

$ sudo dnf install rsync

On SUSE/openSUSE:

$ sudo zypper install rsync

Resume Partially Transferred Files Using Rsync

Now let us see an example. I am going to copy Ubuntu ISO file from my local system to a remote system using the following command:

$ scp ubuntu.iso sk@192.168.225.22:/home/sk/

Here,

  • sk is my remote system's username
  • 192.168.225.22 is the IP address of the remote machine.
  • /home/sk/ is the remote destination.

Now, I terminated the copy process forcibly by pressing CTRL+C.

Sample output:

sk@192.168.43.2's password: 
ubuntu-16.04-desktop-amd64.iso 26% 372MB 26.2MB/s 00:39 ETA^c
Copy file using scp command on Linux
Copy file using scp command on Linux

As you see in the above output, I terminated the copy process when it reached 33%.

If I re-run the above command, it will simply overwrite the existing file. In other words, the copy process will not resume where I left it off.

In order to resume the copy process to copy the partially transferred ISO file, use rsync command as shown below.

$ rsync -P -rsh=ssh ubuntu.iso sk@192.168.225.22:/home/sk/

Sample output:

sk@192.168.225.22's password: 
sending incremental file list
ubuntu.iso
834.00M 100% 14.81MB/s 0:00:56 (xfr#1, to-chk=0/1)
Resume Partially Transferred Files Using Rsync Over SSH On Linux
Resume Partially Transferred Files Using Rsync Over SSH On Linux

See? Now, the copying process is resumed where we left it off earlier.

You can also use "--partial" instead of parameter "-P" like below.

$ rsync --partial -rsh=ssh ubuntu.iso sk@192.168.225.22:/home/sk/

Here, the parameters "--partial" or "-P" tells the rsync command to keep the partially downloaded file and resume the process.

Check if the file is really copied or not to the remote system using command:

$ ssh sk@192.168.225.22 ls -lh
sk@192.168.225.22's password: 
total 835M
-rw-rw-r-- 1 sk sk 834M Jul 26 2019 ubuntu.iso
Run commands on remote systems over SSH
Run commands on remote systems over SSH

As you see in the above output, the file Ubuntu.iso is copied to the remote system and its size is 835MB.

Alternatively, we can use the following commands as well to resume partially transferred files over SSH.

$ rsync -avP ubuntu.iso sk@192.168.225.22:/home/sk/

Or,

$ rsync -av --partial ubuntu.iso sk@192.168.225.22:/home/sk/

That's it. You know now how to resume the cancelled, interrupted, and partially downloaded files using rsync command. As you can see, it is not that difficult. If rsync is installed on both systems, we can easily resume file copy process over SSH as described above.

You May Also Like

4 comments

Jalal January 26, 2020 - 7:27 pm

Hi,
Very useful article.
Thanks a lot

Reply
anon August 24, 2020 - 6:04 am

Hi,

A better way to verify the files are the same is to use a checksum like md5sum or sha1sum.
Thanks for the article!

Reply
anon December 22, 2021 - 4:06 am

great article, worked exactly as described after i was half way through a 17GB download with scp that had already taken 3 hours! appreciate the help.

Reply
sk December 22, 2021 - 1:03 pm

Glad it worked out for you. Happy to help.

Reply

Leave a Comment

* By using this form you agree with the storage and handling of your data by this website.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies to improve your experience. By using this site, we will assume that you're OK with it. Accept Read More