Home Linux Commands How To Monitor The Progress Of Data Through A Pipe Using ‘pv’ Command

How To Monitor The Progress Of Data Through A Pipe Using ‘pv’ Command

By sk
Published: Last Updated on 9K views

Picture this scenario. You are trying to copy a large file from one directory to another or one system to another through command line. As you probably know, you will not see any indication of copy task if you're using 'cp' command. You'll just see an empty screen. You can't be sure whether the copy process is going on or not, until the copy process is completed. In such cases, you can use pv command to monitor the progress of data. The pv, acronym of pipe viewer, command allows a user to see the progress of data through a pipeline, by giving the following information:

  • total time elapsed,
  • percentage completed (with progress bar),
  • current throughput rate,
  • total data transferred,
  • and ETA.

In this brief tutorial, we will see how and where to use pv command to monitor the progress of data in Unix-like systems.

Install pv in Linux

Most modern Linux distributions comes pre-installed with pv command. Just in case if it is not installed already, you can install it by running the following command from your Terminal depending upon the Linux distribution you use.

On Arch Linux, Antergos, Manjaro Linux:

$ sudo pacman -S pv

On RHEL, CentOS, Fedora:

$ sudo yum install pv

Or,

$ sudo dnf install pv

On Debian, Linux Mint, Ubuntu:

$ sudo apt-get install pv

On SUSE, openSUSE:

$ sudo zypper in pv

Now, we have installed pv. Let me show you some examples.

Monitor The Progress Of Data Through A Pipe Using 'pv' Command

Look at the output of the following command:

$ cp CentOS7.iso Downloads/
cp command

cp command

This command copies CentOS 7 ISO image to Downloads directory in my local system. As you see in the above output, it doesn't show any progress bar. You will only see the cursor blinking. We don't know how much time the copy process will take. This could be annoying sometimes as we don't see any sign of when this command will complete. This is where pv command comes in help. The pv command will show you the progress of Linux commands, and show you when the Linux command will complete. So, you can leave the Terminal and do some other tasks and come back when the task is done.

Now, let us do the same with pipe viewer (pv), and see what is the output.

$ pv CentOS7.iso > Downloads/centos.iso
Monitor The Progress Of Data Through A Pipe Using 'pv' Command

Monitor The Progress Of Data Through A Pipe Using 'pv' Command

See, you get the difference? Like I said, the pv command displays total time elapsed, percentage completed (with progress bar), current throughput rate, total data transferred, and ETA.

Let me give you some more examples.

We can limit the data transfer rate using -L switch as shown in the below example.

$ pv -L 2m CentOS7.iso > Downloads/centos.iso

The above command will limit the data transfer rate to 2MB per second. Look at the following output.

Limit the data transfer rate using pv command

Limit the data transfer rate using pv command

If you have multiple files or directories you can use pv with tar command as shown below:

$ tar c Notebooks/ | pv | tar x -C Documents/

The above command will compress the directory Notebooks, and copy it to Documents directory and finally extract the contents.

Here is another example. This command will will create an archive of CentOS7.iso file and save it in the current working directory.

$ pv CentOS7.iso | zip > centos.zip
create an archive file using pv command

create an archive file using pv command

You can also use pv with dd command to copy files between directories like below. This command copies CentOS7.iso file from the current directory to Downloads directory.

$ dd if=CentOS7.iso | pv | dd of=Downloads/centos7.iso
use pv command with dd command to copy files between directories

use pv command with dd command to copy files between directories

Another useful command to display the progress of data is dialog. We can use it along with pv command to show the progress of data like below.

$ tar -czf - Official/ | (pv -n > mybackup.tgz) 2>&1 | dialog --gauge "Compressing files, please wait..." 10 70 0

Sample output:

use dialog command with pv command to show the progress of data

use dialog command with pv command to show the progress of data

For more details, refer the man pages.

$ man pv

Also, check our previous guide to quickly transfer large files between between two remote systems over Network using netcattar, and pv commands in Linux and Unix operating systems.

Update:

The progress bar for "cp" and "mv" commands can be easily added using a patch named "Advanced Copy". For details, check the following link.

You May Also Like

1 comment

tatsu March 8, 2019 - 8:34 pm

you may want to update the dialog –gauge example. I couldn’t get it to work. dialog’s progress bar said “unknown command”. I think you’re missing an option. I tried many varients and such also most people nowadays would simply create an archive with : `tar czf archive.tar.gz folder` creating a `archive.tgz` straight-up, won’t be supported without supplementary installs, at least on ubuntu

Reply

Leave a Comment

* By using this form you agree with the storage and handling of your data by this website.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies to improve your experience. By using this site, we will assume that you're OK with it. Accept Read More