Bash Redirection Explained With Examples

Redirection and piping are two useful features in bash scripting that sysadmins and developers use often. In this guide, we will discuss what is Bash Redirection and how to work with Redirection in Bash with example commands.

Before understanding how redirection works, it is important to learn what file descriptor is.

What is a File descriptor?

When you open a file in Linux, each file will be assigned with an Integer and this information is stored in the kernel. This way the kernel knows what files are opened and which process has opened the files. The assigned Integer number is what we call file descriptor (shortly FD).

By default, every program will start with three file descriptors.

• FD 0 -> Standard Input(stdin) -> Keyboard
• FD 1 -> Standard Output(Stdout) -> Display(Terminal)
• FD 2 -> Standard Error(Stderr) -> Display(Terminal)

You can see the file descriptors in /dev directory:

$ ls -l /dev/std*
lrwxrwxrwx 1 root root 15 Aug 27 21:32 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 Aug 27 21:32 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 Aug 27 21:32 /dev/stdout -> /proc/self/fd/1

Stdin (FD0) gets input from the keyboard. stdout (FD1) and stderr (FD2) are sent to the terminal. When using Pipes and Redirections, you can change the way where input is passed from and output and errors are sent to.

Redirecting output to a file

As stated earlier, output (stdout) and error (stderr) of any program is sent to the terminal. You can use the redirection operator ">” to write the stdout and stderr to a file.

Take a look at the below example. I am running the uname -mrs command and redirecting the output to a file named uname.log.

$ uname -mrs > uname.log

$ cat uname.log
Linux 5.10.0-8-amd64 x86_64

Heads Up: When using ">" operator, if the file is not available, it will be created. And if the file exists already, it will be overridden with new contents.

You can also use the file descriptor number for stdout(1) before the redirection operator to redirect output to a file.

$ uname -mrs 1> uname.log

Like I already mentioned, a single redirection operator (>) will only override the contents of a file, if the file already exists. However, if you want to append the contents instead of overwriting to the same file, use double redirection operator (i.e. >>). You can also use stdout file descriptor(1) here too.

$ whoami >> uname.log

$ echo $SHELL 1>> uname.log

$ cat uname.log
Linux 5.10.0-8-amd64 x86_64
ostechnix
/bin/bash

How to handle Stderr

Every program generates both output and error and both are sent to the terminal. In many cases, you may wish to either ignore the error or isolate the errors and redirect it to a separate file for troubleshooting.

Let's take a simple example of running the ls command to see how this works. I am trying to list two files out of which only one is present. I am getting both my error and output in the terminal as I already said.

$ ls -l uname.log u1.log
ls: cannot access 'u1.log': No such file or directory
-rw-r--r-- 1 ostechnix ostechnix 48 Aug 27 07:37 uname.log

I am running the same ls command again but this time redirecting the output to a file. As you can see from the output > operator redirects stdout(1) to a file but stderr(2) is sent to the terminal.

$ ls -l uname.log u1.log > ls.op
ls: cannot access 'u1.log': No such file or directory

To redirect stderr to a file use the "2>" operator. It is mandatory to use the file descriptor for stderr(2) before the redirection operator > that will send the error alone to a file.

$ ls -l uname.log u1.log > ls.op 2> ls.err

$ cat ls.err
ls: cannot access 'u1.log': No such file or directory

Now, stdout and stderr are written to separate files. You can also send stdout and stderr to a single file too.

$ ls -l uname.log u1.log 1> ls.op 2> ls.op

From bash 4.4, you can also use &> sign to redirect both stdout and stderr to a file.

$ ls -l uname.log u1.log &> ls.op

What is /dev/null?

Null is a character special file that accepts input and discards the input and produces no output. To put this simply, null discards whatever you redirect to it.

$ ls -l /dev/null
crw-rw-rw- 1 root root 1, 3 Aug 27 06:01 /dev/null

Why does the null matter in redirection? You might wonder. In some cases, you may wish not to print or store stdout or stderr. In that case, you can redirect either stdout or stderr to /dev/null which will discard the stream of input.

$ date > /dev/null
$ date 1> /dev/null    # Stdout to Null
$ dateee 2> /dev/null  # WRONG COMMAND, Stderr to Null
$ date &> /dev/null    # Stdout/Stderr to Null

Input Redirection in Bash

Similar to how you are redirecting the output and error to a file, you can also pass inputs to a command using input redirection operator (<).

Let’s start with a simple word count program. Here, I am redirecting the contents of ostechnix.txt file to the word count to find the number of lines.

$ wc -l < ostechnix.txt 
10

You can also pass stdin file descriptor (0) when redirecting the input.

$ wc -l 0< ostechnix.txt 
10

You can combine the input and output redirection operators as shown below.

$ wc -l < ostechnix.txt &> /tmp/wc.op

$ cat /tmp/wc.op 
10

Input redirection will be used along with a while loop to read the content of a file line by line.

Take a look at the below example. I am passing the /etc/passwd file as the input to the while command. Here the read command will read line by line and store it in the variable VAL and further down in the loop condition is written to check if the user is available.

while read VAL; do   
  NAME=$(echo $VAL | awk -F  ":" '/1/ {print $1}' )
  if [[ $NAME = "karthick" ]]
  then
   echo "User spotted"
  fi
done < /etc/passwd

A script to find system user

Replace "karthick" in the above code with your system username. Save the file and close it.

This is not an optimal way to achieve the task of finding the user but for demonstration purposes, this will do.

Now, make the script executable and run it:

$ chmod +x ostechnix.sh 

$ ./ostechnix.sh 
User spotted

Bash supports another form of input redirection known as Heredoc. For more details, refer the following guide:

• Bash Heredoc Tutorial For Beginners

Summary

In this guide, we learned a few important scripting concepts such as what is Bash redirection, what is file descriptor, how to redirect an output to a file, how to handle stderr, what is /dev/null and how it is used in Bash redirection, and finally we wound up the guide with Input redirection in Bash.

If you wish to learn Bash scripting, you should learn and get familiar with the concepts explained here. To make yourself comfortable, I suggest you to try all the commands that we have shown in this article in your system.

If you have any feedback or any tips that will help the Linux community and our readers to learn bash scripting, feel free to let us know in the comment section below.