Home Bash scripting Bash Heredoc Tutorial For Beginners

Bash Heredoc Tutorial For Beginners

By Karthick
Published: Last Updated on 4.2k views

When working with Bash scripts, you may end up in a situation where you have to process a series of inputs using the same command. Fortunately, there is a way in Bash to achieve this in a more optimal way using HereDoc.

HereDoc, acronym for Here Document, is an input Redirection method to pass multiple inputs to a program or command. The concept of heredoc is not exclusively related to Bash alone. Many popular programming languages like Perl, Ruby, PHP support heredoc.

In this article, we will take a look at the syntax and usage of heredoc with some real-world use cases. All the examples in this article are created to be simple, so even a newbie can understand this concept easily. Let’s jump in and start playing with heredoc in Bash.

HereDoc Syntax

The following diagram illustrates the syntax of heredoc.

Graphical Illustration Of HereDoc Syntax
Graphical Illustration Of HereDoc Syntax

Here,

  • Command - Any command (cat, wc, mail, etc..) that accepts redirection.
  • Redirection Operator (<<) - The default operator for HereDoc is <<. It redirects the block of code to the command for processing.
  • Delimiter Token - The delimiter token denotes the start and end of the document (code block). The delimiter token can be anything but it should be identical. Typically you will see EOF is used as delimiter tokens which mean "End Of File Stream".

Printing MultiLine String Using HereDoc In Bash

Let’s start with a simple example of redirecting a multiline string and printing it to the terminal.

The cat command accepts a stream of inputs and with heredoc, and you can redirect the block of code to print it to the terminal.

$ cat << EOF
Something is wrong with the input file received for today.
Contact the downstream team to get it corrected.
==> SLA MISSED <==
EOF

Output:

Something is wrong with the input file received for today.
Contact the downstream team to get it corrected.
==> SLA MISSED <==

Take a look at the above code snippet. I have three lines that are redirected to the cat command. I am using EOF as the delimiter. However, you can use anything as you wish but keep the starting and ending delimiter identical.

Let’s try with one more simple example. I am redirecting the same three-line to word count program. I am using a different delimiter (BLK) here.

$ wc -l << BLK
Something is wrong with the input file received for today.
Contact the downstream team to get it corrected.
==> SLA MISSED <==
BLK

Sample output:

3
Printing multiline string using heredoc

Redirecting And Pipe In HereDoc

You can combine the output redirection operator with heredoc and redirect the output to a file instead of printing it to the terminal.

I am using the same example which I used in the previous section and redirecting the output to a file named log_op.txt.

$ cat << EOF > /tmp/log_op.txt
Something is wrong with the input file received for today.
Contact the downstream team to get it corrected.
==> SLA MISSED <==
EOF

The output of heredoc can be sent to the pipe operator for further processing.

$ cat << EOF | grep -i sla
Something is wrong with the input file received for today.
Contact the downstream team to get it corrected.
==> SLA MISSED <==
EOF

Tab Suppression In HereDoc

When there are white spaces (tabs) in your code block and if you wish to suppress it, then use "-" after the redirection operator. An important point to be noted is that only tabs will be suppressed, not spaces.

Take a look at the below example. I have added a conditional statement to the same example that we have seen in previous sections. The first two lines in heredoc are tabbed (4) and the third line is spaced (2).

if [[ $x = "err" ]]
then
 cat <<- err_msg
     1. Something is wrong with the input file received for today.
     2. Contact the downstream team to get it corrected.
   3. ==> SLA MISSED <==
err_msg
fi

When the code snippet is submitted, then my output will be as follows.

1. Something is wrong with the input file received for today.
2. Contact the downstream team to get it corrected.
   3. ==> SLA MISSED <==

As you can see, line 1 and 2 tabs are suppressed, but in line 3 since spaces are used it is not suppressed.

Variable And Command Expansion In HereDoc

It is not that you can pass only strings within the heredoc code block. You can pass user-defined and environmental variables and run commands within the code block.

Take a look at the below example. Within the code block, I have one user-defined variable "${AUTHOR}", one environmental variable "${SHELL}", an external command "whoami".

When this snippet is submitted, variables and commands will be expanded then it will be redirected to the cat command.

AUTHOR="OSTechNix"

cat << EOF
Author: ${AUTHOR}                  # USER DEFINED VARIABLE
Article: Bash Heredoc
I am using the ${SHELL} shell      # ENV VARIABLE
$(whoami)                          # EXTERNAL COMMAND
EOF

Sample output:

Author: OsTechnix
Article: Bash Heredoc
I am using the /bin/bash shell
karthick

You can enclose the starting delimiter with single quotes to suppress the expansion within the code block. This way everything within the code block will be treated as a string literal.

cat << 'EOF'
Author: ${AUTHOR}                  # USER DEFINED VARIABLE
Article: Bash Heredoc
I am using the ${SHELL} shell      # ENV VARIABLE
$(whoami)                          # EXTERNAL COMMAND
EOF

Sample output:

Author: ${AUTHOR}            
Article: Bash Heredoc
I am using the ${SHELL} shell
$(whoami)

MultiLine Comments With HereDoc

As you may already know, Bash does not support multi-line comments. Using heredoc, you can create multi-line comments by redirecting the block of code to the no-op command (:).

The no-op is bash built-in which takes the input and returns exit code zero. You can consider this as a synonym to the bash built-in "true" which also exits exit code zero.

: << 'COMMENTS'
Author : OStechnix
Article : Bash Heredoc
BashV : 5.1.4
OS : PoP!_OS
COMMENTS

Heads Up: Almost every text editor has the feature of selecting multiple lines and allows you to comment or uncomment using a keystroke. It is better to stick with this approach.

Escaping Special Characters In HereDoc

Code blocks may contain special characters. If you wish to escape the special characters, there are few ways to get it done.

You can enclose the delimiter with single or double quotes or prefix backslash with the delimiter. This way all the special characters will be escaped.

# SINGLE QUOTES ESCAPE
cat << 'EOF'
I am using the ${SHELL} shell      # ENV VARIABLE
$(whoami)                          # EXTERNAL COMMAND
EOF
# DOUBLE QUOTES ESCAPE
cat << "EOF"
I am using the ${SHELL} shell      # ENV VARIABLE
$(whoami)                          # EXTERNAL COMMAND
EOF
# BACKSLASH ESCAPE
cat << \EOF
I am using the ${SHELL} shell      # ENV VARIABLE
$(whoami)                          # EXTERNAL COMMAND
EOF

Instead of escaping all the special characters, you can also escape particular special characters within the block by adding a backslash before any special character.

cat << EOF
I am using the \${SHELL} shell      # ENV VARIABLE
$(whoami)                          # EXTERNAL COMMAND
EOF

Sample output:

I am using the ${SHELL} shell      
karthick      

HereDoc Use Case

Till now we have seen the core construct of heredoc and its basic usage. Now let’s see some real-life use cases. In my experience, I have used heredoc when working with ssh, and database clients where I have to run a group of commands.

Example 1: Running As Different User Inside The Script

In some cases, you may want to run certain parts of your code as a different user. In that case, you can use heredoc to redirect the commands to run as a different user.

Take a look at the below example. I am redirecting the block of code to the su command which will switch the user to "ostechnix" and will create a file named test if it does not exist.

su - ostechnix << EOF
if [[ ! -f /home/ostechnix/test ]];then
 touch /home/ostechnix/test
 echo "File Created"
else
 echo "File exists"
fi
EOF
HereDoc with different user
HereDoc with different user

Example 2: Using HereDoc with DB Client

When you want to run a series of commands against a database, heredoc will come in handy.

Take a look at the below example. I am interacting with MongoDB client mongosh and within the heredoc, code block commands are passed to create a new database, collection, and adding a sample document.

mongosh << EOF
use ostechnix
db.data.insertOne({
    "Site" : "OsTechnix",
    "DB"  : "Mongo"
})
db.data.find().pretty()
EOF
HereDoc with MongoDB client
HereDoc with MongoDB client

You may or may not be knowing MongoDB but that is fine. This is just to show how to use heredoc to interact with db clients. You can use any db client like MySQL, psql, sqlite depending upon which database you are working with.

Example 3: Executing Remote Commands with HereDoc and SSH

When you wish to run commands over the remote server, then you can use heredoc in combination with ssh command. Normally using ssh command, you can run commands in the remote server in the following way.

$ ssh user@host "command"

You have to repeat the same command again and again if you wish to run more commands in the remote host. With heredoc, you can group all the commands and run them.

$ ssh -T user@host << EOF
Command 1...
Command 2..
.....
Command N..
EOF

When the same code needs to be executed across multiple nodes you can add for loop along with heredoc. I am using the same file creation snippet which we have seen in the first example.

  • The array variable "server" holds the list of server names.
  • For loop iterates over the array variable.
  • The file creating command is passed to the SSH command that will iterate over and create files in each server. Make sure you add -T flag to ssh command which will disable pseudo-terminal allocation.
declare -a server=( host1 host2 host3 )
for host in ${server[@]}
do
  ssh -T user@${host} << EOF
  echo "Running at host - ${host}"
  if [[ ! -f /home/ostechnix/test ]];then
    touch /home/ostechnix/test
    echo "File Created"
  else
    echo "File exists"
  fi
EOF
done

Conclusion

Heredoc is an important concept to understand and use in Bash scripts. When you write a lot of scripts you will come to know more about heredoc and ways to optimally use it.

If you have never used heredoc before, start the terminal and try all the code snippets from the article to have a better understanding.

You May Also Like

Leave a Comment

* By using this form you agree with the storage and handling of your data by this website.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies to improve your experience. By using this site, we will assume that you're OK with it. Accept Read More