How To Avoid Duplicate Entries In Bash History In Linux

In this brief guide, we will learn how to avoid duplicate entries in Bash history in Linux. When working from commandline, you might have run some Linux commands multiple times. As you already know, each command we run in the terminal is saved in the history file. So executing the same commands over and over would fill up the history file. Sooner or later, you will end up with too many duplicate entries in your Bash history file.

By default, 500 command entries are stored in the history file. In recent Ubuntu systems, the file size is 1000. Once the number of command entries crossed the default limit, the oldest entries will be eliminated from the history list. Hence you can't retrieve the old entries. You can, of course, increase the history file size and save as many entries as you want. However, instead of increasing the history file size, it is much better to avoid saving duplicates in history file.

Find History File Location

Command history is a shell-specific feature stored on a per-user basis. Depending upon the SHELL you use, the history file is saved in a specific file. In Linux systems that uses Bash, the commands are saved in ~/.bash_history file by default. You can check the location of the history file of your shell by running the following command:

$ echo $HISTFILE

Here, HISTFILE is a variable that is used to define the name of the file in which command history is saved.

Sample output from my Ubuntu desktop:

/home/sk/.bash_history

In CentOS 8, I got the following output:

/root/.bash_history

Change History File Location

If you want to change the default location, run:

$ echo "export HISTFILE=~/.custom_file" >>~/.bashrc

Replace "~/.custom_file" with your own in the above command.

Run the following command to apply the changes:

$ source ~/.bashrc

Now let us see how to avoid saving the same commands multiple times in ~/.bash_history file.

Avoid Duplicate Entries In Bash History In Linux

We retrieve the last executed commands using "history" command. Take a look at the following example:

$ history | grep ls

Sample output:

    8  lsb_release -a
   13  ls -l
   14  lsb_release -a
   17  ls -l
   20  ls -l
   23  lsb_release -a
   27  ls
   29  lsb_release -a
   32  ls
   36  ls
   42  ls
   44  lsb_release -a
   62  history | grep ls

As you can see in the above output, the "ls" command has been recorded multiple times. We can control these duplicates using HISTCONTROL variable. HISTCONTROL can have the following values:

• ignorespace - lines beginning with a space will not be saved in history.
• ignoredups - lines matching the previous history entry will not be saved. In other words, duplicates are ignored.
• ignoreboth - It is shorthand for "ignorespace" and "ignoredups" values. If you set these two values to HISTCONTROL variable, the lines beginning with a space and the duplicates will not be saved.
• erasedups - eliminate duplicates across the whole history.

So, to avoid duplicate entries in Bash history in Linux, edit your ~/.bashrc file:

$ nano ~/.bashrc

Add the following line at the end:

export HISTCONTROL=ignoredups

Save and close the file. Here, the we prefixed the HISTCONTROL variable with "export". It means the variable is available to all sub-processes. More details can be viewed here.

Alternatively, use the following one-liner:

$ echo "export HISTCONTROL=ignoredups" >>~/.bashrc

Now run the following command to take effect the changes:

$ source ~/.bashrc

Or, log out and log back in to apply the changes.

From now on, the duplicate entries will not be recorded. You can verify it by running the same command multiple times and view the history list using the following command:

$ history

You can also set multiple values to HISTCONTROL variable with colon-separated like below:

export HISTCONTROL=ignoredups:erasedups

For more details, refer Bash man page:

$ man bash