Home BASH How To Avoid Duplicate Entries In Bash History In Linux

How To Avoid Duplicate Entries In Bash History In Linux

By sk
Published: Last Updated on 11.6K views

In this brief guide, we will learn how to avoid duplicate entries in Bash history in Linux. When working from commandline, you might have run some Linux commands multiple times. As you already know, each command we run in the terminal is saved in the history file. So executing the same commands over and over would fill up the history file. Sooner or later, you will end up with too many duplicate entries in your Bash history file.

By default, 500 command entries are stored in the history file. In recent Ubuntu systems, the file size is 1000. Once the number of command entries crossed the default limit, the oldest entries will be eliminated from the history list. Hence you can't retrieve the old entries. You can, of course, increase the history file size and save as many entries as you want. However, instead of increasing the history file size, it is much better to avoid saving duplicates in history file.

Find History File Location

Command history is a shell-specific feature stored on a per-user basis. Depending upon the SHELL you use, the history file is saved in a specific file. In Linux systems that uses Bash, the commands are saved in ~/.bash_history file by default. You can check the location of the history file of your shell by running the following command:

$ echo $HISTFILE

Here, HISTFILE is a variable that is used to define the name of the file in which command history is saved.

Sample output from my Ubuntu desktop:


In CentOS 8, I got the following output:


Change History File Location

If you want to change the default location, run:

$ echo "export HISTFILE=~/.custom_file" >>~/.bashrc

Replace "~/.custom_file" with your own in the above command.

Run the following command to apply the changes:

$ source ~/.bashrc

Now let us see how to avoid saving the same commands multiple times in ~/.bash_history file.

Avoid Duplicate Entries In Bash History In Linux

We retrieve the last executed commands using "history" command. Take a look at the following example:

$ history | grep ls

Sample output:

    8  lsb_release -a
   13  ls -l
   14  lsb_release -a
   17  ls -l
   20  ls -l
   23  lsb_release -a
   27  ls
   29  lsb_release -a
   32  ls
   36  ls
   42  ls
   44  lsb_release -a
   62  history | grep ls
Duplicate entries in Bash history
Duplicate entries in Bash history

As you can see in the above output, the "ls" command has been recorded multiple times. We can control these duplicates using HISTCONTROL variable. HISTCONTROL can have the following values:

  • ignorespace - lines beginning with a space will not be saved in history.
  • ignoredups - lines matching the previous history entry will not be saved. In other words, duplicates are ignored.
  • ignoreboth - It is shorthand for "ignorespace" and "ignoredups" values. If you set these two values to HISTCONTROL variable, the lines beginning with a space and the duplicates will not be saved.
  • erasedups - eliminate duplicates across the whole history.

So, to avoid duplicate entries in Bash history in Linux, edit your ~/.bashrc file:

$ nano ~/.bashrc

Add the following line at the end:

export HISTCONTROL=ignoredups
Avoid duplicate entries in Bash history in Linux
Avoid duplicate entries in Bash history in Linux

Save and close the file. Here, the we prefixed the HISTCONTROL variable with "export". It means the variable is available to all sub-processes. More details can be viewed here.

Alternatively, use the following one-liner:

$ echo "export HISTCONTROL=ignoredups" >>~/.bashrc

Now run the following command to take effect the changes:

$ source ~/.bashrc

Or, log out and log back in to apply the changes.

From now on, the duplicate entries will not be recorded. You can verify it by running the same command multiple times and view the history list using the following command:

$ history

You can also set multiple values to HISTCONTROL variable with colon-separated like below:

export HISTCONTROL=ignoredups:erasedups

For more details, refer Bash man page:

$ man bash

You May Also Like


Fred H Olson January 6, 2022 - 5:09 pm

When and how is the history file updated? I use multiple instances of bash , is there a way to save all commands from all simultaneous instances to the history file?

sk January 7, 2022 - 1:55 pm

I haven’t tried it personally. Check it yourself and let me know if it works. https://unix.stackexchange.com/questions/1288/preserve-bash-history-in-multiple-terminal-windows


Leave a Comment

* By using this form you agree with the storage and handling of your data by this website.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies to improve your experience. By using this site, we will assume that you're OK with it. Accept Read More