How To Synchronize Local And Remote Directories Using Lsyncd In Linux

This tutorial describes how to synchronize local and remote directories in Linux using Lsyncd (Live Syncing Daemon). Using Lsyncd, we can mirror local to local directories or local to remote directories. It comes in handy when you want to sync data between directories which are frequently updated with new contents. Lsyncd is designed to synchronize a local directory tree with low profile of expected changes to a remote mirror. It is especially useful to sync data from a secure area to a not-so-secure area.

Install Lsyncd in Linux

On Debian and its derivatives like Ubuntu, Linux Mint run the following command to install Lsyncd:

$ sudo apt-get install lsyncd

On RHEL, CentOS, you need to enable EPEL repository first.

$ sudo yum install epel-release

Then, install Lsyncd using command:

$ sudo yum install lsyncd

Lsyncd Configuration on Debian and Ubuntu and derivatives

Lsyncd does not provide any default configuration file in DEB based systems. We need to manually create it. However, Lsyncd includes some sample config files that are useful to get a basic idea of what/how it does the synchronization. The example configuration files will be found under "/usr/share/doc/lsyncd-*/examples" directory.

$ ls /usr/share/doc/lsyncd/examples/

Sample output:

lbash.lua lgforce.lua lpostcmd.lua lrsyncssh.lua
lecho.lua limagemagic.lua lrsync.lua

As you see in the above output, each config file ends with .lua extension. It is because lsync configuration file is written using Lua programming language. Let us take a look at a sample config file.

$ cat /usr/share/doc/lsyncd/examples/lrsync.lua

This is how Lrsync configuration file looks like:

----
-- User configuration file for lsyncd.
--
-- Simple example for default rsync.
--
settings = {
 statusFile = "/tmp/lsyncd.stat",
 statusInterval = 1,
}

sync{
 default.rsync,
 source="src",
 target="trg",
}

Lsyncd Configuration on RHEL and CentOS

On RHEL and CentOS systems, Lsyncd configuration file will be created automatically. You can view the contents of this file using command:

$ cat /etc/lsyncd.conf

And, the example configuration files are available in the following location.

$ ls /usr/share/doc/lsyncd-2.1.5/examples/

Synchronize Local And Remote Directories In Linux Using Lsyncd

As stated already, Lyncd can able to sync contents two local directories and a local directory to a remote directory. First, we will see how to synchronize local directries.

1. Synchronize local directories on Debian, Ubuntu

Now, we will synchronize two local directories.

Let us create two directories with some example contents.

$ sudo mkdir source_dir

$ sudo mkdir dest_dir

Create some random files in source_dir directory:

$ sudo touch source_dir/file{1..10}

Optionally, you can create some log files for Lysyncd. This is just optional. If you don't want log files, skip this step.

$ sudo mkdir /var/log/lsyncd

$ sudo touch /var/log/lsyncd/lsyncd.{log,status}

Next, create configuration file for Lsyncd:

$ sudo mkdir /etc/lsyncd

$ sudo nano /etc/lsyncd/lsyncd.conf.lua

Edit/modify the following lines:

settings {
        logfile = "/var/log/lsyncd/lsyncd.log",
        statusFile = "/var/log/lsyncd/lsyncd.status"
}

sync {
        default.rsync,
        source = "/home/sk/source_dir",
        target = "/home/sk/dest_dir",
}

Replace the source and target directories path with your own values. Save and close the file.

Enable and start or restart Lsyncd service using commands:

$ sudo systemctl enable lsyncd

$ sudo systemctl restart lsyncd

Now, check the contents of source_dir and dest_dir directories.

$ ls source_dir/

Sample output:

file1 file10 file2 file3 file4 file5 file6 file7 file8 file9

Check dest_dir contents:

$ ls dest_dir/

Sample output:

file1 file10 file2 file3 file4 file5 file6 file7 file8 file9

See? We have created files in source_dir directory only and the contents of this directory have been automatically synchronized to dest_dir directory.

Also, you can review the log files to verify if the replication is completed or not.

$ tail -10 /var/log/lsyncd/lsyncd.log

Sample output:

Sat Apr 15 17:07:34 2017 Normal: recursive startup rsync: /home/sk/source_dir/ -> /home/sk/dest_dir/
Sat Apr 15 17:07:34 2017 Normal: Startup of "/home/sk/source_dir/" finished.

Check the Lsyncd status file:

$ more /var/log/lsyncd/lsyncd.status

Sample output:

Lsyncd status report at Sat Apr 15 17:07:44 2017

Sync1 source=/home/sk/source_dir/
There are 0 delays
Excluding:
 nothing.


Inotify watching 1 directories
 1: /home/sk/source_dir/

2. Synchronize local directories on RHEL, CentOS

Edit Lsyncd configuration file:

$ sudo nano /etc/lsyncd.conf

Edit and modify the following lines:

----
-- User configuration file for lsyncd.
--
-- Simple example for default rsync, but executing moves through on the target.
--
-- For more examples, see /usr/share/doc/lsyncd*/examples/
-- 
settings {
 logfile = "/var/log/lsyncd.log",
 statusFile = "/var/log/lsyncd.status"
}

sync {
 default.rsync,
 source = "/root/source_dir",
 target = "/root/dest_dir",

Replace the source and target directories path with your own values. Save and close the file.

Enable and start Lsyncd service using commands:

$ sudo systemctl enable lsyncd

$ sudo systemctl start lsyncd

Now, check the contents of source_dir and dest_dir directories.

$ ls source_dir/

Sample output:

file1 file10 file2 file3 file4 file5 file6 file7 file8 file9

Check dest_dir contents:

$ ls dest_dir/

Sample output:

file1 file10 file2 file3 file4 file5 file6 file7 file8 file9

Congrats! The contents of source_dir have been successfully synchronized to dest_dir directory.

Also, you can review the log files to verify if the replication is completed or not.

$ tail -10 /var/log/lsyncd.log

Check the Lsyncd status file:

$ more /var/log/lsyncd.status

3. Synchronize remote directories on Debian, Ubuntu systems

We need to setup password-less SSH login. This is because Lsyncd will automatically replicate the contents of the local directory to a remote directory without user intervention.

$ sudo su

# ssh-keygen -t rsa

Do not enter any passphrase. Just press ENTER to accept the default values.

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/home/sk/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:dHc2aKk5F2wBq7CIc5sVHJwYNjoPdwgxDKb1KuBBkgs sk@ubuntuserver
The key's randomart image is:
+---[RSA 2048]----+
|o*=.++.. ... |
|E..*.+o. o + |
|+o+ + = . o O + |
|+ .B o = o * + . |
|..+ + o S + . |
| . o + o |
| o |
| |
| |
+----[SHA256]-----+

Now, copy the SSH public key to your remote system:

# ssh-copy-id root@192.168.43.150

Sample output:

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.43.150 (192.168.43.150)' can't be established.
ECDSA key fingerprint is SHA256:U7I0O1OOzzbHFlhIG0HoGDr1usHzLBju6Jmr6bUB9Es.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.43.150's password: 

Number of key(s) added: 1

Now try logging into the machine, with: "ssh 'root@192.168.43.150'"
and check to make sure that only the key(s) you wanted were added.

Here, 192.168.43.150 is the remote system's IP address.

Now, you will be able to SSH to your remote system without having to enter the password:

# ssh root@192.168.43.150

Create a target directory in the remote system with command:

# mkdir remotesync

Log out from the remote system:

# exit

Next, edit Lsyncd config file in the source machine:

$ sudo nano /etc/lsyncd/lsyncd.conf.lua

Edit/modify the following lines:

settings {
 logfile = "/var/log/lsyncd/lsyncd.log",
 statusFile = "/var/log/lsyncd/lsyncd.status"
}

sync {
 default.rsync,
 source = "/home/sk/source_dir",
 host = "192.168.43.150",
 targetdir = "/root/remotesync",
}

Here, 192.168.43.150 is my remote host's IP address. And, /root/remotesync/ is the destination directory in my remote system. Replace the values with your own. Save and close the file.

Restart Lsyncd service to replicate the contents of source directory to destination.

$ sudo systemctl restart lsyncd

Now, SSH to your remote system:

$ ssh root@192.168.43.150

And, check the contents of the target directory (i.e /root/remotesync/ in our case ). You will see all files from source directory from the local system have been replicated to the target directory in the destination system.

root@server1 ~]# ls remotesync/
file1 file10 file2 file3 file4 file5 file6 file7 file8 file9

This procedure is same for RHEL and CentOS systems. Setup passwordless SSH login and make sure you have modified the correct source and target paths in Lsyncd main config file (/etc/lsyncd.conf) in your RHEL or CentOS system, and the rest of the steps are same.

You know how to synchronize local and remote directories in Linux using Lsyncd. As you can see, this is very easy and straight forward method.

Resource:

• Lsyncd GitHub page