Back up from QNAP to Linux with rsync

Introduction

The other day, I backed up the internal server, so I will briefly explain the meaning as a memorandum.

The content is as in the title. The goal is to synchronize the QNAP NAS used as a server with Linux (CentOS 8) using rsync and take regular backups. This time, assuming that the basic construction of Linux has been completed, I will explain the part from the installation of rsync to the completion of synchronization.

By the way, Linux prepared as a backup destination this time is built on HP's MicroServer.

Premise

--CentOS 8 is used as the OS of Linux this time. --A fixed IP address is assigned to both QNAP and Linux. --QNAP, Linux are on the same network

let's try it

Build rsync

Since the backup application is installed in the GUI on the QNAP NAS, most of the work to be done is on the Linux side. (Unless specified, interpret it as work on the basic Linux side)

Let's start with the installation.

Install rsync on Linux

I'm using the dnf command here, but I think it depends on your OS.

python


dnf -y install rsync rsync-daemon

Edit configuration file

python


vi /etc/rsyncd.conf

Add the following to the opened rsync configuration file.

/etc/rsyncd.conf


pig file = /var/run/rsyncd.pig
log file = /var/log/rsyncd.log
max connections = 4
transfer logging = yes

Boolean change, firewall service authorization

Change the Boolean value and Firewall settings to allow rsync access. Persistence is not required, but be aware that if you don't, you'll have to do the same for each boot.

python


setsebool -P rsync_full_access on         //-Persistence with P option
firewall-cmd --add-service=rsyncd --permanent  //Permanent with permanent
firewall-cmd --reload
firewall-cmd --list-all               //Confirm that rsyncd has been added to the service

Start rsync

Let's start it right away. Here, we start rsync with a daemon and check the free port and LISTEN status.

python


rsync --daemon --config /etc/rsyncd.conf
grep rsync /etc/services | head  //Check rsync port
lsof -i:873              //rsync tcp port(No. 873)Make sure that is LISTEN
ps `cat /var/run/rsyncd.pid`    //Make sure rsync is running as a daemon
systemctl enable rsyncd       //Set rsync autostart

So far, rsync installation, configuration, and startup are complete.

rsync test

When backing up an important server, we recommend that you carefully test each one to prevent accidents. Here, we will test in two stages.

Rsync in Linux

First, sync from Linux to Linux.

Create a test directory in Linux. For / var, select the directory you want to use as the backup destination.

python


mkdir -p /var(Arbitrary path)/test/origin
mkdir /var(Arbitrary path)/test/destination

Create a test file in the created directory and describe the contents arbitrarily

python


vi /var(The path you set earlier)/test/origin/test.txt  //Anything is fine, so describe the contents and save

Added test description to /etc/rsyncd.conf file

/etc/rsyncd.conf


[test]
path = /var(Same path set earlier)/test
hosts allow = "Linux IP address"
hosts deny = *
list = true
uid = root
gid = root
read only = false

Test sync from origin to destination and check success / failure

python


rsync -avz /var/test/origin/test.txt /var/test/destination
tree /var/test
cat /var/test/destination/test.txt

If the test.txt file is duplicated in the destination directory, the test is successful.

Synchronize test files from QNAP to Linux

Next, we will perform a synchronization test from QNAP to Linux.

Added more test description to /etc/rsyncd.conf file

/etc/rsyncd.conf


[test2]
path = /var(Same path set earlier)/test
hosts allow = "IP address of QNAP you want to back up"
hosts deny = *
list = true
uid = root
gid = root
read only = false

This is where the QNAP NAS finally comes into play.

Log in to QNAP's Web-UI and perform the operations in order.

From the menu tab in the upper left, select the application called Backup Station and go to Rsync for remote replication. After that, proceed to the remote replication job creation screen and perform the setting work.

The name of the job is arbitrary. I think it's a good idea to use a name that is easy to understand at a glance.

Also, in the remote site field, set the Linux information that was set earlier. For the user name, specify a user who has rewrite authority for the backup destination folder.

** Be sure to check the success or failure of the test here. ** ** The process itself takes only a few seconds, so if the process stops or fails, there is a high possibility that something is wrong with the settings up to this point.

When the process is completed and the average transmission time is displayed, it is successful.

Set the backup target folder in the source folder and the backup destination folder in the destination folder.

Since this is a test, specify a folder that can be damaged by any chance (** If not, create it separately **) as the source, and specify the test folder set as the path earlier as the destination.

Then set any options, apply the settings and start syncing. スクリーンショット-2020-09-28-16.30.47.png If replication is started in this way and completed successfully, it is successful. (Please check with the tree command etc. on the Linux side as well)

Execute this synchronization

If the test is completed successfully and there are no problems, it will move to this synchronization.

Since it is not possible to set the entire data at once when synchronizing from QNAP to Linux, it will be synchronized one by one from the source folder displayed when the job was created.

Therefore, it is necessary to increase the description in the rsync configuration file on the Linux side by that number. Edit the name and path based on the format below, and add as many as the number of synchronization targets.

Also, you need to create the same number of destination folders. It is easier to manage if you create the source directory and store all the destination folders in the same directory.

/etc/rsynd.conf


["Any name"]
path = "Backup destination folder"
hosts allow = "IP address of QNAP you want to back up"
hosts deny = *
list = true
uid = root
gid = root
read only = false

Let's synchronize immediately after finishing the description. However, the procedure is almost the same as when testing. Since this is not a test, set the source folder and destination folder for production.

The options are optional. The content is as written, so I think you should add what you want.

This time it is checked by default --Copy only files that are different from the destination file --Treat sparse files efficiently

In addition, I checked the following two options. --Enable file compression --Delete extra files on remote destination

After that, synchronization is started, and when replication is completed, synchronization is successful. As with the test, let's check the success or failure on the Linux side.

Finally, set the schedule at the desired pace and enable the schedule to enable regular automatic backups. (QNAP → Linux Rsync does not support constant synchronization, so the shortest is every hour)

Finally

There are many materials that pack up QNAP, such as from QNAP to QNAP, from Linux to QNAP, but there are not many materials that do the opposite, so it was quite difficult.

I hope it will be useful for those who want to do Linux from QNAP.

Recommended Posts

Back up from QNAP to Linux with rsync
ODBC access to SQL Server from Linux with Python
Create folders from '01' to '12' with python
Back up Qiita posts to GitHub
Experience Linux speeding up with Page cache
Convert from PDF to CSV with pdfplumber
Data integration from Python app on Linux to Amazon Redshift with ODBC
Connect to GNU / Linux with Remote Desktop
Set up reverse proxy to https server with CentOS Linux 8 + Apache mod_ssl
How to operate Linux from the console
How to power off Linux with Ultra96-V2
Introducing Docker Engine to Linux From Scratch
[Amazon Linux] Switching from Python 2 series to Python 3 series
How to get started with laravel (Linux)
Useful for changing permissions on Linux! How to count up to 31 with one hand.
[Linux command] From moving to macintosh (top directory) with cd command to connecting vagrant ssh
Mathematics memorandum to keep up with the field # 4
Install Windows 10 from a Linux server with PXE
Easy script migration from CentOS to Oracle Linux
Set up golang with goenv on GNU / Linux
How to operate Linux from the outside Procedure
How to handle Linux commands well from Python
Mathematics memorandum to keep up with the field # 1
Asynchronous processing with Arduino (Asynchronous processing of processing requests from Linux)
Mathematics memorandum to keep up with the field # 2
I tried to operate Linux with Discord Bot
From Python environment construction to virtual environment construction with anaconda
How to make Linux compatible with Japanese keyboard
Mathematics memorandum to keep up with the field # 3
Sum from 1 to 10
I came up with a way to make a 3D model from a photo. 0 Projection to 3D space
[Linux] Introduction to Linux
From setting up Raspberry Pi to installing Python environment
Images created with matplotlib shift from dvi to pdf
How to "cache enabled" access to FPGA memory from Linux
From Kafka to KSQL --Easy environment construction with docker
Yum command to access MySQL with Python 3 on Linux
Set up Docker on Oracle Linux (7.x) with Vagrant
I want to inherit to the back with python dataclass
Copy files directly from Amazon EC2 (Amazon linux) to S3
Speeding up when connecting from cx_Oracle to Autonomous Database
Passing confidential information from SSM to ECS with CloudFormation
Try to tamper with requests from iphone with Burp Suite
What I did to get started with Linux commands
MVC --Model edition to learn from 0 with prejudice only
[Amazon Linux 2] Change from public key authentication to password authentication
I came up with a way to create a 3D model from a photo Part 04 Polygon generation