Chapter 19. Disaster Recovery with Rear¶
Contents
Abstract
Relax-and-Recover (formerly “ReaR”, in this chapter abbreviated as Rear) is a disaster recovery framework for use by system administrators. It is a collection of Bash scripts that need to be adjusted to the specific production environment that is to be protected in case of disaster.
No disaster recovery solution will just work out-of-the-box. Therefore it is essential to take preparations before any disaster happens.
19.1. Conceptual Overview¶
The following sections describe the general disaster recovery concept and the basic steps you need to execute for successful recovery with Rear. They also provide some guidance on Rear requirements, Rear versions shipped with the current product, some limitations to be aware of, and scenarios and backup tools.
19.1.1. Creating a Disaster Recovery Plan¶
Before the worst scenario happens, take action: analyze your IT infrastructure for any substantial risks, evaluate your budget, and create a disaster recovery plan. If you do not already have a disaster recovery plan at hand, find some information on each step below:
Risk Analysis. Conduct a solid risk analysis of your infrastructure. List all the possible threats and evaluate how serious they are. Determine how likely these threats are and prioritize them. It is recommended to use a simple categorization: probability and impact.
Budget Planning. The outcome of the analysis is an overview, which risks can be tolerated and which are critical for your business. Ask yourself how you can minimize risks and how much will it cost. Depending on how big your company is, spend two to fifteen percent of the overall IT budget on disaster recovery.
Disaster Recovery Plan Development. Make checklists, test procedures, establish and assign priorities, and inventory your IT infrastructure. Define how to deal with a problem when some services in your infrastructure fail.
Test. After defining an elaborate plan, test it. Test it at least once a year. Use the same testing hardware as your main IT infrastructure.
19.1.2. What Does Disaster Recovery Mean?¶
If a system in a production environment has been destroyed (for whatever reasons—be it broken hardware, a misconfiguration or software problems), you need to recreate the system. The recreation can be done either on the same hardware or on compatible replacement hardware. Recreating a system means more than just restoring files from a backup. It also includes preparing the system's storage (with regards to file systems, partitioning, or mount points), and reinstalling the boot loader.
19.1.3. How Does Disaster Recovery With Rear Work?¶
While the system is up and running, create a backup of the files and create a recovery system on a recovery medium. The recovery system contains a recovery installer.
In case the system has been destroyed, replace broken hardware (if needed), boot the recovery system from the recovery medium and launch the recovery installer. The recovery installer recreates the system: First, it prepares the storage (partitioning, file systems, mount points), then it restores the files from the backup. Finally, it reinstalls the boot loader.
19.1.4. Rear Requirements¶
To use Rear you need at least two identical systems: the machine that runs your production environment and an identical test machine. “Identical” in this context means that you can, for example, replace a network card with another one using the same Kernel driver.
![[Warning]](static/images/warning.png) | Identical Drivers Required |
|---|---|
If a hardware component does not use the same driver like the one in your production environment, it is not considered identical by Rear. | |
19.1.5. Rear Versions¶
SUSE Linux Enterprise High Availability Extension 11 SP4 ships two Rear versions in parallel:
Rear version
1.10.0(included in RPM package:rear). For the documentation of this Rear version, see the High Availability Guide for SUSE Linux Enterprise High Availability Extension 11 SP3. It is available from http://www.suse.com/documentation/.Rear version
1.16(included in RPM packagerear116). This version is documented in the current chapter.
![[Important]](static/images/important.png) | |
If you have a tested and fully functional disaster recovery procedure with Rear version 1.10.0 there is no need to upgrade. In that case, keep the Rear package and do not change your disaster recovery method! However, if Rear version 1.10.0 does not fulfill your particular
needs (see also Section 19.1.6), you can manually upgrade to Rear
version 1.16. The packages For each Rear version upgrade you must carefully re-validate whether your particular disaster recovery procedure still works. | |
19.1.6. Limitations with Btrfs¶
The following limitations apply if you use Btrfs.
- Your System Includes Subvolumes, but No Snapshots Subvolumes
Rear version 1.16 is required. This version supports recreating “normal” Btrfs subvolume structure (no snapshot subvolumes).
- Your System Includes Snapshot Subvolumes
Btrfs snapshot subvolumes cannot be backed up and restored as usual with file-based backup software.
While recent snapshots in subvolumes on Btrfs file systems need almost no disk space (because of Btrfs' copy-on-write functionality), those files would be backed up as complete files when using file-based backup software. They would end up twice in the backup with their original file size. Therefore, it would be impossible to restore the snapshots as they have been before on the original system.
19.1.7. Scenarios and Backup Tools¶
Rear can create a disaster recovery system (including a specific recovery installer) that can be booted from a local medium (like a hard disk, a flash disk, a DVD/CD-R) or via PXE. The backup data can be stored on a network file system, for example NFS, as described in Example 19.1.
Rear does not replace a file backup, but complements it. By default, Rear supports the generic tar command, and several third-party backup tools (such as Tivoli Storage Manager, QNetix Galaxy, Symantec NetBackup, EMC NetWorker, or HP DataProtector). Refer to Example 19.2 for an example configuration of using Rear with EMC NetWorker as backup tool.
19.1.8. Basic Steps¶
For a successful recovery with Rear in case of disaster, you need to execute the following basic steps:
- Setting Up Rear and Your Backup Solution
This includes tasks like editing the Rear configuration file, adjusting the Bash scripts, and configuring the backup solution that you want to use.
- Creating the Recovery Installation System
While the system to be protected is up and running use the rear mkbackup command to create a file backup and to generate a recovery system that contains a system-specific Rear recovery installer.
- Testing the Recovery Process
Whenever you have created a disaster recovery medium with Rear, test the disaster recovery process thoroughly. It is essential to use a test machine that has identical hardware like the one that is part of your production environment. For details, refer to Section 19.1.4, “Rear Requirements”.
- Recovering from Disaster
After a disaster has occurred, replace any broken hardware (if necessary). Then boot the Rear recovery system and start the recovery installer with the rear recover command.
19.2. Setting Up Rear and Your Backup Solution¶
To set up Rear, you need to edit at least the Rear configuration file
/etc/rear/local.conf and, if needed, the Bash scripts that are part
of the Rear framework.
In particular, you need to define the following tasks that Rear should do:
How to back up files and how to create and store the disaster recovery system. This needs to be configured in
/etc/rear/local.conf.What to recreate exactly (partitioning, file systems, mount points etc.). This can be defined in
/etc/rear/local.conf(for example, what to exclude). To recreate non-standard systems, you may have to enhance the Bash scripts.How the recovery process works. To change how Rear generates the recovery installer, or to adapt what the Rear recovery installer does, you have to edit the Bash scripts.
To configure Rear, add your options to the
/etc/rear/local.conf configuration file. (The former
configuration file /etc/rear/sites.conf has been removed
from the package. However, if you have such a file from your last setup,
Rear will still use it).
A default Rear configuration for reference is available at
/usr/share/rear/conf/default.conf. Other example
configurations (*example.conf) are available in the same
directory. Find more information in the Rear man page.
After you have changed the Rear configuration file, run the following command and check its output:
rear dump
Example 19.1. Using an NFS Server to Store the File Backup¶
Rear can be used in different scenarios. The following example uses an NFS server as storage for the file backup.
Set up an NFS server with YaST as described in the SUSE Linux Enterprise Server 11 SP4 Administration Guide, chapter Sharing File Systems with NFS. It is available from http://www.suse.com/documentation/.
Define the configuration for your NFS server in the
/etc/exportsfile. Make sure the directory on the NFS server (where you want the backup data to be available), has the right mount options. For example:/srv/nfs*(fsid=0,crossmnt,rw,no_root_squash,sync,no_subtree_check)If needed, replace
/srv/nfswith the path to your backup data on the NFS server and adjust the mount options. You will probably needno_root_squashto access the backup data as the rear mkbackup command runs asroot.Adjust the various
BACKUPparameters in the configuration file/etc/rear/local.confto make Rear store the file backup on the respective NFS server. Find an example in your installed system under/usr/share/rear/conf/SLE11-ext3-example.conf.
Example 19.2. Using Third-Party Backup Tools Like EMC NetWorker¶
Using third-party backup tools instead of tar requires appropriate settings in the Rear configuration file.
The following is an example configuration for EMC NetWorker. Add this
configuration snippet to /etc/rear/local.conf and
adjust it according to your setup:
BACKUP=NSR
OUTPUT=ISO
BACKUP_URL=nfs://host.example.com/path/to/your/rear/backup
OUTPUT_URL=nfs://host.example.com/path/to/your/rear/backup
NSRSERVER=backupserver.example.com
RETENTION_TIME="Month"19.3. Creating the Recovery Installation System¶
After you have configured Rear as described in Section 19.2, create the recovery installation system (including the Rear recovery installer) plus the file backup with the following command:
rear -d -D mkbackup
It executes the following steps:
Analyzing the target system and gathering information, in particular about the disk layout (partitioning, file systems, mount points) and about the boot loader.
Creating a bootable recovery system with the information gathered in the first step. The resulting Rear recovery installer is specific to the system that you want to protect from disaster. It can only be used to recreate this specific system.
Calling the configured backup tool to back up system and user files.
19.4. Testing the Recovery Process¶
After having created the recovery system, test the recovery process on a test machine with identical hardware. See also Section 19.1.4, “Rear Requirements”. Make sure the test machine is correctly set up and can serve as a replacement for your main machine.
| Extensive Testing on Identical Hardware |
|---|---|
Thorough testing of the disaster recovery process on machines is required. Test the recovery procedure on a regular basis to ensure everything works as expected. | |
Procedure 19.1. Performing a Disaster Recovery on a Test Machine¶
Create a recovery medium by burning the recovery system that you have created in Section 19.3 to a DVD or CD . Alternatively, you can use a network boot via PXE.
Boot the test machine from the recovery medium.
From the menu, select .
Log in as
root(no password needed).Enter the following command to start the recovery installer:
rear -d -D recover
For details about the steps that Rear takes during the process, see Recovery Process.
After the recovery process has finished, check if the system has been successfully recreated and can serve as a replacement for your original system in the production environment.
19.5. Recovering from Disaster¶
In case a disaster has occurred, exchange any broken hardware if necessary. Then proceed as described in Procedure 19.1, using either the repaired machine (or a tested, identical machine that can serve as a replacement for your original system).
The rear recover command will execute the following steps:
Recovery Process¶
Restoring the disk layout (partitions, file systems, and mount points).
Restoring the system and user files from the backup.
Restoring the boot loader.
19.6. For More Information¶
rear man page
/usr/share/doc/packages/rear/README