Chapter 7. Managing Software RAIDs 6 and 10 with mdadm / 7.2. Creating Nested RAID 10 Devices with mdadm
PrevChapter 7. Managing Software RAIDs 6 and 10 with mdadm7.3. Creating a Complex RAID 10 with mdadmNext

Creating Nested RAID 10 Devices with mdadm

Understanding Nested RAID Devices

A nested RAID device consists of a RAID array that uses another RAID array as its basic element, instead of using physical disks. The goal of this configuration is to improve the performance and fault tolerance of the RAID.

Linux supports nesting of RAID 1 (mirroring) and RAID 0 (striping) arrays. Generally, this combination is referred to as RAID 10. To distinguish the order of the nesting, this document uses the following terminology:

  • RAID 1+0: RAID 1 (mirror) arrays are built first, then combined to form a RAID 0 (stripe) array.

  • RAID 0+1: RAID 0 (stripe) arrays are built first, then combined to form a RAID 1 (mirror) array.

The following table describes the advantages and disadvantages of RAID 10 nesting as 1+0 versus 0+1. It assumes that the storage objects you use reside on different disks, each with a dedicated I/O capability.

Table 7.2. RAID Levels Supported in EVMS

RAID Level

Description

Performance and Fault Tolerance

10 (1+0)

RAID 0 (stripe) built with RAID 1 (mirror) arrays

RAID 1+0 provides high levels of I/O performance, data redundancy, and disk fault tolerance. Because each member device in the RAID 0 is mirrored individually, multiple disk failures can be tolerated and data remains available as long as the disks that fail are in different mirrors.

You can optionally configure a spare for each underlying mirrored array, or configure a spare to serve a spare group that serves all mirrors.

10 (0+1)

RAID 1 (mirror) built with RAID 0 (stripe) arrays

RAID 0+1 provides high levels of I/O performance and data redundancy, but slightly less fault tolerance than a 1+0. If multiple disks fail on one side of the mirror, then the other mirror is available. However, if disks are lost concurrently on both sides of the mirror, all data is lost.

This solution offers less disk fault tolerance than a 1+0 solution, but if you need to perform maintenance or maintain the mirror on a different site, you can take an entire side of the mirror offline and still have a fully functional storage device. Also, if you lose the connection between the two sites, either site operates independently of the other. That is not true if you stripe the mirrored segments, because the mirrors are managed at a lower level.

If a device fails, the mirror on that side fails because RAID 1 is not fault-tolerant. Create a new RAID 0 to replace the failed side, then resynchronize the mirrors.


Creating Nested RAID 10 (1+0) with mdadm

A nested RAID 1+0 is built by creating two or more RAID 1 (mirror) devices, then using them as component devices in a RAID 0.

[Important]Important

If you need to manage multiple connections to the devices, you must configure multipath I/O before configuring the RAID devices. For information, see Chapter 5, Managing Multipath I/O for Devices.

The procedure in this section uses the device names shown in the following table. Make sure to modify the device names with the names of your own devices.

Table 7.3. Scenario for Creating a RAID 10 (1+0) by Nesting

Raw Devices

RAID 1 (mirror)

RAID 1+0 (striped mirrors)

/dev/sdb1

/dev/md0

/dev/md2

/dev/sdc1

/dev/sdd1

/dev/md1

/dev/sde1


  1. Open a terminal console, then log in as the root user or equivalent.

  2. Create 2 software RAID 1 devices, using two different devices for each RAID 1 device. At the command prompt, enter these two commands:

    mdadm --create /dev/md0 --run --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1

    mdadm --create /dev/md1 --run --level=1 --raid-devices=2 /dev/sdd1 /dev/sde1

  3. Create the nested RAID 1+0 device. At the command prompt, enter the following command using the software RAID 1 devices you created in Step 2:

    mdadm --create /dev/md2 --run --level=0 --chunk=64 --raid-devices=2 /dev/md0 /dev/md1

    The default chunk size is 64 KB.

  4. Create a file system on the RAID 1+0 device /dev/md2, such as a Reiser file system (reiserfs). For example, at the command prompt, enter

    mkfs.reiserfs /dev/md2

    Modify the command if you want to use a different file system.

  5. Edit the /etc/mdadm.conf file to add entries for the component devices and the RAID device /dev/md2.

  6. Edit the /etc/fstab file to add an entry for the RAID 1+0 device /dev/md2.

  7. Reboot the server.

    The RAID 1+0 device is mounted to /local.

  8. (Optional) Add hot spares to service the underlying RAID 1 mirrors.

    For information, see Section 6.4, “Adding or Removing a Spare Disk”.

Creating Nested RAID 10 (0+1) with mdadm

A nested RAID 0+1 is built by creating two to four RAID 0 (striping) devices, then mirroring them as component devices in a RAID 1.

[Important]Important

If you need to manage multiple connections to the devices, you must configure multipath I/O before configuring the RAID devices. For information, see Chapter 5, Managing Multipath I/O for Devices.

In this configuration, spare devices cannot be specified for the underlying RAID 0 devices because RAID 0 cannot tolerate a device loss. If a device fails on one side of the mirror, you must create a replacement RAID 0 device, than add it into the mirror.

The procedure in this section uses the device names shown in the following table. Make sure to modify the device names with the names of your own devices.

Table 7.4. Scenario for Creating a RAID 10 (0+1) by Nesting

Raw Devices

RAID 0 (stripe)

RAID 0+1 (mirrored stripes)

/dev/sdb1

/dev/md0

/dev/md2

/dev/sdc1

/dev/sdd1

/dev/md1

/dev/sde1


  1. Open a terminal console, then log in as the root user or equivalent.

  2. Create 2 software RAID 0 devices, using two different devices for each RAID 0 device. At the command prompt, enter these two commands:

    mdadm --create /dev/md0 --run --level=0 --chunk=64 --raid-devices=2 /dev/sdb1 /dev/sdc1

    mdadm --create /dev/md1 --run --level=0 --chunk=64 --raid-devices=2 /dev/sdd1 /dev/sde1

    The default chunk size is 64 KB.

  3. Create the nested RAID 0+1 device. At the command prompt, enter the following command using the software RAID 0 devices you created in Step 2:

    mdadm --create /dev/md2 --run --level=1 --raid-devices=2 /dev/md0 /dev/md1

  4. Create a file system on the RAID 0+1 device /dev/md2, such as a Reiser file system (reiserfs). For example, at the command prompt, enter

    mkfs.reiserfs /dev/md2

    Modify the command if you want to use a different file system.

  5. Edit the /etc/mdadm.conf file to add entries for the component devices and the RAID device /dev/md2.

  6. Edit the /etc/fstab file to add an entry for the RAID 0+1 device /dev/md2.

  7. Reboot the server.

    The RAID 0+1 device is mounted to /local.

SUSE® Linux Enterprise Server Storage Administration Guide 10
PrevChapter 7. Managing Software RAIDs 6 and 10 with mdadmHome7.3. Creating a Complex RAID 10 with mdadmNext