Chapter 7. Advanced Disk Setup

Contents

7.1. LVM Configuration
7.2. Soft RAID Configuration

Sophisticated system configurations require particular disk setups. All common partitioning tasks can be done with YaST. To get persistent device naming with block devices, use the block devices below /dev/disk/by-id/. Logical Volume Management (LVM) is a disk partitioning scheme that is designed to be much more flexible than the physical partitioning used in standard setups. Its snapshot functionality enables easy creation of data backups. Redundant Array of Independent Disks (RAID) offers increased data integrity, performance, and fault tolerance. SUSE® Linux Enterprise Server also supports multipath I/O. For details, see the chapter about multipath I/O in Storage Administration Guide. Starting with SUSE Linux Enterprise 10, there is also the option to use iSCSI as a networked disk. Read more about iSCSI in Chapter 12, Mass Storage over IP Networks—iSCSI .

7.1. LVM Configuration

This section briefly describes the principles behind LVM and its basic features that make it useful under many circumstances. In Section 7.1.2, “LVM Configuration with YaST”, learn how to set up LVM with YaST.

[Warning]

Using LVM might be associated with increased risk, such as data loss. Risks also include application crashes, power failures, and faulty commands. Save your data before implementing LVM or reconfiguring volumes. Never work without a backup.

7.1.1. The Logical Volume Manager

The Logical Volume Manager (LVM) enables flexible distribution of hard disk space over several file systems. It was developed because sometimes the need to change the segmentation of hard disk space arises only after the initial partitioning during installation has already been done. Because it is difficult to modify partitions on a running system, LVM provides a virtual pool (volume group, VG for short) of memory space from which logical volumes (LVs) can be created as needed. The operating system accesses these LVs instead of the physical partitions. Volume groups can span more than only one disk so that several disks or parts of them may constitute one single VG. This way, LVM provides a kind of abstraction from the physical disk space that allows its segmentation to be changed in a much easier and safer way than physical repartitioning does. Background information regarding physical partitioning can be found in Section 8.5.7.1, “Partition Types” and Section 8.5.7, “Using the YaST Partitioner”.

Figure 7.1. Physical Partitioning versus LVM

Physical Partitioning versus LVM

Figure 7.1, “Physical Partitioning versus LVM” compares physical partitioning (left) with LVM segmentation (right). On the left side, one single disk has been divided into three physical partitions (PART), each with a mount point (MP) assigned so that the operating system can access them. On the right side, two disks have been divided into two and three physical partitions each. Two LVM volume groups (VG 1 and VG 2) have been defined. VG 1 contains two partitions from DISK 1 and one from DISK 2. VG 2 contains the remaining two partitions from DISK 2. In LVM, the physical disk partitions that are incorporated in a volume group are called physical volumes (PVs). Within the volume groups, four logical volumes (LV 1 through LV 4) have been defined, which can be used by the operating system via the associated mount points. The border between different logical volumes need not be aligned with any partition border. See the border between LV 1 and LV 2 in this example.

LVM features:

  • Several hard disks or partitions can be combined in a large logical volume.

  • Provided the configuration is suitable, an LV (such as /usr) can be enlarged when the free space is exhausted.

  • Using LVM, it is possible to add hard disks or LVs in a running system. However, this requires hot-swappable hardware that is capable of such actions.

  • It is possible to activate a "striping mode" that distributes the data stream of a logical volume over several physical volumes. If these physical volumes reside on different disks, this can improve the reading and writing performance just like RAID 0.

  • The snapshot feature enables consistent backups (especially for servers) in the running system.

With these features, using LVM already makes sense for heavily used home PCs or small servers. If you have a growing data stock, as in the case of databases, music archives, or user directories, LVM is just the right thing for you. This would allow file systems that are larger than the physical hard disk. Another advantage of LVM is that up to 256 LVs can be added. However, keep in mind that working with LVM is different from working with conventional partitions. Instructions and further information about configuring LVM is available in the official LVM HOWTO at http://tldp.org/HOWTO/LVM-HOWTO/.

Starting from kernel version 2.6, LVM version 2 is available, which is downward-compatible with the previous LVM and enables the continued management of old volume groups. When creating new volume groups, decide whether to use the new format or the downward-compatible version. LVM 2 does not require any kernel patches. It makes use of the device mapper integrated in kernel 2.6. This kernel only supports LVM version 2. Therefore, when talking about LVM, this section always refers to LVM version 2.

Instead of LVM 2, you can use EVMS (Enterprise Volume Management System), which offers a uniform interface for logical volumes and RAID volumes. Like LVM 2, EVMS makes use of the device mapper in kernel 2.6.

7.1.2. LVM Configuration with YaST

The YaST LVM configuration can be reached from the YaST Expert Partitioner (see Section 8.5.7, “Using the YaST Partitioner”). This partitioning tool enables you to edit and delete existing partitions and create new ones that should be used with LVM. There, create an LVM partition by first clicking Create+Do not format then selecting 0x8E Linux LVM as the partition identifier. After creating all the partitions to use with LVM, click LVM to start the LVM configuration.

7.1.2.1. Creating Volume Groups

If no volume group exists on your system yet, you are prompted to add one (see Figure 7.2, “Creating a Volume Group”). It is possible to create additional groups with Add group, but usually one single volume group is sufficient. system is suggested as a name for the volume group in which the SUSE Linux Enterprise® system files are located. The physical extent size defines the size of a physical block in the volume group. All the disk space in a volume group is handled in chunks of this size. This value is normally set to 4 MB and allows for a maximum size of 256 GB for physical and logical volumes. The physical extent size should only be increased, for example, to 8, 16, or 32 MB, if you need logical volumes larger than 256 GB.

Figure 7.2. Creating a Volume Group

Creating a Volume Group

7.1.2.2. Configuring Physical Volumes

Once a volume group has been created, the following dialog lists all partitions with either the Linux LVM or Linux native type. No swap or DOS partitions are shown. If a partition is already assigned to a volume group, the name of the volume group is shown in the list. Unassigned partitions are indicated with --.

If there are several volume groups, set the current volume group in the selection box to the upper left. The buttons in the upper right enable creation of additional volume groups and deletion of existing volume groups. Only volume groups that do not have any partitions assigned can be deleted. All partitions that are assigned to a volume group are also referred to as a physical volumes (PV).

Figure 7.3. Physical Volume Setup

Physical Volume Setup

To add a previously unassigned partition to the selected volume group, first click the partition then Add Volume. At this point, the name of the volume group is entered next to the selected partition. Assign all partitions reserved for LVM to a volume group. Otherwise, the space on the partition remains unused. Before exiting the dialog, every volume group must be assigned at least one physical volume. After assigning all physical volumes, click Next to proceed to the configuration of logical volumes.

7.1.2.3. Configuring Logical Volumes

After the volume group has been filled with physical volumes, define the logical volumes the operating system should use in the next dialog. Set the current volume group in a selection box to the upper left. Next to it, the free space in the current volume group is shown. The list below contains all logical volumes in that volume group. All normal Linux partitions to which a mount point is assigned, all swap partitions, and all already existing logical volumes are listed here. Add, Edit, and Remove logical volumes as needed until all space in the volume group has been exhausted. Assign at least one logical volume to each volume group.

Figure 7.4. Logical Volume Management

Logical Volume Management

To create a new logical volume, click Add and fill out the pop-up that opens. As for partitioning, enter the size, file system, and mount point. Normally, a file system, such as reiserfs or ext2, is created on a logical volume and is then designated a mount point. The files stored on this logical volume can be found at this mount point on the installed system. Additionally it is possible to distribute the data stream in the logical volume among several physical volumes (striping). If these physical volumes reside on different hard disks, this generally results in a better reading and writing performance (like RAID 0). However, a striping LV with n stripes can only be created correctly if the hard disk space required by the LV can be distributed evenly to n physical volumes. If, for example, only two physical volumes are available, a logical volume with three stripes is impossible.

[Warning]Striping

YaST has no chance at this point to verify the correctness of your entries concerning striping. Any mistake made here is apparent only later when the LVM is implemented on disk.

Figure 7.5. Creating Logical Volumes

Creating Logical Volumes

If you have already configured LVM on your system, the existing logical volumes can be entered now. Before continuing, assign appropriate mount points to these logical volumes too. With Next, return to the YaST Expert Partitioner and finish your work there.

7.1.2.4. Direct LVM Management

If you already have configured LVM and only want to change something, there is an alternative way to do that. In the YaST Control Center, select System+LVM. Basically this dialog allows the same actions as described above with the exception of physical partitioning. It shows the existing physical volumes and logical volumes in two lists and you can manage your LVM system using the methods already described.

7.1.3. Storage Management with EVMS

The Enterprise Volume Management System 2 (EVMS2) is a rich, extensible volume manager with built-in cluster awareness. Its plug-in framework allows plug-ins to add functionality for support and knowledge of any partition type. Being cluster-aware, EVMS2 guarantees that managed devices are named identically on each node in the cluster for easier management.

EVMS2 provides a unified interface (evmsgui and command line) for managing the following storage resources:

  • Physical disks and logical devices on local media and SAN-based media, including iSCSI

  • Software RAIDs 0, 1, 4, and 5 for high availability

  • Cluster-aware multipath I/O for fault tolerance

  • Cluster storage objects with the Cluster Segment Manager (CSM) plug-in

  • Volumes for all file systems that have a file system interface module (FSIM) for EVMS2

  • Snapshots of volumes

In SUSE Linux Enterprise Server 10, new features include the following:

  • EVMS2 and CLVM2 (Cluster Linux Volume Manager 2) use the same multidisk (MD) drivers and device mapper (DM) drivers in the kernel.

  • File system plug-ins are available for Heartbeat 2 Cluster Manager and Oracle Cluster File System 2.

7.1.3.1. EVMS Devices

The EVMS Administration Utility distinguishes five different levels of devices:

Disks

This is the lowest level of device. All devices that may be accessed as a physical disk are treated as disks.

Segments

Segments consist of partitions and other memory regions on a disk, such as the master boot record (MBR).

Containers

These are the counterparts of volume groups in LVM.

Regions

The available devices are grouped into LVM2 and RAID here.

Volumes

All devices, regardless of whether they are represented by a real partition, a logical volume, or a RAID device are available with their respective mount points.

If you choose to use EVMS, you must replace your device names with the EVMS device names. Simple partitions are found in /dev/evms/, logical volumes in /dev/evms/lvm/, and RAID devices in /dev/evms/md. To activate EVMS at boot time, add boot.evms to the boot scripts in the YaST runlevel editor. See also Section 20.2.3, “Configuring System Services (Runlevel) with YaST”.

7.1.3.2. For More Information

For information about using EVMS to manage storage resources, see the Storage Administration Guide that is available in /usr/share/doc/manual/sles-stor_evms_en after installing the package sles-stor_evms_en. More common information about EVMS is also available in EVMS User Guide at the EVMS project, hosted on SourceForge*.

7.2. Soft RAID Configuration

The purpose of RAID (redundant array of independent disks) is to combine several hard disk partitions into one large virtual hard disk to optimize performance, data security, or both. Most RAID controllers use the SCSI protocol because it can address a larger number of hard disks in a more effective way than the IDE protocol and is more suitable for parallel processing of commands. There are some RAID controllers that support IDE or SATA hard disks. Soft RAID provides the advantages of RAID systems without the additional cost of hardware RAID controllers. However, this requires some CPU time and has memory requirements that make it unsuitable for real high performance computers.

7.2.1. RAID Levels

SUSE® Linux Enterprise offers the option of combining several hard disks into one soft RAID system with the help of YaST—a very reasonable alternative to hardware RAID. RAID implies several strategies for combining several hard disks in a RAID system, each with different goals, advantages, and characteristics. These variations are commonly known as RAID levels.

Common RAID levels are:

RAID 0

This level improves the performance of your data access by spreading out blocks of each file across multiple disk drives. Actually, this is not really a RAID, because it does not provide data backup, but the name RAID 0 for this type of system has become the norm. With RAID 0, two or more hard disks are pooled together. The performance is very good, but the RAID system is destroyed and your data lost if even one hard disk fails.

RAID 1

This level provides adequate security for your data, because the data is copied to another hard disk 1:1. This is known as hard disk mirroring. If a disk is destroyed, a copy of its contents is available on another one. All of them except one could be damaged without endangering your data. However, if damage is not detected, it also may happen that damaged data is mirrored to the correct disk and data corruption happens that way. The writing performance suffers a little in the copying process compared to when using single disk access (10 to 20 % slower), but read access is significantly faster in comparison to any one of the normal physical hard disks, because the data is duplicated so can be parallel scanned. Generally it can be said that Level 1 provides nearly twice the read transaction rate of single disks and almost the same write transaction rate as single disks.

RAID 2 and RAID 3

These are not typical RAID implementations. Level 2 stripes data at the bit level rather than the block level. Level 3 provides byte-level striping with a dedicated parity disk and cannot service simultaneous multiple requests. Both levels are only rarely used.

RAID 4

Level 4 provides block-level striping just like Level 0 combined with a dedicated parity disk. In the case of a data disk failure, the parity data is used to create a replacement disk. However, the parity disk may create a bottleneck for write access. Nevertheless, Level 4 is sometimes used.

RAID 5

RAID 5 is an optimized compromise between Level 0 and Level 1 in terms of performance and redundancy. The hard disk space equals the number of disks used minus one. The data is distributed over the hard disks as with RAID 0. Parity blocks, created on one of the partitions, are there for security reasons. They are linked to each other with XOR, enabling the contents to be reconstructed by the corresponding parity block in case of system failure. With RAID 5, no more than one hard disk can fail at the same time. If one hard disk fails, it must be replaced as soon as possible to avoid the risk of losing data.

Other RAID Levels

Several other RAID levels have been developed (RAIDn, RAID 10, RAID 0+1, RAID 30, RAID 50, etc.), some of them being proprietary implementations created by hardware vendors. These levels are not very widespread, so are not explained here.

7.2.2. Soft RAID Configuration with YaST

The YaST soft RAID configuration can be reached from the YaST Expert Partitioner, described in Section 8.5.7, “Using the YaST Partitioner”. This partitioning tool enables you to edit and delete existing partitions and create new ones that should be used with soft RAID. There, create RAID partitions by first clicking Create+Do not format then selecting 0xFD Linux RAID as the partition identifier. For RAID 0 and RAID 1, at least two partitions are needed—for RAID 1, usually exactly two and no more. If RAID 5 is used, at least three partitions are required. It is recommended to take only partitions of the same size. The RAID partitions should be stored on different hard disks to decrease the risk of losing data if one is defective (RAID 1 and 5) and to optimize the performance of RAID 0. After creating all the partitions to use with RAID, click RAID+Create RAID to start the RAID configuration.

In the next dialog, choose between RAID levels 0, 1, and 5 (see Section 7.2.1, “RAID Levels” for details). After Next is clicked, the following dialog lists all partitions with either the Linux RAID or Linux native type (see Figure 7.6, “RAID Partitions”). No swap or DOS partitions are shown. If a partition is already assigned to a RAID volume, the name of the RAID device (for example, /dev/md0) is shown in the list. Unassigned partitions are indicated with --.

Figure 7.6. RAID Partitions

RAID Partitions

To add a previously unassigned partition to the selected RAID volume, first click the partition then Add. At this point, the name of the RAID device is entered next to the selected partition. Assign all partitions reserved for RAID. Otherwise, the space on the partition remains unused. After assigning all partitions, click Next to proceed to the settings dialog where you can fine-tune the performance (see Figure 7.7, “File System Settings”).

Figure 7.7. File System Settings

File System Settings

As with conventional partitioning, set the file system to use as well as encryption and the mount point for the RAID volume. After completing the configuration with Finish, see the /dev/md0 device and others indicated with RAID in the expert partitioner.

7.2.3. Troubleshooting

Check the file /proc/mdstats to find out whether a RAID partition has been destroyed. In the event of a system failure, shut down your Linux system and replace the defective hard disk with a new one partitioned the same way. Then restart your system and enter the command mdadm /dev/mdX --add /dev/sdX. Replace 'X' with your particular device identifiers. This integrates the hard disk automatically into the RAID system and fully reconstructs it.

7.2.4. For More Information

Configuration instructions and more details for soft RAID can be found in the HOWTOs at:

Linux RAID mailing lists are also available, such as http://marc.theaimsgroup.com/?l=linux-raid&r=1&w=2.