Contents
When managing shared storage on a cluster, every node must be informed about changes that are done to the storage subsystem. The Linux Volume Manager 2 (LVM2), which is widely used to manage local storage, has been extended to support transparent management of volume groups across the whole cluster. Clustered volume groups can be managed using the same commands as local storage.
To implement cLVM, a shared storage setup — such as provided by a Fire Channel, FCoE, SCSI or iSCSI SAN or DRBD — must be available.
Internally, cLVM uses the Distributed Lock Manager (DLM) component of the cluster stack to coordinate access to the LVM2 metadata. As DLM in turn integrates with the other components of the stack such as fencing, the integrity of the shared storage is protected at all times.
cLVM does not coordinate access to the shared data itself; to do so, you must configure OCFS2 or other cluster-aware applications on top of the cLVM-managed storage.
To setup a cluster-aware volume group, several tasks must be completed successfully:
Change the locking type of LVM2 to be cluster-aware.
Edit the file /etc/lvm/lvm.conf and locate the
line:
locking_type = 1
Change the locking type to 3, and write the
configuration to disk. Copy this configuration to all nodes.
Include the clvmd resource as a clone in the pacemaker configuration, and make it depend on the DLM clone resource. A typical snippet from the crm configuration shell would look like this:
primitive dlm ocf:pacemaker:controld
primitive clvm ocf:lvm2:clvmd \
params daemon_timeout="30"
clone dlm-clone dlm \
meta target-role="Started" interleave="true" ordered="true"
clone clvm-clone clvm \
meta target-role="Started" interleave="true" ordered="true"
colocation colo-clvm inf: clvm-clone dlm-clone
order order-clvm inf: dlm-clone clvm-clone
...
Before proceeding, confirm that these resources have started successfully in your cluster. You may use crm_mon or the GUI to check the running services.
Prepare the physical volume for LVM with the command:
pvcreate <physical volume path>
Create a cluster aware volume group:
vgcreate --clustered y <volume group name> <physical volume path>
Create logical volumes as needed, for example:
lvcreate --name testlv -L 4G <volume group name>
To ensure that the volume group is activated cluster-wide, configure a LVM resource as follows:
primitive vg1 ocf:heartbeat:LVM \
params volgrpname="<volume group name>"
clone vg1-clone vg1 \
meta interleave="true" ordered="true"
colocation colo-vg1 inf: vg1-clone clvm-clone
order order-vg1 inf: clvm-clone vg1-clone
If you want the volume group to only be activated exclusively on one node, use the following example; in this case, cLVM will protect all logical volumes within the VG from being activated on multiple nodes, as an additional measure of protection for non-clustered applications:
primitive vg1 ocf:heartbeat:LVM \
params volgrpname="<volume group name>" exclusive="yes"
colocation colo-vg1 inf: vg1 clvm-clone
order order-vg1 inf: clvm-clone vg1
The logical volumes within the VG are now available as file system mounts or raw usage. Ensure that services using them must have proper dependencies to collocate them with and order them after the VG has been activated.
After finishing these configuration steps, the LVM2 configuration can be done just like on any standalone workstation.