Heartbeat > ◀ ▶

Chapter 4. Configuring and Managing Cluster Resources

Contents

4.1. Graphical HA Management Client
4.2. Creating Cluster Resources
4.3. Configuring Resource Constraints
4.4. Specifying Resource Failover Nodes
4.5. Specifying Resource Failback Nodes (Resource Stickiness)
4.6. Configuring Resource Monitoring
4.7. Starting a New Cluster Resource
4.8. Removing a Cluster Resource
4.9. Configuring a Heartbeat Cluster Resource Group
4.10. Configuring a Heartbeat Clone Resource
4.11. Migrating a Cluster Resource

Abstract

Cluster resources must be created for every resource or application you run on servers in your cluster. Cluster resources can include Web sites, e-mail servers, databases, file systems, virtual machines, and any other server-based applications or services you want to make available to users at all times.

You can either use the graphical HA Management Client utility, or the cibadmin command line utility to create resources.

This chapter introduces the graphical HA Management Client and then covers several topics you need when configuring a cluster: creating resources, configuring constraints, specifying failover nodes and failback nodes, configuring resource monitoring, starting or removing resources, configuring resource groups or clone resources, and migrating resources manually.

4.1. Graphical HA Management Client

When starting the Heartbeat GUI, you need to connect to a cluster.

[Note]Password for the hacluster User

The Heartbeat installation creates a linux user named hacluster. Prior to using the HA Management Client, you must set the password for the hacluster user. To do this, enter passwd hacluster at the command line and enter a password for the hacluster user.

Do this on every node where you will run the HA Management Client utility.

To start the HA Management Client, enter hb_gui at the command line of a Linux server or workstation. Log in to the cluster by selecting Connection+Login. You are prompted for a username and password.

If you are running the HA Management Client on a Heartbeat cluster server, the server and username fields should already be filled in. If not, specify the server IP address of a cluster server and hacluster as the username.

After being connected, your main window shows the cluster components, resources and constraints:

Depending on which entry you select in the left pane, several tabs appear in the right pane of the main window. For example, if you select the topmost entry, linux-ha, you can access three tabs on the right in the main window, allowing you to view general Information on the cluster or to change certain options and aspects on the Configurations and Advanced tabs.

In the following, find some examples how to create and manage cluster resources with the HA Management Client.

4.2. Creating Cluster Resources

To add a cluster resource:

  1. Start the HA Management Client and log in to the cluster as described in Section 4.1, “Graphical HA Management Client”.

  2. Select Resources+Add New Item, or click the + button.

  3. Choose the resource item type you want to create, then click OK.

    The possible options are:

    • Native

    • Group

    • Location (Constraint)

    • Order (Constraint)

    • Colocation (Constraint)

    All but the Native and Group resource item types are constraints. You must have a resource or group created in order to create a constraint and add that constraint to a resource.

  4. Enter the Resource ID (name) and if desired, choose a group to add the resource to.

  5. Select the resource Type from the list. To display a brief description of that resource type, double-click any of the resource types in the list.

  6. Conditional: After selecting a resource type, a line for that resource type might be added to the Parameters section of the screen. If a line is added, click the line once and then click the line again under the Value heading to open a field where you can add the needed value.

    1. Add the needed value for the resource.

      For example, if you selected an IP address as the resource type above, enter the IP address for the resource.

    2. Click Add Parameter, specify the parameter Name and Value, then click OK.

  7. Click Add at the bottom of the screen to add the resource to the cluster.

4.3. Configuring Resource Constraints

Resource constraints let you specify which cluster nodes resources will run on, what order resources will load, and what other resources a specific resource is dependent on. For information on configuring resource constraints, see http://www.linux-ha.org/ClusterInformationBase/SimpleExamples.

4.4. Specifying Resource Failover Nodes

Cluster resources will automatically fail over to another node in the cluster in the event of a software or hardware failure. If you have more than two nodes in your cluster, the node a particular resource fails over to is chosen by the Heartbeat software. If you want to choose which node a resource will fail over to, you must configure a location constraint for that resource.

  1. Start the HA Management Client and log in to the cluster as described in Section 4.1, “Graphical HA Management Client”.

  2. Select the desired resource or group, select Resources+Add New Item, or click the + button.

  3. Choose Location as the resource item type, then click OK.

  4. Specify the ID of the resource constraint. The resource name for the resource you selected should already be displayed in the Resource field.

  5. Click OK to add the constraint. The score will automatically be set to 0 by default to avoid unwanted effects caused by an incomplete rule.

    Score indicates the value you are assigning to this resource constraint. Constraints with higher scores are applied before those with lower scores. By creating additional location constraints with different scores for a given resource, you can specify an order for the nodes that a resource will fail over to.

  6. If you need to modify the score, select the added constraint in the left pane of the main window and enter a new Score value in the Attributes section in the right pane.

  7. On the left side of the page, select the resource constraint you just created and click Add Expression.

  8. Select #uname as the Attribute, eq as the Operation, and the server name of the failover node as the Value.

  9. Click OK, then click Apply to save your changes.

  10. Conditional: If you want to specify additional failover nodes for a resource, create additional location constraints by performing Step 1 through Step 9 for each constraint.

4.5. Specifying Resource Failback Nodes (Resource Stickiness)

A resource will automatically fail back to its original node when that node is back online and in the cluster. If you want to prevent a resource from failing back to the node it was running on prior to a failover, or if you want to specify a different node for the resource to fail back to, you must change its resource stickiness value. You can specify resource stickiness when you are creating a resource, or after.

To specify resource stickiness for a resource after it has been created:

  1. Start the HA Management Client and log in to the cluster as described in Section 4.1, “Graphical HA Management Client”.

  2. Select the desired resource or group, click the Attributes tab, then click Add Attribute.

  3. In the Name field, type resource_stickiness.

  4. In the Value field, specify a value between -100,000 and 100,000.

Consider the following when specifying a resource stickiness value:

Value is 0:

This is the default. The resource will be placed optimally in the system. This may mean that it is moved when a “better” or less loaded node becomes available. This option is almost equivalent to automatic failback, except that the resource may be moved to a node that is not the one it was previously active on.

Value is greater than 0:

The resource will prefer to remain in its current location, but may be moved if a more suitable node is available. Higher values indicate a stronger preference for a resource to stay where it is.

Value is less than 0:

The resource prefers to move away from its current location. Higher absolute values indicate a stronger preference for a resource to be moved.

Value is 100,000:

The resource will always remain in its current location unless forced off because the node is no longer eligible to run the resource (node shutdown, node standby, or configuration change). This option is almost equivalent to completely disabling automatic failback, except that the resource may be moved to other nodes than the one it was previously active on.

Value is -100,000:

The resource will always move away from its current location.

See also the crm_failcount command: crm_failcount(8).

4.6. Configuring Resource Monitoring

Although Heartbeat can detect a node failure, it also has the ability to detect when an individual resource on a node has failed. If you want Heartbeat to ensure that a resource is running, you must configure resource monitoring for that resource. Resource monitoring consists of specifying a timeout and/or start delay value, and an interval. The interval tells Heartbeat how often it should check the resource status.

To configure resource monitoring:

  1. Start the HA Management Client and log in to the cluster as described in Section 4.1, “Graphical HA Management Client”.

  2. Select the resource, click the Operations tab, then click Add Operation.

  3. Select Monitor as the operation name.

  4. Add the desired values in the Interval, Timeout, and Start Delay fields, and a description if desired.

  5. Click OK, then click Apply to start the monitoring operation.

If you do not configure resource monitoring, resource failures after a successful start will not be communicated, and the cluster will always show the resource as healthy.

If the resource monitor detects a failure, the following actions will take place:

  • Log file messages will be generated

  • The failure will be reflected in the hb_gui and crm_mon tools, and in the CIB status section

  • The cluster will initiate noticeable recovery actions which may include stopping the resource to repair the failed state and restarting the resource locally or on another node. The resource also may not be restarted at all, depending on the configuration and state of the cluster.

4.7. Starting a New Cluster Resource

Some cluster resources do not start automatically after being created. To start a new cluster resource after creating it, in the HA Management Client, select the resource in the left pane, then click Resource+Start.

4.8. Removing a Cluster Resource

You can remove a cluster resource using the HA Management Client. To do this, select the resource in the left pane, and click Resource +Delete.

4.9. Configuring a Heartbeat Cluster Resource Group

Some cluster resources are dependent on other components or resources, and require that each component or resource are started in a specific order and run together on the same server. An example of this is a Web server that requires an IP address and a file system. In this case each component is a separate cluster resource that is combined into a cluster resource group. The resource group would then run on a server or servers, and in case of a software or hardware malfunction, fail over to another server in the cluster the same as an individual cluster resource.

Figure 4.1. Group Resource

Group Resource

To configure a resource group using the above example:

  1. Start the HA Management Client and log in to the cluster as described in Section 4.1, “Graphical HA Management Client”.

  2. Select Resources+Add New Item, or click the + button.

  3. Choose Group as the resource item type, then click OK.

  4. Specify a Group name (ID), leave the default of True for both the Ordered and Collocated fields, then click OK.

    The Ordered value specifies, if the resources in the group will load in order you specified. The Collocated value specifies whether the resources in the group will run on the same server.

  5. Specify a resource name (ID) for the IP address resource portion of the group.

  6. In the Type section of the page, scroll down the list and select IPaddr2 (OCF Resource Agent) as the resource type.

  7. In the Parameters section of the page, find the line that was added for the IP address resource, click the line once, then click the line again under the Value heading to open a text field.

  8. Add the IP address for the IP address cluster resource.

  9. Click the Add Parameter button, select nic as the name and enter eth0 as the value, then click OK.

    The name and value are dependent on your hardware configuration and what you chose in Step 3.

  10. Click the Add button at the bottom of the page to add the resource to the group.

4.9.1. Adding a File System to the Resource Group

  1. Using the HA Management Client, select Resources, and then select Add New Item, or click the + button.

  2. Choose Native as the resource item type, then click OK.

  3. Specify a resource name (ID) for the file system resource, and in the Belong to Group field, select the group you created above.

  4. In the Type section of the page, scroll down the list and select Filesystem as the resource type.

  5. In the Parameters section of the page, find the line that was added for the file system resource, click the line once, then click the line again under the Value heading to open a text field.

  6. Add the name of the file system. For example, Reiser.

  7. Click the Add Parameter button and select Device as the name. For the value, specify the device where the file system is located, then click OK.

    The name and value are dependent on your hardware configuration. For example, you could specify /dev/sdc as the value if the file system is on that device.

4.9.2. Adding the Web Server to the Resource Group

  1. Using the HA Management Client, select Resources, and then select Add New Item, or click the + button.

  2. Choose Native as the resource item type, then click OK.

  3. Specify a resource name (ID) for the Web Server resource, and in the Belong to group field, select the group you created above.

  4. In the Type section of the page, scroll down the list and select Apache as the resource type.

  5. In the Parameters section of the page, find the line that was added for the Apache Web server resource, accept or change the path to the Apache Web server, then click OK.

[Note]

If you want to enable resource monitoring for a group resource, you must configure monitoring separately for each resource in the group that you want monitored.

4.10. Configuring a Heartbeat Clone Resource

You may want certain resources to run simultaneously on multiple nodes in your cluster. To do this you must configure a resource as a clone. Examples of resources that might be configured as clones include STONITH and cluster file systems like OCFS2. You can clone any resource provided it is supported by the resource’s ResourceAgent. Clone resources may even be configured differently depending on which nodes they are hosted.

There are three types of resource clones:

  • Anonymous Clones:  These are the simplest type of clones. They behave identically anywhere they are running.

  • Globally Unique Clones:  These resources are distinct entities. An instance of the clone running on one node is not equivalent to another instance on another node; nor would any two instances on the same node be equivalent.

  • Stateful Clones:  Active instances of these resources are divided into two states, active and passive. These are also sometimes referred to as primary and secondary, or master and slave. Stateful clones can be either anonymous or globally unique.

  1. To configure a resource as a clone, follow Step 1 through Step 6 in Section 4.2, “Creating Cluster Resources”.

  2. Select the Clone check box. This will allow the resource to simultaneously run on multiple cluster nodes.

  3. In the clone_node_max field, enter the number of instances of the resource that will run on a given node.

  4. In the clone_max field, enter the number of nodes in the cluster that will run the resource.

  5. Click Add to add the clone resource to the cluster.

  6. Select the resource in the left pane and click Resource+Start to start the resource. Starting the clone resource will cause it to start on the nodes that have been specified with its resource location constraints.

    After creating the resource, you must create location constraints for the resource. The location constraints determine which nodes the resource can run on. For information on configuring resource constraints, see http://www.linux-ha.org/ClusterInformationBase/SimpleExamples.

4.11. Migrating a Cluster Resource

Although resources are configured to automatically fail over (or migrate) to other nodes in the cluster in the even of a hardware or software failure, you can also manually migrate a resource to another node in the cluster using either the HA Management Client or the command line.

  1. At the command line, use the following command:

    crm_resource -M -r resource_name -H hostname -f

    For example, to migrate a resource named ipaddress1 to a cluster node named node2, you would need to enter:

    crm_resource -M -r ipaddress1 -H node2 -f

    To get a list of parameters and switches that can be used with the crm_resource command, enter crm_resource without any options.

  2. To use the HA Management Client for migration, start the HA Management Client and log in to the cluster as described in Section 4.1, “Graphical HA Management Client”.

  3. Select the resource you want to migrate in the left pane of the main window and select Resources+Migrate Resource.

  4. In the new window, select ipaddress1 as Resource to migrate and select node1 from the To Node drop-down list.

Heartbeat Guide

Heartbeat > ◀ ▶