This global option defines what to do when the cluster does not have quorum (no majority of nodes is part of the partition).

Allowed values are:

This global option defines if to apply fencing, allowing STONITH devices to shoot failed nodes and nodes with resources that cannot be stopped. By default, this global option is set to true, because for normal cluster operation it is necessary to use STONITH devices. According to the default value, the cluster will refuse to start any resources if no STONITH resources have been defined.

If you need to disable fencing for any reasons, set stonith-enabled to false.

For an overview of all global cluster options and their default values, see Pacemaker 1.0—Configuration Explained, available from http://clusterlabs.org/wiki/Documentation. Refer to section Available Cluster Options.

Before you can use a resource in the cluster, it must be set up. For example, if you want to use an Apache server as a cluster resource, set up the Apache server first and complete the Apache configuration before starting the respective resource in your cluster.

If a resource has specific environment requirements, make sure they are present and identical on all cluster nodes. This kind of configuration is not managed by the High Availability Extension. You must do this yourself.

	Do Not Touch Services Managed by the Cluster
When managing a resource with the High Availability Extension, the same resource must not be started or stopped otherwise (outside of the cluster, for example manually or on boot or reboot). The High Availability Extension software is responsible for all service start or stop actions.

However, if you want to check if the service is configured properly, start it manually, but make sure that it is stopped again before High Availability takes over.

After having configured the resources in the cluster, use the cluster management tools to start, stop, clean up, remove or migrate any resources manually. For details how to do so, refer to Chapter 5, Configuring and Managing Cluster Resources (GUI) or to Chapter 6, Configuring and Managing Cluster Resources (Command Line).

For each cluster resource you add, you need to define the standard that the resource agent conforms to. Resource agents abstract the services they provide and present an accurate status to the cluster, which allows the cluster to be non-committal about the resources it manages. The cluster relies on the resource agent to react appropriately when given a start, stop or monitor command.

Typically, resource agents come in the form of shell scripts. The High Availability Extension supports the following classes of resource agents:

The agents supplied with the High Availability Extension are written to OCF specifications.

The following types of resources can be created:

Whereas primitives are the simplest kind of resources and therefore easy to configure, you will probably also need more advanced resource types for cluster configuration, such as groups, clones or masters.

Groups

Some cluster resources are dependent on other components or resources, and require that each component or resource starts in a specific order and runs together on the same server. To simplify this configuration, you can use groups.

Example 4.1. Resource Group for a Web Server

An example of a resource group would be a Web server that requires an IP address and a file system. In this case, each component is a separate cluster resource that is combined into a cluster resource group. The resource group would then run on a server or servers, and in case of a software or hardware malfunction, fail over to another server in the cluster the same as an individual cluster resource.

Figure 4.1. Group Resource

Groups have the following properties:

Starting and Stopping

Resources are started in the order they appear in and stopped in the reverse order.

Dependency

If a resource in the group cannot run anywhere, then none of the resources located after that resource in the group is allowed to run.

Contents

Groups may only contain a collection of primitive cluster resources. Groups must contain at least one resource, otherwise the configuration is not valid. To refer to the child of a group resource, use the child’s ID instead of the group’s ID.

Constraints

Although it is possible to reference the group’s children in constraints, it is usually preferable to use the group’s name instead.

Stickiness

Stickiness is additive in groups. Every active member of the group will contribute its stickiness value to the group’s total. So if the default resource-stickiness is 100 and a group has seven members (five of which are active), then the group as a whole will prefer its current location with a score of 500.

Resource Monitoring

To enable resource monitoring for a group, you must configure monitoring separately for each resource in the group that you want monitored.

Learn how to create groups with the GUI in Procedure 5.12, “Adding a Resource Group”. If you prefer the command line approach, see Section 6.3.9, “Configuring a Cluster Resource Group”.

For each resource you add, you can define options. Options are used by the cluster to decide how your resource should behave—they tell the CRM how to treat a specific resource. Resource options can be set with the crm_resource --meta command or with the GUI as described in Procedure 5.3, “Adding or Modifying Meta and Instance Attributes”.

The scripts of all resource classes can be given parameters which determine how they behave and which instance of a service they control. If your resource agent supports parameters, you can add them with the crm_resource command or with the GUI as described in Procedure 5.3, “Adding or Modifying Meta and Instance Attributes”. In the crm command line utility, instance attributes are called params. The list of instance attributes supported by an OCF script can be found by executing the following command as root:

crm ra info [class:[provider:]]resource_agent

or, even shorter:

crm ra info resource_agent

The output lists all the supported attributes, their purpose and default values.

For example, the command

crm ra info Ipaddr

returns the following output:

Manages virtual IPv4 addresses (portable version) (ocf:heartbeat:IPaddr)
    
This script manages IP alias IP addresses
It can add an IP alias, or remove one.   
    
Parameters (* denotes required, [] the default):
    
ip* (string): IPv4 address
The IPv4 address to be configured in dotted quad notation, for example
"192.168.1.1".                                                        
    
nic (string, [eth0]): Network interface
The base network interface on which the IP address will be brought
online.                                                           
    
If left empty, the script will try and determine this from the    
routing table.                                                    
    
Do NOT specify an alias interface in the form eth0:1 or anything here;
rather, specify the base interface only.                              
    
cidr_netmask (string): Netmask
The netmask for the interface in CIDR format. (ie, 24), or in
dotted quad notation  255.255.255.0).                        
    
If unspecified, the script will also try to determine this from the
routing table.                                                     
    
broadcast (string): Broadcast address
Broadcast address associated with the IP. If left empty, the script will
determine this from the netmask.                                        
    
iflabel (string): Interface label
You can specify an additional label for your IP address here.
    
lvs_support (boolean, [false]): Enable support for LVS DR
Enable support for LVS Direct Routing configurations. In case a IP
address is stopped, only move it to the loopback device to allow the
local node to continue to service requests, but no longer advertise it
on the network.                                                       
    
local_stop_script (string): 
Script called when the IP is released
    
local_start_script (string): 
Script called when the IP is added
    
ARP_INTERVAL_MS (integer, [500]): milliseconds between gratuitous ARPs
milliseconds between ARPs                                         
    
ARP_REPEAT (integer, [10]): repeat count
How many gratuitous ARPs to send out when bringing up a new address
    
ARP_BACKGROUND (boolean, [yes]): run in background
run in background (no longer any reason to do this)
    
ARP_NETMASK (string, [ffffffffffff]): netmask for ARP
netmask for ARP - in nonstandard hexadecimal format.
    
Operations' defaults (advisory minimum):
    
start         timeout=90
stop          timeout=100
monitor_0     interval=5s timeout=20s

	Instance Attributes for Groups, Clones or Masters
Note that groups, clones and masters do not have instance attributes. However, any instance attributes set will be inherited by the group's, clone's or master's children.

By default, the cluster will not ensure that your resources are still healthy. To instruct the cluster to do this, you need to add a monitor operation to the resource’s definition. Monitor operations can be added for all classes or resource agents. For more information, refer to Section 4.3, “Resource Monitoring”.

There are three different kinds of constraints available:

For more information on configuring constraints and detailed background information about the basic concepts of ordering and collocation, refer to the following documents available at http://clusterlabs.org/wiki/Documentation:

Learn how to add the various kinds of constraints with the GUI in Section 5.3.3, “Configuring Resource Constraints”. If you prefer the command line approach, see Section 6.3.4, “Configuring Resource Constraints”.

When defining constraints, you also need to deal with scores. Scores of all kinds are integral to how the cluster works. Practically everything from migrating a resource to deciding which resource to stop in a degraded cluster is achieved by manipulating scores in some way. Scores are calculated on a per-resource basis and any node with a negative score for a resource cannot run that resource. After calculating the scores for a resource, the cluster then chooses the node with the highest score.

INFINITY is currently defined as 1,000,000. Additions or subtractions with it stick to the following three basic rules:

When defining resource constraints, you specify a score for each constraint. The score indicates the value you are assigning to this resource constraint. Constraints with higher scores are applied before those with lower scores. By creating additional location constraints with different scores for a given resource, you can specify an order for the nodes that a resource will fail over to.

A resource will be automatically restarted if it fails. If that cannot be achieved on the current node, or it fails N times on the current node, it will try to fail over to another node. Each time the resource fails, its failcount is raised. You can define a number of failures for resources (a migration-threshold), after which they will migrate to a new node. If you have more than two nodes in your cluster, the node a particular resource fails over to is chosen by the High Availability software.

However, you can specify the node a resource will fail over to by configuring one or several location constraints and a migration-threshold for that resource. For detailed instructions how to achieve this with the GUI, refer to Section 5.3.4, “Specifying Resource Failover Nodes”. If you prefer the command line approach, see Section 6.3.5, “Specifying Resource Failover Nodes”.

There are two exceptions to the migration threshold concept, occurring when a resource either fails to start or fails to stop:

For details on using migration thresholds and resetting failcounts, refer to Section 5.3.4, “Specifying Resource Failover Nodes”. If you prefer the command line approach, see Section 6.3.5, “Specifying Resource Failover Nodes”.

A resource might fail back to its original node when that node is back online and in the cluster. If you want to prevent a resource from failing back to the node it was running on prior to failover, or if you want to specify a different node for the resource to fail back to, you must change its resource stickiness value. You can either specify resource stickiness when you are creating a resource, or afterwards.

Consider the following implications when specifying resource stickiness values:

Not all resources are equal. Some, such as Xen guests, require that the node hosting them meets their capacity requirements. If resources are placed such that their combined need exceed the provided capacity, the resources diminish in performance (or even fail).

To take this into account, the High Availability Extension allows you to specify the following parameters:

Currently, these settings are static and must be configured by the administrator—they are not dynamically discovered or adjusted.

Learn how to configure these settings with the GUI in Section 5.3.6, “Configuring Placement of Resources Based on Load Impact”. If you prefer the command line approach, see Section 6.3.7, “Configuring Placement of Resources Based on Load Impact”.

A node is considered eligible for a resource if it has sufficient free capacity to satisfy the resource's requirements. The nature of the required or provided capacities is completely irrelevant for the High Availability Extension, it just makes sure that all capacity requirements of a resource are satisfied before moving a resource to a node.

To configure the resource's requirements and the capacity a node provides, use utilization attributes. You can name the utilization attributes according to your preferences and define as many name/value pairs as your configuration needs. However, the attribute's values must be integers.

The placement strategy can be specified with the placement-strategy property (in the global cluster options). The following values are available:

	Configuring Resource Priorities
The available placement strategies are best-effort—they do not yet use complex heuristic solvers to always reach optimum allocation results. Thus, set your resource priorities in a way that makes sure that your most important resources are scheduled first.

node node1 utilization memory="4000"
node node2 utilization memory="4000"
node node3 utilization memory="4000"
primitive xenA ocf:heartbeat:Xen utilization memory="3500" \
     meta priority="10"
primitive xenB ocf:heartbeat:Xen utilization memory="2000" \
     meta priority="1"
primitive xenC ocf:heartbeat:Xen utilization memory="2000" \
     meta priority="1"
primitive xenD ocf:heartbeat:Xen utilization memory="1000" \
     meta priority="5"
property placement-strategy="minimal"