Load Balancing with Linux Virtual Server

Contents

10.1. Conceptual Overview
10.2. Configuring IP Load Balancing with YaST
10.3. Further Setup
10.4. For More Information

The goal of Linux Virtual Server (LVS) is to provide a basic framework that directs network connections to multiple servers that share their workload. Linux Virtual Server is a cluster of servers (one or more load balancers and several real servers for running services) which appears to be one large, fast server to an outside client. This apparent single server is called a virtual server. The Linux Virtual Server can be used to build highly scalable and highly available network services, such as Web, cache, mail, FTP, media and VoIP services.

The real servers and the load balancers may be interconnected by either high-speed LAN or by geographically dispersed WAN. The load balancers can dispatch requests to the different servers. They make parallel services of the cluster appear as a virtual service on a single IP address (the virtual IP address or VIP). Request dispatching can use IP load balancing technologies or application-level load balancing technologies. Scalability of the system is achieved by transparently adding or removing nodes in the cluster. High availability is provided by detecting node or daemon failures and reconfiguring the system appropriately.

Conceptual Overview

The following sections give an overview of the main LVS components and concepts.

Director

The main component of LVS is the ip_vs (or IPVS) kernel code. It implements transport-layer load balancing inside the Linux kernel (layer-4 switching). The node that runs a Linux kernel including the IPVS code is called director. The IPVS code running on the director is the essential feature of LVS.

When clients connect to the director, the incoming requests are load-balanced across all cluster nodes: The director forwards packets to the real servers, using a modified set of routing rules that make the LVS work. For example, connections do not originate or terminate on the director, it does not send acknowledgments. The director acts as a specialized router that forwards packets from end-users to real servers (the hosts that run the applications that process the requests).

By default, the kernel does not have the IPVS module installed. The IPVS kernel module is included in the cluster-network-kmp-default package.

User Space Controller and Daemons

The ldirectord daemon is a user-space daemon for managing Linux Virtual Server and monitoring the real servers in an LVS cluster of load balanced virtual servers. A configuration file, /etc/ha.d/ldirectord.cf, specifies the virtual services and their associated real servers and tells ldirecord how to configure the server as a LVS redirector. When the daemon is initialized, it creates the virtual services for the cluster.

By periodically requesting a known URL and checking the responses, the ldirectord daemon monitors the health of the real servers. If a real server fails, it will be removed from the available server list at the load balancer. When the service monitor detects that the dead server has recovered and is working again, it will add the server back to the available server list. For the case that all real servers should be down, a fall-back server can be specified to which to redirect a Web service. Typically the fall-back server is localhost, presenting an emergency page about the Web service being temporarily unavailable.

Packet Forwarding

There are three different methods of how the director can send packets from the client to the real servers:

Network Address Translation (NAT): Incoming requests arrive at the virtual IP and are forwarded to the real servers by changing the destination IP address and port to that of the chosen real server. The real server sends the response to the load balancer which in turn changes the destination IP address and forwards the response back to the client, so that the end user receives the replies from the expected source. As all traffic goes through the load balancer, it usually becomes a bottleneck for the cluster.
IP Tunneling (IP-IP Encapsulation): IP tunneling enables packets addressed to an IP address to be redirected to another address, possibly on a different network. The LVS sends requests to real servers through an IP tunnel (redirecting to a different IP address) and the real servers reply directly to the client using their own routing tables. Cluster members can be in different subnets.
Direct Routing: Packets from end users are forwarded directly to the real server. The IP packet is not modified, so the real servers must be configured to accept traffic for the virtual server's IP address. The response from the real server is sent directly to the client. The real servers and load balancers have to be in the same physical network segment.

Scheduling Algorithms

Deciding which real server to use for a new connection requested by a client is implemented using different algorithms. They are available as modules and can be adapted to specific needs. For an overview of available modules, refer to the ipvsadm(8) man page. Upon receiving a connect request from a client, the director assigns a real server to the client based on a schedule. The scheduler is the part of the IPVS kernel code which decides which real server will get the next new connection.

Configuring IP Load Balancing with YaST

You can configure kernel-based IP load balancing with the YaST iplb module. It is a front-end for ldirectord.

To access the IP Load Balancing dialog, start YaST as root and select High Availability+IP Load Balancing. Alternatively, start the YaST cluster module as root on a command line with yast2 iplb.

The YaST module writes its configuration to /etc/ha.d/ldirectord.cf. The tabs available in the YaST module correspond to the structure of the /etc/ha.d/ldirectord.cf configuration file, defining global options and defining the options for the virtual services.

For an example configuration and the resulting processes between load balancers and real servers, refer to Example 10.1, “Simple ldirectord Configuration”.

	Global Parameters and Virtual Server Parameters
If a certain parameter is specified in both the virtual server section and in the global section, the value defined in the virtual server section overrides the value defined in the global section.

Procedure 10.1. Configuring Global Parameters

The following procedure describes how to configure the most important global parameters. For more details about the individual parameters (and the parameters not covered here), click Help or refer to the ldirectord man page.

With Check Interval, define the interval in which ldirectord will connect to each of the real servers to check if they are still online.
With Check Timeout, set the time in which the real server should have responded after the last check.
With Check Count you can define how many times ldirectord will attempt to request the real servers until the check is considered as failed.
With Negotiate Timeout define a timeout in seconds for negotiate checks.
In Fallback, enter the hostname or IP address of the Web server onto which to redirect a Web service in case all real servers are down.
If you want to use an alternative path for logging, specify a path for the logs in Log File. By default, ldirectord writes its logs to /var/log/ldirectord.log.
If you want the system to send alerts in case the connection status to any real server changes, enter a valid e-mail address in Email Alert.
With Email Alert Frequency, define after how many seconds the e-mail alert should be repeated if any of the real servers remains inaccessible.
In Email Alert Status specify the server states for which email alerts should be sent. If you want to define more than one state, use a comma-separated list.
With Auto Reload define, if ldirectord should continuously monitor the configuration file for modification. If set to yes, the configuration is automatically reloaded upon changes.
With the Quiescent switch, define if to remove failed real servers from the kernel's LVS table or not. If set to Yes, failed servers are not removed. Instead their weight is set to 0 which means that no new connections will be accepted. Already established connections will persist until they time out.

Figure 10.1. YaST IP Load Balancing—Global Parameters

Procedure 10.2. Configuring Virtual Services

You can configure one or more virtual services by defining a couple of parameters for each. The following procedure describes how to configure the most important parameters for a virtual service. For more details about the individual parameters (and the parameters not covered here), click Help or refer to the ldirectord man page.

In the YaST iplb module, switch to the Virtual Server Configuration tab.
Add a new virtual server or Edit an existing virtual server. A new dialog shows the available options.
In Virtual Server enter the shared virtual IP address and port under which the load balancers and the real servers are accessible as LVS. Instead of IP address and port name, you can also specify a hostname and a service. Alternatively, you can also use a firewall mark. A firewall mark is a way of aggregating an arbitrary collection of VIP:port services into one virtual service.
To specify the Real Servers, you need to enter the IP address (or hostnames) of the servers, the ports (or service names) and the forwarding method. The forwarding method must either be gate, ipip or masq, see Section 10.1.3, “Packet Forwarding”.
Click the Add button and enter the required arguments for each real server.
As Check Type, select the type of check that should be performed to test if the real servers are still alive. For example, to send a request and check if the response contains an expected string, select Negotiate.
If you have set the Check Type to Negotiate, you also need to define the type of service to monitor. Select it from the Service drop-down list.
In Request, enter the URI to the object that is requested on each real server during the check intervals.
If you want to check if the response from the real servers contains a certain string (“I'm alive” message), define a regular expression that needs to be matched. Enter the regular expression into Receive. If the response from a real server contains this expression, the real server is considered to be alive.
Depending on the type of Service you have selected in Step 6, you also need to specify further parameters like Login, Password, Database, or Secret. For more information, refer to the YaST help text or to the ldirectord man page.
Select the Scheduler to be used for load balancing. For information on the available schedulers, refer to the ipvsadm(8) man page.
Select the Protocol to be used. If the virtual service is specified as an IP address and port, it must be either tcp or udp. If the virtual service is specified as a firewall mark, the protocol must be fwm.
Define further parameters, if needed. Confirm your configuration with OK. YaST writes the configuration to /etc/ha.d/ldirectord.cf.

Figure 10.2. YaST IP Load Balancing—Virtual Services

Example 10.1. Simple ldirectord Configuration

The values shown in Figure 10.1, “YaST IP Load Balancing—Global Parameters” and Figure 10.2, “YaST IP Load Balancing—Virtual Services”, would lead to the following configuration, defined in /etc/ha.d/ldirectord.cf:

autoreload = yes 
checkinterval = 5 
checktimeout = 3 
quiescent = yes 
    virtual = 192.168.0.200:80 
    checktype = negotiate 
    fallback = 127.0.0.1:80 
    protocol = tcp 
    real = 192.168.0.110:80 gate 
    real = 192.168.0.120:80 gate 
    receive = "still alive" 
    request = "test.html" 
    scheduler = wlc 
    service = http

	Defines that `ldirectord` should continuously check the configuration file for modification.
	Interval in which `ldirectord` will connect to each of the real servers to check if they are still online.
	Time in which the real server should have responded after the last check.
	Defines not to remove failed real servers from the kernel's LVS table, but to set their weight to `0` instead.
	Virtual IP address (VIP) of the LVS. The LVS is available at port `80`.
	Type of check that should be performed to test if the real servers are still alive.
	Server onto which to redirect a Web service all real servers for this service are down.
	Protocol to be used.
	Two real servers defined, both available at port `80`. The packet forwarding method is `gate`, meaning that direct routing is used.
	Regular expression that needs to be matched in the response string from the real server.
	URI to the object that is requested on each real server during the check intervals.
	Selected scheduler to be used for load balancing.
	Type of service to monitor.

This configuration would lead to the following process flow: The ldirectord will connect to each real server once every 5 seconds and request 192.168.0.110:80/test.html or 192.168.0.120:80/test.html as specified in and . If it does not receive the expected still alive string from a real server within 3 seconds of the last check, it will remove the real server from the available pool. However, because of the quiescent=yes setting , the real server will not be removed from the LVS table, but its weight will be set to 0 so that no new connections to this real server will be accepted. Already established connections will be persistent until they time out.

Further Setup

Apart from the configuration of ldirectord with YaST, you need to make sure the following conditions are fulfilled to complete the LVS setup:

The real servers are set up correctly to provide the needed services.
The load balancing server (or servers) must be able to route traffic to the real servers using IP forwarding. The network configuration of the real servers depends on which packet forwarding method you have chosen.
To prevent the load balancing server (or servers) from becoming a single point of failure for the whole system, you need to set up one or several backups of the load balancer. In the cluster configuration, configure a primitive resource for ldirectord, so that ldirectord can fail over to other servers in case of hardware failure.
As the backup of the load balancer also needs the ldirectord configuration file to fulfill its task, make sure the /etc/ha.d/ldirectord.cf is available on all servers that you want to use as backup for the load balancer. You can synchronize the configuration file with Csync2 as described in Section 3.2.3, “Transferring the Configuration to All Nodes”.

For More Information

To learn more about Linux Virtual Server, refer to the project home page available at http://www.linuxvirtualserver.org/.

For more information about ldirectord, refer to its comprehensive man page.