SUSE Linux Enterprise Server 11 SP4: Chapter 10. Configuring Virtual Machines

Chapter 10. Configuring Virtual Machines¶

Contents

10.1. Enabling Seamless and Synchronized Cursor Movement
10.2. Adding a CD/DVD-ROM Device with Virtual Machine Manager
10.3. Adding a Floppy Device with Virtual Machine Manager
10.4. Ejecting and Changing Floppy or CD/DVD-ROM Media with Virtual Machine Manager
10.5. Adding a PCI Device with Virtual Machine Manager
10.6. Adding a PCI Device with virsh
10.7. Adding SR-IOV Devices
10.8. Clock Settings

Abstract

Virtual Machine Manager's Details view offers in-depth information about the VM Guest's complete configuration and hardware equipment. Using this view, you can also change the guest configuration or add and modify virtual hardware. To access this view, open the guests console in Virtual Machine Manager and either choose View+Details from the menu, or click the blue information icon in the toolbar.

Figure 10.1. Details View of a VM Guest

10.1. Enabling Seamless and Synchronized Cursor Movement¶

When you click within a VM Guest's console with the mouse, the cursor is captured by the console window and cannot be used outside the console unless it is explicitly released (by pressing Alt+Ctrl ). To prevent the console from grabbing the key and to enable seamless cursor movement between host and guest instead, add a tablet to the VM Guest.

Adding a tablet has the additional advantage of synchronizing the cursor movement between VM Host Server and VM Guest when using a graphical environment on the guest. With no tablet configured on the guest, you will often see two cursor symbols with one dragging behind the other.

Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the Details view with View+Details.
Click Add Hardware and choose Input and then EvTouch USB Graphics Tablet in the pop-up window. Proceed with Finish.
If you try to add the tablet while the guest is still running, you will be asked whether to enable the tablet after the next reboot. Confirm with Yes.
Once you (re)start the VM Guest, the tablet is available in the VM Guest.

10.2. Adding a CD/DVD-ROM Device with Virtual Machine Manager¶

KVM supports CD or DVD-ROMs in VM Guest either by directly accessing a physical drive on the VM Host Server or by accessing ISO images. To create an ISO image from an existing CD or DVD, use dd:

dd if=/dev/cd_dvd_device of=my_distro.iso bs=2048

To add a CD/DVD-ROM device to your VM Guest, proceed as follows:

Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the Details view with View+Details.
Click Add Hardware and choose Storage in the pop-up window. Proceed with Forward.
Change the Device Type to IDE CDROM.
Select Select Managed or Other Existing Storage.
1. To assign the device to a physical medium, enter the path to the VM Host Server's CD/DVD-ROM device (for example, /dev/cdrom) next to the Browse button. Alternatively you may use the Browse button to open a file browser and then click Browse Local to select the device. Assigning the device to a physical medium is only possible when the Virtual Machine Manager was started on the VM Host Server.
2. To assign the device to an existing image, click Browse to choose an image from a storage pool. If the Virtual Machine Manager was started on the VM Host Server, you may alternatively choose an image from another location on the file system by clicking Browse Local. Select an image and close the file browser with Choose Volume.
Proceed with Forward to review the settings. Apply them with Finish, Yes, and Apply.
Reboot the VM Guest to make the new device available. For further information also see Section 10.4, “Ejecting and Changing Floppy or CD/DVD-ROM Media with Virtual Machine Manager”.

10.3. Adding a Floppy Device with Virtual Machine Manager¶

Currently KVM only supports the use of floppy disk images—using a physical floppy drive is not supported. Create a floppy disk image from an existing floppy using dd:

dd if=/dev/fd0 of=/var/lib/libvirt/images/floppy.img

To create an empty floppy disk image use one of the following commands:

Raw Image

dd if=/dev/zero of=/var/lib/libvirt/images/floppy.img bs=512 count=2880

FAT Formatted Image

mkfs.msdos -C /var/lib/libvirt/images/floppy.img 1440

To add a floppy device to your VM Guest proceed as follows:

Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the Details view with View+Details.
Click Add Hardware and choose Storage in the pop-up window. Proceed with Forward.
Change the Device Type to Floppy Disk.
Choose Select Managed or Other Existing Storage and click Browse to choose an existing image from a storage pool. If Virtual Machine Manager was started on the VM Host Server, you may alternatively choose an image from another location on the file system by clicking Browse Local. Select an image and close the file browser with Choose Volume.
Proceed with Forward to review the settings. Apply them with Finish, Yes, and Apply.
Reboot the VM Guest to make the new device available. For further information also see Section 10.4, “Ejecting and Changing Floppy or CD/DVD-ROM Media with Virtual Machine Manager”.

10.4. Ejecting and Changing Floppy or CD/DVD-ROM Media with Virtual Machine Manager¶

Regardless of whether you are using the VM Host Server's physical CD/DVD-ROM device or an ISO/floppy image, before you can change the media or image of an existing device in the VM Guest, you first need to disconnect the media from the guest.

Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the Details view with View+Details.
Choose the Floppy or CD/DVD-ROM device and “eject” the media by clicking Disconnect.
To “insert” a new media, click Connect.
1. If using the VM Host Server's physical CD/DVD-ROM device, first change the media in the device (this may require unmounting it on the VM Host Server before it can be ejected). Then choose CD-ROM or DVD and select the device from the drop-down list.
2. If using an ISO image, choose ISO image Location and select an image by clicking Browse. When connecting from a remote host, you may only choose images from existing storage pools.
Click OK to finish. The new media can now be accessed in the VM Guest.

10.5. Adding a PCI Device with Virtual Machine Manager¶

You can add PCI devices to guests using the graphical virt-manager tool. Once the PCI device is assigned to one VM Guest, it cannot be used by another one unless re-assigned. The following procedure adds a USB controller to a virtualized guest.

Double-click a VM Guest entry in the Virtual Machine Manager to open its console and switch to the Details view with View+Details.
Click Add Hardware and choose the PCI Host Device category in the left pane. A list of host PCI devices appears in the right part of the window.

From the list of available PCI devices, choose the USB controller to assign to the VM Guest. It may read for example USB2 Enhanced Host Controller. Confirm with Finish. Finish.


You cannot assign a PCI device live to a running VM Guest. If you are trying to add a PCI device to a running machine, the Virtual Machine Manager will ask if you want to assign the PCI device after the next VM Guest shutdown. Confirm with Yes, and after you reboot the VM Guest, the device assignment will be finished.

10.6. Adding a PCI Device with virsh¶

To dedicate and assign a PCI device to VM Guest with virsh, follow these steps:

Identify the PCI device.
Use the virsh nodedev-list or lspci -n commands to identify the PCI device designated for pass-through to VM Guest.
The following command lists available PCI devices only:
```
virsh nodedev-list | grep pci
```
Note that PCI devices are identified by a string in the following format (8086 is a variable that represents Intel architecture, and **** stands for a four-digit hexadecimal code specific to each device):
```
pci_8086_****
```
Remember the PCI device number—you will need it in future steps.

Gather the information about the domain, bus, and function:

# virsh nodedev-dumpxml pci_8086_1d26
<device>
  <name>pci_8086_1d26</name>
  <parent>computer</parent>
  <driver>
    <name>ehci_hcd</name>
  </driver>
  <capability type='pci'>
    <domain>0</domain>
    <bus>0</bus>
    <slot>29</slot>
    <function>0</function>
    <product id='0x1d26'>Patsburg USB2 Enhanced Host Controller #1</product>
    <vendor id='0x8086'>Intel Corporation</vendor>
    <capability type='virt_functions'>
    </capability>
  </capability>
</device>

Detach the device from the host system prior to attaching it to VM Guest.
```
# virsh nodedev-detach pci_8086_1d26 
  Device pci_8086_1d26 detached
```
Convert the bus, slot, and function value from decimal to hexadecimal, and prepend '0x' to tell the system that the value is hexadecimal. In our example, bus = 0, slot = 29, and function = 0. Their hexadecimal values are:
```
# printf %x 0
0
# printf %x 29
1d
```
Bus and function hexadecimal numbers are '0x00', while slot number is '0x1d'.

Run virsh edit on your domain, and add the following device entry in the <devices> section.

<hostdev mode='subsystem' type='pci' managed='no'>
  <source>
    <address domain='0x0000' bus='0x00' slot='0x1d' function='0x00'/>
  </source>
</hostdev>

'managed' vs. 'unmanaged'

	'managed' vs. 'unmanaged'
`libvirt` recognizes two modes for handling PCI devices: they can be either 'managed' or 'unmanaged'. In the managed case, `libvirt` will handle all the details of unbinding the device from the existing driver if needed, resetting the device, binding it to `pci-stub` before starting the domain, etc. When the domain is terminated or the device is removed from the domain, `libvirt` will unbind from `pci-stub` and rebind to the original driver in the case of a managed device. If the device is unmanaged, the user must take care to ensure all of these management aspects of the device are done before assigning it to a domain, and after the device is no longer used by the domain. In our example, the `managed='no'` option means that the device is 'unmanaged', and we need to take care of the related driver with the virsh nodedev-detach and virsh nodedev-attach commands. To switch the device mode to 'managed', replace the snippet with `managed='yes'`, and skip the remaining steps (apart from starting the guest).

libvirt recognizes two modes for handling PCI devices: they can be either 'managed' or 'unmanaged'. In the managed case, libvirt will handle all the details of unbinding the device from the existing driver if needed, resetting the device, binding it to pci-stub before starting the domain, etc. When the domain is terminated or the device is removed from the domain, libvirt will unbind from pci-stub and rebind to the original driver in the case of a managed device. If the device is unmanaged, the user must take care to ensure all of these management aspects of the device are done before assigning it to a domain, and after the device is no longer used by the domain.

In our example, the managed='no' option means that the device is 'unmanaged', and we need to take care of the related driver with the virsh nodedev-detach and virsh nodedev-attach commands. To switch the device mode to 'managed', replace the snippet with managed='yes', and skip the remaining steps (apart from starting the guest).

Once the VM Guest system is ready to use the PCI device, tell the host to stop using it. First check what driver the host system is using for the PCI device.
```
# readlink /sys/bus/pci/devices/0000\:00\:1d.0/driver
../../../bus/pci/drivers/pci-stub
```

In our case, the pci-stub driver is loaded, so you can start the virtual machine. It will be able to use the PCI device automatically.

# virsh start sles11

When using a multi-function PCI device that does not support FLR (function level reset) or PM (power management) reset, you need to detach all its functions from the VM Host Server. The device must be reset for security reasons, and without FLR or PM reset, you must reset the whole device. libvirt will refuse to do this if a function of the device is still in use by the VM Host Server or another VM Guest.

You can safely detach a device function from the VM Guest with the virsh nodedev-detach command.

If your PCI device is not 'managed', and the driver controlling the PCI device is not pci-stub, you have to detach it from the device first:

virsh nodedev-detach pci_8086_1d26

If you are running SELinux on your host, you need to disable it for now with

# setsebool -P virt_use_sysfs 1

and then start the virtual machine.

10.7. Adding SR-IOV Devices¶

Single Root I/O Virtualization (SR-IOV) capable PCIe devices are able to replicate their resources, so they appear to be multiple devices. Each of these "pseudo-devices" can be assigned to a VM Guest.

SR-IOV is an industry specification that was created by the Peripheral Component Interconnect Special Interest Group (PCI-SIG) consortium. It introduces physical functions (PF) and virtual functions (VF). PFs are full PCIe functions used to manage and configure the device. PFs also have the ability to move data. VFs lack the configuration and management part—they only have the ability to move data and a reduced set of configuration functions. Since VFs do not have all PCIe functions, the host operating system or the hypervisor must support SR-IOV in order to be able to access and initialize VFs. The theoretical maximum for VFs is 256 per device (consequently the maximum for a dual port Ethernet card would be 512). In practice this maximum is much lower, since each VF consumes resources.

10.7.1. Requirements¶

The following requirements must be met in order to be able to use SR-IOV:

An SR-IOV-capable network card (as of SUSE Linux Enterprise Server 11 SP4 only network cards support SR-IOV)
.n x86_64 host supporting hardware virtualization (AMD-V or Intel VT-x), see Section 1.1, “Hardware Requirements” for more information
a chipset that supports device assignment (AMD-Vi or Intel VT-d)
libvirt-0.9.10 or better
SR-IOV drivers must be loaded and configured on the host system
iommu enabled in the hypervisor (e.g. intel_iommu=on on the linux commandline of the host)
a list of the PCI addresses of the VF(s) that will be assigned to VM Guests

	Checking if a Device is SR-IOV Capable
The information whether a device is SR-IOV capable can be obtained from it's PCI descriptor by running lspci. A device that supports SR-IOV reports a capability similar to the following: Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)

Checking if a Device is SR-IOV Capable

The information whether a device is SR-IOV capable can be obtained from it's PCI descriptor by running lspci. A device that supports SR-IOV reports a capability similar to the following:

Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)

10.7.2. Load and configure the SR-IOV Host drivers¶

In order to be able to access and initialize VFs, an SR-IOV capable driver has to be loaded on the host system.

Before loading the driver, make sure the card is properly detected by running lspci. The following example shows the lspci output for the dual port Intel 82576NS network card:

~ > sudo /sbin/lspci | grep 82576
01:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
04:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
04:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)

In case the card is not detected, it is likely that the hardware virtualization support in the BIOS/EFI has not been enabled.

Check whether the SR-IOV driver is already loaded by running lsmod. In the following example a check for the igb driver (for the Intel 82576NS network card) returns a result. That means the driver is already loaded. If the command returns nothing, the driver is not loaded.
```
~ > sudo /sbin/lsmod | egrep "^igb "
igb                   185649  0
```
Skip this step if the driver is already loaded.
If the SR-IOV driver is not yet loaded, the non-SR-IOV driver needs to be removed first, before loading the new driver. Use rmmod to unload a driver. The following examples unload the non-SR-IOV driver for the Intel 82576NS network card:
```
sudo /sbin/rmmod igbvf
```
Load the SR-IOV driver subsequently using the modprobe command:
```
sudo /sbin/modprobe igb
```
Configure the driver by adding the number of VFs you would like to make available and—if necessary— by blacklisting the non-SR-IOV driver:
```
sudo echo -e "options igb max_vfs=8\nblacklist igbvf" >> /etc/modprobe.d/50-igb/
```
Make sure to replace the example values igb, igbvf and 50-igb by values appropriate for your driver.
Now make sure the driver is loaded on boot. Edit /etc/sysconfig/kernel and add the driver to MODULES_LOADED_ON_BOOT:
```
MODULES_LOADED_ON_BOOT="igb"
```
Make sure to replace the example value igb by a value appropriate for your driver.

Reboot the machine and check if the SR-IOV driver is loaded by re-running the lspci command from the first step of this procedure. If the SR-IOV driver was loaded successfully you should see additional lines for the VFs:

01:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
01:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
01:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
01:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
[...]
04:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
04:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
04:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
04:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
04:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
[...]

10.7.3. Adding a VF network device to an Existing VM Guest¶

Once the SR-IOV hardware is properly set up on the VM Host Server, you can add VFs to VM Guests. In order to do so, you need to collect some data first.

Note: The following procedure is using example data. Make sure to replace it by appropriate data from your setup.

Use the virsh nodedev-list command to get the PCI address of the VF you you want to assign and it's corresponding PF. Numerical values from the lspci output shown in Section 10.7.2, “Load and configure the SR-IOV Host drivers” (for example 01:00.0 or 04:00.1) are transformed by adding the prefix "pci_0000_" and by replacing colons and dots with underscores. So a PCI ID listed as "04:00.0" by lspci is listed as "pci_0000_04_00_0" by virsh. The following example lists the PCI IDs for the second port of the Intel 82576NS network card:
```
~ > virsh nodedev-list | grep 0000_04_
pci_0000_04_00_0
pci_0000_04_00_1
pci_0000_04_10_0
pci_0000_04_10_1
pci_0000_04_10_2
pci_0000_04_10_3
pci_0000_04_10_4
pci_0000_04_10_5
pci_0000_04_10_6
pci_0000_04_10_7
pci_0000_04_11_0
pci_0000_04_11_1
pci_0000_04_11_2
pci_0000_04_11_3
pci_0000_04_11_4
pci_0000_04_11_5
```
The first two entries represent the PFs whereas the other entries represent the VFs.

Get more data that will be needed by running the command virsh nodedev-dumpxml on the PCI ID of the VF you want to add:

~ > virsh nodedev-dumpxml pci_0000_04_10_0
<device>
  <name>pci_0000_04_10_0</name>
  <parent>pci_0000_00_02_0</parent>
  <capability type='pci'>
    <domain>0</domain>
    <bus>4</bus>
    <slot>16</slot>
    <function>0</function>
    <product id='0x10ca'>82576 Virtual Function</product>
    <vendor id='0x8086'>Intel Corporation</vendor>
    <capability type='phys_function'>
      <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </capability>
  </capability>
</device>

The following data is needed for the next step:

<domain>0</domain>
<bus>4</bus>
<slot>16</slot>
<function>0</function>

Create a temporary XML file (for example /tmp/vf-interface.xml containing the data necessary to add a VF network device to an existing VM Guest. The minimal content of the file needs to look like the following:

<interface type='hostdev'>
 <source>
  <address type='pci' domain='0' bus='11' slot='16' function='0'2/>
 </source>
</interface>

VFs do not get a fixed MAC address, it changes everytime the host reboots. When adding network devices the “traditional” way with <hostdev>, it would require to reconfigure the VM Guest's network device after each reboot of the host, because of the MAC address change. To avoid this kind of problems, libvirt introduced the “interface type='hostdev'” directive, which sets up network specific data before assigning the device.

Specify the data you acquired in the previous step here.

In case a device is already attached to the host, it cannot be attached to a guest. To make it available for guests, detach it from the host first:
```
virsh nodedev-detach pci_0000_04_10_0
```
Last, add the VF interface to an existing VM Guest:
```
virsh attach-device GUEST /tmp/vf-interface.xml --OPTION
```
GUEST needs to be replace by the domain name, id or uuid of the VM Guest and --OPTION can be one of the following:
--persistent
This option will always add the device to the domain's persistent XML. In addition, if the domain is running, it will be hotplugged.
--config
This option will only affect the persistent XML, even if the domain is running., The device will only show up in the guest on next boot
--live
This option will only affect a running domain. If the domain is inactive, the operation will fail. The device is not persisted in the XML and won't be available in the guest on next boot.
--current
This option affects the current state of the domain. If the domain is inactive, the device is added to the persistent XML and will be available on next boot. If the domain is active, the device is hotplugged but not added to the persistent XML.
To detach a VF interface, use the virsh detach-device command which also takes the options listed above.

10.8. Clock Settings¶

Keeping the correct time in a VM Guest is one of the more difficult aspects of virtualization. Keeping the correct time is especially important for network applications and is also a prerequisite to do a live migration of a VM Guest.

	Time Keeping on the VM Host Server
It is strongly recommended to ensure the VM Host Server keeps the correct time as well, for example, by utilizing NTP (see Chapter 24, Time Synchronization with NTP (↑Administration Guide) for more information).

10.8.1. Using `kvm_clock`¶

KVM provides a paravirtualized clock which is currently supported by SUSE Linux Enterprise Server 10 SP3 and newer and RedHat Enterprise Linux 5.4 and newer via the kvm_clock driver. It is strongly recommended to use kvm_clock when available.

Use the following command inside a VM Guest running Linux to check whether the driver kvm_clock has been loaded:

~ # dmesg | grep kvm-clock
[    0.000000] kvm-clock: cpu 0, msr 0:7d3a81, boot clock
[    0.000000] kvm-clock: cpu 0, msr 0:1206a81, primary cpu clock
[    0.012000] kvm-clock: cpu 1, msr 0:1306a81, secondary cpu clock
[    0.160082] Switching to clocksource kvm-clock

To check which clock source is currently used, run the following command in the VM Guest. It should output kvm-clock:

cat /sys/devices/system/clocksource/clocksource0/current_clocksource

	`kvm-clock` and NTP
When using `kvm-clock`, it is not recommended to use NTP in the VM Guest, as well. Using NTP on the VM Host Server, however, is still recommended.

10.8.2. Other Time Keeping Methods¶

The paravirtualized kvm-clock is currently not available for SUSE Linux Enterprise Server 9 and Windows operating systems. For Windows, use the Windows Time Service Tools for time synchronization (see http://technet.microsoft.com/en-us/library/cc773263%28WS.10%29.aspx for more information).

Correct time keeping in SUSE Linux Enterprise Server 9 SP4 can be achieved by using special boot parameters:

32-bit Kernel:clock=pmtmr

64-bit Kernel:ignore_lost_ticks