One of the central tasks in computer centers and when operating servers is providing hard disk capacity for server systems. Fiber channel is often used for this purpose in the mainframe sector. So far, UNIX computers and the majority of servers are not connected to central storage solutions.
linux-iSCSI provides an easy and reasonably inexpensive solution for connecting Linux computers to central storage systems. In principle, iSCSI represents a transfer of SCSI commands on the IP level. If a program starts an inquiry for such a device, the operating system produces the necessary SCSI commands. These are then embedded in IP packages and encrypted as necessary by software that is commonly known as an iSCSI initiator. The packages are then transferred to the corresponding iSCSI remote station, also called iSCSI target.
Many storage solutions provide access over iSCSI, but it is also possible to run a Linux server that provides an iSCSI target. In this case, it is important to set up the Linux server optimized for file system services. The iSCSI target just accesses block devices in Linux. Therefore it is possible to use RAID solutions to increase disk space as well as a lot of memory to improve data caching. For more information about RAID, also see Section 7.2, “Soft RAID Configuration”.
SUSE® Linux Enterprise Server comes with an open source iSCSI target solution that evolved from the Ardis iSCSI target. A basic setup can be done with YaST, but to take full advantage of iSCSI, a manual setup is required.
The iSCSI target configuration exports existing block devices or
file system images to iSCSI initiators. First
create the needed block devices with YaST or create file system
images. For an overview of partitioning, see Section 8.5.7, “Using the YaST Partitioner”. File system images must be
created manually. For example, if you want to create the image
/var/lib/xen/images/xen-0 with the size 4GB,
first make sure that the directory is there then create the
image itself:
mkdir -p /var/lib/xen/images dd if=/dev/zero of=/var/lib/xen/images/xen-0 seek=1M bs=4096 count=1
To configure the iSCSI target, run the
module
in YaST. The configuration is split into three tabs. In the
tab, select the start mode
and the firewall settings. If you want to access the iSCSI target
from a remote machine, select . If a iSNS server should
manage the discovery and access control, activate
and enter the IP address of
your iSNS Server. Note, that you cannot use even valid hostnames,
but must use the IP address. For more about
iSNS, read Chapter 13, iSNS for Linux Overview.
The tab provides settings for the iSCSI server. The authentication set here is used for the discovery of services, not for accessing the targets. If you do not want to restrict the access to the discovery, use .
If authentication is needed, there are two possibilities to consider. One is that an initiator must prove that it has the permissions to run a discovery on the iSCSI target. This is done with . The other is that the iSCSI target must prove to the initiator that it is the expected target. Therefore, the iSCSI target can also provide a username and password. This is done with . Find more information about authentication in RFC 3720 (see http://www.ietf.org/rfc/rfc3720.txt).
The targets are defined in the tab. Use to create a new iSCSI target. The first dialog asks for information about the device to export.
The line has a fixed syntax that looks like the following:
iqn.yyyy-mm.<reversed domain name>
It always starts with iqn. yyyy-mm is the format of the date when this target is activated. Find more about naming conventions in RFC 3722 (see http://www.ietf.org/rfc/rfc3722.txt).
The is freely selectable. It should follow some scheme to make the whole system more structured.
It is possible to assign several LUNs to a target. To do this, select a target in the tab then click . There, add new LUNs to an existing target.
Add the path to the block device or file system image to export.
The next menu configures the access restrictions of the target. The configuration is very similar to the configuration of the discovery authentication. In this case, at least an incoming authentication should be setup.
finishes the configuration of the new target, and brings you back to the overview page of the tab. Activate your changes by clicking on .
Configure an iSCSI target in
/etc/ietd.conf. All parameters in this file
before the first Target declaration are global
for the file. Authentication information in this portion has a
special meaning—it is not global, but is used for the discovery of
the iSCSI target.
If you have access to an iSNS server, the first
thing to configure is telling the target about this server. Note,
that the address of the iSNS server must always
be given as IP address. A normal domain name is not sufficient.
The configuration for this functionality looks like the following:
iSNSServer 192.168.1.111 iSNSAccessControl no
This configuration makes the iSCSI target
register itself with the iSNS server, which in
turn provides the discovery for initiators. For more about
iSNS read Chapter 13, iSNS for Linux Overview.
Note that the access control for the
iSNS discovery is not supported. Just keep
iSNSAccessControl no.
All direct iSCSI authentication may be done in two directions.
The iSCSI target can require the iSCSI initiator to authenticate with the
IncomingUser, which can be added multiple times.
The iSCSI initiator may also require the iSCSI target to
authenticate. Use OutgoingUser for this. Both have the
same syntax:
IncomingUser <username> <password> OutgoingUser <username> <password>
The authentication is followed by one or several target
definitions. For each target, add a
Target section. This section
always starts with a Target identifier followed
by definitions of logical unit numbers:
Target iqn.yyyy-mm.<reversed domain name>[:identifier]
Lun 0 Path=/dev/mapper/system-v3
Lun 1 Path=/dev/hda4
Lun 2 Path=/var/lib/xen/images/xen-1,Type=fileio
In the Target line, yyyy-mm is the date when this target
is activated, and
identifier is freely selectable. Find more about naming conventions in RFC
3722 (see http://www.ietf.org/rfc/rfc3722.txt).
Three different block devices are exported in this example.
The first one is a logical volume (see also Section 7.1, “LVM Configuration”), the second is an IDE partition,
and the third is an image available in the local file system.
All these look like block devices to an iSCSI initiator.
Before activating the iSCSI target, add at least one
IncomingUser after the
Lun definitions. It does the authentication for
the use of this target.
To activate all your changes, restart the iscsitarget daemon
with rcopen-iscsi restart.
Check your configuration in the /proc
file system:
cat /proc/net/iet/volume
tid:1 name:iqn.2006-02.com.example.iserv:systems
lun:0 state:0 iotype:fileio path:/dev/mapper/system-v3
lun:1 state:0 iotype:fileio path:/dev/hda4
lun:2 state:0 iotype:fileio path:/var/lib/xen/images/xen-1
There are many more options that control the behavior of the iSCSI
target. Find them in the manual page of
ietd.conf.
Active sessions are also displayed in the
/proc file system. For each connected initiator,
an extra entry is added to
/proc/net/iet/session:
cat /proc/net/iet/session
tid:1 name:iqn.2006-02.com.example.iserv:system-v3
sid:562949957419520 initiator:iqn.2005-11.de.suse:cn=rome.example.com,01.9ff842f5645
cid:0 ip:192.168.178.42 state:active hd:none dd:none
sid:281474980708864 initiator:iqn.2006-02.de.suse:01.6f7259c88b70
cid:0 ip:192.168.178.72 state:active hd:none dd:none
When changes to the iSCSI target configuration are necessary, you
always must restart the target to activate changes that are
done in the configuration file.
Unfortunately, all active sessions are interrupted in this
process. To maintain an undisturbed operation, the changes should
be done in the main configuration file
/etc/ietd.conf, but also made manually to the
current configuration with the administration utility
ietadm.
To create a new iSCSI target with a LUN, first update your configuration file. The additional entry could be:
Target iqn.2006-02.com.example.iserv:system2
Lun 0 Path=/dev/mapper/system-swap2
IncomingUser joe secret
To set up this configuration manually, proceed as follows:
Create a new target with the command ietadm --op new --tid=2 --params Name=iqn.2006-02.com.example.iserv:system2.
Add a logical unit with ietadm --op new --tid=2 --lun=0 --params Path=/dev/mapper/system-swap2.
Set the username and password combination on this target with ietadm --op new --tid=2 --user --params=IncomingUser=joe,Password=secret.
Check the configuration with cat /proc/net/iet/volume.
It is also possible to delete active connections. First, check all active connections with the command cat /proc/net/iet/session. This may look like:
cat /proc/net/iet/session
tid:1 name:iqn.2006-03.com.example.iserv:system
sid:281474980708864 initiator:iqn.1996-04.com.example:01.82725735af5
cid:0 ip:192.168.178.72 state:active hd:none dd:none
To delete the session with the session ID 281474980708864, use the command ietadm --op delete --tid=1 --sid=281474980708864 --cid=0. Be aware that this makes the device unaccessible on the client system and processes accessing this device are likely to hang.
ietadm can also be used to change various configuration parameters. Obtain a list of the global variables with ietadm --op show --tid=1 --sid=0. The output looks like:
InitialR2T=Yes ImmediateData=Yes MaxConnections=1 MaxRecvDataSegmentLength=8192 MaxXmitDataSegmentLength=8192 MaxBurstLength=262144 FirstBurstLength=65536 DefaultTime2Wait=2 DefaultTime2Retain=20 MaxOutstandingR2T=1 DataPDUInOrder=Yes DataSequenceInOrder=Yes ErrorRecoveryLevel=0 HeaderDigest=None DataDigest=None OFMarker=No IFMarker=No OFMarkInt=Reject IFMarkInt=Reject
All of these parameters may be changed easily. For example, if you
want to change the maximum number of connections to
two, use ietadm --op update --tid=1
--params=MaxConnections=2. In the file
/etc/ietd.conf, the associated line should
look like MaxConnections 2.
![]() | Update ietd.conf According to Changes with ietadm |
|---|---|
The changes that you make with the command
ietadm are not permanent for the system. These
changes are lost at the next reboot if they are not added to the
configuration file | |
There are several more options available for the command
ietadm. Find an overview with
ietadm -h. The abbreviations there are target
ID (tid), session ID (sid), and connection ID (cid). They can also be
found in /proc/net/iet/session.
iSCSI initiator, also called client, can be used to connect to any iSCSI target. This is not restricted to the iSCSI target solution explained above. The configuration of iSCSI initiator involves two major steps—the discovery of available iSCSI targets and the setup of an iSCSI session. Both can be done with YaST.
The configuration is divided into three tabs. The
tab may be used to enable the iSCSI
initiator at boot time. It also offers to set a unique
and an iSNS
server to use for the discovery. The default port for
iSNS is 3205. The
tab gives an overview of the
currently connected iSCSI targets. Like the tab, it gives the option to add new targets to the
system. is the tab to start
with. It provides the possibility of discovering iSCSI targets in the
network.
Use to open the discovery dialog.
Enter the IP address and change the port if necessary.
If necessary, add the or authentication.
Use to start the discovery.
After a successful discovery, use to activate the target. You will be asked for authentication information to use the selected iSCSI target. finishes the configuration. If everything went well, the target now appears in .
The virtual iSCSI device is now available. Find the actual device with lsscsi:
lsscsi [1:0:0:0] disk IET VIRTUAL-DISK 0 /dev/sda
Both the discovery and the configuration of iSCSI connections
require a running iscsid. When running the discovery the first
time, the internal database of the iSCSI initiator is created in the
directory /var/lib/open-iscsi.
If your discovery is password protected, provide the
authentication information to iscsid. Because the internal
database does not exist when doing the first discovery, it cannot
be used at this time. Instead, the configuration file
/etc/iscsid.conf must be edited to provide
the information. To add your password information for the
discovery, add the following lines to the end of
/etc/iscsid.conf:
discovery.sendtargets.auth.authmethod = CHAP discovery.sendtargets.auth.username = <username> discovery.sendtargets.auth.password = <password>
The discovery stores all received values in an internal persistent
database. In addition, it displays all detected targets. Run this
discovery with the command iscsiadm -m discovery
--type=st --portal=<targetip>. The output
should look like:
149.44.171.99:3260,1 iqn.2006-02.com.example.iserv:systems
To discover the available targets on a iSNS
server, use the command iscsiadm --mode discovery --type
isns --portal <targetip>
For each target defined on the iSCSI target, one line appears. Learn how to obtain more information about the stored data in Section 12.2.4, “The iSCSI Client Databases”.
The special --login option of
iscsiadm creates all needed devices:
iscsiadm -m node -n iqn.2006-02.com.example.iserv:systems --login
The newly generated devices show up in the output of lsscsi and can now be accessed by mount.
LVM startup is supported by udev, so that any LVM volume groups will be activated automatically via udev once all required physical volumes have been detected.
LVM autoassembly in udev makes use of the udev helper program
collect. This program takes as the first argument an
abstract ID to be checked, followed by a list of
component IDs. Once this program has been called with each of the
component IDs as the first argument it'll return 0.
So for autoassembly the physical volume UUIDs for the given volume group is registered as argument list for collect. udev (or rather vol_id) is capable of detecting the physical volume UUID on a device and hence it can be passed as the first argument to collect.
Once collect has been called with all physical volume
UUIDs (i.e. udev has received events for all component devices) the next
rule triggers which just calls vgchange -a y
<vgname> and the volume group will be
activated.
Use the script
/usr/share/doc/packages/lvm2/lvm-vg-to-udev-rules.sh.
It takes as argument the volume group you want to start automatically.
This script will generate the required udev rules. Now restart iSCSI to
activate the volume groups. If you want to have the array started
automatically on boot you have to switch the iSCSI component devices to
automatic, so that the initiator will log into the
target automatically on boot.
All information that was discovered by the iSCSI initiator is stored
in two database files that reside in
/var/lib/open-iscsi. There is one database for
the discovery of targets and one for the discovered nodes.
When accessing a database, you first must select if you want to
get your data from the discovery or from the node database. Do this with the
-m discovery and -m node parameters of
iscsiadm. Using
iscsiadm just with one of these parameters gives
an overview of the stored records:
iscsiadm -m discovery 149.44.171.99:3260,1 iqn.2006-02.com.example.iserv:systems
The target name in this example is
iqn.2006-02.com.example.iserv:systems. This name
is needed for all actions that relate to this special data set. To
examine the content of the data record with the ID
iqn.2006-02.com.example.iserv:systems, use the following command:
iscsiadm -m node --targetname iqn.2006-02.com.example.iserv:systems node.name = iqn.2006-02.com.example.iserv:systems node.transport_name = tcp node.tpgt = 1 node.active_conn = 1 node.startup = manual node.session.initial_cmdsn = 0 node.session.reopen_max = 32 node.session.auth.authmethod = CHAP node.session.auth.username = joe node.session.auth.password = ******** node.session.auth.username_in = <empty> node.session.auth.password_in = <empty> node.session.timeo.replacement_timeout = 0 node.session.err_timeo.abort_timeout = 10 node.session.err_timeo.reset_timeout = 30 node.session.iscsi.InitialR2T = No node.session.iscsi.ImmediateData = Yes ....
To edit the value of one of these variables, use the
command iscsiadm with the update
operation. For example, if you
want iscsid to log in to the iSCSI target when it initializes,
set the variable node.startup to the value
automatic:
iscsiadm -m node -n iqn.2006-02.com.example.iserv:systems --op=update --name=node.startup --value=automatic
Remove obsolete data sets with the operation delete. If
the target iqn.2006-02.com.example.iserv:systems is no longer a valid record, delete
this record with the command iscsiadm
-m node -n iqn.2006-02.com.example.iserv:systems --op=delete.
Use this option with caution because it deletes the record without any additional
confirmation prompt.
To get a list of all discovered targets, run the command iscsiadm -m node.
The iSCSI protocol has been available for several years. There are many reviews and additional documentation comparing iSCSI with SAN solutions, doing performance benchmarks, or just describing hardware solutions. Important pages for more information about open-iscsi are:
http://www.open-iscsi.org/
http://www.open-iscsi.org/cgi-bin/wiki.pl
http://www.novell.com/coolsolutions/appnote/15394.html
There is also some online documentation available. See the
manual pages of iscsiadm, iscsid,
ietd.conf, and ietd and the example
configuration file /etc/iscsid.conf.