14-Oct-2003			    iscsi_initiator							RPM 1.4.01
						---------------

	vi: set autoindent tabstop=4 shiftwidth=4 :

Note:	this document is written in ordinary ASCII text with tabs set
		to a width of 4.

This directory contains the C source code for the UNH initiator
reference implementation.  It depends on code contained in
../common and ../security that is shared by both the reference
initiator and the reference target.

The current code supports draft 20 of the proposed iSCSI standard.

The current code supports the following features of the standard:
	-	header and data digests
	-	CHAP and SRP authentication methods
	-	multiple simultaneous sessions
	-	multiple LUNs per session
	-	multiple connections per session
	-	normal and discovery sessions
	-	redirection on login
	-	all combinations of InitialR2T and ImmediateData keys
	-	arbitrary (legal) values for MaxRecvDataSegmentLength,
		MaxBurstLength and FirstBurstLength
	-	arbitrary number of outstanding R2Ts
	-	data and data-sequences in any order
	-	error recovery levels 0, 1.
	-	optional periodic nop "ping" to/from target
	-	all PDU types, including SNACK
	-   ipV6 addressing 
	-   Initial SNMP iSCSI MIB statistics.

We do not yet support:
	-	markers
	-	bidirectional commands
	-	Additional Header Segments (AHS)

	
The following sequence of steps details the process necessary to
use this module.  You must run as root (super user) to perform
steps 1 through 5 inclusive, and steps 7 through 9 inclusive
(i.e., an ordinary user can only perform step 6, which is to
use a target device after it has been set up).


1. Making the loadable unh_iscsi_initiator module
-------------------------------------------------

Doing a "make" in the directory above this will create a module
called unh_iscsi_initiator.o which must be dynamically loaded
into the Linux kernel using /sbin/insmod:

		/sbin/insmod unh_iscsi_initiator.o

(See also the file ../cmd/ini-install)
This module has been tested for various kernel versions up through 2.4.20.
The Makefile expects that the path "/lib/modules/`uname -r`/build" contains
the source for the running kernel (where `uname -r` produces a string
such as "2.4.20-18.9").


2. Loading the unh_iscsi_initiator module
-------------------------------------

While running as part of the kernel, the unh_iscsi_initiator writes
information to the file /var/log/messages.  The best way to see what
the unh_iscsi_initiator is doing is to open a separate window and in
that window type:
	tail -f /var/log/messages
which will "hang" on the /var/log/messages file and will write to
the screen each new line added to the end of that file.

As part of loading the module, unh_iscsi_initiator registers itself
with the Linux SCSI subsystem under the name "iscsi_initiator" as a
Host Bus Adapter (HBA) with 1 device.  The Linux SCSI subsystem will
assign a "host" number to this HBA at this time.
Be careful -- this host number is simply a sequential counter
maintained by the Linux SCSI subsystem and the value assigned to the
unh_iscsi_initiator HBA depends on the number of other SCSI HBAs
registered with the Linux SCSI subsystem prior to the time the
unh_iscsi_initiator module is loaded.  If your system has no other SCSI
HBAs, then the unh_iscsi_initiator will be assigned host number 0.  If
you have a SCSI disk adaptor, then that adaptor will be host 0 and the
unh_iscsi_initiator will be number 1. etc.

Next, the SCSI subsystem attempts to probe the target devices attached to
the unh_iscsi_initiator HBA, but since this is entirely a software entity,
there are currently no devices, so a number of
	Target n not logged in
messages are written to /var/log/messages.  The SCSI subsystem
attempts no further action on any of these targets until the
user takes the actions described next.


3. Configuring the unh_iscsi_initiator module
---------------------------------------------

As part of loading the unh_iscsi_initiator module, an internal table
of parameter keys is initialized to the default values.  These can be
changed (configured) using the program ../cmd/iscsi_manage.
In addition, this program can also be used to force certain behavior
from the unh_iscsi_initiator.  (See the file ../cmd/ini-manage which
contains lots of examples of this.)

For example, to set the value of a parameter key when the
unh_iscsi_initiator has been assigned host number 1, type the following
shell command after the unh_iscsi_initiator module has been loaded:

	../cmd/iscsi_manage init set InitialR2T=No host=1

When the login process is started (see section 4), all keys set in
this manner will be offered to the target as part of the login
negotiation process.  Note that the first command line parameter
to this program is "init" to indicate "initiator" because the same
iscsi_manage program is also used to configure the reference target,
in which case the first command line parameter would be "target".


This command can be repeated any number of times with different keys.

In order to have a valid login to a normal session, at least the
TargetName and InitiatorName keys should be set (although for testing
purposes, these can be omitted).  If no keys are set by this
configuration tool, then no keys will be offered to the target during
the login phase.  All keys defined by the draft standard can be
configured in this way.

A second form of configuration is to use "restore" instead of "set",
and to omit any "key=value" after the "restore".  This restores
all parameters values and forced values (see below) to their
default values.  When writing shell scripts containing manage
commands, the first command should always be:

	../cmd/iscsi_manage init restore host=1

so that scripts will always restore the configuration to a known
default before making changes.

A third form of configuration is to use "force" instead of "set"
as the second command line parameter to the iscsi_manage program.
In this case, the parameter following the "force" will have the
form "option" or "option=value", depending on the "force" option.
For example, giving the shell command:

	../cmd/iscsi_manage init force s host=1

		forces the iscsi_initiator to start in the security
		parameter negotiation stage, even if no security parameter
		keys have been configured to be sent to the target.

Other possible "force" configuration options are:

	../cmd/iscsi_manage init force o host=1

		forces the iscsi_initiator to offer the operational
		parameter negotiation stage, even if no operational parameter
		keys have been configured to be sent to the target.

	../cmd/iscsi_manage init force r host=1

		forces the iscsi_initiator to reply to all boolean key
		offers, even in the 2 special cases when replies to boolean
		offers can be omitted (when the offer to a boolean AND function
		is No, and when the offer to a boolean OR function is Yes.)
		Without this force, the iscsi_initiator will omit these optional
		replies.

	../cmd/iscsi_manage init force f host=1

		forces the iscsi_initiator to use the "flat space
		addressing method", as described in SAM-2 section 4.12.6, when
		packing an ordinary LUN number into the 8-byte LUN structure
		described in SAM-2 section 4.12.3.  Without this force, the
		iscsi_initiator uses the "peripheral device addressing method",
		as described in SAM-2 section 4.12.5, by default.

	../cmd/iscsi_manage init force n=20 host=1

		forces the iscsi_initiator to send NopOut PDUs to the
		target if there has been no activity on a connection for 20
		seconds or more.

	../cmd/iscsi_manage init force p=10 host=1

		forces the iscsi_initiator to use a retransmission wait period
		of 10 seconds during error recovery.

	../cmd/iscsi_manage init force sch=1 host=1

		forces the iscsi_initiator to use one of the following scheduling
		algorithms for sending commands over multiple connections:
				sch=0	all commands send on first connection -- the default
				sch=1	distribute commands over all connections in a
						round-robin fashion
				sch=2	map each LUN onto exactly one connection
						(not implemented yet)

	../cmd/iscsi_manage init force t host=1

		forces the iscsi_initiator to use target confirmation in a
		CHAP or SRP authentication exchange.  The default is not to
		use target confirmation after an initiator successfully
		responds to a target challenge.

	../cmd/iscsi_manage init force b host=1

		forces the iscsi_initiator to use base64 format when sending
		a large-binary-value in CHAP or SRP authentication keys.  The
		default is hex format.

	../cmd/iscsi_manage init force px="peer secret string" host=1

		forces the iscsi_initiator to use the "peer secret string"
		as the secret when generating the CHAP_R response to send
		back to the target in response to a challenge from the
		target.  There is no default value, so a "px" value must be
		forced if CHAP authentication is to be used.

	../cmd/iscsi_manage init force pn="peer name string" host=1

		forces the iscsi_initiator to use the "peer name string" as
		the CHAP_N value to send back to the target in response to a
		challenge from the target.  There is no default value, so a
		"pn" value must be forced if CHAP authentication is to be used.

	../cmd/iscsi_manage init force lx="local secret string" host=1

		forces the iscsi_initiator to use the "local secret string"
		as the secret when checking CHAP_R responses sent from the
		target in response to a confirmation challenge to the target.
		There is no default value, so an "lx" value must be forced if
		CHAP authentication is to be used and the "t" option has also
		been forced.

	../cmd/iscsi_manage init force ln="local name string" host=1

		forces the iscsi_initiator to use the "local name string"
		to check the CHAP_N value sent from the target in response
		to a confirmation challenge to the target.  There is no default
		value, so an "ln" value must be forced if CHAP authentication
		is to be used and the "t" option has also been forced.

	../cmd/iscsi_manage init force cl=256 host=1

		forces the iscsi-initiator to use 256 as the length in bytes
		of the large-binary-value generated as the value of CHAP_C
		when sending a challenge to the target.  The maximum allowed
		value after the "=" is 1024.

	../cmd/iscsi_manage init force sx="SRP secret string" host=1

		forces the iscsi_initiator to use the "SRP secret string" as
		the secret when generating and checking SRP challenges sent
		to/from the target.  There is no default value, so an "sx"
		value must be forced if SRP authentication is to be used.

	../cmd/iscsi_manage init force sn="SRP name string" host=1

		forces the iscsi_initiator to use the "SRP name string" as the
		value of the SRP_U key sent to the target.  There is no
		default value, so an "sn" value must be forced if SRP
		authentication is to be used.

	../cmd/iscsi_manage init force sg=SRP_1536 host=1

		forces the iscsi_initiator to use group SRP_1536 during
		an SRP authentication.  The allowed values after the "="
		are: SRP-768, SRP-1024, SRP-1280, SRP-1536, and SRP-2048.
		The default is SRP_1536 if no "sg" value is forced.

Typically the collection of iscsi_manage commands necessary to configure
the unh_iscsi_initiator module correctly for a certain target are written
in a shell command file that is made executable so that just typing
the name of that single file will run all the necessary iscsi_manage
commands in sequence.  This file should probably also end with the command
described in section 4 to active the module once it has been configured.
A example file called "ini-manage" is included in the "cmd" directory
of the distribution.


4. Opening connections to target devices
----------------------------------------

After the unh_iscsi_initiator module has been loaded and configured,
it must be explicitly directed to make connections to target devices.
This is done using a second program, called iscsi_config.  In order to
use this, the DNS name or IP address of the target must be known.  In
addition, if the target is expecting connections on any port other than
the WKP (3260), this must also be known in advance.  (See section 10
below for a complete discussion of the iscsi_config utility.)  The
shell level command that causes the initiator to attempt to connect to
the target at IP address 123.45.67.089 and WKP 3260 is:

	iscsi_config up ip=123.45.67.089 lun=0 host=1
 or
	iscsi_config up ip=123.45.67.089 port=3260 lun=0 host=1

This will cause iscsi_initiator to create a new session and a new
connection to that target, and if that succeeds, to then start
the login phase negotiations by sending a login PDU to the target.
If security parameter keys have been configured, or if the security
negotiation stage has been forced during the configuration, this
first login will be in the security negotiation stage (CSG=0);
otherwise it will be in the operational negotiation stage (CSG=1).

The login phase will then proceed, using as many exchanges of login
and login reply PDUs as necessary until either the login fails or
the iscsi_initiator receives a valid login response with a transition
to the Full Feature Phase (i.e., T=1 and NSG=3).

On a successful transition to Full Feature Phase, the iscsi_initiator
will enable any digests negotiated during the Login Phase, and then
the SCSI subsystem will be informed that there is now a device
available on the unh_iscsi_initiator HBA.  The SCSI subsystem will then
give the iscsi_initiator an INQUIRY command, which will be sent to the
target and the reply given back to the SCSI subsystem, which will then
report the information it has received by writing appropriate messages
to the /var/log/messages file.  It is via the INQUIRY command that the
SCSI subsystem finds out the type of device represented by the target
(i.e., direct access (disk), sequential access (tape), etc.).

The actions that follow depend on the type of device offered by the
target.  If that device is a disk, the SCSI subsystem will next send,
via the iscsi_initiator, a TEST_UNIT_READY command, and if that is
successful, it is followed by a READ_CAPACITY command.  If that is
successful, the SCSI subsystem will report the size of the disk and
the size of blocks on the disk (again by writing messages to the
/var/log/messages file), and will then issue a READ_10 command to
read block 0 from the disk.

Note that if lun=0 is ommitted from the above iscsi_config command,
then after the transition to FUll Feture Phase ALL luns, not just
lun 0, will be probed via INQUIRY commands, as just described.


As with the assignment of a host number to the unh_iscsi_initiator HBA,
devices activated by the iscsi_config program are assigned the next
device name of the appropriate type according to the current state of
the Linux system.  For example, the first SCSI disk activated on a
Linux system is given the name sda, the second is given the name sdb,
etc.  Therefore, if the iscsi_config program has activated a target
disk, it will be called sda or sdb or sdc, etc., depending on how many
other scsi disks have been activated prior to issuing the iscsi_config
command.  Similarly, if the device activated by the iscsi_config
program is a tape, the names assigned are st0, st1, st2, etc.
These names are printed by the SCSI subsystem in the /var/log/messages
file after a successful login.


5. Setting up the target device
-------------------------------

If the target device is a disk, then the Linux system expects block 0
to contain a partition table in a standard format.  If this is not
found, the message "unknown partition table" will be written to the
/var/log/messages file.  At this point you will have to run the
fdisk program to create a valid partition table.  If your disk has
been assigned the name sdb, then give the shell command:

	/sbin/fdisk	/dev/sdb

to run the fdisk program.  This will interactively prompt you for
the information to create a partition table.  The partitions are
numbered 1, 2, etc., and correspond to the names /dev/sdb1, /dev/sdb2,
etc.

Once the target disk device has a valid partition table on it,
you can mount the various partitions using the mount command.  The
easiest way to do this is to have an entry in the file /etc/fstab and
a mount-point directory in the file system.  For example, to mount the
first partition (/dev/sdb1) on disk sdb, under the name /mnt/scsi,
the directory /mnt/scsi should be created (and be empty) and the
/etc/fstab should contain a line similar to:

	/dev/sdb1	/mnt/scsi	ext2	noauto	0 0

where ext2 is the type of file system expected on that device.

The mount command is then:

	mount /mnt/scsi

The above line in /etc/fstab says that partition /dev/sdb1 should
contain an ext2 file system.  If the mount command fails, it is probably
because the partition does not contain a valid ext2 file system.
To create one, type the shell command:

	/sbin/mke2fs	/dev/sdb1

and then try the mount command again.


If the partition does contain a valid ext2 file system, but it was
corrupted in any way (probably due to a system crash when it was
last mounted), a warning message is written by the Linux File subsystem
to the /var/log/messages file.  To correct the problem, first unmount
the disk using the shell command:

	umount /mnt/scsi

and then type the shell command:

	/sbin/e2fsck	/dev/sdb1

If this program detects any problems it will attempt to correct them,
which may involve prompting you interactively to help it decide what
to do (the best choice is to accept the default action in almost all
cases).

Once the device is mounted properly, the system administrator may
have to create directories on it for use by users.  This is done
using the mkdir command, along with the chown and chgrp commands.
For example, to create a directory called xyz on the newly mounted
/mnt/scsi device for user rdr in group iol, give the sequence of shell
commands:

	mkdir		/mnt/scsi/xyz
	chown	rdr /mnt/scsi/xyz
	chgrp	iol	/mnt/scsi/xyz


6. Using the target device
--------------------------

Once the target disk device has been mounted and directories have been
created on it, ordinary (non-superuser) users can access it like they
would any other disk device.  They do not have to have root access in
order to do this.

For example, to copy a file in his current directory called project.tgz
to his new directory on the disk (and keep the same file name), user rdr
can do:

	cp project.tgz /mnt/scsi/rdr

Presumably this is a tar file.  To untar it and rebuild it, do:

	cd /mnt/scsi/rdr
	tar zxvf project.tgz
	cd project
	make

etc.


7. Bringing down the target device
----------------------------------

Once all users have finished using a device, the System Administrator
(superuser) can unmount it using the shell command:

	umount /mnt/scsi

Note that no user can have any directory on that device as his or her
current directory, or the umount will fail with a busy indication.


8. Closing the session
----------------------

Once the mounted device has been unmounted, the session established
in step 4 above can be closed by giving the shell command:

	iscsi_config down ip=123.45.67.089 host=1

See section 10 below for a complete discussion of the iscsi_config
utility.


9. Unloading the unh_iscsi_initiator module
---------------------------------------

Once the device has been closed, the unh_iscsi_initiator module can be
unloaded by giving the shell command:

	/sbin/rmmod unh_iscsi_initiator

Note that step 8 above can be skipped if the user is going to
immediately unload the module, since all connections to targets
are automatically closed when the module is unloaded.


10. multiple sessions, luns, connections
----------------------------------------

The general form of a shell command to the iscsi_config utility is:

	iscsi_config up ip=123.45.67.089 port=5142 host=1 target=2 lun=3 cid=4
  or
	iscsi_config down host=1 target=2 lun=3 cid=4

Where the "up" or "down" parameter is required, and if any other parameter
is missing, its value is assumed to be 0.

The ip and port numbers are used to locate the target device, as explained
in step 4 above, and are used only when "up" is given.


The SCSI subsystem addresses all its devices by using the quadruple:

	(host, channel, target, lun)

  or in some places (like /proc/scsi/scsi):

  	(host, channel, id, lun)

where the host, target (id) and lun numbers are taken from the parameters
to the iscsi_config command, and the channel number is always 0 for the
reference implementations.  The host number must match that assigned to
the reference module when it was first loaded in step 2 above.  The
target and lun numbers can be chosen by the user, subject to the
restrictions:

	0 <= target < MAX_TARGET_IDS (currently 10)
and
	0 <= lun < MAX_LUNS (currently 8)

where MAX_TARGET_IDS and MAX_LUNS are compile-time constants defined in
the file "initiator/iscsi_initiator.h".


The cid number is a user-chosen number to identify the connection within
the session, subject to the restriction:

	0 <= cid <= 255

where 255 is a hard limit so that the cid value will fit in 1 byte.
(We do not anticipate more than 256 connections per session, even though
the standard allows for a cid value up to 65536!)


Rules for bringing up a new session, lun or connection:

	1.	The host number must match the number assigned to the iscsi
		reference initiator module when it was loaded and registered with
		the SCSI subsystem (step 2 above).

	2.	If the target number does NOT match that of any currently
		configured session, then start a new session (subject to the
		restriction of at most MAX_TARGET_IDS simultaneous sessions.
		The lun associated with this new session will be as given, and
		the cid for the first connection of this session will be as given.
		The new connection will be logged in and a new device will appear
		at the SCSI level (sdc, st4, etc.).

	3.	If the target number matches a currently configured session, then
		if the lun number does NOT match that of any currently configured
		lun for this session, add this lun to this session (subject to the
		restriction of at most MAX_LUNS simultaneous luns per session.
		The cid number is ignored in this case (because PDUs for any lun
		can be sent over any connection belonging to the session).  A new
		device will appear at the SCSI level (sdc, st4, etc.).


	4.	If the target number matches a currently configured session, and
		if the lun number matches a currently configured lun of this
		session, then if the cid number does NOT match that of any
		currently configured connection for this session, add a new
		connection to this session (subject to the restriction of at most
		MaxConnections connections per session, as negotiated during the
		very first login that established this session).  NO new device
		will appear at the SCSI level because the number of connections
		per session is an internal iSCSI matter, and is not seen outside
		iSCSI.

	5.	If the target number matches a currently configured session, and
		if the lun number matches a currently configured lun of this
		session, and if the cid number matches a currently configured
		connection of this session, then this is an error and the "up"
		is not performed.


Rules for bringing down a session, lun or connection:

	1.	The host number must match the number assigned to the iscsi
		reference initiator module when it was loaded and registered with
		the SCSI subsystem (step 2 above).

	2.	If the target number matches a currently configured session, and
		if the lun number matches a currently configured lun of this
		session, and if the cid number matches a currently configured
		connection of this session, then this connection is closed and
		removed from the session.  If this was the last connection
		associated with this session, then the entire session is closed
		too.  There is NO effect at the SCSI level unless the entire
		session is closed, in which case only the device indicated by
		the target and lun numbers will be removed.  This can cause
		problems if other devices were associated with this session,
		since they are not removed at the SCSI level.  This is because
		the iscsi_config program does not maintain any state and does
		not know what these other devices were.

	3.	If the target number matches a currently configured session, and
		if the lun number matches a currently configured lun of this
		session, and if the cid number does NOT match that of any
		currently configured connection for this session, then remove this
		lun from this session.  The corresponding device at the SCSI level
		will also be removed.  If this was the last lun associated with
		this session, then the entire session and all its remaining
		connections are closed too, and the device indicated by the target
		and lun numbers will be removed at the SCSI level.  This can cause
		problems if other devices were associated with this session, since
		they are not removed at the SCSI level.  This is because the
		iscsi_config program does not maintain any state and does not know
		what these other devices were.


	4.	If the target number matches a currently configured session, and
		if the lun number does NOT match a currently configured lun of
		this session, then close the entire session and all its remaining
		connections.  The cid number is ignored in this situation.  The
		device indicated by the target and lun numbers will be removed at
		the SCSI level.  This can cause problems if other devices were
		associated with this session, since they are not removed at the
		SCSI level.  This is because the iscsi_config program does not
		maintain any state and does not know what these other devices were.

	5.	If the target number does NOT match a currently configured
		session, then this is an error and the "down" is not performed.



 ipV6 Notes


 Included are updated cmd/iscsi_config.c to accept ipv6 addresses.

 Of course, several of the files in initiator/ and target/ and
 common/ were also changed to accommodate the ipv6 addresses.
 Now wherever there is an ip=, what follows the = can be
 ipv4 or ipv6.  There are lots of (now commented out) examples
 that are used for testing in the file cmd/ini-m1.
 That is how to tell the initiator to connect to a target using
 ipv6.

 To configure the target to use ipv6, you need to modify the
 file target/iscsi_portal_group.c.  At the end of that file is
 a table, and initialization for it.  The first 3 fields in each
 entry in that table are the IP address as a string, the port number
 as a string, and the portal group tag as a number.  The IP address
 can be ipv4 dotted decimal, or ipv6 in the bracketed notation
 refered to in Draft 20 section 12.8 and RFC 2732.  The first 2
 fields are strings, because these strings are used by the target
 to compose its responses to a SendTargets request during a
 discovery session.  Again there are (now commented out) examples
 in that file of how it is done.


 co-existing a IPv^6 on a V4 network ?

 I found out the hard way that our mods correctly deal with IPv6
 global scope address, but not link scope addresses
 (in IPv4 everything is global scope).  This is because link
 scope addresses need extra information (the interface name,
 like "eth0", "eth1") and we do not now provide a mechanism
 to deal with this extra info.  The global scope addresses
 do not need this, and work fine.

 Setting up IPV6 using only link scope addresses is trivial.
 You only need to do 2 things:
 1. add the line NETWORKING_IPV6=yes
    to the file /etc/sysconfig/network
 2. be sure the module ipv6 is loaded (or compiled in the kernel)
	When you bring up the network, you have IPv6, but with link scope
	addresses only, and this coexists with IPv4 on all the eth0, eth1,
	etc.  The IPv6 address is automatically constructed from the
	ethernet address.  The problem is, we can't use these addresses.
	I didn't find out about this until I tried it -- we did all our
	development on machines that had global scope address configured,
	so I wasn't even aware of the problem.

	To configure real global scope addresses requires a real, separate
	IPv6 network with IPv6 switches.  That's all I know about it --
	the IPv6 group in our lab set everything up for us, and it just
	worked.  But we can't get it to work on the ipv4 network that
	our iscsi group runs because we don't have the ipv6 capable
	switches (and we don't have the know-how to do the configuration,
	although we could learn that from the ipv6 group).

	If we have a chance, we can add the extra info so link scope
	addresses will work.  Then we can test on our own network.
	However, the IPv6 people claim these link scope addresses
	are almost never used (I can see why) and that it probably
	wasn't worth the effort to get them to work.  Certainly
	for the upcoming IPv6 plugfest we will not be needing or
	using them, since everyone there will be using global
	scope addresses.  Apparently the IPv6 group itself uses
	only global scope addresses.


 Notes on FKT.
 
 The initiator includes hooks for a tool called FKT:
 Fast Kernel Tracing, which allows us to trace every
 cycle in the kernel.  Using this we can see the flow
 through all the kernel routines, modules, etc.
 There are 2 ways to use this tool (both methods can be
 used simultaneously)  1 is to plant explicit probe macros
 at various points in the kernel code.  The other, which
 is best used with modules, is to compile the code using
 the gcc option -finstrument-functions.  This causes gcc
 to generate a call to an instrumentation fuction on every
 function entry and function return.  Just by supplying this
 compile-time option our iscsi modules will be traced by our
 FKT tool.  However, many functions are small, auxiliary
 functions that are not of much interest in following the
 flow, and they just clutter up the output generated by the
 tool (which is voluminous anyway).  So by applying the
 __attribute__  (no_instrument_function)
 to those functions, we are telling gcc that if the
 -finstrument_function optiion is in effect, then don't
 instrument this function (and if the -finstrument_function
 option is not in effect, the attribute is ignored).

 This tool is very useful -- I have already used it to find
 several problems with our memory management, have used it
 to reduce the number of function calls that "do nothing"
 because a test in the function causes it to return quickly,
 and even to find a bad gig card that was causing our initiator
 to freeze.



