The configuration file for failover V2 is /etc/failover2.conf. The entries in this file define failover attributes associated with a path to the storage.Entries can be in any order.

In a failover2.conf file, you use the preferred keyword to specify the preferred path for accessing each XVM physvol; there is no default preferred path.The paths to a physvol are assigned an affinity value. This value is used to associate paths to a specific RAID controller, and to determine priority order in which groups of paths from a node to a LUN will be tried in the case of a failure: all affinity 0 paths are tried, then all affinity 1, then all affinity 2, etc.

Usually, all paths to the same controller are configured with the same affinity value and thus only two affinity values are used. You can, however, use more than two affinity values. What is important is that an affinity group for a LUN should not contain paths that go to different RAID groups.

The valid range of affinity values is 0 (lowest) through 15 (highest); the default is affinity 0. Paths with the same affinity number are all tried before failover V2 moves to the next highest affinity number; at 15, failover V2 wraps back to affinity 0 and starts over.

The paths to one controller of the RAID device should be affinity 0, which is the default affinity value. You should set the paths to the second controller to affinity 1.

Since the default affinity is zero, it would be sufficient to include entries only for those paths that are a non-zero affinity. It would also be sufficient to include an entry for the preferred path only. SGI recommends including definitions for all paths, however.

In a multi-host environment, it is recommended that the affinity values for a particular RAID controller be identical on every host in the CXFS cluster.

You can use the affinity value in association with the XVM foswitch command to switch an XVM physvol to a physical path of a defined affinity value, as described in “Switching physvol Path Interactively”.

If a failover.conf file is configured, XVM will employ failover V1 and use the configuration information in that file, even if you configure a failover2.conf file and the RAID devices are in AVT mode. You should remove or comment out an existing failover.conf file when you configure a failover2.conf file.

For instructions on generating a failover2.conf file, see “How to Create a failover2.conf File”.

The following example for SGI ProPack groups the paths for lun3 and the paths for lun4. The order of paths in the file is not significant.

Paths to the same LUN are detected automatically if the LUN has been labeled by XVM.A label command inititates a reprobe to discover new alternate paths. If storage that has already been labeled is connected to a live system, you must run an XVM probe for XVM to recognize the disk as an XVM disk.

Without this file, all paths to each LUN would have affinity 0 and there would be no preferred path.

Setting a preferred path allows the administrator to guarantee that not all traffic to multiple LUNs goes through the same HBA, for example, by selecting preferred paths that spread the load out. Otherwise, the path used is the first one discovered and usually leads to almost all of the load going through the first HBA discovered that is attached to a specific RAID controller.

If no path is designated as preferred, the path used to the LUN is arbitrary based on the order of device discovery. There is no interaction between the preferred path and the affinity values.

This file uses affinity to group the RAID controllers for a particular path. Each controller has been assigned an affinity value. It shows the following:

Given the above, failover will exhaust all paths to lun3 from RAID controller node200900a0b813b982 (with affinity=0 and the preferred path) before moving to RAID controller node200800a0b813b982 paths (with affinity=1)

The following example for SGI ProPack shows an additional grouping of PCI cards. The preferred path has an affinity of 2. If that path is not available, the failover mechanism will try the next path on the same PCI card (with affinity=2). If that is not successful, it will move to affinity=3, which is the other PCI port on the same RAID controller (node200800a0b813b982).

  /dev/xscsi/pci04.01.1/node200900a0b813b982/port1/lun4/disc, affinity=0
  /dev/xscsi/pci04.01.1/node200900a0b813b982/port2/lun4/disc, affinity=1 
  /dev/xscsi/pci04.01.0/node200900a0b813b982/port1/lun4/disc, affinity=0
  /dev/xscsi/pci04.01.0/node200900a0b813b982/port2/lun4/disc, affinity=1
  /dev/xscsi/pci04.01.1/node200800a0b813b982/port1/lun4/disc, affinity=3
  /dev/xscsi/pci04.01.1/node200800a0b813b982/port2/lun4/disc, affinity=2 preferred
  /dev/xscsi/pci04.01.0/node200800a0b813b982/port1/lun4/disc, affinity=3
  /dev/xscsi/pci04.01.0/node200800a0b813b982/port2/lun4/disc, affinity=2

The following example for IRIX shows two RAID controllers, 200800a0b818b4de and 200900a0b818b4de for lun4vol:

 /dev/dsk/200800a0b818b4de/lun4vol/c2p2 affinity=0 preferred
 /dev/dsk/200800a0b818b4de/lun4vol/c2p1 affinity=0
 /dev/dsk/200900a0b818b4de/lun4vol/c2p2 affinity=1
 /dev/dsk/200900a0b818b4de/lun4vol/c2p1 affinity=1

The configuration information in the failover2.conf file becomes available to the system and takes effect when the system is rebooted. You can also parse the failover2.conf file on a running system by means of the XVM foconfig command:

xvm:cluster> 
foconfig -init

You can also execute the foconfig command directly from the shell prompt:

% xvm foconfig -init

The XVM foconfig command allows you to override the default /etc/failover2.conf filename with the -f option. The following command parses the failover information in the file myfailover2.conf:

xvm:cluster> foconfig -f myfailover2.conf

Additionally, the XVM foconfig command provides a -verbose option.

Running the foconfig command does not change any paths, even if new preferred paths are specified in the new failover file. To change the curent path, use the foswitch command, as described in “Switching physvol Path Interactively”.

When using failover V2, you can switch the path used to access an XVM physvol by using the XVM foswitch command. This enables you to set up a new current path on a running system, without rebooting.

---

Note: The XVM foswitch command does not switch paths for storage being managed by XVM V1. For XVM V1, use the (x)scsifo command.

---

xvm:cluster> 
foswitch -preferred phys

% xvm foswitch -preferred phys

xvm:cluster> foswitch -dev 345 phys/lun22

xvm:cluster> foswitch -setaffinity 2 phys/lun33
The following command switches physvol phys/lun33 to the next available path of affinity 2, if there is one available.
xvm:cluster> foswitch -setaffinity 2 -movepath phys/lun33

xvm:cluster> foswitch -cluster -setaffinity 2 phys/lun33

xvm:cluster> foswitch -cluster -preferred phys/lun33

Under failover V2, after you label a device XVM must probe the device to locate the alternate paths to the device. Disks are probed when the system is booted and when you execute an XVM probe command.

When using failover V2, unlabeled disks are probed automatically when the XVM command exits after you label a device.This allows XVM failover to discover alternate paths for newly-labeled devices.

A probe can be slow, and it is necessary to probe a newly-labeled device only once. XVM allows you to disable the automatic probe feature of failover V2.

You can disable automatic probe in the following ways:

You can easily create a failover2.conf file which can be edited to change affinity and preferred path settings with the following command.

xvm show -v phys | grep affinity > /etc/failover2.conf

Values in `< >' within the file are considered comments and can be deleted or ignored.

The entries in the file only apply to already labeled devices. You might want to run the command in both xvm domains, local and cluster, in order to get all defined devices.

To make it easier to understand and maintain the /etc/failover2.conf file, it is best to follow a consistent strategy for setting path affinity. Path failover will occur with preference toward another path of the same affinity as the current path regardless of the affinity value. Here is a simple strategy that works well for most sites:

Note: For SGI Infinite Storage platforms, the WWN of a controller A path always starts with an even number in the first 4 digits (e.g. 2002, 2004, 2006) and the WWN of a controller B path always starts with an odd number in the first 4 digits (e.g. 2003, 2005, 2007).

Make sure that for each LUN in a cluster the same controller is used. Otherwise, a LUN ownership change could result with each I/O process if the hosts are accessing the LUN at the same time.

When setting the preferred path you should have it match the preferred controller owner for the LUN. You can get this information from the TPSSM GUI or from the RAID Array profile.

If you want to have XVM initialize its pathing configuration without going through a reboot, you can do that with the following command.

# xvm foconfig -init

This can be done on a live system without any ill effect because it will not initiate any path failovers. The current path will stay the current path even though the defined preferred path and path affinities may change.

Pay attention to any messages that are generated by this command as it will tell you if you goofed in defining your /etc/failover2.conf file.

You can set all phyvols to their preferred path using the following command. In a cluster configuration be sure all of your /etc/failover2.conf files are correct and consistent and do this from every system to avoid trespass “storms”.

# xvm foswitch -preferred phys