Sunday, 21 October 2018

How to configure Solaris 10 Probe based IPMP By Devan

How to configure Solaris 10 Probe based IPMP

What’s a Probe Based IPMP ?

The failure detection method used by the in.mpathd daemon differentiates the IPMP as probe based or link based. Probe based IPMP uses 2 types of addresses in its configuration.
1. Test address – Used by in.mpathd daemon for detecting the failure (also called as probe address).
2. Data Address – Used by applications for actual data transfer.
In case of probe based IPMP :
– in.mpathd daemon sends out ICMP probe messages on test address to one or more target systems on the same subnet.
– in.mpathd daemon determines the available target systems to probe dynamically. It uses a “all hosts” multicast (224.0.0.1) address to determine the target systems to probe.
examples of target systems :
1. all default routes on same subnet.
2. all host routes on same subnet. (configured with “route -p add” command)
– All test addresses should be in the same subnet.

mpathd Configuration file

mpathd daemon is responsible to detect an interface failure . It uses a configuration file /etc/default/mpathd to set various IPMP parameters.
# cat /etc/default/mpathd
#
#pragma ident   "@(#)mpathd.dfl 1.2     00/07/17 SMI"
#
# Time taken by mpathd to detect a NIC failure in ms. The minimum time
# that can be specified is 100 ms.
#
FAILURE_DETECTION_TIME=10000
#
# Failback is enabled by default. To disable failback turn off this option
#
FAILBACK=yes
#
# By default only interfaces configured as part of multipathing groups
# are tracked. Turn off this option to track all network interfaces
# on the system
#
TRACK_INTERFACES_ONLY_WITH_GROUPS=yes
The important parameters in mpathd configuration file are :
1. FAILURE_DETECTION_TIME : Time taken by mpathd to detect a NIC failure in ms (default value – 10 seconds)
2. FAILBACK : To enable or disable failback after the failed link becomes available (default vaule – yes)
3. TRACK_INTERFACES_ONLY_WITH_GROUPS – If turned on interfaces configured as part of IPMP are only monitored (default vaule – yes)
The command for in.mpathd daemon to re-read the configuration file is :
# pkill -HUP in.mpathd

Failure detection and repair detection time

– The probing rate depends on the FAILURE_DETECTION_TIME set in the /etc/default/mpathd configuration file (default value 10 seconds).
– in.mpathd send 5 probes in every 10 seconds ( or the FAILURE_DETECTION_TIME set ). If 5 consecutive probes fail, in.mpathd considers the interface to be failed.
– The minimum repair detection time is twice the failure detection time. So if the failure detection time is 10 seconds, then the repaire detection time would be 20 seconds (with 10 probes).

Testing IPMP failover

We can check the failure and repair of an interface very easily using if_mpadm command. “-d” detaches the interface whereas “-r” reattaches it.
# if_mpadm -d ce0
# if_mpadm -r ce0

Meanings of FLAGs

You would see flags such as NOFAILOVER, DEPRECATED, STANDBY etc.. in the output of “ifconfig -a” command. The meanings of these flags and parameters to enable them are:
deprecated -> can only be used as test address for IPMP and not for any actual data transfer by applications.
-failover -> does not failover when the interface fails
standby -> makes the interface to be used as standby

Most commonly used Probe-Based IPMP configurations

1. Active – Active configuration
Groupname:             ipmp0
Active interface(s):   e1000g0 
                       e1000g1
Standby interface(s):  -
Data IP addresse(s):   192.168.1.2
Test IP addresse(s):   192.168.1.3
                       192.168.1.4
Command line :
# ifconfig e1000g0 plumb 192.168.1.2 netmask + broadcast + group ipmp0 up addif 192.168.1.3 netmask + broadcast + deprecated -failover up
# ifconfig e1000g1 plumb 192.168.1.4 netmask + broadcast + deprecated -failover group ipmp0 up
To ensure persistent configuration across reboots edit the files :
/etc/hostname.e1000g0:
192.168.1.2 netmask + broadcast + group ipmp0 up 
addif 192.168.1.3 netmask + broadcast + deprecated -failover up

/etc/hostname.e1000g1:
192.168.1.4 netmask + broadcast + deprecated -failover group ipmp0 up
Before failure :
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 9
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
        groupname ipmp0
        ether 0:c:29:f6:ef:67
e1000g0:1: flags=9040843[UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER] mtu 1500 index 9
        inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255
e1000g1: flags=9040843[UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER] mtu 1500 index 10
        inet 192.168.1.4 netmask ffffff00 broadcast 192.168.1.255
        groupname ipmp0
        ether 0:c:29:f6:ef:71
After failure :
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=19000802[BROADCAST,MULTICAST,IPv4,NOFAILOVER,FAILED] mtu 0 index 9
        inet 0.0.0.0 netmask 0
        groupname ipmp0
        ether 0:c:29:f6:ef:67
e1000g0:1: flags=19040803[UP,BROADCAST,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,FAILED] mtu 1500 index 9
        inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255
e1000g1: flags=9040843[UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER] mtu 1500 index 10
        inet 192.168.1.4 netmask ffffff00 broadcast 192.168.1.255
        groupname ipmp0
        ether 0:c:29:f6:ef:71
e1000g1:1: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 10
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
2. Active – Standby
The only difference in case of a active-standby configuration is – the interface configured as standby is not used to send any out bound traffic. Thus disabling the load balancing feature of an active-active configuration.
Groupname:             ipmp0
Active interface(s):   e1000g0             
Standby interface(s):  e1000g1
Data IP addresse(s):   192.168.1.2
Test IP addresse(s):   192.168.1.3
                       192.168.1.4
Command line :
# ifconfig e1000g0 plumb 192.168.1.2 netmask + broadcast + group ipmp0 up addif 192.168.1.3 netmask + broadcast + deprecated -failover up
# ifconfig e1000g1 plumb 192.168.1.4 netmask + broadcast + deprecated -failover group ipmp0 standby up
To ensure persistent configuration across reboots edit the files :
/etc/hostname.e1000g0:
192.168.1.2 netmask + broadcast + group ipmp0 up 
addif 192.168.1.3 netmask + broadcast + deprecated -failover up

/etc/hostname.e1000g1:
192.168.1.4 netmask + broadcast + deprecated -failover group ipmp0 standby up
Before failure :
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 11
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
        groupname ipmp0
        ether 0:c:29:f6:ef:67
e1000g0:1: flags=9040843[UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER] mtu 1500 index 11
        inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255
e1000g1: flags=69040843[UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY,INACTIVE] mtu 1500 index 12
        inet 192.168.1.4 netmask ffffff00 broadcast 192.168.1.255
        groupname ipmp0
        ether 0:c:29:f6:ef:71
After failure :
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=19000802[BROADCAST,MULTICAST,IPv4,NOFAILOVER,FAILED] mtu 0 index 11
        inet 0.0.0.0 netmask 0
        groupname ipmp0
        ether 0:c:29:f6:ef:67
e1000g0:1: flags=19040803[UP,BROADCAST,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,FAILED] mtu 1500 index 11
        inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255
e1000g1: flags=29040843[UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY] mtu 1500 index 12
        inet 192.168.1.4 netmask ffffff00 broadcast 192.168.1.255
        groupname ipmp0
        ether 0:c:29:f6:ef:71
e1000g1:1: flags=21000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4,STANDBY] mtu 1500 index 12
        inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
by at
Tags , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)