How to configure link based IPMP in Solaris 11
Oracle made a tone of changes to its networking stack in Solaris 11. Persistent network configuration is now managed by SMF, and not by editing following files :
/etc/defaultdomain /etc/dhcp.* /etc/hostname.* /etc/hostname.ip*.tun* /etc/nodename /etc/nsswitch.conf
IPMP conponents :
1. mpathd daemon – This daemon is responsible to detect an interface failure or repair. It uses a configuration file /etc/default/mpathd to determine when to failover the interface.
# cat /etc/default/mpathd # #pragma ident "@(#)mpathd.dfl 1.2 00/07/17 SMI" # # Time taken by mpathd to detect a NIC failure in ms. The minimum time # that can be specified is 100 ms. # FAILURE_DETECTION_TIME=10000 # # Failback is enabled by default. To disable failback turn off this option # FAILBACK=yes # # By default only interfaces configured as part of multipathing groups # are tracked. Turn off this option to track all network interfaces # on the system # TRACK_INTERFACES_ONLY_WITH_GROUPS=yes
2. ipmpstat – Shows information about IP interface configured in an IPMP group also the test and data address.
Types of configurations
1. Singleton – IPMP group with single interface
2. Active-Active – All the interfaces in the IPMP group are online.
3. Active-standby – One of the interface in the IPMP configuration will be used as standby.
2. Active-Active – All the interfaces in the IPMP group are online.
3. Active-standby – One of the interface in the IPMP configuration will be used as standby.
Testing IPMP failover
We can check the failure and repair of an interface very easily through command line. “-d” detaches the interface whereas “-r” reattaches it.
# if_mpadm -d ce0 # if_mpadm -r ce0
Prerequisites
1. IPMP requires a manual (fixed) network configuration mode, so we have to set the network configuration profile (NCP) to DefaultFixed.
# netadm enable -p ncp DefaultFixed
The output of netadm list should show the ncp online with DefaultFixed profile.
# netadm list
TYPE PROFILE STATE
ncp Automatic disabled
ncp DefaultFixed online
loc Automatic offline
loc NoNet offline
loc DefaultFixed online
2. Make sure you have unique MAC addresses for all the interfaces being used in the IPMP group
# eeprom local-mac-address?=true
# dladm show-linkprop -p mac-address LINK PROPERTY PERM VALUE DEFAULT POSSIBLE net1 mac-address rw 0:c:29:12:40:79 0:c:29:12:40:79 -- net2 mac-address rw 0:c:29:12:40:83 0:c:29:12:40:83 -- net3 mac-address rw 0:c:29:12:40:8d 0:c:29:12:40:8d -- net0 mac-address rw 0:c:29:12:40:6f 0:c:29:12:40:6f --
By default the link based failure detection is enabled. To check the failure detection method :
# ipmpstat -p ipmpstat: probe-based failure detection is disabled
My virtual machine has 4 NIC cards as follows. I will be using net1 and net2 to configure the IPMP group.
# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 Ethernet unknown 1000 full e1000g0 net1 Ethernet unknown 1000 full e1000g1 net2 Ethernet unknown 1000 full e1000g2 net3 Ethernet unknown 1000 full e1000g3
1. Active/Active Configuration example
Active Interface : e1000g1 (net1)
: e1000g2 (net2)
Data IP : 192.168.1.10
Create the IPMP group “ipmp0”.
# ipadm create-ipmp ipmp0
Create the interfaces net1 and net2 to be used in the IPMP group.
# ipadm create-ip net1 # ipadm create-ip net2
Add the two interfaces to the IPMP group
# ipadm add-ipmp -i net1 -i net2 ipmp0
To check the status of the IPMP group we just created :
# ipmpstat -g GROUP GROUPNAME STATE FDT INTERFACES ipmp0 ipmp0 ok -- net2 net1
Add data IP address to the IPMP group. The syntax to do so is :
Syntax
Syntax
ipadm create-addr -T static -a [hostname or IP address] [IPMP-interface]/[arbitrary-string]
# ipadm create-addr -T static -a 192.168.1.10 ipmp0/data1
To check the status of the IPMP interface :
# ipmpstat -a ADDRESS STATE GROUP INBOUND OUTBOUND ipmp0-data1 up ipmp0 net1 net2 net1
# ipmpstat -an ADDRESS STATE GROUP INBOUND OUTBOUND 192.168.1.10 up ipmp0 net1 net2 net1
To check the status of individual interfaces in the IPMP group.
# ipmpstat -i INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE net2 yes ipmp0 ------- up disabled ok net1 yes ipmp0 --mb--- up disabled ok
To check the IP address of the IPMP group :
# ipadm show-addr ADDROBJ TYPE STATE ADDR lo0/v4 static ok 127.0.0.1/8 ipmp0/data1 static ok 192.168.1.10/24
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
net1: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 12
inet 0.0.0.0 netmask ff000000
groupname ipmp0
ether 0:c:29:12:40:79
net2: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 13
inet 0.0.0.0 netmask ff000000
groupname ipmp0
ether 0:c:29:12:40:83
ipmp0: flags=8001000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4,IPMP] mtu 1500 index 11
inet 192.168.1.10 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp0
All the configuration changes we just made, are saved in the /etc/ipadm/ipadm-DefaultFixed.conf file (As we are using Fixed NCP)
# cat /etc/ipadm/ipadm-DefaultFixed.conf _ifname=lo0;_aobjname=lo0/v4; _ipv4saddr=string,127.0.0.1;prefixlen=string,8;up=string,yes; _ifname=lo0; _family=string,2,26;_class=uint64,2; _ifname=net1; _family=string,2,26;_class=uint64,0;_ipmpif=string,ipmp0; _ifname=ipmp0; _family=string,2,26;_class=uint64,1;_underif=string,net1,net2; _ifname=net2; _family=string,2,26;_class=uint64,0;_ipmpif=string,ipmp0; _ifname=ipmp0;_aobjname=ipmp0/data1; _ipv4saddr=string,192.168.1.10;up=string,yes;
2. Active/Standby Configuration Example
Let us configure an Active/Standby Link-Based IPMP configuration, by setting net2 as the standby interface.
Active Interface : e1000g1 (net1) Standby Interface : e1000g2 (net2) Data IP : 192.168.1.10
To set the net2 interface as standby interface :
# ipadm set-ifprop -p standby=on -m ip net2
Check the status of the individual interfaces. Observe the 2nd column. The net2 interface is not active.
# ipmpstat -i INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE net2 no ipmp0 is----- up disabled ok net1 yes ipmp0 --mb--- up disabled ok
You can also confirm the STANDBY flag in the ifconfig command output.
# ifconfig -a
lo0: flags=2001000849[UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL] mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
net1: flags=1000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4] mtu 1500 index 12
inet 0.0.0.0 netmask ff000000
groupname ipmp0
ether 0:c:29:12:40:79
net2: flags=61000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4,STANDBY,INACTIVE] mtu 1500 index 13
inet 0.0.0.0 netmask ff000000
groupname ipmp0
ether 0:c:29:12:40:83
ipmp0: flags=8001000843[UP,BROADCAST,RUNNING,MULTICAST,IPv4,IPMP] mtu 1500 index 11
inet 192.168.1.10 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp0
Check the status of the IPMP group and observe the net2 interface in (), as a contrast to the Active/Active configuration.
# ipmpstat -g GROUP GROUPNAME STATE FDT INTERFACES ipmp0 ipmp0 ok -- net1 (net2)
Also note the difference in the output of the below command. Bothe the inbound and outbound traffic is on the interface net1.
# ipmpstat -an ADDRESS STATE GROUP INBOUND OUTBOUND 192.168.1.10 up ipmp0 net1 net1
No comments:
Post a Comment