How to configure IPMP in Solaris 11 – Active-Active & Active-Standby
In production environments, it is important to eliminate any single point of failure. IP network multipathing (IPMP) provides a mechanism for building redundant network interfaces to guard against failures with network interfaces, cables, switches, or other networking hardware. In addition to eliminating any single point of failure, the IPMP load spreading feature increases the machine’s bandwidth by spreading the outbound load among all the cards in the same IPMP group.
When compared with the IPMP configuration in Oracle Solaris 10, Oracle Solaris 11 groups the IP interfaces into a virtual IP interface (for example, ipmp0). The virtual IP interface serves all of the data IP addresses, while test addresses that are used for probe-based failure detection are assigned to an underlying interface, such as net0.
With IPMP, you can assign two or more NICs to a failover group. Each interface is assigned a static test IP address, which is used by Solaris to verify the operational state of the interface. The interfaces with the static test IP addresses are used to periodically send an Internet Control Message Protocol (ICMP) echo request to a target system and listen for the response. If no response occurs within a given number of tries, the link is marked as failed. IPMP will fail over all application IP addresses currently configured on that physical interface to another physical interface within the IPMP group. In this way, network outages due to failed network hardware are eliminated.
IPMP Components
IPMP consists of the following software components:
- IPMP daemon: Detects failure by sending ICMP echo probes through the interface and also monitors the RUNNING flag on the interface. If a failure is detected from one of these methods, the daemon chooses the standby IP address or the next appropriate IP address and failover occurs.
- IPMP service: Manages the IPMP daemon. The IPMP service also sets IPMP properties, such as enabling or disabling transitive probing.
- Configuration file: Specifies the daemon’s default behavior. This file can be used to set parameters, such as specifying the interfaces to probe for failure and the duration. This file can also be used to specify what the status of a failed interface should be after it is repaired, or whether to monitor all interfaces, including those that do not belong to an IPMP group.
- IPMP administration command: Administers the IP interfaces of the IPMP group
- IPMP display information command: Provides information about the status of IPMP
IPMP Configurations
An IPMP configuration typically consists of two or more physical interfaces on the same system that are attached to the same LAN. These interfaces can belong to an IPMP group in either of the following configurations:
Active-active
In this configuration, all underlying interfaces are active. An active interface is an IP interface that is currently available for use by the IPMP group. By default, an underlying interface becomes active when you configure the interface to become part of an IPMP group.
Active-standby
In this configuration, at least one interface is administratively configured as a reserve. The reserve interface is called the standby interface. Although idle, the standby IP interface is monitored by the multipathing daemon to track the interface’s availability, depending on how the interface is configured. If link-failure notification is supported by the interface, link-based failure detection is used. If the interface is configured with a test address, probe-based failure detection is also used. If an active interface fails, the standby interface is automatically deployed as needed. You can configure as many standby interfaces as you want for an IPMP group.
Configuring IPMP: Active-Active
Below are the steps to configure an active-active IPMP configuration with flexible datalink names.
1. First, you rename the datalinks net0 and net1 to link0_ipmp0 and link1_ipmp0, respectively. Before these datalinks can be used by IPMP, you must create an IP interface for each one.
# dladm rename-link net0 link0_ipmp0 # dladm rename-link net1 link1_ipmp0
# ipadm create-ip link0_ipmp0 # ipadm create-ip link1_ipmp0
2. Now you are ready to create the IPMP group. This involves two steps. You first create the IPMP group (ipmp0 in this example), and then you add the underlying interfaces (link0_ipmp0 and link1_ipmp0) to the group.
# ipadm create-ipmp ipmp0
# ipadm add-ipmp –i link0_ipmp0 –i link1_ipmp0 ipmp0
3. Next, assign the data IP addresses to the IPMP interface (ipmp0) in the form of IP address objects (ipmp0/v4add1 and ipmp0/v4add2).
# ipadm create-addr –a 192.168.0.112/24 ipmp0/v4add1 # ipadm create-addr –a 192.168.0.113/24 ipmp0/v4add2
4. Finally, assign the test IP addresses to each underlying interface in the form of IP address objects (link0_ipmp0/test and link1_ipmp0/test).
# ipadm create-addr –a 192.168.0.142/24 link0_ipmp0/test # ipadm create-addr –a 192.168.0.143/24 link1_ipmp0/test
Configuring IPMP: Active-Standby
Example below shows you the steps to configure an active-standby IPMP configuration with flexible datalink names. The steps are similar to those shown in Active-Active IPMP configuration.
1. First, you rename the datalinks net0, net1, and net2 to link0_ipmp0, link1_ipmp0, and link2_ipmp0, respectively.
# dladm rename-link net0 link0_ipmp0 # dladm rename-link net1 link1_ipmp0 # dladm rename-link net2 link2_ipmp0
2. You then create an IP interface for each one.
# ipadm create-ip link0_ipmp0 # ipadm create-ip link1_ipmp0 # ipadm create-ip link2_ipmp0
3. Now you create the IPMP group. This involves two steps. You first create the IPMP group (ipmp0 in this example), and then you add the underlying interfaces (link0_ipmp0, link1_ipmp0, and link2_ipmp0) to the group.
# ipadm create-ipmp ipmp0 # ipadm add-ipmp –i link0_ipmp0 –i link1_ipmp0 –i link2_ipmp0 ipmp0
4. After the IPMP group is created, you set the standby property in one of the underlying interfaces (link2_ipmp0 in this example) to on.
# ipadm set-ifprop -p standby=on -m ip link2_ipmp0
5. Next, assign the data IP addresses to the IPMP interface (ipmp0) in the form of IP address objects (ipmp0/v4add1 and ipmp0/v4add2).
# ipadm create-addr –a 192.168.0.112/24 ipmp0/v4add1 # ipadm create-addr –a 192.168.0.113/24 ipmp0/v4add2
6. Finally, assign the test IP addresses to each underlying interface in the form of IP address objects (link0_ipmp0/test, link1_ipmp0/test, and link2_ipmp0).
# ipadm create-addr –a 192.168.0.142/24 link0_ipmp0/test # ipadm create-addr –a 192.168.0.143/24 link1_ipmp0/test # ipadm create-addr –a 192.168.0.144/24 link2_ipmp0/test
Failure Detection in IPMP
To ensure continuous availability of the network to send or receive traffic, IPMP performs failure detection on the IPMP group’s underlying IP interfaces. Failed interfaces remain unusable until they are repaired. Remaining active interfaces continue to function while any existing standby interfaces are deployed as needed.
IPMP performs failure detection on the IPMP group’s underlying IP interfaces to ensure continuous availability of the network to send or receive traffic.
The in.mpathd daemon handles the following types of failure detection:
- Two types of probe-based failure detection:
- No test addresses are configured.
- Test addresses are configured.
- Link-based failure detection, if supported by the NIC driver.
Monitoring IPMP
You use the ipmpstat command to monitor IPMP group activity and health. Shown below are three examples of the ipmpstat usage. The examples that you see here are taken from an IPMP active-standby configuration created by the procedure shown in the previous section. Here, one of the underlying interfaces has failed.
The first example (ipmpstat –g) displays information about the IPMP group. The IPMP group is named ipmp0. It has three underlying interfaces: link0_ipmp0, link1_impm0, and link2_ipmp0. Note that the state of the IPMP group is degraded and the underlying interface link0_ipmp0 has brackets around it (boxed) indicating that it has failed.
# ipmpstat -g GROUP GROUPNAME STATE FDT INTERFACES ipmp0 ipmp0 degraded 10.00s link2_ipmp0 link1_ipmp0 [link0_ipmp0]
The second example (ipmpstat –i) displays information about the IP interfaces. Here, link2_ipmp0 is in the Active state and link0_ipmp0 is in the Failed state.
# ipmpstat -i INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE link2_ipmp0 yes ipmp0 -s----- up ok ok link1_ipmp0 yes ipmp0 --mbM-- up ok ok link0_ipmp0 no ipmp0 ------- up failed failed
The third example (ipmpstat –an) displays information about the IPMP data addresses. IP address 192.168.0.112 is currently assigned to the standby interface (link2_ipmp0) and 192.168.0.113 is assigned to link1_ipmp0 for all INBOUND data traffic. OUTBOUND data traffic is spread across both active interfaces for each IP address.
# ipmpstat -an ADDRESS STATE GROUP INBOUND OUTBOUND :: down ipmp0 -- -- 192.168.0.113 up ipmp0 link1_ipmp0 link2_ipmp0 link1_ipmp0 192.168.0.112 up ipmp0 link2_ipmp0 link2_ipmp0 link1_ipmp0
The example below (ipmpstat –pn) shows information about the IPMP probe. For IPMP probing to work correctly, the IPMP group must be connected to the local area network and at least one other host (the probe target) must also be connected to the same network.
# ipmpstat -pn TIME INTERFACE PROBE NETRTT RTT RTTAVG TARGET 0.06s link2_ipmp0 i163 0.26ms 0.49ms 0.33ms 192.168.0.100 0.90s link1_ipmp0 i162 0.26ms 0.39ms 0.31ms 192.168.0.100 0.92s link2_ipmp0 i164 0.19ms 0.36ms 0.34ms 192.168.0.100 0.49s link0_ipmp0 i161 -- -- -- 192.168.0.100 -0.49s link0_ipmp0 i160 -- -- -- 192.168.0.100 ....
Here, link2_ipmp0 (standby) and link1_ipmp0 interfaces are actively probing target 192.168.0.100. Interface link0_ipmp0 probing is failing.
No comments:
Post a Comment