1 .\" $OpenBSD: carp.4,v 1.16 2004/12/07 23:41:35 jmc Exp $
3 .\" Copyright (c) 2003, Ryan McBride. All rights reserved.
5 .\" Redistribution and use in source and binary forms, with or without
6 .\" modification, are permitted provided that the following conditions
8 .\" 1. Redistributions of source code must retain the above copyright
9 .\" notice, this list of conditions and the following disclaimer.
10 .\" 2. Redistributions in binary form must reproduce the above copyright
11 .\" notice, this list of conditions and the following disclaimer in the
12 .\" documentation and/or other materials provided with the distribution.
14 .\" THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND
15 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
16 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
17 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE
18 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
19 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
20 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
21 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
22 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
23 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
33 .Nd Common Address Redundancy Protocol
39 interface is a pseudo-device that implements and controls the
41 CARP allows multiple hosts on the same local network to share a set of IP addresses.
42 Its primary purpose is to ensure that these
43 addresses are always available, but in some configurations
45 can also provide load balancing functionality.
49 interface can be created at runtime using the
50 .Nm ifconfig Li carp Ns Ar N Cm create
51 command or by configuring
60 the administrator needs to configure at minimum a common virtual host ID (VHID)
61 and virtual host IP address on each machine which is to take part in the virtual
63 Additional parameters can also be set on a per-interface basis:
67 which are used to control how frequently the host sends advertisements when it
68 is the master for a virtual host, and
70 which is used to authenticate
76 .Dq "advertisement base" .
77 It is measured in seconds and specifies the base of the advertisement interval.
81 .Dq "advertisement skew" .
82 It is measured in 1/256 of seconds.
83 It is added to the base advertisement interval to make one host advertise
84 a bit slower that the other does.
89 are put inside CARP advertisements.
90 These configurations can be done using
96 Additionally, there are a number of global parameters which can be set using
98 .Bl -tag -width ".Va net.inet.carp.arpbalance"
99 .It Va net.inet.carp.allow
104 .It Va net.inet.carp.preempt
105 Allow virtual hosts to preempt each other.
106 It is also used to failover
108 interfaces as a group.
109 When the option is enabled and one of the
111 enabled physical interfaces
114 is changed to 240 on all
117 See also the first example.
119 .It Va net.inet.carp.log
120 Value of 0 disables any logging.
121 Value of 1 enables logging state changes of
124 Values above 1 enable logging of bad
128 .It Va net.inet.carp.arpbalance
129 Balance local traffic using ARP (see below).
131 .It Va net.inet.carp.suppress_preempt
132 A read only value showing the status of preemption suppression.
133 Preemption can be suppressed if link on an interface is down
136 interface is not synchronized.
137 Value of 0 means that preemption is not suppressed, since no
138 problems are detected.
139 Every problem increments suppression counter.
141 .Sh ARP level load balancing
144 has limited abilities for load balancing the incoming connections
145 between hosts in Ethernet network.
146 For load balancing operation, one needs several CARP interfaces that
147 are configured to the same IP address, but to a different VHIDs.
148 Once an ARP request is received, the CARP protocol will use a hashing
149 function against the source IP address in the ARP request to determine
150 which VHID should this request belong to.
151 If the corresponding CARP interface is in master state, the ARP request
152 will be replied, otherwise it will be ignored.
155 section for a practical example of load balancing.
157 The ARP load balancing has some limitations.
158 First, ARP balancing only works on the local network segment.
159 It cannot balance traffic that crosses a router, because the
160 router itself will always be balanced to the same virtual host.
161 Second, ARP load balancing can lead to asymmetric routing
162 of incoming and outgoing traffic, and thus combining it with
164 is dangerous, because this creates a race condition between
165 balanced routers and a host they are serving.
166 Imagine an incoming packet creating state on the first router, being
167 forwarded to its destination, and destination replying faster
168 than the state information is packed and synced with the second router.
169 If the reply would be load balanced to second router, it will be
170 dropped due to no state.
171 .Sh STATE CHANGE NOTIFICATIONS
172 Sometimes it is useful to get notified about
174 status change events.
175 This can be accomplished by using
178 Master/slave events are signalled as
189 section for more information.
191 For firewalls and routers with multiple interfaces, it is desirable to
194 interfaces together, when one of the physical interfaces goes down.
195 This is achieved by the preempt option.
196 Enable it on both host A and B:
198 .Dl sysctl net.inet.carp.preempt=1
200 Assume that host A is the preferred master and 192.168.1.x/24 is
201 configured on one physical interface and 192.168.2.y/24 on another.
202 This is the setup for host A:
203 .Bd -literal -offset indent
204 ifconfig carp0 create
205 ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.1/24
206 ifconfig carp1 create
207 ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.2.1/24
210 The setup for host B is identical, but it has a higher
212 .Bd -literal -offset indent
213 ifconfig carp0 create
214 ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.1/24
215 ifconfig carp1 create
216 ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.2.1/24
219 Because of the preempt option, when one of the physical interfaces of
222 is adjusted to 240 on all its
225 This will cause host B to preempt on both interfaces instead of
228 In order to set up an ARP balanced virtual host, it is necessary to configure
229 one virtual host for each physical host which would respond to ARP requests
230 and thus handle the traffic.
231 In the following example, two virtual hosts are configured on two hosts to
232 provide balancing and failover for the IP address 192.168.1.10.
236 interfaces on host A are configured.
239 of 100 on the second virtual host means that its advertisements will be sent
240 out slightly less frequently.
241 .Bd -literal -offset indent
242 ifconfig carp0 create
243 ifconfig carp0 vhid 1 pass mekmitasdigoat 192.168.1.10/24
244 ifconfig carp1 create
245 ifconfig carp1 vhid 2 advskew 100 pass mekmitasdigoat 192.168.1.10/24
248 The configuration for host B is identical, except the
250 is on virtual host 1 rather than virtual host 2.
251 .Bd -literal -offset indent
252 ifconfig carp0 create
253 ifconfig carp0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.1.10/24
254 ifconfig carp1 create
255 ifconfig carp1 vhid 2 pass mekmitasdigoat 192.168.1.10/24
258 Finally, the ARP balancing feature must be enabled on both hosts:
260 .Dl sysctl net.inet.carp.arpbalance=1
262 When the hosts receive an ARP request for 192.168.1.10, the source IP address
263 of the request is used to compute which virtual host should answer the request.
264 The host which is master of the selected virtual host will reply to the
265 request, the other(s) will ignore it.
267 This way, locally connected systems will receive different ARP replies and
268 subsequent IP traffic will be balanced among the hosts.
269 If one of the hosts fails, the other will take over the virtual MAC address,
270 and begin answering ARP requests on its behalf.
274 status change events can be set up by using the following devd.conf rules:
275 .Bd -literal -offset indent
277 match "system" "IFNET";
278 match "type" "LINK_UP";
279 match "subsystem" "carp*";
280 action "/root/carpcontrol.sh $type $subsystem";
284 match "system" "IFNET";
285 match "type" "LINK_DOWN";
286 match "subsystem" "carp*";
287 action "/root/carpcontrol.sh $type $subsystem";
300 device first appeared in
304 device was imported into