1 ixl FreeBSD* Base Driver and ixlv VF Driver for the
2 Intel XL710 Ethernet Controller Family
5 ================================================================
16 - Building and Installation
17 - Additional Configurations
24 This file describes the IXL FreeBSD* Base driver and the IXLV VF Driver
25 for the XL710 Ethernet Family of Adapters. The Driver has been developed
26 for use with FreeBSD 10.0 or later, but should be compatible with any
29 For questions related to hardware requirements, refer to the documentation
30 supplied with your Intel XL710 adapter. All hardware requirements listed
31 apply for use with FreeBSD.
37 The drivers in this release are compatible with XL710 and X710-based
38 Intel Ethernet Network Connections.
41 SFP+ Devices with Pluggable Optics
42 ----------------------------------
46 Intel DUAL RATE 1G/10G SFP+ SR (bailed) FTLX8571D3BCV-IT
47 Intel DUAL RATE 1G/10G SFP+ SR (bailed) AFBR-703SDZ-IN2
51 Intel DUAL RATE 1G/10G SFP+ LR (bailed) FTLX1471D3BCV-IT
52 Intel DUAL RATE 1G/10G SFP+ LR (bailed) AFCT-701SDZ-IN2
56 Intel TRIPLE RATE 1G/10G/40G QSFP+ SR (bailed) E40GQSFPSR
57 Intel TRIPLE RATE 1G/10G/40G QSFP+ LR (bailed) E40GQSFPLR
58 QSFP+ 1G speed is not supported on XL710 based devices.
60 X710/XL710 Based SFP+ adapters support all passive and active limiting direct
61 attach cables that comply with SFF-8431 v4.1 and SFF-8472 v10.4 specifications.
65 The VF driver is normally used in a virtualized environment where a host
66 driver manages SRIOV, and provides a VF device to the guest. With this
67 first release the only host environment tested was using Linux QEMU/KVM.
68 Support is planned for Xen and VMWare hosts at a later time.
70 In the FreeBSD guest the IXLV driver would be loaded and will function
71 using the VF device assigned to it.
73 The VF driver provides most of the same functionality as the CORE driver,
74 but is actually a slave to the Host, access to many controls are actually
75 accomplished by a request to the Host via what is called the "Admin queue".
76 These are startup and initialization events however, once in operation
77 the device is self-contained and should achieve near native performance.
79 Some notable limitations of the VF environment: for security reasons
80 the driver is never permitted to be promiscuous, therefore a tcpdump
81 will not behave the same with the interface. Second, media info is not
82 available from the PF, so it will always appear as auto.
84 Tarball Building and Installation
85 =========================
87 NOTE: You must have kernel sources installed to compile the driver tarball.
89 These instructions assume a standalone driver tarball, building the driver
90 already in the kernel source is simply a matter of adding the device entry
91 to the kernel config file, or building in the ixl or ixlv module directory.
93 In the instructions below, x.x.x is the driver version
94 as indicated in the name of the driver tarball. The example is
95 for ixl, the same procedure applies for ixlv.
97 1. Move the base driver tar file to the directory of your choice.
98 For example, use /home/username/ixl or /usr/local/src/ixl.
100 2. Untar/unzip the archive:
101 tar xfz ixl-x.x.x.tar.gz
103 3. To install man page:
105 gzip -c ixl.4 > /usr/share/man/man4/ixl.4.gz
107 4. To load the driver onto a running system:
111 5. To assign an IP address to the interface, enter the following:
112 ifconfig ixl<interface_num> <IP_address>
114 6. Verify that the interface works. Enter the following, where <IP_address>
115 is the IP address for another machine on the same subnet as the interface
116 that is being tested:
120 7. If you want the driver to load automatically when the system is booted:
126 Edit /boot/loader.conf, and add the following line:
129 Edit /etc/rc.conf, and create the appropriate
130 ifconfig_ixl<interface_num> entry:
132 ifconfig_ixl<interface_num>="<ifconfig_settings>"
136 ifconfig_ixl0="inet 192.168.10.1 netmask 255.255.255.0"
138 NOTE: For assistance, see the ifconfig man page.
142 Configuration and Tuning
143 =========================
145 Both drivers supports Transmit/Receive Checksum Offload for IPv4 and IPv6,
146 TSO forIPv4 and IPv6, LRO, and Jumbo Frames on all 40 Gigabit adapters.
150 To enable Jumbo Frames, use the ifconfig utility to increase
151 the MTU beyond 1500 bytes.
153 - The Jumbo Frames setting on the switch must be set to at least
154 22 byteslarger than that of the adapter.
156 - The maximum MTU setting for Jumbo Frames is 9706. This value
157 coincides with the maximum jumbo frames size of 9728.
158 To modify the setting, enter the following:
160 ifconfig ixl<interface_num> <hostname or IP address> mtu 9000
162 - To confirm an interface's MTU value, use the ifconfig command.
163 To confirm the MTU used between two specific devices, use:
165 route get <destination_IP_address>
169 To create a new VLAN pseudo-interface:
171 ifconfig <vlan_name> create
173 To associate the VLAN pseudo-interface with a physical interface
174 and assign a VLAN ID, IP address, and netmask:
176 ifconfig <vlan_name> <ip_address> netmask <subnet_mask> vlan
177 <vlan_id> vlandev <physical_interface>
181 ifconfig vlan10 10.0.0.1 netmask 255.255.255.0 vlan 10 vlandev ixl0
183 In this example, all packets will be marked on egress with
184 802.1Q VLAN tags, specifying a VLAN ID of 10.
186 To remove a VLAN pseudo-interface:
188 ifconfig <vlan_name> destroy
194 Checksum offloading supports IPv4 and IPv6 with TCP and UDP packets
195 and is supported for both transmit and receive. Checksum offloading
196 for transmit and recieve is enabled by default for both IPv4 and IPv6.
198 Checksum offloading can be enabled or disabled using ifconfig.
199 Transmit and receive offloading for IPv4 and Ipv6 are enabled
200 and disabled seperately.
202 NOTE: TSO requires Tx checksum, so when Tx checksum
203 is disabled, TSO will also be disabled.
205 To enable Tx checksum offloading for ipv4:
207 ifconfig ixl<interface_num> txcsum4
209 To disable Tx checksum offloading for ipv4:
211 ifconfig ixl<interface_num> -txcsum4
212 (NOTE: This will disable TSO4)
214 To enable Rx checksum offloading for ipv6:
216 ifconfig ixl<interface_num> rxcsum6
218 To disable Rx checksum offloading for ipv6:
220 ifconfig ixl<interface_num> -rxcsum6
221 (NOTE: This will disable TSO6)
224 To confirm the current settings:
226 ifconfig ixl<interface_num>
232 TSO supports both IPv4 and IPv6 and is enabled by default. TSO can
233 be disabled and enabled using the ifconfig utility.
235 NOTE: TSO requires Tx checksum, so when Tx checksum is
236 disabled, TSO will also be disabled.
240 ifconfig ixl<interface_num> -tso4
244 ifconfig ixl<interface_num> tso4
248 ifconfig ixl<interface_num> -tso6
252 ifconfig ixl<interface_num> tso6
254 To disable BOTH TSO IPv4 and IPv6:
256 ifconfig ixl<interface_num> -tso
258 To enable BOTH TSO IPv4 and IPv6:
260 ifconfig ixl<interface_num> tso
266 Large Receive Offload is enabled by default. It can be enabled
267 or disabled by using the ifconfig utility.
269 NOTE: LRO should be disabled when forwarding packets.
273 ifconfig ixl<interface_num> -lro
277 ifconfig ixl<interface_num> lro
280 Flow Control (IXL only)
282 Flow control is disabled by default. To change flow control settings use sysctl.
284 To enable flow control to Rx pause frames:
286 sysctl dev.ixl.<interface_num>.fc=1
288 To enable flow control to Tx pause frames:
290 sysctl dev.ixl.<interface_num>.fc=2
292 To enable flow control to Rx and Tx pause frames:
294 sysctl dev.ixl.<interface_num>.fc=3
296 To disable flow control:
298 sysctl dev.ixl.<interface_num>.fc=0
301 NOTE: You must have a flow control capable link partner.
303 NOTE: The VF driver does not have access to flow control, it must be
304 managed from the host side.
307 Important system configuration changes:
308 =======================================
310 -Change the file /etc/sysctl.conf, and add the line:
312 hw.intr_storm_threshold: 0 (the default is 1000)
314 -Best throughput results are seen with a large MTU; use 9706 if possible.
316 -The default number of descriptors per ring is 1024, increasing this may
317 improve performance depending on the use case.
319 -The VF driver uses a relatively large buf ring, this was found to eliminate
320 UDP transmit errors, it is a tuneable, and if no UDP traffic is used it can
321 be reduced. It is memory used per queue.
327 Network Memory Buffer allocation
328 --------------------------------
329 FreeBSD may have a low number of network memory buffers (mbufs) by default.
330 If your mbuf value is too low, it may cause the driver to fail to initialize
331 and/or cause the system to become unresponsive. You can check to see if the
332 system is mbuf-starved by running 'netstat -m'. Increase the number of mbufs
333 by editing the lines below in /etc/sysctl.conf:
341 The amount of memory that you allocate is system specific, and may
342 require some trial and error.
344 Also, increasing the follwing in /etc/sysctl.conf could help increase
348 net.inet.tcp.sendspace
349 net.inet.tcp.recvspace
350 net.inet.udp.maxdgram
351 net.inet.udp.recvspace
354 UDP Stress Test Dropped Packet Issue
355 ------------------------------------
356 Under small packet UDP stress test with the ixl driver, the FreeBSD system
357 may drop UDP packets due to the fullness of socket buffers. You may want to
358 change the driver's Flow Control variables to the minimum value for
359 controlling packet reception.
362 Disable LRO when routing/bridging
363 ---------------------------------
364 LRO must be turned off when forwarding traffic.
367 Lower than expected performance
368 -------------------------------
369 Some PCIe x8 slots are actually configured as x4 slots. These slots have
370 insufficient bandwidth for full line rate with dual port and quad port
373 In addition, if you put a PCIe Generation 3-capable adapter into a PCIe
374 Generation 2 slot, you cannot get full bandwidth. The driver detects this
375 situation and writes the following message in the system log:
377 "PCI-Express bandwidth available for this card is not sufficient for
378 optimal performance. For optimal performance a x8 PCI-Express slot
381 If this error occurs, moving your adapter to a true PCIe Generation 3 x8
382 slot will resolve the issue.
388 For general information and support, go to the Intel support website at:
390 http://support.intel.com
392 If an issue is identified with the released source code on the supported kernel
393 with a supported adapter, email the specific information related to the issue
394 to freebsdnic@mailbox.intel.com.
400 This software program is released under the terms of a license agreement
401 between you ('Licensee') and Intel. Do not use or load this software or any
402 associated materials (collectively, the 'Software') until you have carefully
403 read the full terms and conditions of the LICENSE located in this software
404 package. By loadingor using the Software, you agree to the terms of this
405 Agreement. If you do not agree with the terms of this Agreement, do not
406 install or use the Software.
408 * Other names and brands may be claimed as the property of others.