1 .\" Copyright (c) 2015 EMC / Isilon Storage Division
2 .\" All rights reserved.
4 .\" Redistribution and use in source and binary forms, with or without
5 .\" modification, are permitted provided that the following conditions
7 .\" 1. Redistributions of source code must retain the above copyright
8 .\" notice, this list of conditions and the following disclaimer.
9 .\" 2. Redistributions in binary form must reproduce the above copyright
10 .\" notice, this list of conditions and the following disclaimer in the
11 .\" documentation and/or other materials provided with the distribution.
13 .\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
14 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
15 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
16 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
17 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
18 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
19 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
20 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
21 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
22 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
32 .Nd Intel I/O Acceleration Technology
34 To compile this driver into your kernel,
35 place the following line in your kernel configuration file:
36 .Bd -ragged -offset indent
40 Or, to load the driver as a module at boot, place the following line in
42 .Bd -literal -offset indent
49 .Cd hw.ioat.force_legacy_interrupts=0
55 .Cd hw.ioat.enable_ioat_test=0
56 .Cd hw.ioat.debug_level=0
57 (only critical errors; maximum of 3)
60 .Fn (*bus_dmaengine_callback_t) "void *arg" "int error"
63 .Fn ioat_get_dmaengine "uint32_t channel_index"
65 .Fn ioat_put_dmaengine "bus_dmaengine_t dmaengine"
67 .Fn ioat_get_hwversion "bus_dmaengine_t dmaengine"
69 .Fn ioat_get_max_io_size "bus_dmaengine_t dmaengine"
71 .Fn ioat_set_interrupt_coalesce "bus_dmaengine_t dmaengine" "uint16_t delay"
73 .Fn ioat_get_max_coalesce_period "bus_dmaengine_t dmaengine"
75 .Fn ioat_acquire "bus_dmaengine_t dmaengine"
77 .Fn ioat_acquire_reserve "bus_dmaengine_t dmaengine" "uint32_t n" "int mflags"
79 .Fn ioat_release "bus_dmaengine_t dmaengine"
80 .Ft struct bus_dmadesc *
82 .Fa "bus_dmaengine_t dmaengine"
86 .Fa "bus_dmaengine_callback_t callback_fn"
87 .Fa "void *callback_arg"
90 .Ft struct bus_dmadesc *
91 .Fo ioat_copy_8k_aligned
92 .Fa "bus_dmaengine_t dmaengine"
97 .Fa "bus_dmaengine_callback_t callback_fn"
98 .Fa "void *callback_arg"
101 .Ft struct bus_dmadesc *
103 .Fa "bus_dmaengine_t dmaengine"
107 .Fa "uint32_t *initialseed"
108 .Fa "bus_addr_t crcptr"
109 .Fa "bus_dmaengine_callback_t callback_fn"
110 .Fa "void *callback_arg"
113 .Ft struct bus_dmadesc *
115 .Fa "bus_dmaengine_t dmaengine"
118 .Fa "uint32_t *initialseed"
119 .Fa "bus_addr_t crcptr"
120 .Fa "bus_dmaengine_callback_t callback_fn"
121 .Fa "void *callback_arg"
124 .Ft struct bus_dmadesc *
126 .Fa "bus_dmaengine_t dmaengine"
128 .Fa "uint64_t fillpattern"
130 .Fa "bus_dmaengine_callback_t callback_fn"
131 .Fa "void *callback_arg"
134 .Ft struct bus_dmadesc *
136 .Fa "bus_dmaengine_t dmaengine"
137 .Fa "bus_dmaengine_callback_t callback_fn"
138 .Fa "void *callback_arg"
144 driver provides a kernel API to a variety of DMA engines on some Intel server
147 There is a number of DMA channels per CPU package.
149 Each may be used independently.
150 Operations on a single channel proceed sequentially.
152 Blockfill operations can be used to write a 64-bit pattern to memory.
154 Copy operations can be used to offload memory copies to the DMA engines.
156 Null operations do nothing, but may be used to test the interrupt and callback
159 All operations can optionally trigger an interrupt at completion with the
162 For example, a user might submit multiple operations to the same channel and
163 only enable an interrupt and callback for the last operation.
165 The hardware can delay and coalesce interrupts on a given channel for a
166 configurable period of time, in microseconds.
167 This may be desired to reduce the processing and interrupt overhead per
168 descriptor, especially for workflows consisting of many small operations.
169 Software can control this on a per-channel basis with the
170 .Fn ioat_set_interrupt_coalesce
173 .Fn ioat_get_max_coalesce_period
174 API can be used to determine the maximum coalescing period supported by the
175 hardware, in microseconds.
176 Current platforms support up to a 16.383 millisecond coalescing period.
177 Optimal configuration will vary by workflow and desired operation latency.
179 All operations are safe to use in a non-blocking context with the
182 (Of course, allocations may fail and operations requested with
186 Operations that depend on the result of prior operations should use
188 For example, such a scenario can happen when two related DMA operations are
190 First, a DMA copy to one location (A), followed directly by a DMA copy
192 In this scenario, some classes of I/OAT hardware may prefetch A for the second
193 operation before it is written by the first operation.
194 To avoid reading a stale value in sequences of dependent operations, use
197 All operations, as well as
198 .Fn ioat_get_dmaengine ,
199 can return NULL in special circumstances.
202 driver is being unloaded, or the administrator has induced a hardware reset, or
203 a usage error has resulted in a hardware error state that needs to be recovered
206 It is invalid to attempt to submit new DMA operations in a
207 .Fa bus_dmaengine_callback_t
210 The CRC operations have three distinct modes.
211 The default mode is to accumulate.
212 By accumulating over multiple descriptors, a user may gather a CRC over several
213 chunks of memory and only write out the result once.
217 flag causes the operation to emit the CRC32C result.
220 is set, the result is written inline with the destination data (or source in
225 is not set, the result is written to the provided
230 flag causes the operation to compare the CRC32C result to an existing checksum.
233 is set, the result is compared against the inline four bytes trailing the
235 If it is not set, the result is compared against the value pointed to by
239 calculates a CRC32C while copying data.
241 only computes a CRC32C of some data.
244 argument to either routine is non-NULL, the CRC32C engine is initialized with
245 the value it points to.
247 A typical user will lookup the DMA engine object for a given channel with
248 .Fn ioat_get_dmaengine .
249 When the user wants to offload a copy, they will first
253 object for exclusive access to enqueue operations on that channel.
254 Optionally, the user can reserve space by using
255 .Fn ioat_acquire_reserve
258 .Fn ioat_acquire_reserve
259 succeeds, there is guaranteed to be room for
261 new operations in the internal ring buffer.
263 Then, they will submit one or more operations using
266 .Fn ioat_copy_8k_aligned ,
271 After queuing one or more individual DMA operations, they will
275 to drop their exclusive access to the channel.
276 The routine they provided for the
278 argument will be invoked with the provided
280 when the operation is complete.
281 When they are finished with the
282 .Ar bus_dmaengine_t ,
284 .Fn ioat_put_dmaengine .
286 Users MUST NOT block between
290 Users SHOULD NOT hold
292 references for a very long time to enable fault recovery and kernel module
295 For an example of usage, see
296 .Pa src/sys/dev/ioat/ioat_test.c .
299 .It Pa /dev/ioat_test
308 driver first appeared in
313 driver was developed by
314 .An \&Jim Harris Aq Mt jimharris@FreeBSD.org ,
315 .An \&Carl Delsey Aq Mt carl.r.delsey@intel.com ,
317 .An \&Conrad Meyer Aq Mt cem@FreeBSD.org .
318 This manual page was written by
319 .An \&Conrad Meyer Aq Mt cem@FreeBSD.org .
321 Copy operation takes bus addresses as parameters, not virtual addresses.
323 Buffers for individual copy operations must be physically contiguous.
325 Copies larger than max transfer size (1MB, but may vary by hardware) are not
327 Future versions will likely support this by breaking up the transfer into
332 driver only supports blockfill, copy, and null operations at this time.
333 The driver does not yet support advanced DMA modes, such as XOR, that some
334 I/OAT devices support.