5fcdc19a8111 didn't fully resolve the issue. There remains a report
that an ifconfig wlan0 up by itself is insufficient. Ifconfig down
must precede it.
Reported by: Filipe da Silva Santos <contact _ shiori_com_br>
Fixes: 5fcdc19a8111
Some installations may experience CTRL-EVENT-SCAN-FAILED when
associating to an AP. Installations that specify
ifconfig_wlan0="WPA ... up" in rc.conf do not experience
the problem whereas those which specify ifconfig_wlan0="WPA" without
the "up" will experience CTRL-EVENT-SCAN_FAILED.
However those that specify "up" in ifconfig_wlan0 will be able to
reproduce this problem by service netif stop wlan0;
service netif start wlan0. Interestingly The service netif stop/start
problem is reproducible on the older wpa 2.9 as well.
Reported by: dhw
Reported by: "Oleg V. Nauman" <oleg _ theweb_org_ua>
Reported by: Filipe da Silva Santos <contact _ shiori_com_br>
Reported by: Jakob Alvermark <jakob _ alvermark_net>
RSN Preauthentication allows a station autnetnicate to an AP that
it is not associated with yet while associated with a different AP.
This allows athentication to multiple APs simulteneously.
Ed Maste [Thu, 21 Oct 2021 15:09:58 +0000 (11:09 -0400)]
iscsid: set max_recv_data_segment_length to what we advertise
Previously we updated the conection's conn_max_recv_data_segment_length
only when we received a response containing MaxRecvDataSegmentLength
from the target. If the target did not send MaxRecvDataSegmentLength
then we left conn_max_recv_data_segment_length at the default (i.e.,
8192). A target could then send more data than that defult (up to our
advertised maximum), and we would drop the connection.
RFC 7143 specifies that MaxRecvDataSegmentLength is Declarative, not
negotiated. Just set conn_max_recv_data_segment_length to our
advertised value in login_negotiate().
PR: 259355
Reviewed by: mav
MFC after: 1 week
Fixes: a15fbc904a4d ("Alike to r312190 decouple iSCSI...")
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32605
Mark Johnston [Thu, 21 Oct 2021 15:46:25 +0000 (11:46 -0400)]
vm_page: Break reservations to handle noobj allocations
vm_reserv_reclaim_*() will release pages to the default freepool, not
the direct freepool from which noobj allocations are drawn. But if both
pools are empty, the noobj allocator variants must break reservations to
make progress.
Reported by: cy
Reviewed by: kib (previous version)
Fixes: b498f71bc56a ("vm_page: Add a new page allocator interface for unnamed pages")
Sponsored by: The FreeBSD Foundation
Mark Johnston [Wed, 20 Oct 2021 00:24:21 +0000 (20:24 -0400)]
Introduce vm_page_alloc_noobj_contig()
This is the same as vm_page_alloc_noobj(), but allocates physically
contiguous runs of memory. For now it is implemented in terms of
vm_page_alloc_contig(), with the difference that
vm_page_alloc_noobj_contig() implements VM_ALLOC_ZERO by zeroing the
page.
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Mark Johnston [Wed, 20 Oct 2021 00:23:39 +0000 (20:23 -0400)]
Convert vm_page_alloc() callers to use vm_page_alloc_noobj().
Remove page zeroing code from consumers and stop specifying
VM_ALLOC_NOOBJ. In a few places, also convert an allocation loop to
simply use VM_ALLOC_WAITOK.
Mark Johnston [Wed, 20 Oct 2021 00:22:12 +0000 (20:22 -0400)]
vm_page: Add a new page allocator interface for unnamed pages
The diff adds vm_page_alloc_noobj() and vm_page_alloc_noobj_domain().
These mostly correspond to vm_page_alloc() and vm_page_alloc_domain()
when no VM object is specified, with the exception that they handle
VM_ALLOC_ZERO by zeroing the page, rather than by preserving PG_ZERO.
This simplifies callers and will permit simplification of the
vm_page_alloc_domain() definition.
Since the new allocator variant is similar to vm_page_alloc_freelist(),
implement both of them using a common backend allocator function. No
functional change intended.
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Ryan Stone [Fri, 29 Jan 2021 21:13:57 +0000 (16:13 -0500)]
Add a VM flag to prevent reclaim on a failed contig allocation
If a M_WAITOK contig alloc fails, the VM subsystem will try to
reclaim contiguous memory twice before actually failing the
request. On a system with 64GB of RAM I've observed this take
400-500ms before it finally gives up, and I believe that this
will only be worse on systems with even more memory.
In certain contexts this delay is extremely harmful, so add a flag
that will skip reclaim for allocation requests to allow those
paths to opt-out of doing an expensive reclaim.
Mark Johnston [Wed, 20 Oct 2021 00:50:06 +0000 (20:50 -0400)]
vlapic: Schedule callouts on the local CPU
The virtual LAPIC driver uses callouts to implement the LAPIC timer.
Callouts are armed using callout_reset_sbt(), which currently puts
everything on CPU 0. On systems running many bhyve VMs this results in
a large amount of contention for CPU 0's callout lock.
Modify vlapic to schedule callouts on the local CPU instead. This
allows timer interrupts to be scheduled more evenly among CPUs where
bhyve is running.
Reviewed by: grehan, jhb
Sponsored by: The FreeBSD Foundation
Mark Johnston [Tue, 10 Aug 2021 20:25:39 +0000 (16:25 -0400)]
amd64: Define KVA regions for KMSAN shadow maps
KMSAN requires two shadow maps, each one-to-one with the kernel map.
Allocate regions of the kernels PML4 page for them. Add functions to
create mappings in the shadow map regions, these will be used by the
KMSAN runtime.
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
usb(4): Fix for use after free in combination with EVDEV_SUPPORT.
When EVDEV_SUPPORT was introduced, the USB transfers may be running
after the main FIFO is closed. In connection to this a race may appear
which can lead to use-after-free scenarios. Fix this for all FIFO
consumers by initializing and resetting the FIFO queues under the
lock used by the client. Then the client driver will see an empty
queue in all cases a race may appear.
Alexander Motin [Sun, 3 Oct 2021 00:57:55 +0000 (20:57 -0400)]
sleepqueue(9): Remove sbinuptime() from sleepq_timeout().
Callout c_time is always bigger or equal than the scheduled time. It
is also smaller than sbinuptime() and can't change while the callback
is running. So we reliably can use it instead of sbinuptime() here.
In case there was a race and the callout was rescheduled to the later
time, the callback will be called again.
According to profiles it saves ~5% of the timer interrupt time even
with fast TSC timecounter.
Mark Johnston [Wed, 24 Mar 2021 23:43:05 +0000 (19:43 -0400)]
Generalize sanitizer interceptors for memory and string routines
Similar to commit 3ead60236f ("Generalize bus_space(9) and atomic(9)
sanitizer interceptors"), use a more generic scheme for interposing
sanitizer implementations of routines like memcpy().
Mark Johnston [Tue, 23 Mar 2021 01:44:55 +0000 (21:44 -0400)]
Generalize bus_space(9) and atomic(9) sanitizer interceptors
Make it easy to define interceptors for new sanitizer runtimes, rather
than assuming KCSAN. Lay a bit of groundwork for KASAN and KMSAN.
When a sanitizer is compiled in, atomic(9) and bus_space(9) definitions
in atomic_san.h are used by default instead of the inline
implementations in the platform's atomic.h. These definitions are
implemented in the sanitizer runtime, which includes
machine/{atomic,bus}.h with SAN_RUNTIME defined to pull in the actual
implementations.
Mark Johnston [Fri, 23 Jul 2021 14:41:00 +0000 (10:41 -0400)]
KASAN: Disable checking before triggering a panic
KASAN hooks will not generate reports if panicstr != NULL, but then
there is a window after the initial panic() call where another report
may be raised. This can happen if a false positive occurs; to simplify
debugging of such problems, avoid recursing.
Mark Johnston [Fri, 23 Jul 2021 14:30:29 +0000 (10:30 -0400)]
redzone: Raise a compile error if KASAN is configured
redzone(9) does some munging of the allocation to insert redzones before
and after a valid memory buffer, but KASAN does not know about this and
will raise false positives if both are configured. Until this is fixed,
do not allow both to be configured. Note that KASAN provides similar
checking on its own but currently does not force the creation of
redzones for all UMA allocations; this should be addressed as well.
Mark Johnston [Sat, 10 Jul 2021 00:38:28 +0000 (20:38 -0400)]
KASAN: Implement __asan_unregister_globals()
It will be called during KLD unload to unpoison the redzones following
global variables. Otherwise, virtual address ranges previously used for
a KLD may be left tainted, triggering false positives when they are
recycled.
Reported by: pho
Sponsored by: The FreeBSD Foundation
Mark Johnston [Sat, 10 Jul 2021 00:38:21 +0000 (20:38 -0400)]
uma: Fix a few problems with KASAN integration
- Ensure that all items returned by UMA are aligned to
KASAN_SHADOW_SCALE (8). This was true in practice since smaller
alignments are not used by any consumers, but we should enforce it
anyway.
- Use a non-zero code for marking redzones that appear naturally in
items that are not a multiple of the scale factor in size. Currently
we do not modify keg layouts to force the creation of redzones.
- Use a non-zero code for marking freed per-CPU items, otherwise
accesses of freed per-CPU items are not detected by the runtime.
Mark Johnston [Fri, 7 May 2021 18:24:37 +0000 (14:24 -0400)]
linker_set: Disable ASAN only in userspace
KASAN does not insert redzones around global variables and so is not
susceptible to the problem that led to us disabling ASAN for linker set
elements in the first place (see commit fe3d8086fb6f).
Reviewed by: andrew, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30126
Mark Johnston [Wed, 5 May 2021 21:05:46 +0000 (17:05 -0400)]
realloc: Fix KASAN(9) shadow map updates
When copying from the old buffer to the new buffer, we don't know the
requested size of the old allocation, but only the size of the
allocation provided by UMA. This value is "alloc". Because the copy
may access bytes in the old allocation's red zone, we must mark the full
allocation valid in the shadow map. Do so using the correct size.
Reported by: kp
Tested by: kp
Sponsored by: The FreeBSD Foundation
Mark Johnston [Tue, 13 Apr 2021 21:40:27 +0000 (17:40 -0400)]
malloc: Add state transitions for KASAN
- Reuse some REDZONE bits to keep track of the requested and allocated
sizes, and use that to provide red zones.
- As in UMA, disable memory trashing to avoid unnecessary CPU overhead.
Mark Johnston [Tue, 13 Apr 2021 21:40:11 +0000 (17:40 -0400)]
vfs: Add KASAN state transitions for vnodes
vnodes are a bit special in that they may exist on per-CPU lists even
while free. Add a KASAN-only destructor that poisons regions of each
vnode that are not expected to be accessed after a free.
Mark Johnston [Tue, 13 Apr 2021 21:40:01 +0000 (17:40 -0400)]
kmem: Add KASAN state transitions
Memory allocated with kmem_* is unmapped upon free, so KASAN doesn't
provide a lot of benefit, but since allocations are always a multiple of
the page size we can create a redzone when the allocation request size
is not a multiple of the page size.
Mark Johnston [Tue, 13 Apr 2021 21:39:50 +0000 (17:39 -0400)]
uma: Add KASAN state transitions
- Add a UMA_ZONE_NOKASAN flag to indicate that items from a particular
zone should not be sanitized. This is applied implicitly for NOFREE
and cache zones.
- Add KASAN call backs which get invoked:
1) when a slab is imported into a keg
2) when an item is allocated from a zone
3) when an item is freed to a zone
4) when a slab is freed back to the VM
In state transitions 1 and 3, memory is poisoned so that accesses will
trigger a panic. In state transitions 2 and 4, memory is marked
valid.
- Disable trashing if KASAN is enabled. It just adds extra CPU overhead
to catch problems that are detected by KASAN.
Mark Johnston [Tue, 13 Apr 2021 21:39:35 +0000 (17:39 -0400)]
amd64: Add MD bits for KASAN
- Initialize KASAN before executing SYSINITs.
- Add a GENERIC-KASAN kernel config, akin to GENERIC-KCSAN.
- Increase the kernel stack size if KASAN is enabled. Some of the
ASAN instrumentation increases stack usage and it's enough to
trigger stack overflows in ZFS.
- Mark the trapframe as valid in interrupt handlers if it is
assigned to td_intr_frame. Otherwise, an interrupt in a function
which creates a poisoned alloca region can trigger false positives.
Mark Johnston [Tue, 13 Apr 2021 20:30:05 +0000 (16:30 -0400)]
amd64: Implement a KASAN shadow map
The idea behind KASAN is to use a region of memory to track the validity
of buffers in the kernel map. This region is the shadow map. The
compiler inserts calls to the KASAN runtime for every emitted load
and store, and the runtime uses the shadow map to decide whether the
access is valid. Various kernel allocators call kasan_mark() to update
the shadow map.
Since the shadow map tracks only accesses to the kernel map, accesses to
other kernel maps are not validated by KASAN. UMA_MD_SMALL_ALLOC is
disabled when KASAN is configured to reduce usage of the direct map.
Currently we have no mechanism to completely eliminate uses of the
direct map, so KASAN's coverage is not comprehensive.
The shadow map uses one byte per eight bytes in the kernel map. In
pmap_bootstrap() we create an initial set of page tables for the kernel
and preloaded data.
When pmap_growkernel() is called, we call kasan_shadow_map() to extend
the shadow map. kasan_shadow_map() uses pmap_kasan_enter() to allocate
memory for the shadow region and map it.
Reviewed by: kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29417
Mark Johnston [Tue, 13 Apr 2021 21:39:19 +0000 (17:39 -0400)]
Add the KASAN runtime
KASAN enables the use of LLVM's AddressSanitizer in the kernel. This
feature makes use of compiler instrumentation to validate memory
accesses in the kernel and detect several types of bugs, including
use-after-frees and out-of-bounds accesses. It is particularly
effective when combined with test suites or syzkaller. KASAN has high
CPU and memory usage overhead and so is not suited for production
environments.
The runtime and pmap maintain a shadow of the kernel map to store
information about the validity of memory mapped at a given kernel
address.
The runtime implements a number of functions defined by the compiler
ABI. These are prefixed by __asan. The compiler emits calls to
__asan_load*() and __asan_store*() around memory accesses, and the
runtime consults the shadow map to determine whether a given access is
valid.
kasan_mark() is called by various kernel allocators to update state in
the shadow map. Updates to those allocators will come in subsequent
commits.
The runtime also defines various interceptors. Some low-level routines
are implemented in assembly and are thus not amenable to compiler
instrumentation. To handle this, the runtime implements these routines
on behalf of the rest of the kernel. The sanitizer implementation
validates memory accesses manually before handing off to the real
implementation.
The sanitizer in a KASAN-configured kernel can be disabled by setting
the loader tunable debug.kasan.disable=1.
Obtained from: NetBSD
Sponsored by: The FreeBSD Foundation
Mark Johnston [Sat, 16 Oct 2021 13:46:55 +0000 (09:46 -0400)]
timecounter: Lock the timecounter list
Timecounter registration is dynamic, i.e., there is no requirement that
timecounters must be registered during single-threaded boot. Loadable
drivers may in principle register timecounters (which can be switched to
automatically). Timecounters cannot be unregistered, though this could
be implemented.
Registered timecounters belong to a global linked list. Add a mutex to
synchronize insertions and the traversals done by (mpsafe) sysctl
handlers. No functional change intended.
Reviewed by: imp, kib
Sponsored by: The FreeBSD Foundation
Mark Johnston [Tue, 21 Sep 2021 15:32:23 +0000 (11:32 -0400)]
bitset(9): Introduce BIT_FOREACH_ISSET and BIT_FOREACH_ISCLR
These allow one to non-destructively iterate over the set or clear bits
in a bitset. The motivation is that we have several code fragments
which iterate over a CPU set like this:
This is slow since CPU_FFS begins the search at the beginning of the
bitset each time. On amd64 and arm64, CPU sets have size 256, so there
are four limbs in the bitset and we do a lot of unnecessary scanning.
A second problem is that this is destructive, so code which needs to
preserve the original set has to make a copy. In particular, we have
quite a few functions which take a cpuset_t parameter by value, meaning
that each call has to copy the 32 byte cpuset_t.
The new macros address both problems.
Reviewed by: cem, kib
Sponsored by: The FreeBSD Foundation
Mark Johnston [Sat, 16 Oct 2021 13:44:40 +0000 (09:44 -0400)]
signal: Add SIG_FOREACH and refactor issignal()
Add a SIG_FOREACH macro that can be used to iterate over a signal set.
This is a bit cleaner and more efficient than calling sig_ffs() in a
loop. The implementation is based on BIT_FOREACH_ISSET(), except
that the bitset limbs are always 32 bits wide, and signal sets are
1-indexed rather than 0-indexed like bitset(9) sets.
issignal() cannot really be modified to use SIG_FOREACH() directly.
Take this opportunity to split the function into two explicit loops.
I've always found this function hard to read and think that this change
is an improvement.
Remove sig_ffs(), nothing uses it now.
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Mark Johnston [Mon, 25 Oct 2021 17:08:38 +0000 (13:08 -0400)]
hyperv: Register hyperv_timecounter later during boot
Previously the MSR-based timecounter was registered during
SI_SUB_HYPERVISOR, i.e., very early during boot, and before SI_SUB_LOCK.
After commit 621fd9dcb2d8 this triggers a panic since the timecounter
list lock is not yet initialized.
The hyperv timecounter does not need to be registered so early, so defer
that to SI_SUB_DRIVERS, at the same time the hyperv TSC timecounter is
registered.
Reported by: whu
Approved by: whu
Fixes: 621fd9dcb2d8 ("timecounter: Lock the timecounter list")
Sponsored by: The FreeBSD Foundation
Rick Macklem [Mon, 18 Oct 2021 22:02:21 +0000 (15:02 -0700)]
nfscl: Handle NFSv4.1/4.2 Close RPC NFSERR_DELAY replies better
Without this patch, if a NFSv4.1/4.2 server replies NFSERR_DELAY to
a Close operation, the client loops retrying the Close while holding
a shared lock on the clientID. This shared lock blocks returns of
delegations, even though the server has issued a CB_RECALL to request
the delegation return.
This patch delays doing a retry of a Close that received a reply of
NFSERR_DELAY until after the shared lock on the clientID is released,
for NFSv4.1/4.2. To fix this for NFSv4.0 would be very difficult and
since the only known NFSv4 server to reply NFSERR_DELAY to Close only
does NFSv4.1/4.2, this fix is hoped to be sufficient.
This problem was detected during a recent IETF working group NFSv4
testing event.
Rick Macklem [Mon, 18 Oct 2021 00:50:56 +0000 (17:50 -0700)]
nfscl: Modify Close RPC so that it does not use "owner" for NFSv4.1/4.2
This patch modifies the function that does the Close RPC (nfsrpc_closerpc)
so that it does not use the open_owner (nfso_own) for NFSv4.1/4.2.
Use of the seqid in the open_owner structure is only needed for NFSv4.0.
Same applies to a NFSERR_STALESTATEID reply, which should only happen
for NFSv4.0. This allows nfsrpc_closerpc() to be called when nfso_own
is no longer valid. This, in turn, allows nfsrpc_closerpc() to be called
after the shared lock on the clientID is released, for NFSv4.1/4.2.
This is being done to prepare the code for a future patch that fixes
the case where an NFSv4.1/4.2 server replies NFSERR_DELAY to a Close
operation.
Felix Johnson [Thu, 28 Oct 2021 18:15:08 +0000 (14:15 -0400)]
config(5): Update upper limit for maxusers on 64-bit systems
The limit of 384 maxusers for auto configuration was only imposed on
32-bit systems. Document that maxusers scales above 384 based on memory
for 64-bit systems.
PR: 204938
Reported by: David Höppner <0xffea@gmail.com>
Ka Ho Ng [Tue, 26 Oct 2021 14:48:57 +0000 (22:48 +0800)]
sh: Set PATH envvar after setting HOME in dotfile
In single-user mode, all env vars are absent, so exptilde() would not be
able to expand ~ correctly.
Place the lines setting PATH below HOME, so exptilde() would work as
expected.
Sponsored by: The FreeBSD Foundation
Reviewed by: jilles, emaste
Differential Revision: https://reviews.freebsd.org/D27003
Mark Johnston [Sat, 16 Oct 2021 17:13:26 +0000 (13:13 -0400)]
bhyve: Fix the WITH_BHYVE_SNAPSHOT build
Note, this breaks compatibility with snapshots generated by older builds
of bhyve(8).
Fixes: 7fa233534736 ("bhyve: Map the MSI-X table unconditionally for passthrough")
Reported by: Greg V <greg@unrelenting.technology>
Reviewed by: grehan, bz
Sponsored by: The FreeBSD Foundation
Mark Johnston [Sat, 9 Oct 2021 15:36:19 +0000 (11:36 -0400)]
bhyve: Map the MSI-X table unconditionally for passthrough
It is possible for the PBA to reside in the same page as the MSI-X
table. And, while devices are not supposed to do this, at least some
Intel wifi devices place registers in a page shared with the MSI-X
table. To handle the first case we currently map the PBA page using
/dev/mem, and the second case is not handled.
Kill two birds with one stone: map the MSI-X table BAR using the
PCIOCBARMMAP ioctl instead of /dev/mem, and map the entire table so that
accesses beyond the bounds of the table can be emulated. Regions of the
BAR not containing the table are left unmapped.
Reviewed by: bz, grehan, jhb
Sponsored by: The FreeBSD Foundation
Rick Macklem [Sat, 16 Oct 2021 22:49:38 +0000 (15:49 -0700)]
nfscl: Move release of the clientID lock into nfscl_doclose()
This patch moves release of the shared clientID lock from nfsrpc_close()
just after the nfscl_doclose() call to the end of nfscl_doclose() call.
This does make the code cleaner, since the shared lock is acquired at
the beginning of nfscl_doclose(). The only semantics change is that
the code no longer drops and reaquires the NFSCLSTATELOCK() mutex,
which I do not believe will have a negative effect on the NFSv4 client.
This is being done to prepare the code for a future patch that fixes
the case where an NFSv4.1/4.2 server replies NFSERR_DELAY to a Close
operation.
John Baldwin [Wed, 15 Sep 2021 20:25:30 +0000 (13:25 -0700)]
iscsi: Abort data-out tasks queued on a terminating session.
cfiscsi_datamove_out() can race with cfiscsi_session_terminate_tasks()
and enqueue a new task after the latter function has aborted existing
tasks. This could result in a deadlock as
cfiscsi_session_terminate_tasks() waited forever for this task to
complete.
John Baldwin [Fri, 10 Sep 2021 22:10:00 +0000 (15:10 -0700)]
cxgbei: Handle errors in PDUs.
When a PDU with an error (bad padding, header digest, or data digest)
is received, log the error via ICL_WARN() and then reset the
connection via the ic_error callback.
John Baldwin [Mon, 30 Aug 2021 22:55:40 +0000 (15:55 -0700)]
cxgbei: Add sysctls to report the maximum data segment lengths.
These sysctls report the maximum data segment lengths supported by an
adapter. These are the values advertised to the remote end during the
login phase.
John Baldwin [Wed, 18 Aug 2021 17:56:28 +0000 (10:56 -0700)]
iscsi: Teach the iSCSI stack about "large" received PDUs.
When using iSCSI PDU offload (cxgbei) on T6 adapters, a burst of
received PDUs can be reported via a single message to the driver.
Previously the driver passed these multi-PDU bursts up to the iSCSI
stack up as a single "large" PDU by rewriting the buffer offset, data
segment length, and DataSN fields in the iSCSI header. The DataSN
field in particular was rewritten so that each of the "large" PDUs
used consecutively increasing values. While this worked, the forged
DataSN values did not match the ExpDataSN value in the subsequent SCSI
Response PDU. The initiator does not currently verify this value, but
the forged DataSN values prevent adding a check.
To avoid this, allow a logical iSCSI PDU (struct icl_pdu) to describe
a burst of PDUs via a new 'ip_additional_pdus' field. Normally this
field is set to zero when 'struct icl_pdu' represents a single PDU.
If logical PDU represents a burst of on-the-wire PDUs, then 'ip_npdus'
contains the count of additional on-the-wire PDUs. The header of this
"large" PDU is still modified, but the DataSN field now contains the
DataSN value of the first on-the-wire PDU in the burst.
John Baldwin [Thu, 12 Aug 2021 15:48:14 +0000 (08:48 -0700)]
cxgbei: Wait for the final CPL to be received in icl_cxgbei_conn_close.
A socket in the FIN_WAIT_1 state is marked disconnected by
do_close_con_rpl() even though there might still receive data pending.
This is because the socket at that point has set SBS_CANTRCVMORE which
causes the protocol layer to discard any data received before the FIN.
However, icl_cxgbei_conn_close needs to wait until all the data has
been discarded. Replace the wait for SS_ISDISCONNECTED with instead
waiting for final_cpl_received() to be called.
John Baldwin [Fri, 6 Aug 2021 21:03:00 +0000 (14:03 -0700)]
iSCSI: Add support for segmentation offload for hardware offloads.
Similar to TSO, iSCSI segmentation offload permits the upper layers to
submit a "large" virtual PDU which is split up into multiple segments
(PDUs) on the wire. Similar to how the TCP/IP headers are used as
templates for TSO, the BHS at the start of a large PDU is used as a
template to construct the specific BHS at the start of each PDU. In
particular, the DataSN is incremented for each subsequent PDU, and the
'F' flag is only set on the last PDU.
struct icl_conn has a new 'ic_hw_isomax' field which defaults to 0,
but can be set to the largest virtual PDU a backend supports. If this
value is non-zero, the iSCSI target and initiator use this size
instead of 'ic_max_send_data_segment_length' to determine the maximum
size for SCSI Data-In and SCSI Data-Out PDUs. Note that since PDUs
can be constructed from multiple buffers before being dispatched, the
target and initiator must wait for the PDU to be fully constructed
before determining the number of DataSN values were consumed (and thus
updating the per-transfer DataSN value used for the start of the next
PDU).
The target generates large PDUs for SCSI Data-In PDUs in
cfiscsi_datamove_in(). The initiator generates large PDUs for SCSI
Data-Out PDUs generated in response to an R2T.
projects / FreeBSD / FreeBSD.git / log