Ed Maste [Sun, 2 May 2021 19:28:36 +0000 (15:28 -0400)]
Restore Cirrus-CI boot smoke test
This reverts commit a7d593dd1da27833b5384349700bc3c7bcae6aad.
We now use compute_engine_instance which allows us to specify a custom
disk size. Also go back to using the default qemu version (rather than
qemu42 or qemu-devel) as any issues were fixed some time ago.
Reviewed by: lwhsu, markj
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30082
Marko Zec [Wed, 5 May 2021 11:45:52 +0000 (13:45 +0200)]
Introduce DXR as an IPv4 longest prefix matching / FIB module
DXR maintains compressed lookup structures with a trivial search
procedure. A two-stage trie is indexed by the more significant bits of
the search key (IPv4 address), while the remaining bits are used for
finding the next hop in a sorted array. The tradeoff between memory
footprint and search speed depends on the split between the trie and
the remaining binary search. The default of 20 bits of the key being
used for trie indexing yields good performance (see below) with
footprints of around 2.5 Bytes per prefix with current BGP snapshots.
Rebuilding lookup structures takes some time, which is compensated for by
batching several RIB change requests into a single FIB update, i.e. FIB
synchronization with the RIB may be delayed for a fraction of a second.
RIB to FIB synchronization, next-hop table housekeeping, and lockless
lookup capability is provided by the FIB_ALGO infrastructure.
DXR works well on modern CPUs with several MBytes of caches, especially
in VMs, where is outperforms other currently available IPv4 FIB
algorithms by a large margin.
Synthetic single-thread LPM throughput test method:
Marko Zec [Wed, 5 May 2021 10:28:17 +0000 (12:28 +0200)]
Revise FIB lookups per second benchmarking routines.
Add a LPS benchmark variant which introduces artificial dependencies
between successive lookups. While here, instead of writing the results
from the lookups to a huge array, add them to an accumulator, in a more
lightweight attempt at preventing the CPU's OOO machinery from
discarding the lookup results if they would be completely unused.
net.route.test.run_lps_rnd measures LPS throughput with independent
uniformly random keys
net.route.test.run_lps_seq measures LPS throughput with uniformly
random keys with artificial interdependencies
Reviewed by: melifaro
MFC after: 7 days
Differential Revision: https://reviews.freebsd.org/D30096
Warner Losh [Thu, 22 Apr 2021 16:44:15 +0000 (10:44 -0600)]
param.h: Document __FreeBSD_version better
Document what __FreeBSD_version means a bit better by documenting the
sorts of events it should be bumped for. Also include a handy shorthand
for what it means. Add a some advice for how frequently to change this
as well.
Added a note about the approved way to parse this from the param.h file,
though that was not in the review. All in-tree users have been updated
to this method prior to this commit. Move and reword the comment that
was on the same line.
Navdeep Parhar [Sat, 1 May 2021 23:53:50 +0000 (16:53 -0700)]
cxgbe(4): Use ifaddr_event_ext instead of ifaddr_event for CLIP management.
The _ext event notification includes the address being added/removed and
that gives the driver an easy way to ignore non-IPv6 addresses. Remove
'tom' from the handler's name while here, it was moved out of t4_tom a
long time ago.
netgraph/ng_bridge: learn MACs via control message
Add a new control message to move ethernet addresses to a given link
in ng_bridge(4). Send this message instead of doing the work directly.
This decouples the read-only activity from the modification under a
more strict writer lock.
Decoupling the work is a prerequisite for multithreaded operation.
Warner Losh [Tue, 4 May 2021 19:49:27 +0000 (13:49 -0600)]
ttys: diff reduction
Back in the day, we used to have a number of entries that were either
'on' or 'off' directly, rather than conditionally on 'onifexists'
etc. Back then, we tried to line up the 'secure' columns by using the
construct 'on secure' or 'off secure' (one space or two). Now that these
have all moved to a conditional construct, remove the second space on
the ttys that still have it. This reduces diffs between the different
ttys and is no functional change. i386 and amd64 did this a long time
ago, and those are the only ones that affected external users (who used
to sed 's/on /off /' the entires as part of the automation).
Mark Johnston [Tue, 4 May 2021 12:53:57 +0000 (08:53 -0400)]
nfsclient: Copy only initialized fields in nfs_getattr()
When loading attributes from the cache, the NFS client is careful to
copy only the fields that it initialized. After fetching attributes
from the server, however, it would copy the entire vattr structure
initialized from the RPC response, so uninitialized stack bytes would
end up being copied to userspace. In particular, va_birthtime (v2 and
v3) and va_gen (v3) had this problem.
Use a common subroutine to copy fields provided by the NFS client, and
ensure that we provide a dummy va_gen for the v3 case.
Reviewed by: rmacklem
Reported by: KMSAN
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30090
AIM (adaptive interrupt moderation) was part of BSD11 driver. Upon IFLIB
migration, AIM feature got lost. Re-introducing AIM back into IFLIB
based IXGBE driver.
One caveat is that in BSD11 driver, a queue comprises both Rx and Tx
ring. Starting from BSD12, Rx and Tx have their own queues and rings.
Also, IRQ is now only configured for Rx side. So, when AIM is
re-enabled, we should now consider only Rx stats for configuring EITR
register in contrast to BSD11 where Rx and Tx stats were considered to
manipulate EITR register.
Mark Johnston [Mon, 3 May 2021 16:51:04 +0000 (12:51 -0400)]
Add missing sockaddr length and family validation to various protocols
Several protocol methods take a sockaddr as input. In some cases the
sockaddr lengths were not being validated, or were validated after some
out-of-bounds accesses could occur. Add requisite checking to various
protocol entry points, and convert some existing checks to assertions
where appropriate.
Reported by: syzkaller+KASAN
Reviewed by: tuexen, melifaro
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29519
Zhenlei Huang [Mon, 3 May 2021 16:46:19 +0000 (12:46 -0400)]
traceroute6: Properly calculate UDP checksum
The revision D25604 capsicumize traceroute6. For UDP the send socket was
changed from SOCK_DGRAM to SOCK_RAW and thus the UDP checksum need be
calculated by application itself other than the kernel.
outpacket is filled with zeros by line 707, thus the first round the UDP
checksum is correct. But subsequent rounds outudp->uh_sum will be left
with garbage.
Mark Johnston [Mon, 3 May 2021 16:43:29 +0000 (12:43 -0400)]
devfs: Avoid comparison with an uninitialized var in devfs_fp_check()
devvn_refthread() will initialize *devp only if it succeeds, so check for
success before comparing with fp->f_data. Other devvn_refthread()
callers are careful to do this.
Reported by: KMSAN
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30068
filt_timerexpire: do not iterate over the interval
User-supplied data might make this loop too time-consuming. Divide
directly, and handle both the possibility that we were woken up earlier,
and arithmetic overflows/underflows from the calculation.
Reported and tested by: pho (previous version)
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D30069
Set a new P2_PTRACEREQ flag around the request Wait for the target .
process P2_PTRACEREQ flag to clear before setting ours .
Otherwise, we rely on the moment that the process lock is not dropped
until the stopped target state is important. This is going to be no
longer true after some future change.
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29955
It unsuspends single suspended thread, passed as the argument.
It is up to the caller to arrange the target thread to suspend later,
since the state of the process is not changed from stopped. In particular,
the unsuspended thread must not leave to userspace, since boundary code
is not prepared to this situation.
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29955
The helper removes the thread from a sleep queue, assuming that it would
need to sleep. The sleepq_remove_nested() function is intended for quite
special case, where suspended thread from traced stopped process is
temporary unsuspended to do some work on behalf of the debugger in the
target context, and this work might require sleep.
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29955
ELF coredump: define several useful flags for the coredump operations
- SVC_ALL request dumping all map entries, including those marked as
non-dumpable
- SVC_NOCOMPRESS disallows compressing the dump regardless of the sysctl
policy
- SVC_PC_COREDUMP is provided for future use by userspace core dump
request
Reviewed by: markj
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D29955
From my understanding this could happen with iSCSI LUNs with
unusually long names. The bug would make CAM fail to retrieve
the full inquiry data. Instead of bumping the size of the local
variable, just use a macro.
Reviewed By: imp, mav
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
X-NetApp-PR: #50
Differential Revision: https://reviews.freebsd.org/D29991
NB (Intel Document Number 336256-004US):
1. The E3900 and A3900 Series Processors support four LPSS_UART ports,
while the N- and J- Series Processors support only LPSS_UART [2:1]
ports.
2. The LPSS_UART1 port is dedicated for discrete Global Navigation
Satellite System (GNSS). This port can be used for generic UART
functionality if GNSS is not used.
3. The LPSS_UART2 port is dedicated for host OS debug.
4. The LPSS_UART0 and LPSS_UART3 ports are for generic UART functionality.
5. Only UART [1:0] ports support DMA.
PR: 255556
Submitted by: Jose Luis Duran <jlduran@gmail.com>
MFC after: 1 week
Alexander Motin [Sun, 2 May 2021 23:35:28 +0000 (19:35 -0400)]
Improve UMA cache reclamation.
When estimating working set size, measure only allocation batches, not free
batches. Allocation and free patterns can be very different. For example,
ZFS on vm_lowmem event can free to UMA few gigabytes of memory in one call,
but it does not mean it will request the same amount back that fast too, in
fact it won't.
Update working set size on every reclamation call, shrinking caches faster
under pressure. Lack of this caused repeating vm_lowmem events squeezing
more and more memory out of real consumers only to make it stuck in UMA
caches. I saw ZFS drop ARC size in half before previous algorithm after
periodic WSS update decided to reclaim UMA caches.
Introduce voluntary reclamation of UMA caches not used for a long time. For
each zdom track longterm minimal cache size watermark, freeing some unused
items every UMA_TIMEOUT after first 15 minutes without cache misses. Freed
memory can get better use by other consumers. For example, ZFS won't grow
its ARC unless it see free memory, since it does not know it is not really
used. And even if memory is not really needed, periodic free during
inactivity periods should reduce its fragmentation.
Reviewed by: markj, jeff (previous version)
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D29790
Michael Tuexen [Sun, 2 May 2021 20:38:27 +0000 (22:38 +0200)]
sctp: improve error handling in INIT/INIT-ACK processing
When processing INIT and INIT-ACK information, also during
COOKIE processing, delete the current association, when it
would end up in an inconsistent state.
Rick Macklem [Sun, 2 May 2021 23:04:27 +0000 (16:04 -0700)]
copy_file_range(2): improve copying of a large hole to EOF
PR#255523 reported that a file copy for a file with a large hole
to EOF on ZFS ran slowly over NFSv4.2.
The problem was that vn_generic_copy_file_range() would
loop around reading the hole's data and then see it is all
0s. It was coded this way since UFS always allocates a data
block near the end of the file, such that a hole to EOF never exists.
This patch modifies vn_generic_copy_file_range() to check for a
ENXIO returned from VOP_IOCTL(..FIOSEEKDATA..) and handle that
case as a hole to EOF. asomers@ confirms that it works for his
ZFS test case.
sbin/ipfw: Fix null pointer deference when printing counters
ipfw -[tT] prints statistics of the last access. If the rule was never
used, the counter might be not exist. This happens unconditionally on
inserting a new rule. Avoid printing statistics in this case.
Andrew Turner [Sun, 2 May 2021 07:43:34 +0000 (07:43 +0000)]
Enable IPIs on CPU 0 on arm and arm64
Not all interrupt controllers enable IPIs by default as the Arm
GIC specs make it an implementation defined option. As at least two
hypervisors have also previously masked the IPIs on boot.
As we already enable these IPIs on the non-boot CPUs it is expected
this is a safe operation.
Andrew Turner [Thu, 8 Apr 2021 11:54:20 +0000 (11:54 +0000)]
Add a way to map arm64 non-posted device memory
On arm64 we currently use a non-posted write for device memory, however
we should move to use posted writes. This is expected to work on most
hardware, however we will need to support a non-posted option for some
broken hardware.
Justin Hibbits [Sat, 1 May 2021 00:58:11 +0000 (19:58 -0500)]
powerpc64: Optimize radix trap handling a little more
Summary:
Since PCPU can live in a GPR for a while longer, let it, rather than
re-getting it in yet another register. MFSPR is an expensive operation,
12 clock latency on POWER9, so the fewer operations we need, the better.
Since the check is tightly coupled to the fetch, by reducing the number
of fetch+check, we reduce the stalls, and improve the performance
marginally. Buildworld was measured at a ~5-7% improvement on a single
run.
Marcin Wojtas [Thu, 29 Apr 2021 09:39:09 +0000 (11:39 +0200)]
gicv3_its: Flush cache after allocating ITT memory
It has to be zeroed before committing it to device.
We do that by allocating it with M_ZERO, but there was no
memory barrier or cache flush to ensure its sees it zeroed.
This fixes MSIX on LS1028A SoC.
Submitted by: Kornel Duleba <mindal@semihalf.com>
Reviewed by: andrew
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D30033
Eric van Gyzen [Mon, 26 Apr 2021 15:01:17 +0000 (10:01 -0500)]
Wait longer for a previous IPI to be sent
When sending an IPI, if a previous IPI is still pending delivery,
native_lapic_ipi_vectored() waits for the previous IPI to be sent.
We've seen a few inexplicable panics with the current timeout of 50 ms.
Increase the timeout to 1 second and make it tunable.
No hardware specification mentions a timeout in this case; I checked
the Intel SDM, Intel MP spec, and Intel x2APIC spec. Linux and illumos
wait forever. In Linux, see __default_send_IPI_shortcut() in
arch/x86/kernel/apic/ipi.c. In illumos, see apic_send_ipi() in
usr/src/uts/i86pc/io/pcplusmp/apic_common.c. However, misbehaving hardware
could hang the system if we wait forever.
Eric van Gyzen [Fri, 30 Apr 2021 17:53:45 +0000 (12:53 -0500)]
EFI secure boot VECTX related changes
When VECTX is enabled as a kernel option and non-EFI loaders are
built, many reads will fail due to the mis-match of whether
LOADER_VERIEXEC_VECTX or not in readin.h. Source that includes
bootstrap.h must ensure the kernel option agrees with the compile
time CFLAGS in the various make related files.
Marcin Wojtas [Tue, 27 Apr 2021 09:00:15 +0000 (11:00 +0200)]
iflib: Take iri_pad into account when processing small frames
Drivers can specify padding of received frames with iri_pad field.
This can be used to enforce ip alignment by hardware.
Iflib ignored that padding when processing small frames,
which rendered this feature inoperable.
I found it while writing a driver for a NIC that can ip align
received packets. Note that this doesn't change behavior of existing
drivers as they all set iri_pad to 0.
If we reassemble a packet we modify the IP header (to set the length and
remove the fragment offset information), but we failed to update the
checksum. On certain setups (mostly where we did not re-fragment again
afterwards) this could lead to us sending out packets with incorrect
checksums.
Yinlong Lu [Thu, 29 Apr 2021 10:04:36 +0000 (05:04 -0500)]
ipmi: support getting address from EFI
The original implementation only supports getting the address from legacy
BIOS (by searching for the SMBIOS_SIG pattern in a fixed address space).
Try to get the SMBIOS table from EFI through efirt (EFI Runtime Services)
firstly. Continue to search in the legacy BIOS if a NULL address is
returned from EFI.
By this way the ipmi function supports both legacy BIOS and UEFI systems.
We do not currently generate armv7 distribution sets, because we don't
build any installer images. However, having such sets available can be
useful for quickly installing a base system, particularly in the case
of creating an armv7 poudriere jail on arm64.
Add a configuration file for the generation of these distribution sets.
pkg(7): replace usage of sbuf(9) with open_memstream(3)
open_memstream(3) is a standard way to obtain the same feature we do get
by using sbuf(9) (aka dynamic size buffer), switching to using it makes
pkg(7) more portable, and reduces its number of dependencies.
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D30005
projects / FreeBSD / FreeBSD.git / log