tcp: Add hash histogram output and validate bucket length accounting
Provide a histogram output to check, if the hashsize or
bucketlimit could be optimized. Also add some basic sanity
checks around the accounting of the hash utilization.
tcp: For hostcache performance, use atomics instead of counters
As accessing the tcp hostcache happens frequently on some
classes of servers, it was recommended to use atomic_add/subtract
rather than (per-CPU distributed) counters, which have to be
summed up at high cost to cache efficiency in a hot codepath.
tcp: Make hostcache.cache_count MPSAFE by using a counter_u64_t
Addressing the underlying root cause for cache_count to
show unexpectedly high values, by protecting all arithmetic on
that global variable by using counter(9).
tcp: reduce memory footprint when listing tcp hostcache
In tcp_hostcache_list, the sbuf used would need a large (~2MB)
blocking allocation of memory (M_WAITOK), when listing a
full hostcache. This may stall the requestor for an indeterminate
time.
A further optimization is to return the expected userspace
buffersize right away, rather than preparing the output of
each current entry of the hostcase, provided by: @tuexen.
This makes use of the ready-made functions of sbuf to work
with sysctl, and repeatedly drain the much smaller buffer.
While sbuf_drain was an internal function, two
KASSERTS checked the sanity of it being called.
However, an external caller may be ignorant if
there is any data to drain, or if an error has
already accumulated. Be nice and return immediately
with the accumulated error.
Nathan Whitehorn [Fri, 26 Mar 2021 15:39:12 +0000 (11:39 -0400)]
Add a new mode to the scripted partition editor for variant disk names.
If the disk parameter "DEFAULT" is set in place of an actual device name,
or no disk is specified for the PARTITIONS parameter, the installer will
follow the logic used in the automatic-partitioning mode, in which it
will either provide a selection dialog for one of several disks if
several are present or automatically select it if there is only one. This
simplifies the creation of fully-automatic installation media for
hardware or VMs with varying disk names.
Ryan Moeller [Fri, 16 Apr 2021 02:32:25 +0000 (22:32 -0400)]
ifconfig: Reuse media state in ifmedia_getstate
This restores behavior lost in code cleanup, fixing a regression after 2803fa471e77dc8f227fe00bbf075de7feb10022 where changing media options
only applies some of the changes, not all.
Mark Johnston [Fri, 2 Apr 2021 21:42:26 +0000 (17:42 -0400)]
fbt: Remove some handling for multiple CTF containers
This was ported from illumos but not completely done. Currently we do
not perform type deduplication between KLDs and the kernel, i.e., kernel
modules have a complete type graph. So, remove it for now since it's
not functional and complicates the task of modifying various CTF type
definitions, and we are hitting some limits in the current format which
necessitate an update.
Rick Macklem [Thu, 1 Apr 2021 22:36:37 +0000 (15:36 -0700)]
nfsv4 client: factor loop contents out into a separate function
Commit fdc9b2d50fe9 replaced a couple of while loops with LIST_FOREACH()
loops. This patch factors the body of that loop out into a separate
function called nfscl_checkown().
This prepares the code for future changes to use a hash table of
lists for open searches via file handle.
This patch should not result in a semantics change.
Rick Macklem [Thu, 1 Apr 2021 22:09:03 +0000 (15:09 -0700)]
nfsd: silence rpcb_unset noise for NFSv4 only servers
An NFSv4 only configuration does not register with
rpcbind(). Without this patch a failure to rpcb_unset()
is reported when the daemon is terminated for this case.
This is harmless noise, but this patch avoids calling
rpcb_unset() for the NFSv4 only case, avoiding the noise.
When called with "-d", it still does the rpcb_unset(),
assuming that the configuration might have been
changed to NFSv4 only and unregistering with
rpcbind() might still be needed.
Avoid raising unexpected floating point exceptions in libm
When using clang with x86_64 CPUs that support AVX, some floating point
transformations may raise exceptions that would not have been raised by
the original code. To avoid this, use the -fp-exception-behavior=maytrap
flag, introduced in clang 10.0.0.
In particular, this fixes a number of test failures with ctanhf(3) and
ctanf(3), when libm is compiled with -mavx. An unexpected FE_INVALID
exception is then raised, because clang emits vdivps instructions to
perform certain divides. (The vdivps instruction operates on multiple
single-precision float operands simultaneously, but the exceptions may
be influenced by unused parts of the XMM registers. In this particular
case, it was calculating 0 / 0, which results in FE_INVALID.)
If -fp-exception-behavior=maytrap is specified however, clang uses
vdivss instructions instead, which work on one operand, and should not
raise unexpected exceptions.
Only use -fp-exception-behavior=maytrap on x86, for now
After 3b00222f156d, it turns out that clang only supports strict
floating point semantics for SystemZ and x86 at the moment, while for
other architectures it is still experimental.
Therefore, only use -fp-exception-behavior=maytrap on x86 for now,
otherwise this option results in "error: overriding currently
unsupported use of floating point exceptions on this target
[-Werror,-Wunsupported-floating-point-opt]" on other architectures.
Avoid -pedantic warnings about using _Generic in __fp_type_select
When compiling parts of math.h with clang using a C standard before C11,
and using -pedantic, it will result in warnings similar to:
bug254714.c:5:11: warning: '_Generic' is a C11 extension [-Wc11-extensions]
return !isfinite(1.0);
^
/usr/include/math.h:111:21: note: expanded from macro 'isfinite'
^
/usr/include/math.h:82:39: note: expanded from macro '__fp_type_select'
^
This is because the block that enables use of _Generic is conditional
not only on C11, but also on whether the compiler advertises support for
C generic selections via __has_extension(c_generic_selections).
To work around the warning without having to pessimize the code, use the
__extension__ keyword, which is supported by both clang and gcc. While
here, remove the check for __clang__, as _Generic has been supported for
a long time by gcc too now.
Mark Johnston [Thu, 1 Apr 2021 14:25:57 +0000 (10:25 -0400)]
powernv: Include NUMA locality information in the CPU topology
ULE uses this topology to try and preserve locality when migrating
threads between CPUs and when performing work stealing. Ensure that on
NUMA systems it will at least take the NUMA topology into account.
Nathan Whitehorn [Wed, 24 Feb 2021 15:31:44 +0000 (10:31 -0500)]
Add GPT PREP-boot type to mkimg(1) from geom_gpt.
This partition type can be used to boot some PowerKVM VMs. We don't
support it well because of some limitations in SLOF, but it's worth at
least have feature parity in geom and mkimg.
Nathan Whitehorn [Wed, 24 Feb 2021 03:17:20 +0000 (22:17 -0500)]
Delete memstick images for PowerPC.
These images only ever worked on Apple Powermacs, which are now a very
old platform, and did so only for a very loose definition of "worked"
(they booted on a small subset of supported machines). Moreover, all
the machines they *did* boot on also would boot from a memstick made
by dd'ing an CD image to a flash drive. Since a flash drive prepared
in this way would also boot all the newer systems we support, the
memstick images were strictly less functional than the CD images, even
for booting from memory sticks.
Reviewed by: jhibbits
MFC after: 1 week
Kristof Provost [Thu, 25 Mar 2021 12:59:14 +0000 (13:59 +0100)]
libnv: Allow use in non-sleepable contexts
44c125c4cebc2fd87c6260b90eddae11201f5232 switched the nvlist allocations
to be M_WAITOK, but this precludes the use in non-sleepable contexts.
(E.g. with a nonsleepable lock held).
All callers for these allocation functions already cope with memory
alloation failures, so there's no reason to allow sleeping during
allocations.
pf tests: make synproxy and nat work correctly even if inetd is running
tests/sys/netfil/pf/synproxy fails if inetd has been running
outside of the jail because pidfile_open() fails with EEXIST.
tests/sys/netfil/pf/nat has the same problem but the test succeeds
because whether inetd is running is not so important.
Fix the problem by changing the pidfile path from the default
location.
Ed Maste [Sun, 4 Apr 2021 00:57:26 +0000 (20:57 -0400)]
freebsd-update: improve mandoc db generation
freebsd-update compares the dates on man pages with mandoc.db, and if
any newer pages are found it regenerates mandoc.db.
Previously, if mandoc.db did not already exist the check failed and
freebsd-update then failed to create one. Now, check that mandoc.db
exists before performing the check for newer pages.
Reported by: bdrewery (in D10482)
Reviewed by: gordon
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29575
Gordon Bergling [Fri, 9 Apr 2021 15:28:18 +0000 (17:28 +0200)]
sysctl.conf(5): Mention sysctl.conf.local in the sysctl.conf(5) manual page
The possibility of using a sysctl.conf.local on a machine that has a shared
sysctl.conf(5) isn't documented. So mention the sysctl.conf.local in the
manual page.
PR: 254901
Submitted by: Jose Luis Duran <jlduran at gmail dot com>
Reported by: Jose Luis Duran <jlduran at gmail dot com>
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D29673
Nathan Whitehorn [Mon, 22 Mar 2021 14:08:55 +0000 (10:08 -0400)]
Improve example install scripts, making them simpler and more robust.
In particular:
- There is no need to do anything with gpart (the installer does that
for you).
- There is no need to specify the network interface, since we have
an option for defaults.
Rick Macklem [Mon, 29 Mar 2021 21:14:51 +0000 (14:14 -0700)]
nfsv4 client: replace while loops with LIST_FOREACH() loops
This patch replaces a couple of while() loops with LIST_FOREACH() loops.
While here, declare a couple of variables "bool".
I think LIST_FOREACH() is preferred and makes the code more readable.
This also prepares the code for future changes to use a hash table of
lists for open searches via file handle.
This patch should not result in a semantics change.
Rick Macklem [Mon, 29 Mar 2021 19:09:19 +0000 (12:09 -0700)]
nfsv4.1/4.2 client: fix handling of delegations for "oneopenown" mnt option
If a delegation for a file has been acquired, the "oneopenown" option
was ignored when the local open was issued. This could result in multiple
openowners/opens for a file, that would be transferred to the server
when the delegation was recalled.
This would not be serious, but could result in more than one openowner.
Since the Amazon/EFS does not issue delegations, this probably never
occurs in practice.
Spotted during code inspection.
This small patch fixes the code so that it checks for "oneopenown"
when doing client local opens on a delegation.
Rick Macklem [Tue, 30 Mar 2021 21:31:05 +0000 (14:31 -0700)]
nfsd: do not implicitly bind the back channel for NFSv4.1/4.2 mounts
The NFSv4.1 (and 4.2 on 13) server incorrectly binds
a new TCP connection to the back channel when first
used by an RPC with a Sequence op in it (almost all of them).
RFC5661 specifies that only the fore channel should be bound.
This was done because early clients (including FreeBSD)
did not do the required BindConnectionToSession RPC.
Unfortunately, this breaks the Linux client when the
"nconnects" mount option is used, since the server
may do a callback on the incorrect TCP connection.
This patch converts the server behaviour to that
required by the RFC. It also makes the server test/indicate
failure of the back channel more aggressively.
Until this patch is applied to the server, the
"nconnects" mount option is not recommended for a Linux
NFSv4.1/4.2 client mount to the FreeBSD server.
Rick Macklem [Tue, 23 Mar 2021 20:04:37 +0000 (13:04 -0700)]
nfsv4 client: fix forced dismount when sleeping in the renew thread
During a recent NFSv4 testing event a test server caused a hang
where "umount -N" failed. The renew thread was sleeping on "nfsv4lck"
and the "umount" was sleeping, waiting for the renew thread to
terminate.
This is the second of two patches that is hoped to fix the renew thread
so that it will terminate when "umount -N" is done on the mount.
This patch adds a 5second timeout on the msleep()s and checks for
the forced dismount flag so that the renew thread will
wake up and see the forced dismount flag. Normally a wakeup()
will occur in less than 5seconds, but if a premature return from
msleep() does occur, it will simply loop around and msleep() again.
The patch also adds the "mp" argument to nfsv4_lock() so that it
will return when the forced dismount flag is set.
While here, replace the nfsmsleep() wrapper that was used for portability
with the actual msleep() call.
epoll: Store epoll_event udata member in ext member of kevent.
Current epoll implementation stores udata fields of epoll_event
structure in special dynamically-sized table rather than in udata field
of backing kevent structure because of 2 reasons:
1. Kevent's udata size is smaller than epoll's on 32-bit archs.
2. Kevent's udata can be clobbered on execution EPOLL_CTL_ADD as kqueue
modifies existing event while epoll returns error in this case.
After r320043 has introduced four new 64bit user data members (ext[]),
we can store epoll udata in one of them and drop aforementioned table.
According to kqueue_register() source code ext members are not updated
when existing kevent is modified that fixes p.2.
Daniel Engerg [Wed, 17 Mar 2021 14:00:57 +0000 (15:00 +0100)]
Remove tmpfs size and properly format generated fstab for arm
Remove tmpfs size limitation, this breaks make installworld and installation of some packages
Format generated fstab using tabs to make it consistent and readable
Stefan Eßer [Tue, 6 Apr 2021 08:44:52 +0000 (10:44 +0200)]
[bc] Update to version 4.0.0
This version fixes an issue (missing pop of top-of-stack value in the
"P" command of the dc program).
This issue did not affect the bc program, since it does not use dc as
an back-end to actually perform the calculations as was the case with
the traditional bc and dc programs.
The major number has been bumped due to Windows support that has been
added to this version. It does not correspond to a major change that
might affect FreeBSD.
Mateusz Guzik [Sat, 27 Feb 2021 22:23:23 +0000 (22:23 +0000)]
cache: temporarily drop the assert that dvp != vp when adding an entry
Historically it was allowed for any names, but arguably should never be
even attempted. Allow it again since there is a release pending and
allowing it is bug-compatible with previous behavior.
The C variant in libkern performs excessive branching to find the zero
byte instead of using the bsfq instruction. The same code patched to use
it is still slower than the routine implemented here as the compiler
keeps neglecting to perform certain optimizations (like using leaq).
On top of that the routine can be used as a starting point for copyinstr
which operates on words intead of bytes.
The previous attempt had an instance of swapped operands to andq when
dealing with fully aligned case, which had a side effect of breaking the
code for certain corner cases. Noted by jrtc27.
When the FreeBSD testsuite runs the libarchive tests it checks that stderr
is empty. Since #1382 this is no longer the case. This change restores
the behaviour of silencing bunzip2 stderr but doesn't bring back the
output text check.
Alex Richardson [Tue, 30 Mar 2021 13:53:41 +0000 (14:53 +0100)]
sys/dev/md: Drop unncessary __GLOBL(mfs_root)
LLVM12 complains if you change the symbol binding:
error: mfs_root_end changed binding to STB_WEAK [-Werror,-Winline-asm]
error: mfs_root changed binding to STB_WEAK [-Werror,-Winline-asm]
Alex Richardson [Tue, 30 Mar 2021 13:52:31 +0000 (14:52 +0100)]
libsa: Remove conflicting .global/.weak directive
LLVM12 complains if you change the symbol binding:
`error: _longjmp changed binding to STB_GLOBAL`
In this case LLVM actually ignored the weak directive and used the
later .global, but GNU as would mark the symbol as weak.
None of the other architectures mark the libsa _setjmp as weak so
just drop this directive.
Alex Richardson [Thu, 25 Mar 2021 11:22:10 +0000 (11:22 +0000)]
lib/libc/net/nsdispatch.c: Fix missing unlock and add locking annotations
The error cases (goto fin) of _nsdispatch were missing the unlock.
This change also drops the checks for __isthreaded since the pthread stubs
are already no-ops if threads are not being used. Dropping those conditionals
allows clang's thread safety analysis to deal with the file and also makes
the code a bit more readable. While touching the file also add a few more
assertions in debug mode that the right locks are held.
Alex Richardson [Thu, 25 Mar 2021 11:15:41 +0000 (11:15 +0000)]
RISC-V: Fix feenableexcept return value
The man page says "The feenableexcept(), fedisableexcept(), and
fegetexcept() functions return a bitmap of the exceptions that were
unmasked prior to the call.", so we should return zero not -1.
Reviewed By: mhorne
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D29386
Alex Richardson [Thu, 25 Mar 2021 11:14:46 +0000 (11:14 +0000)]
ng_macfilter_test: Skip rather than fail if there is no network
This should bring the number of Jenkins failures from 4 down to 3.
Locally kyua now prints `skipped: could not find a valid interface [0.115s]`
when I run it in QEMU without a network device.
Reviewed By: lwhsu
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D29414
Alex Richardson [Fri, 12 Mar 2021 17:15:00 +0000 (17:15 +0000)]
Allow using sanitizers for ssp tests with out-of-tree compiler
With an out-of-tree Clang, we can use the -resource-dir flag when linking
to point it at the runtime libraries from the current SYSROOT.
This moves the path to the clang-internal library directory to a separate
.mk file that can be used by Makefiles that want to find the sanitizer
libraries. I intend to re-use this .mk file for my upcoming changes that
allow building the entire base system with ASAN/UBSAN/MSAN.
Reviewed By: dim
Differential Revision: https://reviews.freebsd.org/D28852
Alex Richardson [Wed, 10 Feb 2021 15:25:14 +0000 (15:25 +0000)]
Always build the sanitizer runtimes when compiling with clang
This allows instrumenting e.g. test binaries even when compiling with an
external clang (e.g. CROSS_TOOLCHAIN=llvm11). I have some upcoming patches
that allow building the entire base system with ASan/UBSan/etc.
instrumentation and this is required in preparation for this.
Alex Richardson [Fri, 5 Mar 2021 10:21:12 +0000 (10:21 +0000)]
tools/build/make.py: Avoid calling brew --prefix on macOS unnecessarily
If all the require variables (XCC/XCXX/XCPP/XLD) are already set in the
environment, we don't have to infer a default value for the cross toolchain
path. This avoids an additional `brew --prefix` call when building with
cheribuild (since it already sets all these variables).
Alex Richardson [Fri, 26 Feb 2021 17:49:03 +0000 (17:49 +0000)]
tools/build/make.py: Don't call brew --prefix if --cross-bindir is set
Also updated the logic to use subprocess.run() instead of the old
subprocess.getoutput() which also includes stderr and therefore
can trigger an exception inside Path().exists().
Alex Richardson [Sat, 13 Feb 2021 13:54:20 +0000 (13:54 +0000)]
tools/build/make.py: drop workaround for cc --version not being parsed
Previously bsd.compiler.mk was not able to detect the compiler type for
Ubuntu's /usr/bin/cc unless we were invoking the /usr/bin/gcc symlink.
This problem has been fixed by 9c6954329a9285547881ddd60e393b7c55ed30c4
so we can drop the workaround from make.py.
Jessica Clarke [Tue, 9 Feb 2021 21:40:24 +0000 (21:40 +0000)]
bsd.compiler.mk: Detect distribution-provided GCC when executed as cc
Clang always prints "clang $VERSION" regardless of the name used to
execute it, whereas GCC prints "$progname $VERSION", meaning if CC is
set to cc and cc is GCC it will print "cc $VERSION". We are able to
detect some of those cases since it then prints "($PKGVERSION)", where
the default is "GCC", but many distributions override that to print
their name and the package version number (e.g. "Debian 10.2.1-6"), so
nothing tells us it's GCC other than the fact that it's not Clang (and
that there's an FSF copyright disclaimer).
However, GCC's -v option will always print "gcc version $VERSION", so
fall back on using that to detect GCC. Whilst Clang also supports this
option, we should never get here, so Clang handling is not added.
Alex Richardson [Fri, 19 Mar 2021 18:34:29 +0000 (18:34 +0000)]
tests/sys/netgraph: Further CI fixes
I was trying to debug why this test is working locally but failing in CI.
While doing so I made some small changes to allow running it with set -e.
It turns out the problem is that find_iface does not return anything in
Jenkins, so all following tests fail with obscure error messages.
To handle this case exit early if $eth is empty.
Alex Richardson [Thu, 18 Mar 2021 20:52:20 +0000 (20:52 +0000)]
Unbreak MSG_CMSG_CLOEXEC
MSG_CMSG_CLOEXEC has not been working since 2015 (SVN r284380) because
_finstall expects O_CLOEXEC and not UF_EXCLOSE as the flags argument.
This was probably not noticed because we don't have a test for this flag
so this commit adds one. I found this problem because one of the
libwayland tests was failing.
Fixes: ea31808c3b07 ("fd: move out actual fp installation to _finstall")
MFC after: 3 days
Reviewed By: mjg, kib
Differential Revision: https://reviews.freebsd.org/D29328
Mateusz Guzik [Fri, 5 Feb 2021 23:16:55 +0000 (00:16 +0100)]
cache: remove the largely obsolete general description
Examples of inconsistencies with the current state:
- references LRU of all entries, removed years ago
- references a non-existent lock (neglist)
- claims negative entries have a NULL target
It will be replaced with a more accurate and more informative
description.
In the meantime take it out so it stops misleading.
projects / FreeBSD / FreeBSD.git / log