We actually do not know is it safe or not to flush cache for random
BAR/register page existing in the system. It is well-known that for
instance LAPICs cannot tolerate cache flush. As report indicates,
there are more such devices.
This issue typically affects AMD machines which do not report self-snoop,
causing real CLFLUSH invocation on the mapped pages. Intels do self-snoop,
so this change should be nop for them, and unsafe devices, if any, are
already ignored.
Reported and tested by: manu
Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32318
Similar to pmap_page_set_memattr() by setting MD page cache attribute
to the argument. Unlike pmap_page_set_memattr(), does not flush cache
for the direct mapping of the page.
Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32318
The default value for LAPIC registers page physical address
is usually right. Having this value available early makes
pmap_force_invalidate_cache_range(), used on non-self-snoop machines,
avoid flushing LAPIC range for early calls.
Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32318
Alexander Motin [Tue, 5 Oct 2021 19:01:16 +0000 (15:01 -0400)]
cam(4): Limit search for disks in SES enclosure by single bus
At least for SAS that we only support now disks are typically
connected to the same bus as the enclosure. Limiting the search
scope makes it much faster on systems with multiple buses and
thousands of disks.
Alexander Motin [Tue, 5 Oct 2021 18:54:03 +0000 (14:54 -0400)]
cam(4): Improve XPT_DEV_MATCH
Remove *_MATCH_NONE enums, making no sense and so never used. Make
*_MATCH_ANY enums 0 (no any match flags set), previously used by
*_MATCH_NONE. Bump CAM_VERSION to 0x1a reflecting those changes and
add compat shims.
When traversing through buses and devices do not descend if we can
already see that requested pattern does not match the bus or device.
It allows to save significant amount of time on system with thousands
of disks when doing limited searches.
sdhci: Fix crash caused by M_WAITOK in sdhci dumps
In some contexts it is illegal to wait for memory allocation, causing
kernel panic. By default sbuf_new passes M_WAITOK to malloc,
which caused crashes when sdhci_dumpcaps or sdhci_dumpregs was callend in
non sutiable context.
Mark Johnston [Mon, 4 Oct 2021 21:48:44 +0000 (17:48 -0400)]
geom_label: Add more validation for NTFS volume tasting
- Ensure that the computed MFT record size isn't negative or larger than
maxphys before trying to read $Volume.
- Guard against truncated records in volume metadata.
- Ensure that the record length is large enough to contain the volume
name.
- Verify that the (UTF-16-encoded) volume name's length is a multiple of
two.
PR: 258833, 258914
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
David Bright [Mon, 4 Oct 2021 15:43:41 +0000 (08:43 -0700)]
RPCBIND: skip ipv6 link local when request is not from link local address
RPCINFO on macOS behaves different compared to other linux clients and
doesn't provide request address in rpcb structure of the
RPCBPROC_GETADDRLIST call which doesn't seem to be forbidden.
In this case RPCBIND uses RPC call's source address and picks a
closest corresponding local address. If there are no addresses in the
same subnet as the source address, return of RPCBIND may vary
depending on the order of addresses returned in getifaddrs. If a link
local precedes global address it may be returned even if the request
comes from neither a link local nor from link local in a different
scope, which will prevent services like nfs from working in tpc6
scenario on macOS clients. Issue can be seen only on FreeBSD rpcbind
port due to changes in workflow of addrmerge call.
Mark Johnston [Mon, 4 Oct 2021 16:28:22 +0000 (12:28 -0400)]
libctf: Improve check for duplicate SOU definitions in ctf_add_type()
When copying a struct or union from one CTF container to another,
ctf_add_type() checks whether it matches an existing type in the
destination container. It does so by looking for a type with the same
name and kind as the new type, and if one exists, it iterates over all
members of the source type and checks whether a member with matching
name and offset exists in the matched destination type. This can
produce false positives, for example because member types are not
compared, but this is not expected to arise in practice. If the match
fails, ctf_add_type() returns an error.
The procedure used for member comparison breaks down in the face of
anonymous struct and union members. ctf_member_iter() visits each
member in the source definition and looks up the corresponding member in
the desination definition by name using ctf_member_info(), but this
function will descend into anonymous members and thus fail to match.
Fix the problem by introducing a custom comparison routine which does
not assume member names are unique. This should also be faster for
types with many members; in the previous scheme, membcmp() would perform
a linear scan of the desination type's members to perform a lookup by
name. The new routine steps through the members of both types in a
single loop.
PR: 258763
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
ncurses: fix path where to find curses.h at bootstrap
after the split, curses.h is now generated by tinfo Makefile, but
still used for a file generated in ncurses lib. Adjust the path to
make sure curses.h is always found
many external program expects libncurses to not be provided as a single
library. Instead of fixing all ports, distribute ncurses the way
upstream distributes it
Turn libncursesw.so into a ldscript which will link automatically as
needed to libtinfow so so this change is seamless at compile time.
Stefan Eßer [Mon, 4 Oct 2021 07:50:44 +0000 (09:50 +0200)]
contrib/bc: temporarily disconnect the tests for 5.0.2
The tests that come with version 5.0.2 have been extended to cover the
line editing functions. It has been found that these tests generate
false negative results in FreeBSD, most likely due to an issue in the
pexpect functionality used.
These history tests are skipped on systems that do not have python and
py-pexpect installed (and thus are unlikely to cause CI test failures),
but in order to not cause irritating failures on systems were these
packages are in fact installed, I temporarily disconnect them.
I had planned to skip this version due to the issue with the history
tests, but some committer has asked me to go ahead since the currently
used version 5.0.0 contains a macro name that collides with a project
he is working on.
No MFC of this version is planned. A version 5.0.3 is expected to be
released soon, and that version will allow to reconnect the tests and
will be MFCed.
Stefan Eßer [Mon, 4 Oct 2021 07:41:06 +0000 (09:41 +0200)]
contrib/bc: remove files ommitted from the release
A number of files have been removed from the release distribution of
this bc implementation. They were mostly relevant for pre release
testing and benchmarking to identify regressions. The Markdown
sources of the man pages are only relevant for combinations of build
options not used in FreeBSD and need non-default conversion tools
(available as ports in FreeBSD).
All the omitted files can be found in the upstream git repository,
and they are fetched when building this software as a port. But they
have never been used in the FreeBSD base system.
Colin Percival [Sun, 3 Oct 2021 21:55:10 +0000 (14:55 -0700)]
loader bcache: Allow readahead up to 256 kB I/Os
Prior to this commit, the loader would perform readaheads of up to
128 kB; when booting on a UFS filesystem this resulted in a series
of 160 kB reads (32 kB request + 128 kB readahead).
This commit allows readaheads to be longer, subject to a total I/O
size limit of 256 kB; i.e. 32 kB read requests will have added
readaheads of up to 224 kB.
In my testing on an EC2 c5.xlarge instance, this change reduces the
boot time by roughly 80 ms.
Colin Percival [Sun, 3 Oct 2021 21:49:41 +0000 (14:49 -0700)]
loader bcache: Track unconsumed readahead
The loader bcache attempts to determine whether readahead is useful,
increasing or decreasing its readahead length based on whether a
read could be serviced out of the cache. This resulted in two
unfortunate behaviours:
1. A series of consecutive 32 kB reads are requested and bcache
performs 16 kB readaheads. For each read, bcache determines that,
since only the first 16 kB is already in the cache, the readahead
was not useful, and keeps the readahead at the minimum (16 kB) level.
2. A series of consecutive 32 kB reads are requested and bcache
starts with a 32 kB readahead resulting in a 64 kB being read on
the first request. The second 32 kB request can be serviced out of
the cache, and bcache responds by doubling its readahead length to
64 kB. The third 32 kB request cannot be serviced out of the cache,
and bcache reduces its readahead length back down to 32 kB.
The first syndrome converts a series of 32 kB reads into a series of
(misaligned) 32 kB reads, while the second syndrome converts a series
of 32 kB reads into a series of 64 kB reads; in both cases we do not
increase the readahead length to its limit (currently 128 kB) no
matter how many consecutive read requests are made.
This change avoids this problem by tracking the "unconsumed
readahead" length; readahead is deemed to be useful (and the
read-ahead length is potentially increased) not only if a request was
completely serviced out of the cache, but also if *any* of the request
was serviced out of the cache and that length matches the amount of
unconsumed readahead. Conversely, we now only reduce the readahead
length in cases where there was unconsumed readahead data.
In my testing on an EC2 c5.xlarge instance, this change reduces the
boot time by roughly 120 ms.
If a device is plugged into the x16 slot that itself has a bridge, such
as many graphics cards, we currently fail to allocate a bus number for
its child bus (and so pcib_attach_child skips adding a child bus for
further enumeration) as, when the new child bridge attaches, it attempts
to allocate a bus number from its parent (pcib7) which in turn attempts
to grow its own bus range by calling bus_adjust_resource on its own
parent (pcib2) whose bus rman cannot accommodate the request and needs
to itself be extended by calling its own parent (pcib1). Note that
pcib3-7 do not face the same issue when they attach since pcib1 ends up
managing bus numbers 1-255 from the beginning and so never needs to grow
its own range.
Jessica Clarke [Sun, 3 Oct 2021 18:34:53 +0000 (19:34 +0100)]
riscv: Add vt and kbdmux to GENERIC for video console support
No in-tree drivers are supported for RISC-V (given it supports UEFI we
could enable the EFI framebuffer, but U-Boot has very limited hardware
support and EDK2 remains a work in progress), but drm-kmod exists with
drivers for video cards that can be used with the HiFive Unmatched.
Jessica Clarke [Sun, 3 Oct 2021 18:34:40 +0000 (19:34 +0100)]
LinuxKPI: Add more #ifdef VM_MEMATTR_WRITE_COMBINING guards
One of the three uses is already guarded; this guards the remaining ones
to support architectures like riscv that do not provide write-combining,
and is needed to build drm-kmod on riscv.
Jessica Clarke [Sun, 3 Oct 2021 18:34:25 +0000 (19:34 +0100)]
libgcc_s: Export 64-bit int to 128-bit float functions
The corresponding 32-bit int and 128-bit int functions were added in 790a6be5a169, as were all combinations of the float to int functions,
but these two were overlooked. __floatditf is needed to build curl for
riscv as there's a signed 64-bit int to 128-bit float conversion in
lib/progress.c's trspeed as of 7.77.0.
Reviewed by: dim
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31997
Jessica Clarke [Sun, 3 Oct 2021 18:33:47 +0000 (19:33 +0100)]
riscv: Add a stub pmap_change_attr implementation
pmap_change_attr is required by drm-kmod so we need the function to
exist. Since the Svpbmt extension is on the horizon we will likely end
up with a real implementation of it, so this stub implementation does
all the necessary page table walking to validate the input, ensuring
that no new errors are returned once it's implemented fully (other than
due to out of memory conditions when demoting L2 entries) and providing
a skeleton for that future implementation.
Kyle Evans [Sat, 2 Oct 2021 05:17:57 +0000 (00:17 -0500)]
vfs: remove dead fifoop VOP_KQFILTER implementations
These began to become obsolete in d6d64f0f2c26 (r137739) and the deal
was later sealed in 003e18aef4c4 (r137801) when vfs.fifofs.fops was
dropped and vop-bypass for pipes became mandatory.
Alexander Motin [Sun, 3 Oct 2021 00:57:55 +0000 (20:57 -0400)]
sleepqueue(9): Remove sbinuptime() from sleepq_timeout().
Callout c_time is always bigger or equal than the scheduled time. It
is also smaller than sbinuptime() and can't change while the callback
is running. So we reliably can use it instead of sbinuptime() here.
In case there was a race and the callout was rescheduled to the later
time, the callback will be called again.
According to profiles it saves ~5% of the timer interrupt time even
with fast TSC timecounter.
Robert Wing [Sat, 2 Oct 2021 23:11:40 +0000 (15:11 -0800)]
ffs: retire unused fsckpid mount option
The fsckpid mount option was introduced in 927a12ae16433b50 along with a
couple sysctl's to support SU+J with snapshots. However, those sysctl's
were never used and eventually removed in f2620e9ceb3ede02.
There are no in-tree consumers of this mount option.
Rick Macklem [Sat, 2 Oct 2021 21:11:15 +0000 (14:11 -0700)]
nfsd: Fix pNFS handling of Deallocate
For a pNFS server configuration, an NFSv4.2 Deallocate operation
is proxied to the DS(s). The code that parsed the reply for the
proxy RPC is broken and did not process the pre-operation attributes.
This patch fixes this problem.
This bug would only affect pNFS servers built from recent main/FreeBSD14
sources.
Alexander Motin [Sat, 2 Oct 2021 03:47:18 +0000 (23:47 -0400)]
sched_ule(4): Fix possible significance loss.
Before this change kern.sched.interact sysctl setting above 32 gave
all interactive threads identical priority of PRI_MIN_INTERACT due to
((PRI_MAX_INTERACT - PRI_MIN_INTERACT + 1) / sched_interact) turning
zero. Setting the sysctl lower reduced the range of used priority
levels up to half, that is not great either.
Change of the operations order should fix the issue, always using full
range of priorities, while overflow is impossible there since both
score and priority values are small. While there, make the variables
unsigned as they really are.
Warner Losh [Fri, 1 Oct 2021 17:32:48 +0000 (11:32 -0600)]
nvme: Use adaptive spinning when polling for completion or state change
We only use nvme_completion_poll in the initialization path. The
commands they queue and wait for finish quickly as they involve no I/O
to the drive's media. These command take about 20-200 microsecnds
each. Set the wait time to 1us and then increase it by 1.5 each
successive iteration (max 1ms). This reduces initialization time by
80ms in cpervica's tests.
Use this same technique waiting for RDY state transitions. This saves
another 20ms. In total we're down from ~330ms to ~2ms.
Li-Wen Hsu [Fri, 1 Oct 2021 20:59:10 +0000 (04:59 +0800)]
Update Azure release bits
Imports the changes for building official images on Azure Marketplace,
which fulfill the requirements of Azure and FreeBSD cloud images like
disk layout and UEFI for Gen2 VM, along with some minor improvements like
configurations to speed up booting.
"CLOUDWARE" list will be updated after some more collaborations with re
completed.
Reviewed by: re (gjb)
Sponsored by: The FreeBSD Foundation
Technical assistance from: Microsoft
Differential Revision: https://reviews.freebsd.org/D23804
Justin Hibbits [Fri, 1 Oct 2021 18:39:18 +0000 (13:39 -0500)]
Fix segment size in compressing core dumps
A core segment is bounded in size only by memory size. On 64-bit
architectures this means a segment can be much larger than 4GB.
However, compress_chunk() takes only a u_int, clamping segment size to
4GB-1, resulting in a truncated core. Everything else, including the
compressor internally, uses size_t, so use size_t at the boundary here.
This dates back to the original refactor back in 2015 (r279801 / aa14e9b7).
MFC after: 1 week
Sponsored by: Juniper Networks, Inc.
Faraz Vahedi [Fri, 1 Oct 2021 18:48:57 +0000 (13:48 -0500)]
freebsd-update(8): Add -j flag to support jails
Make freebsd-update(8) support jails by adding the -j flag which takes
a jail jid or name as an argument. This takes advantage of the recently
added -j support to freebsd-version(8) in order to get the version of
the installed userland.
Faraz Vahedi [Fri, 1 Oct 2021 18:46:23 +0000 (13:46 -0500)]
freebsd-version(1): Add -j flag to support jails
Make freebsd-version(1) support jails by adding the -j flag which takes
a jail jid or name as an argument. As with other options, -j
flags stack and display in the order requested.
Kyle Evans [Fri, 1 Oct 2021 16:59:31 +0000 (11:59 -0500)]
kqueue: fix recent assertion
NOTE_ABSTIME may also have a zero timeout, which indicates that we
should still fire immediately as an absolute time in the past. A test
has been added for this one as well.
Gleb Smirnoff [Fri, 1 Oct 2021 17:24:50 +0000 (10:24 -0700)]
Don't set GELI UMA zone as UMA_ZONE_NOFREE.
That fixes memory leak on last GELI provider destroyed, introduced
in 2dbc9a388ee. This patch was originally developed late 2019 and
the flag was necessary to prevent zone drainage under memory pressure.
Today, with f09cbea31a3f the UMA is fixed not to drain into reserves.
Warner Losh [Fri, 1 Oct 2021 17:09:34 +0000 (11:09 -0600)]
nvme: Only reset once on attach.
The FreeBSD nvme driver has reset the nvme controller twice on attach to
address a theoretical issue assuring the hardware is in a known
state. However, exierence has shown the second reset is unnecessary and
increases the time to boot. Eliminate the second reset. Should there be
a situation when you need a second reset (for buggy or at least somewhat
out of the mainstream hardware), the hardware option NVME_2X_RESET will
restore the old behavior. Document this in nvme(4).
If there's any trouble at all with this, I'll add a sysctl tunable to
control it.
Warner Losh [Fri, 1 Oct 2021 17:09:05 +0000 (11:09 -0600)]
nvme: Remove pause while resetting
After some study of the code and the standard, I think we can just drop
the pause(), unconditionally. If we're not initialized, then there's
nothing to wait for from a software perspective. If we are initialized,
then there might be outstanding I/O. If so, then the qpair 'recovery
state' will transition to WAITING in nvme_ctrlr_disable_qpairs, which
will ignore any interrupts for items that complete before we complete
the reset by setting cc.en=0.
If we go on to fail the controller, we'll cancel the outstanding I/O
transactions. If we reset the controller, the hardware throws away
pending transactions and we retry all the pending I/O transactions. Any
transactions that happend to complete before cc.en=0 will have the same
effect in the end (doing the same transaction twice is just inefficient,
it won't affect the state of the device any differently than having done
it once).
The standard imposes no wait times here, so it isn't needed from that
perspective.
Unanswered Question: Do we may need to disable interrupts while we
disable in legacy mode since those are level-sensitive.
Warner Losh [Fri, 1 Oct 2021 16:51:24 +0000 (10:51 -0600)]
nvme_ctrlr_enable: Small style nits
Rewrite the nested if's using the preferred FreeBSD style for branches
of ifs that return. NFC. Minor tweaks to the comments to better fit new
code layout.
Warner Losh [Fri, 1 Oct 2021 16:47:27 +0000 (10:47 -0600)]
nvme_ctrlr_enable: Remove unnecessary 5ms delays
Remove the 5ms delays after writing the administrative queue
registers. These delays are from the very earliest days of the driver
(they are in the first commit) and were most likely vestiges of the
Chatham NVMe prototype card that was used to create this driver. Many of
the workarounds necessary for it aren't necessary for standards
compliant cards. The original driver had other areas marked for Chatham,
but these were not. They are unneeded. There's three lines of supporting
evidence.
First, the NVMe standards make no mention of a delay time after these
registers are written. Second, the Linux driver doesn't have them, even
as an option. Third, all my nvme cards work w/o them.
To be safe, add a write barrier between setting up the admin queue and
enabling the controller.
Eric van Gyzen [Fri, 1 Oct 2021 11:25:48 +0000 (06:25 -0500)]
sem_clockwait_np test: fix usage of ATF API
ATF_REQUIRE_ERRNO requires the given errno iff the given expression is
true. These test cases used it incorrectly, potentially allowing
sem_clockwait_np to succeed when it was expected to fail. Use separate
ATF calls to require failure and the expected errno.
Eric van Gyzen [Fri, 1 Oct 2021 10:37:17 +0000 (05:37 -0500)]
sem_clockwait_np test: relax time constraint on VMs
In a guest on a busy hypervisor, the time remaining after an
interrupted sleep could be much lower than other environments.
Relax the lower bound on VMs.
Randall Stewart [Fri, 1 Oct 2021 14:32:30 +0000 (10:32 -0400)]
tcp: Make dsack stats available in netstat and also make sure its aware of TLP's.
DSACK accounting has been for quite some time under a NETFLIX_STATS ifdef. Statistics
on DSACKs however are very useful in figuring out how much bad retransmissions you
are doing. This is further complicated, however, by stacks that do TLP. A TLP
when discovering a lost ack in the reverse path will cause the generation
of a DSACK. For this situation we introduce a new dsack-tlp-bytes as well
as the more traditional dsack-bytes and dsack-packets. These will now
all display in netstat -p tcp -s. This also updates all stacks that
are currently built to keep track of these stats.
Reviewed by: tuexen
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D32158
projects / FreeBSD / FreeBSD.git / log