Alan Somers [Thu, 20 May 2021 01:10:15 +0000 (19:10 -0600)]
fusefs: correctly set lock owner during FUSE_SETLK
During FUSE_SETLK, the owner field should uniquely identify the calling
process. The fusefs module now sets it to the process's pid.
Previously, it expected the calling process to set it directly, which
was wrong.
libfuse also apparently expects the owner field to be set during
FUSE_GETLK, though I'm not sure why.
Alan Somers [Fri, 18 Jun 2021 00:04:59 +0000 (18:04 -0600)]
fusefs: ensure that FUSE ops' headers' unique values are actually unique
Every FUSE operation has a unique value in its header. As the name
implies, these values are supposed to be unique among all outstanding
operations. And since FUSE_INTERRUPT is asynchronous and racy, it is
desirable that the unique values be unique among all operations that are
"close in time".
Ensure that they are actually unique by incrementing them whenever we
reuse a fuse_dispatcher object, for example during fsync, write, and
listextattr.
Alan Somers [Thu, 24 Dec 2020 19:21:00 +0000 (19:21 +0000)]
fusefs: delete some dead code
The original fusefs GSoC project seems to have envisioned exchanging two
types of messages with FUSE servers. Perhaps vectored and non-vectored?
But in practice only one type has ever been used. Delete the other type.
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27770
Alan Somers [Tue, 15 Jun 2021 20:24:05 +0000 (14:24 -0600)]
fusefs: improve warnings about buggy FUSE servers
The fusefs driver will print warning messages about FUSE servers that
commit protocol violations. Previously it would print those warnings on
every violation, but that could spam the console. Now it will print
each warning no more than once per lifetime of the mount. There is also
now a dtrace probe for each violation.
We failed to list the new pf_syncookies.c file in sys/conf/files. This
worked for the usual configurations, where pf is a module, but not for
LINT builds.
Kristof Provost [Wed, 2 Jun 2021 16:16:03 +0000 (18:16 +0200)]
pf tests: Forwarding syncookie test
Test syncookies on a forwarding host. That is, in a setup where the
machine (or vnet) running pf is not the same as the machine (or vnet)
running the server it's protecting.
Kristof Provost [Thu, 20 May 2021 09:54:41 +0000 (11:54 +0200)]
pf: syncookie support
Import OpenBSD's syncookie support for pf. This feature help pf resist
TCP SYN floods by only creating states once the remote host completes
the TCP handshake rather than when the initial SYN packet is received.
This is accomplished by using the initial sequence numbers to encode a
cookie (hence the name) in the SYN+ACK response and verifying this on
receipt of the client ACK.
This can be used for variables which are only used with either
INVARIANTS or WITNESS. Without any annotation they run into dead store
warnings from cc --analyze and always annotating with __unused may hide
bad vars when it should not.
Warner Losh [Thu, 15 Jul 2021 22:17:23 +0000 (16:17 -0600)]
nvme: Enable interrupts after qpair fully constructed
To guard against the ill effects of a spurious interrupt during
construction (or one that was bogusly pending), enable interrupts after
the qpair is completely constructed. Otherwise, we can die with null
pointer dereferences in nvme_qpair_process_completions. This has been
observed in at least one pre-release NVMe drive where the MSIX interrupt
fired while the queue was being created, before we'd started the NVMe
controller card.
The alternative of only turning on the interrupts after the rest was
tried, but was insufficient to work around this bug and made the code
more complicated w/o benefit.
Warner Losh [Thu, 6 May 2021 19:05:09 +0000 (13:05 -0600)]
boot: fix OBJS to not include BTX's crt0.o
According to comments in the Makefile, to make pxeboot work we need to
have crt0.o first. This is needed because the simplified loader in
pxeboot assumes that the startup code is at offset 0 in this binary. In
normal booting, the start address can be obtained from headers of the
binary, but since pxeboot encodes this as a pure binary, it has no way
of knowing where that is and assumes 0. Added comments to that effect
in the Makefile.
We've done this by adding it to OBJS before all the other .o's are
added. However, there's a problem. This also adds it to the CLEANFILES
variable, which causes it to be removed from multiple places. The
dependencies may also cause it to be re-built at a time that's after
boot2 is built. This causes installs to fail because at install time
boot2 is considered to be out of date and the programs to rebuild it are
no longer in the path.
Cope with this problem by just adding it to LDFLAGS instead.
Glanced at by: kevans ("I thought that went in ages ago")
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D28876
nanobsd: Use gpart and create code image before full disk image
The attached patch brings two main changes to the nanobsd script:
1- gpart is used instead of fdisk;
2- the code image is created first, and then used to ``assemble'' the
full disk image.
The patch was first proposed on the freebsd-embedded list:
http://lists.freebsd.org/pipermail/freebsd-embedded/2012-June/001580.html
and is currently under discussion:
http://lists.freebsd.org/pipermail/freebsd-embedded/2014-January/002216.html
Another effect is that the -f option ("suppress code slice extraction")
now imples the -i option ("suppress disk image build").
imp@ applied Patch by hand to new legacy.sh, plus tweaked for NANO_LOG vs
NANO_OBJ confusion in original.
Warner Losh [Thu, 15 Jul 2021 03:06:08 +0000 (21:06 -0600)]
loader: make sure CPUTYPE is ignored when building
CPUTYPE?=native causes -march=native to be added to the command
line. When the host machine is haswell, this causes some versions of
clang to generate code that can't execute in the efi boot loader
environment. Set _CPUCFLAGS= to undo what's done bsd.cpu.mk. bsd.cpu.mk
is included too early to control with NO_CPU_CFLAGS here. The only other
option is to put that in all the Makefiles, and this is less tedious and
error prone.
Alfonso Gregory [Wed, 14 Jul 2021 21:48:35 +0000 (15:48 -0600)]
Remove incorrect __restricted labels from strcspn
strcspn should never have had the __restrict keywords. While both of
these strings are const, it may have unindended side effects. While this
is the kernel, the POSIX definition also omits restrict.
These issues have low impact because they require precise circumstances
to trigger one of them. The disk must be > 2 TiB in size and either:
- The primary GPT header is dammaged.
- The freebsd-boot partiton is located farther than the first 2 TiB of
the disc and one of its sectors takes place at a lba value that makes
the higher 32 bits of this very value change.
Errors and corrections folow:
- decl and incl don't affect CF, so replace with subl/addl $1
- repe uses %cx, so move size to it with movw
- moving a 64-bit value with %cx of 2 (should be 4) so addresses
> 2TB will work.
PR: 233180
Reviewed by: imp@ (applied patch using description in bug)
Differential Revision: https://reviews.freebsd.org/D31100
Warner Losh [Tue, 13 Jul 2021 06:00:33 +0000 (00:00 -0600)]
cam_iosched: use tunable flag and make a bool really a bool
kern.cam.do_dynamic_iosched is really a bool, so change its type to
bool. While I'm here, also use the CTLFLAG_TUN flag instead of a
separate tunable line for it and kern.cam.iosched_alpha_bits.
Young Xiao [Tue, 21 May 2019 07:36:29 +0000 (15:36 +0800)]
Fix potential NULL pointer dereference of device physical path
In ata_dev_advinfo() and nvme_dev_advinfo(), if the physical path is
being stored and there is a malloc failure (malloc(9) is called with
M_NOWAIT), we could wind up in a situation where the device's
physpath_len is set to the length the user provided, but the physpath
itself is NULL.
If another context then comes in to fetch the physical path value, we
would wind up trying to memcpy a NULL pointer into the caller's buffer.
So, set the physpath_len to 0 when we free the physpath on entry into
the store case for the physical path. Reset the length to a non-zero
value only after we've successfully malloced a buffer to hold it.
This code mirrors scsi_xpt.c does already as well.
Signed-off-by: Young Xiao <92siuyang@gmail.com>
Reviewed by: imp
PR: 238014
John Hood [Sun, 11 Jul 2021 14:44:12 +0000 (08:44 -0600)]
loader: support.4th resets the read buffer incorrectly
Large nextboot.conf files (over 80 bytes) are not read correctly by the
Forth loader, causing file parsing to abort, and nextboot configuration
fails to apply.
Simple repro:
nextboot -e foo=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
shutdown -r now
That will cause the bug to cause a parse failure but shouldn't otherwise
affect the boot. Depending on your loader configuration, you may also
have to set beastie_disable and/or reduce the number of modules loaded
to see the error on a small console screen. 12.0 or CURRENT users will
also have to explicitly use the Forth loader instead of the Lua loader.
The error will look something like:
Warning: syntax error on file /boot/loader.conf.local
foo="xxxxxxxxxxxxxxnextboot_enable="YES"
^
/boot/support.4th has crude file I/O buffering, which uses a buffer
'read_buffer', defined to be 80 bytes by the 'read_buffer_size'
constant. The loader first tastes nextboot.conf, reading and parsing
the first line in it for nextboot_enable="YES". If this is true, then
it reopens the file and parses it like other loader .conf files.
Unfortunately, the file I/O buffering code does not fully reset the
buffer state in the reset_line_reading word. If the last file was read
to the end, that doesn't matter; the file buffer is treated as empty
anyway. But in the nextboot.conf case, the loader will not read to the
end of file if it is over 80 bytes, and the file buffer may be reused
when reading the next file. When the file is reread, the corrupt text
may cause file parsing to abort on bad syntax (if the corrupt line has
<>2 quotes in it), the wrong variable to be set, no variable to be set
at all, or (if the splice happens to land at a line ending) something
approximating normal operation.
The bug is very old, dating back to at least 2000 if not before, and is
still present in 12.0 and CURRENT r345863 (though it is now hidden by
the Lua loader by default).
Suggested one-line attached. This does change the behavior of the
reset_line_reading word, which is exported in the line-reading
dictionary (though the export is not documented in loader man pages).
But repo history shows it was probably exported for the PNP support
code, which was never included in the loader build, and was removed 5
months ago.
One thing that puzzles me: how has this bug gone unnoticed/unfixed for
nearly 2 decades? I find it hard to believe that nobody's tried to do
something interesting with nextboot, like load a kernel and filesystem,
which is what I'm doing.
Andriy Gapon [Mon, 21 Jun 2021 08:05:04 +0000 (11:05 +0300)]
rtwn: make sure to not write in upper bits of txdseq
ni_txseqs is kept as 16-bit counter, but we need to trim the upper four
bits as they may have special meanings for the firmware / hardware.
For instance, bit 15 enables hardware / firmware generation of sequence
numbers that overrides sequence numbers programmed by the driver.
Warner Losh [Thu, 8 Jul 2021 19:53:18 +0000 (13:53 -0600)]
devmatch: don't announce autoloading so much
devmatch rc script would announce it was loading a module multiple
times. It used kldload -n so it really wasn't loading it that many
times, but the message is confusing. Use kldstat to see if we need to
load the module before saying we do. This fixes the vast majority of the
problems. It may be possible to race devmatch with a user invocation and
devd, though quite hard. In that case we'll announce things twice, but
still only load it once. No attempt is made to fix this.
Warner Losh [Thu, 8 Jul 2021 19:44:21 +0000 (13:44 -0600)]
devmatch: Be tolerant of .ko being present.
We document that we did not need .ko on the module names in
devmatch_blocklist, but we really needed them. Keep the documentation
the same, but strip the .ko when we need to use the names so you can
specify either.
devmatch loads a number of things automatically. Allow the list of
things to load to happen first in case those drivers affect what would
be loaded. Normally, this will produce the same results, but there's
some special cases that may not when drivers are loaded that report
other drivers missing, like virtio_pci.
Warner Losh [Thu, 11 Mar 2021 15:42:09 +0000 (08:42 -0700)]
config_intrhook: provide config_intrhook_drain
config_intrhook_drain will remove the hook from the list as
config_intrhook_disestablish does if the hook hasn't been called. If it has,
config_intrhook_drain will wait for the hook to be disestablished in the normal
course (or expedited, it's up to the driver to decide how and when
to call config_intrhook_disestablish).
This is intended for removable devices that use config_intrhook and might be
attached early in boot, but that may be removed before the kernel can call the
config_intrhook or before it ends. To prevent all races, the detach routine will
need to call config_intrhook_train.
Sponsored by: Netflix, Inc
Reviewed by: jhb, mav, gde (in D29006 for man page)
Differential Revision: https://reviews.freebsd.org/D29005
Rather than allocating however much memory userspace asks for we only
allocate enough for a handful of states, and copy to userspace for each
completed row.
We start out with enough space for 16 states (per row), but grow that as
required. In most configurations we expect at most a handful of states
per row (more than that would have other negative effects on packet
processing performance).
Happily this wasn't a real bug, because pf_killstates() never fails, but
we should check the return value anyway, in case it does ever start
returning errors.
pidx is never NULL, and is used unconditionally later on in the
function.
Add an assertion, as documentation for the requirement to provide an idx
pointer.
Eugene Grosbein [Mon, 17 May 2021 21:03:15 +0000 (04:03 +0700)]
ipfw: reload sysctl.conf variables if needed
Currently ipfw has multiple components that are not parts
of GENERIC kernel like dummynet etc. They can bring in important
sysctls if enabled with rc.conf(5) and loaded with ipfw startup script
by means of "required_modules" after initial consult
with /etc/sysctl.conf at boot time. Here is an example of one
increasing limit for dummynet hold queues that defaults to 100:
net.inet.ip.dummynet.pipe_slot_limit=1000
This makes it possible to use ipfw/dummynet rules such as:
ipfw pipe 1 config bw 50Mbit/s queue 1000
Such rule is rejected unless above sysctl is applied.
Another example is a group of net.inet.ip.alias.* sysctls
created after libalias.ko loaded as dependency of ipfw_nat.
This is not a problem if corresponding code compiled in custom kernel
so sysctls exist when sysctl.conf is read early or kernel modules
loaded with a loader. This change makes it work also for GENERIC
and modules loaded by means of rc.conf(5) settings.
Eugene Grosbein [Wed, 19 May 2021 13:02:31 +0000 (20:02 +0700)]
rc.d: unbreak sysctl lastload
/etc/rc.d/securelevel is supposed to run /etc/rc.d/sysctl lastload
late at boot time to apply /etc/sysctl.conf settings that fail
to apply early. However, this does not work in default configuration
because of kern_securelevel_enable="NO" by default.
Add new script /etc/rc.d/sysctl_lastload that starts unconditionally.
Warner Losh [Fri, 16 Jul 2021 04:05:31 +0000 (22:05 -0600)]
UPDATING: Not unusual side effect of the awk bug fixed in 40a925385fa6b
You might not be able to build the kernel if you have an awk between Jul
10th and today. In stable/12, more (all) platforms than -current or
stable/13 have the first offset of the thread structure in the 0xfxx
range, which triggers this problem. If you've updated awk, you'll likely
need to follow the instructions here to install a fixed version before
you'll be able to build a kernel.
Warner Losh [Thu, 15 Jul 2021 22:46:06 +0000 (16:46 -0600)]
awk: revert upstream's attempt to disallow hex strings
Upstream one-true-awk decided to disallow hex strings as numbers. This
is in line with awk's behavior prior to C99, and allowed by the POSIX
standard. The standard, however, allows them to be treated as numbers
because that's what the standard said in the 2001 through 2004 editions.
Since 2001, the nawk in FreeBSD has treated them as numbers, so restore
that behavior, allowed by the standard.
A number of scripts in the FreeBSD tree depend on this interpretation,
including scripts to build the kernel which had mysteriously started
failing for some people and not others. By re-allowing 0x hex numbers,
this fixes those scripts and restores POLA.
Upstream issue: https://github.com/onetrueawk/awk/issues/126
Sponsored by: Netflix
Reviewed by: kevans
MFC After: asap due to regression alrady merged to stable
Differential Revision: https://reviews.freebsd.org/D31199
pf: allow table stats clearing and reading with ruleset rlock
Instead serialize against these operations with a dedicated lock.
Prior to the change, When pushing 17 mln pps of traffic, calling
DIOCRGETTSTATS in a loop would restrict throughput to about 7 mln. With
the change there is no slowdown.
Creating tables and zeroing their counters induces excessive IPIs (14
per table), which in turns kills single- and multi-threaded performance.
Work around the problem by extending per-CPU counters with a general
counter populated on "zeroing" requests -- it stores the currently found
sum. Then requests to report the current value are the sum of per-CPU
counters subtracted by the saved value.
Sample timings when loading a config with 100k tables on a 104-way box:
stock:
pfctl -f tables100000.conf 0.39s user 69.37s system 99% cpu 1:09.76 total
pfctl -f tables100000.conf 0.40s user 68.14s system 99% cpu 1:08.54 total
patched:
pfctl -f tables100000.conf 0.35s user 6.41s system 99% cpu 6.771 total
pfctl -f tables100000.conf 0.48s user 6.47s system 99% cpu 6.949 total
Stefan Eßer [Sat, 10 Jul 2021 11:00:56 +0000 (13:00 +0200)]
libalias: fix divide by zero causing panic
The packet_limit can fall to 0, leading to a divide by zero abort in
the "packets % packet_limit".
An possible solution would be to apply a lower limit of 1 after the
calculation of packet_limit, but since any number modulo 1 gives 0,
the more efficient solution is to skip the modulo operation for
packet_limit <= 1.
Michael Tuexen [Fri, 11 Jun 2021 18:14:34 +0000 (20:14 +0200)]
tcp: tolerate missing timestamps
Some TCP stacks negotiate TS support, but do not send TS at all
or not for keep-alive segments. Since this includes modern widely
deployed stacks, tolerate the violation of RFC 7323 per default.
Alexander Motin [Tue, 6 Jul 2021 02:19:48 +0000 (22:19 -0400)]
nvme(4): Report NPWA before NPWG as stripesize.
New Samsung 980 SSDs report Namespace Preferred Write Alignment of
8 (4KB) and Namespace Preferred Write Granularity of 32 (16KB).
My quick tests show that 16KB is a minimal sequential write size
when the SSD reaches peak IOPS, so writing much less is very slow.
But writing slightly less or slightly more does not change much,
so it seems not so much a size granularity as minimum I/O size.
Thinking about different stripesize consumers:
- Partition alignment should be based on NPWA by definition.
- ZFS ashift in part of forcing alignment of all I/Os should also
be based on NPWA. In part of forcing size granularity, if really
needed, it may be set to NPWG, but too big value can make ZFS too
space-inefficient, and the 16KB is actually the biggest supported
value there now.
- ZFS recordsize/volblocksize could potentially be tuned up toward
NPWG to work as I/O size granularity, but enabled compression makes
it too fuzzy. And those are normally user-configurable things.
- ZFS I/O aggregation code could definitely use Optimal Write Size
value and may be NPWG, but we don't have fields in GEOM now to report
the minimal and optimal I/O sizes, and even maximal is not reported
outside GEOM DISK to be used by ZFS.
Import the latest bsd-features branch of the one-true-awk upstream:
o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)
Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.
Alex Richardson [Mon, 21 Sep 2020 19:03:07 +0000 (19:03 +0000)]
awk: Fix subobject out-of-bounds access
When matching a regex with ^, it would attempt to access
gototab[NSTATES][NCHARS+2], and therefore access the state for the \002
character instead. This change is required to run awk under CHERI (with
sub-object bounds) and when running with UBSan instrumentation.
Warner Losh [Mon, 3 Jun 2019 05:25:22 +0000 (05:25 +0000)]
Another partial revert of r301289.
In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.
Warner Losh [Mon, 3 Jun 2019 05:25:16 +0000 (05:25 +0000)]
Fix mismerge that crept into r301289.
The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.
Warner Losh [Sun, 2 Jun 2019 16:25:07 +0000 (16:25 +0000)]
Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git
Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).
Chuck Tuffli [Sun, 27 Jun 2021 22:14:52 +0000 (15:14 -0700)]
bhyve: Fix NVMe iovec construction for large IOs
The UEFI driver included with Rocky Linux 8.4 uncovered an existing bug
in the NVMe emulation's construction of iovec's.
By default, NVMe data transfer operations use a scatter-gather list in
which all entries point to a fixed size memory region. For example, if
the Memory Page Size is 4KiB, a 2MiB IO requires 512 entries. Lists
themselves are also fixed size (default is 512 entries).
Because the list size is fixed, the last entry is special. If the IO
requires more than 512 entries, the last entry in the list contains the
address of the next list of entries. But if the IO requires exactly 512
entries, the last entry points to data.
The NVMe emulation missed this logic and unconditionally treated the
last entry as a pointer to the next list. Fix is to check if the
remaining data is greater than the page size before using the last entry
as a pointer to the next list.
Kristof Provost [Tue, 29 Jun 2021 08:26:40 +0000 (10:26 +0200)]
pf: Reduce the data returned in DIOCGETSTATESNV
This call is particularly slow due to the large amount of data it
returns. Remove all fields pfctl does not use. There is no functional
impact to pfctl, but it somewhat speeds up the call.
It might affect other (i.e. non-FreeBSD) code that uses the new
interface, but this call is very new, so there's unlikely to be any. No
releases contained the previous version, so we choose to live with the
ABI modification.
projects / FreeBSD / FreeBSD.git / log