Kristof Provost [Tue, 29 Aug 2023 15:16:19 +0000 (17:16 +0200)]
snmp_pf: use libpfctl's pfctl_get_status() rather than DIOCGETSTATUS
Prefer libpfctl functions over direct access to the ioctl whenever
possible. This will allow subsequent removal of DIOCGETSTATUS (in 15) as
there already is an nvlist-based alternative.
Kristof Provost [Tue, 29 Aug 2023 15:04:17 +0000 (17:04 +0200)]
libpfctl: implement status counter accessor functions
The new nvlist-based status call allows us to easily add new counters.
However, the libpfctl interface defines a TAILQ, so it's not quite
trivial to find the counter consumers are interested in.
Provide convenience functions to access the counters.
Kristof Provost [Tue, 29 Aug 2023 15:00:44 +0000 (17:00 +0200)]
pf (t)ftp-proxy: use libpfctl instead of DIOCGETSTATUS
Prefer libpfctl functions over direct access to the ioctl whenever
possible. This will allow subsequent removal of DIOCGETSTATUS (in 15) as
there already is an nvlist-based alternative.
Kristof Provost [Thu, 31 Aug 2023 07:32:54 +0000 (09:32 +0200)]
vmxnet3: do restart on VLAN changes
At least one user reports issues with vmx interfaces after 725e4008ef,
where we default to not resetting the interface on VLAN changes. This
was on an ESXi 7.0.3 setup.
nss_tacplus: Provide dummy setpwent(), getpwent_r(), endpwent().
These aren't really needed, since TACACS+ does not support enumeration, but providing placeholders keeps nsdispatch() from complaining that they're missing.
Simon J. Gerraty [Wed, 30 Aug 2023 14:46:08 +0000 (07:46 -0700)]
Add sys.dirdeps.mk to share/mk FILES
A few recent makefiles should have been added to FILES.
Rename sys.machine.mk to local.sys.machine.mk as it is very
tree specific so does not belong in /usr/share/mk/
Zhenlei Huang [Wed, 30 Aug 2023 09:36:38 +0000 (17:36 +0800)]
net: Remove vlan metadata on pcp / vlan encapsulation
For oubound traffic, the flag M_VLANTAG is set in mbuf packet header to
indicate the underlaying interface do hardware VLAN tag insertion if
capable, otherwise the net stack will do 802.1Q encapsulation instead.
Commit 868aabb4708d introduced per-flow priority which set the priority ID
in the mbuf packet header. There's a corner case that when the driver is
disabled to do hardware VLAN tag insertion, and the net stack do 802.1Q
encapsulation, then it will result double tagged packets if the driver do
not check the enabled capability (hardware VLAN tag insertion).
Unfortunately some drivers, currently known cxgbe(4) re(4) ure(4) igc(4)
and vmx(4), have this issue. From a quick review for other interface
drivers I believe a lot more drivers have the same issue. It makes more
sense to fix in net stack than to try to change every single driver.
Kristof Provost [Tue, 29 Aug 2023 09:23:49 +0000 (11:23 +0200)]
igmp: do not upgrade IGMP version beyond net.inet.igmp.default_version
IGMP requires hosts to use the lowest version they've seen on the
network. When the IGMP timers expire we take the opportunity to upgrade again.
However, we did not take the net.inet.igmp.default_version sysctl
setting into account, so we could end up switching to IGMPv3 even if the
user had requested IGMPv2 or IGMPv1 via the sysctl.
Check V_igmp_default_version before we upgrade the IGMP version.
John Baldwin [Tue, 29 Aug 2023 21:39:36 +0000 (14:39 -0700)]
libcrypto: Refactor Makefile.asm so it can be run outside of buildenv
Currently Makefile.asm relies on the current buildenv to set CFLAGS
for i386. The current approach also leaves various temporary *.s
files around in the current directory. To make this a bit better:
- Instead of using CFLAGS from buildenv for i386, define the actual
flags the perl scripts need: -DOPENSSL_IA32_SSE2 to enable SSE2.
- Change i386 to have the perl scripts write to /dev/stdout to avoid
creating temporaries. Previously i386 was generating the temporary
files in the OpenSSL contrib src.
- Cleanup temporary *.s files in the all target after generating the
real *.S files for architectures which need them.
Tom Cosgrove [Tue, 29 Aug 2023 21:38:11 +0000 (14:38 -0700)]
OpenSSL: Fix handling of the "0:" label in arm-xlate.pl
When $label == "0", $label is not truthy, so `if ($label)` thinks there isn't
a label. Correct this by looking at the result of the s/// command.
Verified that there are no changes in the .S files created during a normal
build, and that the "0:" labels appear in the translation given in the error
report (and they are the only difference in the before and after output).
Ed Maste [Fri, 18 Aug 2023 03:39:08 +0000 (23:39 -0400)]
x86: Introduce APIC ID limit by default on AMD hardware
Lack of an AMD IOMMU driver means we cannot successfully route
interrupts to APIC IDs 255 and over. Do not add the corresponding CPUs
to the per-domain lists of CPUs to which interrupts can be assigned.
This change should be reverted (or, at least the APIC ID limit) once we
have an AMD IOMMU / interrupt remapping driver.
See also commits fa5f94140a83 ("msi: handle error from BUS_REMAP_INTR in
msi_assign_cpu") and 4258eb5a0d97 ("x86: handle domains with no CPUs
usable for intr delivery.").
Reviewed by: markj, jhb
Tested by: cperciva (earlier version)
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41618
Mark Johnston [Tue, 29 Aug 2023 01:26:53 +0000 (21:26 -0400)]
aesni: Push FPU sections down further
After commit 937b4473be21 aesni_cipher_crypt() and aesni_cipher_mac()
execute in a FPU_KERN_NOCTX section, which means that they must run with
preemption disabled. These functions handle discontiguous I/O buffers
by allocating a contiguous buffer and copying as necessary, but this
allocation cannot happen with preemption disabled. Fix the problem by
pushing the FPU section down into aesni_cipher_crypt() and
aesni_cipher_mac(). In particular, encrypt-then-auth transforms need
not be handled with a single FPU section.
Reported by: syzbot+78258dbb02eb92157357@syzkaller.appspotmail.com
Discussed with: jhb
Fixes: 937b4473be21 ("aesni: Switch to using FPU_KERN_NOCTX.")
Justin Hibbits [Mon, 28 Aug 2023 23:27:11 +0000 (19:27 -0400)]
spibus: Make ofw_spibus probe just a little more favored
With ade70a1ad(svn r332196) ofw_spibus probes at the BUS_PROBE_DEFAULT
instead of 0. However, this races with spibus, resulting in ofw_spibus
often times losing the race and the OFW node not being referenced. This
in turn causes child device tree nodes to not be attached. Solve this
by returning 1 higher than spibus, just like acpi_spibus.
Sponsored by: Juniper Networks, Inc.
MFC after: 1 week
Jamie Gritton [Mon, 28 Aug 2023 18:22:36 +0000 (11:22 -0700)]
jail: make jail(8) man page more readable and more correct
The synopsis section of jail(8) is fine at showing everything that could
be on the command line, but doesn't make much sense. Add some sub-
ections for the different uses of the command.
Also fix up the paragraph about command-line parameter specification,
including removing some clearly erroneous information.
Reviewed by: dvl
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D41606
This adds formatted input/output of binary integer numbers to the printf(), scanf(), and strtol() families, including their wide-character counterparts.
libzfs uses librt as a dependency. Following 315ee00fa961 systems with
a separate / and /usr will fail to load the libzfs.so library because
librt.so is not available due to the fact that /usr is not mounted yet.
Install librt in /lib making it available to libzfs.
Pietro Cerutti [Thu, 17 Aug 2023 11:42:23 +0000 (11:42 +0000)]
netcat: add --crlf to convert LF into CRLF
This adds the --crlf option to netcat, which triggers translation of \n
characters into \r\n sequences in the input -> network direction.
The Linux version of nc also supports this functionality with --crlf and
-C. The OpenBSD version uses -C to specify client certificates. Our
version is too old and doesn't have it, but I avoided adding -C anyway
to ease future syncs with upstream.
Attempts to upstream the feature were unsuccessful:
https://marc.info/?t=169282068500001
Wei Hu [Mon, 28 Aug 2023 09:15:16 +0000 (09:15 +0000)]
mana: batch ringing RX queue doorbell on receiving packets
It's inefficient to ring the doorbell page every time a WQE is posted to
the received queue. Excessive MMIO writes result in CPU spending more
time waiting on LOCK instructions (atomic operations), resulting in
poor scaling performance.
Move the code for ringing doorbell page to where after we have posted all
WQEs to the receive queue in mana_poll_rx_cq().
In addition, use the correct WQE count for ringing RQ doorbell.
The hardware specification specifies that WQE_COUNT should set to 0 for
the Receive Queue. Although currently the hardware doesn't enforce the
check, in the future releases it may check on this value.
Tested by: whu
MFC after: 1 week
Sponsored by: Microsoft
Notable upstream pull request merges:
#15024 Add missed DMU_PROJECTUSED_OBJECT prefetch
#15029 Do not request data L1 buffers on scan prefetch
#15036 FreeBSD: catch up to __FreeBSD_version 1400093
#15039 Fix raw receive with different indirect block size
#15047 FreeBSD: Fix build on stable/13 after 1302506
#15049 Fix the ZFS checksum error histograms with larger record sizes
#15052 Reduce bloat in ereport.fs.zfs.checksum events
#15056 Avoid extra snprintf() in dsl_deadlist_merge()
#15061 Ignore pool ashift property during vdev attachment
#15063 Don't panic if setting vdev properties is unsupported for this vdev type
#15067 spa_min_alloc should be GCD, not min
#15071 Add explicit prefetches to bpobj_iterate()
#15072 Adjust prefetch parameters
#15076 Refactor dmu_prefetch()
#15079 set autotrim default to 'off' everywhere
#15080 ZIL: Fix config lock deadlock
#15088 metaslab: tuneable to better control force ganging
#15096 Avoid waiting in dmu_sync_late_arrival()
#15097 BRT should return EOPNOTSUPP
#15103 Remove zl_issuer_lock from zil_suspend()
#15107 Remove fastwrite mechanism
#15113 libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1
#15122 ZIL: Second attempt to reduce scope of zl_issuer_lock
#15129 zpool_vdev_remove() should handle EALREADY error return
#15132 ZIL: Replay blocks without next block pointer
#15148 zfs_clone_range should return descriptive error codes
#15153 ZIL: Avoid dbuf_read() before dmu_sync()
#15172 copy_file_range: fix fallback when source create on same txg
#15180 Update outdated assertion from zio_write_compress
Justin Hibbits [Fri, 25 Aug 2023 15:36:35 +0000 (11:36 -0400)]
dtsec: Support multicast receive.
Implemented based on the tsec(4) multicast support. This is the minimum
required to support VLANs. The hardware does support vlan tagging,
among other acceleration features, which will be added at a later time.
Doug Rabson [Sat, 26 Aug 2023 09:32:32 +0000 (10:32 +0100)]
Fix MNT_IGNORE for devfs, fdescfs and nullfs
The MNT_IGNORE flag can be used to mark certain filesystem mounts so
that utilities such as df(1) and mount(8) can filter out those mounts by
default. This can be used, for instance, to reduce the noise from
running container workloads inside jails which often have at least three
and sometimes as many as ten mounts per container.
The flag is supplied by the nmount(2) system call and is recorded so
that it can be reported by statfs(2). Unfortunately several filesystems
override the default behaviour and mask out the flag, defeating its
purpose. This change preserves the MNT_IGNORE flag for those filesystems
so that it can be reported correctly.
Kyle Evans [Sat, 26 Aug 2023 01:01:47 +0000 (20:01 -0500)]
caroot: update the root bundle
Summary:
- Six (6) new roots
- Four (4) distrusted roots
Note that this was intentionally generated with OpenSSL 1.1.1 to avoid
mixing updates and non-functional changes -- there will be some churn
with OpenSSL 3. The next commit will update the current batch of
trusted certs with the format OpenSSL 3 produces, which I've tested
against OpenSSL 1.1.1 to be sure that that doesn't hurt us in older
branches.
Kyle Evans [Sat, 26 Aug 2023 00:55:32 +0000 (19:55 -0500)]
caroot: drop VERSION tags from certs
With this change, we'll drop the "with $FreeBSD$" lines from trusted/
certs in the next update. untrusted/ will need to be done manually, but
I'll likely just do them all manually, commit, then run the script and
commit any legitimate updates after confirming the output matches what
I did manually.
Mike Karels [Fri, 25 Aug 2023 20:11:46 +0000 (15:11 -0500)]
clk_fixed: reduce and clarify messages due to missing clk-frequency
The current dts for Raspberry 3B+ and 4B have 2 disabled fixed-clock
nodes which have no clock-frequency value. Such nodes are non-standard.
This results in about 150 messages during autoconfiguration:
clk_fixed2: <Fixed clock> disabled on ofwbus0
clk_fixed2: Cannot FDT parameters.
device_attach: clk_fixed2 attach returned 6
Defensively check for clock-frequency in the probe routine, and if
none, return an error with a more direct error message (1 line per
occurrence, 50 lines total):
clk_fixed2: clock-fixed has no clock-frequency
Based on a patch by manu.
MFC after: 1 week
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D41594
lib/libc/amd64/string/strchrnul.S: fix edge case in scalar code
When the buffer is immediately preceeded by the character we
are looking for and begins with one higher than that character,
and the buffer is misaligned, a match was errorneously detected
in the first character. Fix this by changing the way we prevent
matches before the buffer from being detected: instead of
removing the corresponding bit from the 0x80..80 mask, set the
LSB of bytes before the buffer after xoring with the character we
look for.
The bug only affects amd64 with ARCHLEVEL=scalar (cf. simd(7)).
The change comes at a 2% performance impact for short strings
if ARCHLEVEL is set to scalar. The default configuration is not
affected.
Gleb Smirnoff [Fri, 25 Aug 2023 17:31:55 +0000 (10:31 -0700)]
ifconfig: fix 'ifconfig -l link'
The rtnetlink(4) RTM_GETADDR does not list link level addresses, thus
the correct match for interfaces that have a link level address should
be based on what was returned by RTM_GETLINK.
Rob N [Fri, 25 Aug 2023 17:31:29 +0000 (03:31 +1000)]
tests/block_cloning: rename and document get_same_blocks helper
`get_same_blocks` is a helper to compare two files and return a list of
the blocks that are clones of each other. Its very necessary for block
cloning tests.
Previously it was incorrectly called `unique_blocks`, which is the
_inverse_ of what it does (an early version did list unique blocks; it
was changed but the name was not). So if nothing else, it should be
called `duplicate_blocks`.
But, keeping the details of a clone operation in your head is actually
quite difficult, without the additional overhead of wondering how the
tools work. So I've renamed it to better describe what it does, added a
usage note, and changed it to return block indexes from 0 instead of 1,
to match how L0 blocks are normally counted.
Reviewed-by: Umer Saleem <usaleem@ixsystems.com> Reviewed-by: Kay Pedersen <mail@mkwg.de> Signed-off-by: Rob Norris <robn@despairlabs.com>
Closes #15181
Gleb Smirnoff [Fri, 25 Aug 2023 17:31:26 +0000 (10:31 -0700)]
ifconfig: fix 'ifconfig -l ether'
When matching interfaces for being Ethernet, use same trick that
the link module does - pass if_type through convert_iftype().
That restores historicaly behaviour of listing lagg(4) ports.
As part of some internal gang block testing within Delphix
we hit the assertion removed by this patch. The assertion
was triggered by a ZIO that had two copies and was a gang
block making the following expression equal to 3:
```
MIN(zp->zp_copies + BP_IS_GANG(bp), spa_max_replication(spa))
```
and failing when we expected the above to be equal to
`BP_GET_NDVAS(bp)`.
The assertion is no longer valid since the following commit:
```
commit 14872aaa4f909d72c6b5e4105dadcfa13c7d9d66
Author: Matthew Ahrens <matthew.ahrens@delphix.com>
Date: Mon Feb 6 09:37:06 2023 -0800
EIO caused by encryption + recursive gang
```
The above commit changed gang block headers so they can't
have more than 2 copies but the assertion in question from
this PR was never updated.
Reviewed-by: George Wilson <george.wilson@delphix.com> Reviewed-by: Matthew Ahrens <mahrens@delphix.com> Signed-off-by: Serapheim Dimitropoulos <serapheim@delphix.com>
Closes #15180
Summary:
These came in the original DrvAPI commits in 2014, and are obsoleted by
bpf_mtap_if() and ether_bpf_mtap_if(). The `_if` suffix, rather than
prefix, conveys that it's operating on the bpf of the interface, instead
than the interface itself.
Warner Losh [Fri, 25 Aug 2023 16:10:16 +0000 (10:10 -0600)]
nvme: Add exclusion for ISR
Add a basically uncontended spinlock that we take out while the ISR is
running. This has two effects: First, when we get a timeout, we can
safely call the nvme_qpair_process_completions w/o racing any ISRs.
Second, we can use it to ensure that we don't reset the card while
the ISRs are active (right now we just sleep and hope for the best,
which usually is fine, but not always).
Warner Losh [Fri, 25 Aug 2023 16:10:08 +0000 (10:10 -0600)]
nvme: Greatly improve error recovery
Next phase of error recovery: Eliminate the REOVERY_START phase, since
we don't need to wait to start recovery. Eliminate the RECOVERY_RESET
phase since it is transient, we now transition from RECOVERY_NORMAL into
RECOVERY_WAITING.
In normal mode, read the status of the controller. If it is in failed
state, or appears to be hot-plugged, jump directly to reset which will
sort out the proper things to do. This will cause all pending I/O to
complete with an abort status before the reset.
When in the NORMAL state, call the interrupt handler. This will complete
all pending transactions when interrupts are broken or temporarily
misbehaving. We then check all the pending completions for timeouts. If
we have abort enabled, then we'll send an abort. Otherwise we'll assume
the controller is wedged and needs a reset. By calling the interrupt
handler here, we'll avoid an issue with the current code where we
transitioned to RECOVERY_START which prevented any completions from
happening. Now completions happen. In addition and follow-on I/O that is
scheduled in the completion routines will be submitted, rather than
queued, because the recovery state is correct. This also fixes a problem
where I/O would timeout, but never complete, leading to hung I/O.
Resetting remains the same as before, just when we chose to reset has
changed.
A nice side effect of these changes is that we now do I/O when
interrupts to the card are totally broken. Followon commits will improve
the error reporting and logging when this happens. Performance will be
aweful, but will at least be minimally functional.
There is a small race when we're checking the completions if interrupts
are working, but this is handled in a future commit.
Warner Losh [Fri, 25 Aug 2023 15:45:28 +0000 (09:45 -0600)]
smartpqi: Fix build issues highlighted by gcc12
Make inline functions defined in a header file as static inline.
Mark inline functions used only in the compilation unit they are
defined in as merely static (the compiler can figure out it these
days).
Simon J. Gerraty [Fri, 25 Aug 2023 00:41:22 +0000 (17:41 -0700)]
Add mac_grantbylabel
This module allows controlled privilege escallation via mac labels
securely associated with a process via mac_veriexec.
There are over 700 PRIV_* but we can compress many of them into
a single GBL_* thus constraining the size of gbl labels.
The goal is to allow a daemon to run as an unprivileged process while
still being able a set of privileged operations needed.
We add APIs to libveriexec so that userland processes can check labels
and an exec_script API that allows a suitably labeled process to run
something like a python interpreter directly if necessary;
overcomming the 'indirect' flag applied to the interpreter.
Alexander Motin [Fri, 25 Aug 2023 00:08:49 +0000 (20:08 -0400)]
ZIL: Second attempt to reduce scope of zl_issuer_lock.
The previous patch #14841 appeared to have significant flaw, causing
deadlocks if zl_get_data callback got blocked waiting for TXG sync. I
already handled some of such cases in the original patch, but issue
#14982 shown cases that were impossible to solve in that design.
This patch fixes the problem by postponing log blocks allocation till
the very end, just before the zios issue, leaving nothing blocking after
that point to cause deadlocks. Before that point though any sleeps are
now allowed, not causing sync thread blockage. This require slightly
more complicated lwb state machine to allocate blocks and issue zios
in proper order. But with removal of special early issue workarounds
the new code is much cleaner now, and should even be more efficient.
Since this patch uses null zios between write, I've found that null
zios do not wait for logical children ready status in zio_ready(),
that makes parent write to proceed prematurely, producing incorrect
log blocks. Added ZIO_CHILD_LOGICAL_BIT to zio_wait_for_children()
fixes it.
Reviewed-by: Rob Norris <rob.norris@klarasystems.com> Reviewed-by: Mark Maybee <mark.maybee@delphix.com> Reviewed-by: George Wilson <george.wilson@delphix.com>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored by: iXsystems, Inc.
Closes #15122
Mateusz Guzik [Thu, 24 Aug 2023 05:34:08 +0000 (05:34 +0000)]
vfs: try harder to find free vnodes when recycling
The free vnode marker can slide past eligible entries.
Artificially reducing vnode limit to 300k and spawning 104 workers each
creating a million files results in all of them trying to recycle, which
often fails when it should not have to.
Because of the excessive traffic in this scenario, the trylock to
requeue is virtually guaranteed to fail, meaning nothing gets pushed
forward.
Since no vnodes were found, the most unfortunate sleep for 1 second is
induced (see vn_alloc_hard, the "vlruwk" msleep).
Without the fix the machine is mostly idle with almost everyone stuck
off CPU waiting for the sleep to finish. With the fix it is busy
creating files.
Unrelated to the above problem the marker could have landed in a
similarly problematic spot for because of any failure in vtryrecycle.
Originally reported as poudriere builders stalling in a vnode-count
restricted setup.
Fixes: 138a5dafba31 ("vfs: trylock vnode requeue")
Reported by: Mark Millard
Kevin Bowling [Thu, 24 Aug 2023 20:42:23 +0000 (13:42 -0700)]
iflib: invert default restart on VLAN changes
In rS360398, a new iflib device method was added to opt out of VLAN
events needing an interface reset.
I am switching the default to not requiring a restart for:
* VLAN events
* unknown events
After fixing various bugs, I do not think this would be a common need
of hardware and it is undesirable from the user's perspective causing
link flaps and much slower VLAN configuration. Currently, there are no
other restart events besides VLAN events, and setting the
ifdi_needs_restart default to false will alleviate the need to churn
every driver if an odd event is added in the future for specific
hardware.
markj points out this could cause churn in the other direction; I will
solve that problem with an event registration system as he mentions in
the review should we need it in the future.
These drivers will opt into restart and need further inspection or work:
* ixv (needs code audit, 61a8231 fixed principal issue; re-init probably
not necessary)
* axgbe (needs code audit; re-init probably not necessary)
* iavf - (needs code audit; interaction with Malicious Driver Detection
mentioned in rS360398)
* mgb - no VLAN functions are currently implemented. Left a comment.
Kevin Bowling [Thu, 24 Aug 2023 20:16:24 +0000 (13:16 -0700)]
bnxt: Don't restart on VLAN changes
In rS360398, a new iflib device method was added with default of opt out
for VLAN events needing an interface reset.
This is unintentional for bnxt(4) and is causing another bug in its VLAN
initialization code to affect the common case of adding and removing
VLANs on an existing interface.
Jake Freeland [Thu, 24 Aug 2023 04:39:54 +0000 (22:39 -0600)]
timerfd: Move implementation from linux compat to sys/kern
Move the timerfd impelemntation from linux compat code to sys/kern. Use
it to implement the new system calls for timerfd. Add a hook to kern_tc
to allow timerfd to know when the system time has stepped. Add kqueue
support to timerfd. Adjust a few names to be less Linux centric.
Tony Hutter [Thu, 24 Aug 2023 18:59:03 +0000 (11:59 -0700)]
zed: Add zedlet to power off slot when drive is faulted
If ZED_POWER_OFF_ENCLOUSRE_SLOT_ON_FAULT is enabled in zed.rc, then
power off the drive's slot in the enclosure if it becomes FAULTED.
This can help silence misbehaving drives. This assumes your drive
enclosure fully supports slot power control via sysfs.
Reviewed-by: @AllKind Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tony Hutter <hutter2@llnl.gov>
Closes #15200
projects / FreeBSD / FreeBSD.git / log