Brooks Davis [Wed, 17 Nov 2021 20:12:26 +0000 (20:12 +0000)]
freebsd32: sync return types with default ABI
This consists of int -> ssize_t where required and one int -> mode_t.
As a rule, return types are informative rather than functional as the
actual return is in a register.
Brooks Davis [Wed, 17 Nov 2021 20:12:26 +0000 (20:12 +0000)]
freebsd32: rename 32-bit compat pads to _pad
Some 32-bit architectures pass 64-bit values in aligned
register pairs (a0,a1), (a2,a3) etc. In freebsd32 we add these pads
explicitly from compat code. We also sometimes add pads in the default
ABI. Differentiate the two by making the freebsd32 ones int _pad.
In a future commit the 32-bit ones will be automatically generated.
Brooks Davis [Wed, 17 Nov 2021 20:12:26 +0000 (20:12 +0000)]
freebsd32: fix getfsstat sign extension bugs
Add freebsd32 versions of getfsstat and freebsd11_getfsstat so that
bufsize is properly sign-extended if a negative value is passed.
Reject negative values before passing to kern_getfsstat as a size_t.
Brooks Davis [Wed, 17 Nov 2021 20:12:26 +0000 (20:12 +0000)]
freebsd32: signed long corrections
Syscalls that take signed longs need to treat the 32-bit versions as
signed int so that sign extension happens correctly. Improve
decleration quality and add a few minimal syscall implementations.
Brooks Davis [Wed, 17 Nov 2021 20:12:25 +0000 (20:12 +0000)]
freebsd32: add stubs for ofreebsd32_(send|recv)msg
The upcoming change to generate freebsd32 generated files from
sys/kern/syscalls.master doesn't have a way to handle disabling
this one without disabling the non-COMPAT counterpart so just add
a stub for now.
Brooks Davis [Wed, 17 Nov 2021 20:12:25 +0000 (20:12 +0000)]
freebsd32: add feed foward clock syscalls
These are required when supporting i386 because time_t is 32-bit which
reduces struct bintime to 12-bytes when combined with the fact that 64-bit
integers only requiring 32-bit alignment on i386. Reusing the default
ABI version resulted in 4-byte overreads or overwrites to userspace.
Brooks Davis [Wed, 17 Nov 2021 20:12:25 +0000 (20:12 +0000)]
freebsd32: don't implement kldsym
Previously we fell back to sys_kldsym, but because we'd always
mismatch on the version field we'd return EINVAL. A freebsd32
implementation is impossible with the current ABI as there simply
isn't space to store a kernel virtual address in a uint32_t.
ofreebsd32_sigprocmask, ofreebsd32_sigblock, ofreebsd32_sigsetmask,
and ofreebsd32_sigsuspend were all duplicates of the default ABI
versions and there are no type concerns as all arguments are the
same.
Brooks Davis [Wed, 17 Nov 2021 20:12:24 +0000 (20:12 +0000)]
freebsd32: remove freebsd32_recvfrom
The freebsd32_recvfrom() serves no purpose as no arguments require
translation. The prototype was mis-declared and the implementation
contained (relatively harmless) errors.
Brooks Davis [Wed, 17 Nov 2021 20:12:24 +0000 (20:12 +0000)]
freebsd32: remove redundant no-arg syscalls
pipe requires no special handling.
ofreebsd32_sigpending did differ from osigpending in that it acted
on the siglist rather than the sigqueue, but this appears to be an
oversight in 3fbdb3c21524d9d95278ada1d61b4d1e6bee654b.
ogetpagesize could theoretically have ABI-dependent results, but in
practice does not. If it does it would be easy handle in the central
implementation and be the least of the problems in changing the value of
PAGE_SIZE.
Follow common convention and put the `32` on the end of the struct
name. This is a step toward generating freebsd32 syscall files
from sys/kern/syscalls.master.
Brooks Davis [Wed, 17 Nov 2021 20:12:23 +0000 (20:12 +0000)]
freebsd32: add a union semun_old32
Use this for COMPAT7 support. In practice it's the same as
union semun32 since the pointers become uint32_t's the it's more
symetric and is the logical thing to generate from semun_old.
Brooks Davis [Wed, 17 Nov 2021 20:12:23 +0000 (20:12 +0000)]
freebsd32: fix type size of pointer array args
Make pointers to arrays of pointers `uint32_t *` so the sizes of the
array elements are correct. In an ideal world we'd use something
like __ptr32 annotations instead.
Brooks Davis [Wed, 17 Nov 2021 20:12:23 +0000 (20:12 +0000)]
freebsd32: thread IDs are int32_t
Thread IDs are of type long which means int32_t on 32-bit systems.
While this detail is handled without compat functions, expose it
here as code to generate prototypes from the default syscalls.master
will do so.
Brooks Davis [Wed, 17 Nov 2021 20:12:23 +0000 (20:12 +0000)]
freebsd32: fix types on statfs syscalls
Rename struct statfs32 to struct ostatfs32 to mirror struct ostatfs.
These structs are use for COMPAT4 support. Stop using struct statfs32
for modern implementations as struct statfs uses fixed-width types
and it the same on all architectures.
Rename struct freebsd4_freebsd32_ucontext to struct freebsd4_ucontext32
allowing conversion from the default ABI's struct freebsd4_ucontext
by appending "32". This has no practical effect as this type does not
actually exist.
Give freebsd4_freebsd32_sigreturn an ANSI C prototype.
Allan Jude [Wed, 17 Nov 2021 15:07:05 +0000 (15:07 +0000)]
Fix ZFS module build
resolves: link_elf_obj: symbol abd_checksum_edonr_native undefined
The required module-build bits were originally identified in the
upstream pull request: https://github.com/openzfs/zfs/pull/12735
But were missed when the code was imported (since they are not
committed upstream).
Randall Stewart [Wed, 17 Nov 2021 14:45:51 +0000 (09:45 -0500)]
tcp: Rack ack war with a mis-behaving firewall or nat with resets.
Previously we added ack-war prevention for misbehaving firewalls. This is
where the f/w or nat messes up its sequence numbers and causes an ack-war.
There is yet another type of ack war that we have found in the wild that is
like unto this. Basically the f/w or nat gets a ack (keep-alive probe or such)
and instead of turning the ack/seq around and adding a TH_RST it does something
real stupid and sends a new packet with seq=0. This of course triggers the challenge
ack in the reset processing which then sends in a challenge ack (if the seq=0 is within
the range of possible sequence numbers allowed by the challenge) and then we rinse-repeat.
This will add the needed tweaks (similar to the last ack-war prevention using the same sysctls and counters)
to prevent it and allow say 5 per second by default.
Reviewed by: Michael Tuexen
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D32938
Martin Matuska [Wed, 17 Nov 2021 08:35:14 +0000 (09:35 +0100)]
zfs: merge openzfs/zfs@269b5dadc (master) into main
Notable upstream pull request merges:
#12285 Introduce a tunable to exclude special class buffers from L2ARC
#12689 Check l2cache vdevs pending list inside the vdev_inuse()
#12735 Enable edonr in FreeBSD
#12743 FreeBSD: fix world build after de198f2
#12745 Restore dirty dnode detection logic
Kristof Provost [Mon, 15 Nov 2021 20:14:00 +0000 (21:14 +0100)]
pf: add COMPAT_FREEBSD13 for DIOCKEEPCOUNTERS
DIOCKEEPCOUNTERS used to overlap with DIOCGIFSPEEDV0, which has been
fixed in 14, but remains in stable/12 and stable/13.
Support the old, overlapping, call under COMPAT_FREEBSD13.
Kristof Provost [Tue, 16 Nov 2021 19:46:26 +0000 (20:46 +0100)]
riscv: add COMPAT_FREEBSD12 option
Turn on compat option for older FreeBSD versions (i.e. 12). We do not
enable the compat options for 11 or older because riscv was never
supported in those versions.
Rick Macklem [Wed, 17 Nov 2021 00:02:53 +0000 (16:02 -0800)]
nfsd: Add a new rc variable nfs_server_maxio
Since vfs.nfsd.srvmaxio can only be set when nfsd.ko
is loaded, but nfsd is not running, setting it in
/etc/sysctl.conf is not feasible when "options NFSD"
was not specified for the kernel.
This patch adds a new rc variable nfs_server_maxio,
which sets vfs.nfsd.srvmaxio at the correct time.
Warner Losh [Tue, 16 Nov 2021 23:10:25 +0000 (16:10 -0700)]
Skip -flto for all MIPS ports
There likely should be a macro for the ports that support lto, but I'm
making sure that all the mips things build before decommissioning it and
this is the only thing that's broken...
Because now ASLR is enabled by default for 64-bit architectures
and the purpose of the installation menu is to allow choosing
additional 'mitigation'/'hardening' options that are originally
disabled, remove the ASLR knob from bsdinstall.
Marcin Wojtas [Sun, 24 Oct 2021 14:53:06 +0000 (16:53 +0200)]
Enable ASLR by default for 64-bit executables
Address Space Layout Randomization (ASLR) is an exploit mitigation
technique implemented in the majority of modern operating systems.
It involves randomly positioning the base address of an executable
and the position of libraries, heap, and stack, in a process's address
space. Although over the years ASLR proved to not guarantee full OS
security on its own, this mechanism can make exploitation more difficult.
Tests on the tier 1 64-bit architectures demonstrated that the ASLR is
stable and does not result in noticeable performance degradation,
therefore it should be safe to enable this mechanism by default.
Moreover its effectiveness is increased for PIE (Position Independent
Executable) binaries. Thanks to commit 9a227a2fd642 ("Enable PIE by
default on 64-bit architectures"), building from src is not necessary
to have PIE binaries. It is enough to control usage of ASLR in the
OS solely by setting the appropriate sysctls.
This patch toggles the kernel settings to use address map randomization
for PIE & non-PIE 64-bit binaries. It also disables SBRK, in order
to allow utilization of the bss grow region for mappings. The latter
has no effect if ASLR is disabled, so apply it to all architectures.
As for the drawbacks, a consequence of using the ASLR is more
significant VM fragmentation, hence the issues may be encountered
in the systems with a limited address space in high memory consumption
cases, such as buildworld. As a result, although the tests on 32-bit
architectures with ASLR enabled were mostly on par with what was
observed on 64-bit ones, the defaults for the former are not changed
at this time. Also, for the sake of safety keep the feature disabled
for 32-bit executables on 64-bit machines, too.
The committed change affects the overall OS operation, so the
following should be taken into consideration:
* Address space fragmentation.
* A changed ABI due to modified layout of address space.
* More complicated debugging due to:
* Non-reproducible address space layout between runs.
* Some debuggers automatically disable ASLR for spawned processes,
making target's environment different between debug and
non-debug runs.
In order to confirm/rule-out the dependency of any encountered issue
on ASLR it is strongly advised to re-run the test with the feature
disabled - it can be done by setting the following sysctls
in the /etc/sysctl.conf file:
kern.elf64.aslr.enable=0
kern.elf64.aslr.pie_enable=0
Mark Johnston [Tue, 16 Nov 2021 18:36:30 +0000 (13:36 -0500)]
sctp: Use m_apply() to calcuate a checksum for an mbuf chain
m_apply() works on unmapped mbufs, so this will let us elide
mb_unmapped_to_ext() calls preceding sctp_calculate_cksum() calls in
the network stack.
Modify sctp_calculate_cksum() to assume it's passed an mbuf header.
This assumption appears to be true in practice, and we need to know the
full length of the chain.
No functional change intended.
Reviewed by: tuexen, jhb
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32941
Mark Johnston [Tue, 16 Nov 2021 18:31:04 +0000 (13:31 -0500)]
mbuf: Only allow extpg mbufs if the system has a direct map
Some upcoming changes will modify software checksum routines like
in_cksum() to operate using m_apply(), which uses the direct map to
access packet data for unmapped mbufs. This approach of course does not
work on platforms without a direct map, so we have to disallow the use
of unmapped mbufs on such platforms.
I believe this is the right tradeoff: we only configure KTLS on amd64
and arm64 today (and one KTLS consumer, NFS TLS, requires a direct map
already), and the use of unmapped mbufs with plain sendfile is a recent
optimization. If need be, m_apply() could be modified to create
CPU-private mappings of extpg mbuf pages as a fallback.
So, change mb_use_ext_pgs to be hard-wired to zero on systems without a
direct map. Note that PMAP_HAS_DMAP is not a compile-time constant on
some systems, so the default value of mb_use_ext_pgs has to be
determined during boot.
Reviewed by: jhb
Discussed with: gallatin
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32940
Colin Percival [Tue, 16 Nov 2021 18:24:05 +0000 (10:24 -0800)]
rtsol/rtsold: Add option to skip random delay
In accordance with a SHOULD in RFC 4861, rtsol and rtsold wait a
random time between zero and one (aka MAX_RTR_SOLICITATION_DELAY)
seconds before sending a Router Solicitation, in order to avoid
network congestion if many hosts come online at once. (The
question of how many hosts would be required to cause congestion
by each sending a single packet on a Gbps+ network is left to the
reader.)
The new option -i disables this wait and instructs rtsol and rtsold
to send the Router Solicitation immediately.
Colin Percival [Sat, 13 Nov 2021 16:38:09 +0000 (08:38 -0800)]
randomdev: Remove 100 ms sleep from write routine
This was introduced in 2014 along with the comment (which has since
been deleted):
/* Introduce an annoying delay to stop swamping */
Modern cryptographic random number generators can ingest arbitrarily
large amounts of non-random (or even maliciously selected) input
without losing their security.
Depending on the number of "boot entropy files" present on the system,
this can speed up the boot process by up to 1 second.
The inline function vn_flush_cached_data() in vnode.h
must not be compiled when building BASE.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@ixsystems.com> Reviewed-by: Allan Jude <allan@klarasystems.com> Signed-off-by: Martin Matuska <mm@FreeBSD.org>
Closes #12743
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@ixsystems.com> Signed-off-by: Pawel Jakub Dawidek <pawel@dawidek.net>
Closes #12748
George Amanakis [Thu, 11 Nov 2021 20:52:16 +0000 (21:52 +0100)]
Introduce a tunable to exclude special class buffers from L2ARC
Special allocation class or dedup vdevs may have roughly the same
performance as L2ARC vdevs. Introduce a new tunable to exclude those
buffers from being cacheable on L2ARC.
Reviewed-by: Don Brady <don.brady@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: George Amanakis <gamanakis@gmail.com>
Closes #11761
Closes #12285
наб [Thu, 11 Nov 2021 20:27:37 +0000 (21:27 +0100)]
Remove basename(1). Clean up/shorten some coreutils pipelines
Basenames that remain, in cmd/zed/zed.d/statechange-led.sh:
dev=$(basename "$(echo "$therest" | awk '{print $(NF-1)}')")
vdev=$(basename "$ZEVENT_VDEV_PATH")
I don't wanna interfere with #11988
Fedor Uporov [Thu, 11 Nov 2021 19:54:15 +0000 (11:54 -0800)]
Check l2cache vdevs pending list inside the vdev_inuse()
The l2cache device could be added twice because vdev_inuse() does not
check spa_l2cache for added devices. Make l2cache vdevs inuse checking
logic more closer to spare vdevs.
Reviewed-by: George Amanakis <gamanakis@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Fedor Uporov <fuporov.vstack@gmail.com>
Closes #9153
Closes #12689
Fedor Uporov [Thu, 11 Nov 2021 19:26:18 +0000 (11:26 -0800)]
zhack: Add repair label option
In case if all label checksums will be invalid on any vdev, the pool
will become unimportable. The zhack with newly added cli options could
be used to restore label checksums and make pool importable again.
Palash Gandhi [Thu, 11 Nov 2021 15:46:44 +0000 (07:46 -0800)]
ZTS: zfs_list_004_neg should not check paths that belong to ZFS
When ZFS is on root, /tmp is a ZFS. This causes zfs_list_004_neg to
fail since `zfs list` on /tmp passes when the test expects it not to.
The fix is to exclude paths that belong to ZFS.
Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Palash Gandhi <pbg4930@rit.edu>
Closes #12744
Brian Behlendorf [Thu, 11 Nov 2021 00:14:32 +0000 (16:14 -0800)]
Restore dirty dnode detection logic
In addition to flushing memory mapped regions when checking holes,
commit de198f2d95 modified the dirty dnode detection logic to check
the dn->dn_dirty_records instead of the dn->dn_dirty_link. Relying
on the dirty record has not be reliable, switch back to the previous
method.
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #11900
Closes #12745
Fedor Uporov [Wed, 10 Nov 2021 19:22:00 +0000 (11:22 -0800)]
zdb: Report bad label checksum
In case if all label checksums will be invalid on any vdev, the pool
will become unimportable. From other side zdb with -l option will not
provide any useful information why it happened. Add notifications
about corrupted label checksums.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Fedor Uporov <fuporov.vstack@gmail.com>
Closes #2509
Closes #12685
Emmanuel Vadot [Mon, 15 Nov 2021 11:12:50 +0000 (12:12 +0100)]
loader: lsefi: Print more information
Printing the EFI_HANDLE pointer isn't very useful.
If the handle have a IMAGE_DEVICE_PATH or a DEVICE_PATH protocol print it.
This makes it easier to see which devices are present and what protocol they
expose.
Warner Losh [Tue, 16 Nov 2021 03:35:58 +0000 (20:35 -0700)]
mpr: Minor formatting changes to match mps.
Minor reformatting nits to make mprsas_scsiio_timeout match
mpssas_scsiio_timeout more closely. The differences aren't necessary and
are distracting when comparing the routines. No functional changes.
Kirk McKusick [Tue, 16 Nov 2021 00:03:08 +0000 (16:03 -0800)]
Allow forced r/w mount of UFS/FFS filesystem with a bad check hash.
Normally a UFS/FFS filesystem with a bad check hash can only be
mounted read only. With this commit the mount(8) -f (force) option
can be used to force a read-write mount of a UFS/FFS filesystem with
a bad check hash. Conveniently the filesystem will proceed to
update its on-disk superblock with a corrected check hash.
Kirk McKusick [Mon, 15 Nov 2021 23:45:06 +0000 (15:45 -0800)]
Allow the MNT_FORCE flag to be passed through to an initial mount.
When doing an initial mount(8) with its -f (force) flag, the MNT_FORCE
flag is not passed through to the underlying filesystem mount routine.
MNT_FORCE is only passed through on later updates to an existing
mount. With this commit the MNT_FORCE flag is now passed through on the
initial mount.
Mark Johnston [Mon, 15 Nov 2021 20:31:21 +0000 (15:31 -0500)]
x86: Implement deferred TSC calibration
There is no universal way to find the TSC frequency. Newer Intel CPUs
may report it via CPUID leaves 0x15 and 0x16. Sometimes it can be
obtained from the PLATFORM_INFO MSR as well, though we never use that.
On older platforms we derive the frequency using a DELAY(1000000) call,
which uses the 8254 PIT. On some newer platforms the 8254 is apparently
non-functional, leading to bogus calibration results. On such platforms
the TSC frequency must be available from CPUID. It is also possible to
disable calibration with a tunable, in which case we try to parse the
brand string if the TSC freq is not available from CPUID.
CPUID 0x15 provides an authoritative TSC frequency value, but even that
is not always available on new Intel platforms. CPUID 0x16 provides the
specified processor base frequency, which is not the same as the TSC
frequency. Empirically, it is close enough for early boot, but too far
off for timekeeping: on a Comet Lake NUC, CPUID 0x16 yields 1600MHz but
the TSC frequency is rougly 1608MHz, leading to frequent clock stepping
when NTP is in use.
Thus we have a situation where we cannot calibrate using the PIT and
cannot obtain a precise frequency from CPUID (or MSRs). This change
seeks to address that by using the CPUID 0x16 value during early boot
and refining the calibration later once ACPI-based timecounters are
available. TSC frequency detection is thus split into two phases:
Early phase:
- On Intel platforms, query CPUID 0x15 and 0x16 and use that value
initially if available.
- Otherwise, get an estimate using the PIT, reducing the delay loop to
100ms from 1s.
- Continue to register the TSC as the CPU ticks provider early, even
though the frequency may be off. Otherwise any code executed during
boot that uses cpu_ticks() (e.g., context switching) gets tripped up
when the ticks provider changes.
Later phase:
- In SI_SUB_CLOCKS, once the timehands are initialized, load the current
TSC and timecounter (sbinuptime()) values at the beginning and end of
a 1s interval and use the timecounter frequency (typically from
kvmclock, HPET or the ACPI PM timer) to estimate the TSC frequency.
- Update the TSC timecounter, global tsc_freq and CPU ticker with the
new frequency and finally register the TSC as a timecounter.