Warner Losh [Wed, 28 Feb 2024 15:16:16 +0000 (08:16 -0700)]
Merge one true awk from 2024-01-22 for the Awk Second Edition support
This brings in Unicode support, CSV support and a number of bug fixes.
They are described in _The AWK Programming Language_, Second Edition, by
Al Aho, Brian Kernighan, and Peter Weinberger (Addison-Wesley, 2024,
ISBN-13 978-0138269722, ISBN-10 0138269726).
Warner Losh [Thu, 29 Feb 2024 16:36:31 +0000 (09:36 -0700)]
firmware: Allow binary files to be loaded by /boot/loader
Files loaded "-t firmware" (or module_type="firmware"). They are
registered with the firmware system using the full path to the file.
There's only one firmware per file, and it is the entire file. We do an
extra firmware_get() on any firmware we find here to prevent them from
ever being unloaded (we can't handle that case sanely).
Warner Losh [Thu, 29 Feb 2024 16:36:20 +0000 (09:36 -0700)]
firmware: load binary firmware files
When we can't find a .ko module to satisfy the firmware request, try
harder by looking for a file to read in directly. We compose this file's
name by appending the imagename parameter to the firmware path
(currently hard-wired to be /boot/firmware, future plans are for a
path). Allow this file to be unloaded when firmware_put() releases the
last reference, but we don't need to do the indirection and dance we
need to do when unloading the .ko that will unregister the firmware.
Warner Losh [Thu, 29 Feb 2024 16:30:37 +0000 (09:30 -0700)]
kernel: Sync ldscript files wrt gnu.attributes and .note.GNU-stack
Bring the keeping of .gnu.attributes to all architectures. Also discard
.note.GNU-stack on all archtiectures. Plus delete obsolete comment that
was removed from i386 in 2010.
Warner Losh [Tue, 27 Feb 2024 05:47:38 +0000 (22:47 -0700)]
loader/zfs: Fix to actually return the last error
The last fix, to try to return the last error, really returns the first
return code after the last error, which could be zero. Instead, return
the last error. Also, change rc to err to make it visually distinct from
rv, which is the cause of my error in e54bb0ad8058.
Reported by: Bill Sommerfeld <sommerfeld@hamachi.org>
Fixes: e54bb0ad8058
Sponsored by: Netflix
Different implementations of `nm` have different ways of telling you
that your file is not a valid object or library, but they all seem to
have “not recognized” as a common substring.
Shawn Anastasio [Tue, 27 Feb 2024 19:40:50 +0000 (14:40 -0500)]
powerpc: Bump maximum number of FDT reserved mem entries
Newer firmware on POWER systems, including v2.10 of the Talos II and
Blackbird firmware can end up reserving more than 32 memory regions in
the device tree, which exceeded an assumption made by ofw_machdep.c's
excise_fdt_reserved(). Bump the maximum number of FDT reservations to
the next power of 2 in order to fix booting on newer firmware.
rtld: unconditionally generate map file during build
It is needed at least to ensure that undesirable code is not linked into
rtld from libsys/libc, and adding the map file option each time is not
productive.
Reviewed by: brooks, emaste
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D44136
Gleb Smirnoff [Wed, 28 Feb 2024 23:57:04 +0000 (15:57 -0800)]
sockets: in solisten_proto() don't call sbdestroy() on a PR_SOCKBUF
A socket marked with PR_SOCKBUF has protocol specific socket buffers
and will take care of the in its pr_listen method. Right now we don't
have any sockets that can listen and are PR_SOCKBUF, but that will
change soon.
Brooks Davis [Wed, 28 Feb 2024 23:23:33 +0000 (23:23 +0000)]
lib{c,sys}: expose cap_sandboxed from libc/gen
It's a thin wrapper on cap_getmode() implemented in libc, not a system
call so the symbol should have been exposed by libc/gen/Symbol.map
alongside the implementation.
Gleb Smirnoff [Wed, 28 Feb 2024 22:32:46 +0000 (14:32 -0800)]
tests/unix_seqpacket: test that data can be sent before accept(2)
This is undocumented feature of PF_UNIX/SOCK_STREAM and thus of
PF_UNIX/SOCK_SEQPACKET, too. Put the test into this file, since this file
is most advanced and has all primitives to write this test in minimum
number of lines.
Gleb Smirnoff [Wed, 28 Feb 2024 22:32:46 +0000 (14:32 -0800)]
tests/unix_seqpacket: provide function that returns bound & listening sock
Use the function in mk_pair_of_sockets(), and in four existing tests -
'listen_bound', 'connect', 'shutdown_send' and 'shutdown_send_sigpipe'.
While here make mk_pair_of_sockets() return pointer to sockaddr_un instead
of path. This also fixes bug of returning pointer to stack memory of
returning function. However, the only caller that cares about this return
is temporarily ifdefed out. No functional change intended.
Gleb Smirnoff [Wed, 28 Feb 2024 22:32:46 +0000 (14:32 -0800)]
tests/unix_seqpacket: remove EMSGSIZE tests
These tests were not testing conformance to the specification, rather than
the limitation of our implementation. The specification doesn't say that
a SOCK_SEQPACKET shall ever return EMSGSIZE. It says:
The SOCK_SEQPACKET socket type is similar to the SOCK_STREAM type,
and is also connection-oriented. The only difference between these
types is that record boundaries are maintained using the
SOCK_SEQPACKET type. A record can be sent using one or more output
operations and received using one or more input operations, but a
single operation never transfers parts of more than one record.
Record boundaries are visible to the receiver via the MSG_EOR flag
in the received message flags returned by the recvmsg() function. It
is protocol-specific whether a maximum record size is imposed.
The EMSGSIZE is specified as 'message is too large to be sent all at once,
as the socket requires'. Indeed existing implementation that has
unix/seqpacket marked as PR_ATOMIC has such a limitation. But future
implementation won't have, thus remove the tests.
Gleb Smirnoff [Wed, 28 Feb 2024 22:32:46 +0000 (14:32 -0800)]
socket tests: remove MSG_TRUNC test for unix/seqpacket
The PF_UNIX/SOCK_SEQPACKET was marked as PR_ATOMIC and that made
soreceive_generic() to treat it pretty much as a datagram socket.
However, POSIX says:
The SOCK_SEQPACKET socket type is similar to the SOCK_STREAM type,
and is also connection-oriented. The only difference between these
types is that record boundaries are maintained using the
SOCK_SEQPACKET type. A record can be sent using one or more output
operations and received using one or more input operations, but a
single operation never transfers parts of more than one record.
Record boundaries are visible to the receiver via the MSG_EOR flag
in the received message flags returned by the recvmsg() function. It
is protocol-specific whether a maximum record size is imposed.
What the test was doing is checking if MSG_TRUNC would report the space
required to return up the end of next mbuf record in the socket buffer.
Apparently the test assumed that this boundary is defined by the write(2)
size on the peer socket. This was true in test conditions, but I'm not
sure it would always be true - sbcompress() may merge mbufs. Anyway, the
mbuf boundaries are internal socket buffer implementation, they are not
SOCK_SEQPACKET records. The records need to be explicitly marked with
MSG_EOR by sender, and the test definitely wasn't doing that.
Stefan Eßer [Wed, 28 Feb 2024 17:49:29 +0000 (18:49 +0100)]
ldconfig: support hints files of either byte-order
Make the ldconfig program accept hints files in little-endian and
big-endian format on all architectures.
The default format is the native byte-order of the respective host.
This is expected to change when a version of the pkg command is
available that implements support for either byte-order in its
internal ldconfig function. (Already committed in the development
tree of the pkg utility, a release is expected at the end of Q1/2024).
This update adds the -B option to the ldconfig program. It enforces
the creation of a big-endian hints file on a little-endian host.
The main purpose to is support of tests with non-native byte-order
files on little-endian hosts. It will be removed when all supported
FreeBSD releases use little-endian hints files by default.
When little-endian hints files are generally used, support of
either byte-order in libexec/rtld can also be removed.
When support for big-endian hints files is no longer required,
the COND_SWAP macro in ldconfig and rtld shall be replaced by
le32toh(), which just return their argument on little-endian
architectures.
While lorder is not required by our current toolchain (or any toolchain
we've used in the past decade or two), it still occasionally shows up
in build systems of third party software, including The Open Group's
UNIX conformance test suite, and the maintenance cost is negligible.
* Ensure that errors go to stderr and result in a non-zero exit.
* Drop the special-case code for a single argument, as it will produce
the wrong outcome if the file does not exist or is corrupted.
* Don't print anything until after we've collected all the data.
* Always create all temporary files before setting the trap. This
ensures that the trap can safely fire at any moment, regardless of any
previous definition of `T`.
* Use a temporary file rather than a pipe between `nm` and `sed` to
ensure proper termination if `nm` fails due to a missing or invalid
input.
* The check for self-referential entries was conditional on testing our
argument list against a regex looking for archives. This was a
needless and unreliable optimization; make the check unconditional.
* Document that lorder will not work properly if any of its inputs have
spaces in their name.
Jan 22, 2024:
Restore the ability to compile with g++. Thanks to
Arnold Robbins.
Dec 24, 2023:
Matchop dereference after free problem fix when the first
argument is a function call. Thanks to Oguz Ismail Uysal.
Fix inconsistent handling of --csv and FS set in the
command line. Thanks to Wilbert van der Poel.
Casting changes to int for is* functions.
Nov 27, 2023:
Fix exit status of system on MacOS. Update to REGRESS.
Thanks to Arnold Robbins.
Fix inconsistent handling of -F and --csv, and loss of csv
mode when FS is set.
Warner Losh [Wed, 28 Feb 2024 14:09:17 +0000 (07:09 -0700)]
gicv3: If the LPI is already allocated, remember it
If the LPI Configuration Tabel has been pre-allocated by the boot
loader, then we have to remember PROPBASER and use it rather than
allocating memory for it ourselves. Linux provides us with a reserved
table that contains all the gicv3 allocations, so make sure what we read
from PROPBASER matches something in that table. Normally, bare metal
boot loaders leave the gic in a reset state. However, Linux brings it up
fully so it can do I/O to boot the next kernel via kexec. Since the
gicv3 PENDBASER can't be reset while running due to undefined behavior,
we must reuse what's there for both PENDBASER and PROPBASER.
With this commit, the workaround is complete. Details are at
https://lkml.iu.edu/hypermail/linux/kernel/1809.2/06246.html
and pointers in the thread.
Sponsored by: Netflix
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D44038
Warner Losh [Wed, 28 Feb 2024 14:09:10 +0000 (07:09 -0700)]
gicv3: In its_init_cpu_lpi record each cpu's PENDBASER
When we're using the preallocated memory for gicv3, record each cpu's
PENDBASER where we'd normally allocate memory for it. Make sure that
memory is in the excluded list and map the PA to VA and store that, to
mimic what we do with the allocation case.
Sponsored by: Netflix
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D44037
Warner Losh [Wed, 28 Feb 2024 14:08:55 +0000 (07:08 -0700)]
gicv3: Report PENDBASE when bootverbose
Report some stats about PENDBASE when we're running under
bootverbose. We don't do this by default because experience has been
excess output in this routine hangs the system.
Sponsored by: Netflix
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D44036
Warner Losh [Wed, 28 Feb 2024 14:08:24 +0000 (07:08 -0700)]
gicv3: Panic if the gicv3 already running
Due to undefined behavior, it's impossible to re-program a gicv3 ITS
table once it's programmed once. Memory corruption happens otherwise.
Panic if we detect the LPI is already enabled.
Sponsored by: Netflix
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D44033
Warner Losh [Wed, 28 Feb 2024 14:08:15 +0000 (07:08 -0700)]
gicv3: Change how we initialize its children.
The current code is written such that all the attach routines can do so
in parallel. However, newbus serializes children today, and is likely to
do so in the future. Only allocate memory for the first time. Add an
assertion that this memory is allocated for larger units.
Sponsored by: Netflix
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D44032
Stefan Schlosser [Wed, 28 Feb 2024 08:31:28 +0000 (09:31 +0100)]
md5.1: Fix an example
The last example in the manpage md5(1) wants to demonstrate
GNU mode (md5sum), but uses BSD mode (md5) instead:
In GNU mode, the -c option does not compare against a hash string
passed as parameter. Instead, it expects a digest file,
as created under the name digest for /boot/loader.conf in
the example above.
Emmanuel Vadot [Tue, 27 Feb 2024 17:33:52 +0000 (18:33 +0100)]
rc: Set var_run_enable to enable by default
This will load/save the /var/run directories at boot/shutdown if
and only if /var/run/ is a tmpfs mount so it is a win for tmpfs
users and a no-op for everyone else.
Mitchell Horne [Tue, 27 Feb 2024 21:59:53 +0000 (17:59 -0400)]
conf: deduplicate dwc config logic
Move the core dwc(4) file entries to the MI conf/files.
Platform-specific versions e.g. rk_dwc are retained in their current
position. On arm64 this necessitates adding an additional 'device dwc'
option.
This hardware is found on several available/emerging RISC-V SoCs, so we
will soon need it on this architecture.
Reviewed by: manu
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44103
John Baldwin [Tue, 27 Feb 2024 19:59:52 +0000 (11:59 -0800)]
netlink: Don't use a zero-length array
Define SNL_DECLARE_FIELD_PARSER* macros to create a parser that has
no output attributes only input fields and use this to define the
snl_donemsg_parser.
This removes the need for the zero-length nla_p_donemsg[] variable.
Zero length arrays are not valid in ISO C.
John Baldwin [Tue, 27 Feb 2024 19:44:03 +0000 (11:44 -0800)]
bhnd_usb: Use bus_generic_rman_*
For SYS_RES_MEMORY, use bus_generic_rman_* for
activate/deactivate_resource methods as well as custom
map/unmap_resource methods that request submappings of the sc_mem
resource allocated from the parent bus.
John Baldwin [Tue, 27 Feb 2024 19:43:34 +0000 (11:43 -0800)]
rman: Remove rman_set_start/end
These functions are not safe as the rman implementation assumes that
all regions (including allocated resources) are sorted by address in
the internal linked-list.
Alexander Motin [Tue, 27 Feb 2024 18:28:44 +0000 (13:28 -0500)]
CTL: Drop Format Device and Rigid Disk Geometry mode pages
Those mode pages are obsolete since SBC-2 specification almost 20
years ago. First I was trying to understand possible relations
between physical block and physical sector terms in different specs.
Then was thinking about possible relations to device CHS geometry
and compatibility issues. Finally I just decided that none of it
worth the efforts and should rest in piece.
Kristof Provost [Tue, 27 Feb 2024 13:24:25 +0000 (14:24 +0100)]
pf: fix packet-to-big for route-to as well
When we handle a packet via route-to (i.e. pf_route6()) we still need to
verify the MTU. However, we only run that check in the forwarding case.
Set the PFIL_FWD tag when running the pf_test6(PF_OUT) check from
pf_route6(). We are in fact forwarding, so should call the test function
as such. This will cause us to run the MTU check, and generate an ICMP6
packet-too-big error when required.
Olivier Certner [Fri, 9 Feb 2024 18:03:22 +0000 (19:03 +0100)]
sched: Simplify sched_lend_user_prio_cond()
If 'td_lend_user_pri' has the expected value, there is no need to check
the fields that sched_lend_user_prio() modifies, they either are already
good or soon will be ('td->td_lend_user_pri' has just been changed by
a concurrent update).
Reviewed by: kib
Approved by: emaste (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44050
Olivier Certner [Thu, 22 Feb 2024 09:13:38 +0000 (10:13 +0100)]
PP mutexes: lock: Reduce 'umtx_lock' holding before taking the user lock
There is no need to have it for the priority check (that the thread
doesn't have a higher priority than the mutex's ceiling), and there's
also no need to take it if the thread doesn't have privileges to set its
priority to the mutex's ceiling.
While here, turn 'su' into a 'bool' and compute the internal priority
corresponding to the mutex's ceiling once and for all, putting it in new
'new_pri'.
Reviewed by: kib
Approved by: emaste (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44045
Olivier Certner [Thu, 22 Feb 2024 08:53:51 +0000 (09:53 +0100)]
PP mutexes: lock: Check if priority is too high against base one
Doing this instead of using the current (user) priority, which includes
current lendings, prevents gratuitous failures for threads involved in
multiple locking groups, where each group is defined as the threads that
can lock a particular PP or PI mutex. No deadlock can occur in this
case. Indeed, if a thread holds such a lock A giving it a higher
priority than the ceiling of some other lock B that is PP, and B is
acquired by another thread, effectively the latter may not be able to
run but this situation can only last until the first thread releases A,
which it will do eventually.
Reviewed by: kib
Approved by: emaste (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D44044
Olivier Certner [Tue, 20 Feb 2024 13:52:40 +0000 (14:52 +0100)]
PP mutexes: unlock: Reset inherited prio regardless of privileges
'uq_inherited_pri' contains the current priority inherited from Priority
Protection mutexes. If -1 is passed through 'm_ceilings[1]', meaning
that there are no such mutexes held anymore, this must be reflected into
it by setting it to PRI_MAX, regardless of whether the thread has
privilege to set realtime priorities (PRI_MAX is also obviously not
a realtime priority level). By contrast, it shall not be updated and
the computed 'new_inherited_pri' shall stay unused if the thread doesn't
have the ability to set a realtime priority, possibly keeping an older
such priority acquired previously.
Reviewed by: kib
Approved by: emaste (mentor)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43984
Warner Losh [Tue, 27 Feb 2024 03:19:58 +0000 (20:19 -0700)]
loader: Add loader.exit
Add loader.exit(status). While one can get alomst this behavior with
loader.perform("quit"), quit doesn't allow a value to be returned to the
firmware. The interpretation of 'status' is firmware specific. This can
be used when autobooting doesn't work in scripts, for example, to allow
the firmware to try something else...
Florian Walpen [Mon, 26 Feb 2024 23:27:47 +0000 (00:27 +0100)]
snd_uaudio(4): Fix sample rate selection after 42fdcd9fd917.
The sample rate selection of snd_uaudio(4) at runtime was implicitly
relying on a specific order in the device config list. In case a default
was set through the hw.usb.uaudio.default_rate sysctl tunable, commit 42fdcd9fd917 removed a duplicate sample rate entry from that list, which
inadvertently broke sample rate selection at runtime. Implement sample
rate selection in a way that works for any order in the device config
list.
Stefan Eßer [Mon, 26 Feb 2024 22:18:12 +0000 (23:18 +0100)]
rtld-elf: support either byte-order of hints file
Accept either little-endian or big-endian representation of the ELF
hints magic number in the header of a hints file and convert the
parameters to the native byte-order of the repsective system.
This is a pre-requisite for a planned change to always write the byte
order in little-endian format on all architectures. The only relvant
architecture that uses big-endian data is powerpc64, and it is not
likely that new architectures will choose that representation of data
in memory.
When all supported architectures use little-endian data in the hints
file, the byte swap logic can be enabled for big-endian CPUs at
compile time. Up to that point, there is a very small run-time penalty
that is paid on all systems to check the byte-order of the hints file
and to provide the option to byte-swap the parameters read from the
hints file header.
This commit contains the changes from review D44080 (which had been
split off from this patch for easier review),
* Overhaul column width and padding calculation.
* Rewrite print_space() so it is now a) correct and b) understandable.
* Rewrite tab expansion in fetch() for the same reason.
This brings us in line with GNU diff for all cases I could think of.
Sponsored by: Klara, Inc.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D44014
Emmanuel Vadot [Thu, 22 Feb 2024 05:46:08 +0000 (06:46 +0100)]
pam: Always use pam_xdg for session and xdm
This ensure that the XDG_RUNTIME_DIR is always created and the environment
variable is present which is a must for any wayland user (at least).
The only "cost" of it is a directory is created and unused.
Reviewed by: des
Differential Revision: https://reviews.freebsd.org/D44012
Sponsored by: Beckhoff Automation GmbH & Co. KG
Emmanuel Vadot [Wed, 21 Feb 2024 14:51:05 +0000 (15:51 +0100)]
pam: Add pam_xdg module
This is a module to setup the XDG directories and environment variables.
For now the only usage is to have a XDG_RUNTIME_DIR environment setup at
user login.
All other environment variable have a default fallback so no need to export
them in this module.
The directory is created according to the XDG Base directory specification.
The default base directory is /var/run/xdg/<username> but can be configured
using the runtime_dir=<dir> module option.
According to the spec the directory *must* not survive a reboot so adding
var_run_enable="YES" to rc.conf is highly recommanded.
It is possible that SA was removed while processing packed, in which
case it is changed to the DEAD state and it index is removed from the
tree. Dereferencing sav->sah then touches freed memory.
Toomas Soome [Sun, 25 Feb 2024 22:34:00 +0000 (00:34 +0200)]
loader: we can only env_discard() existing variable
While dropping nvpair from nvstore, we also remove the corresponding
environment variable. By doing so, we should be careful not to try
to unset non-existing variable.
Florian Walpen [Sun, 25 Feb 2024 21:39:36 +0000 (21:39 +0000)]
snd_hdspe(4): Optional unified pcm device.
Add a sysctl tunable to unify all physical ports of an HDSPe sound card
into one pcm device, with up to 14 (AIO) or 36 (RayDAT) channels. This
makes all ports available in multi-channel audio software.
When knob is zero, intent is that no SIGSYS signals are delivered.
Comparing zero to zero does not test much, we should compare the count
of delivered SIGSYSs to zero.
Reviewed by: dchagin, imp
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D44077
Warner Losh [Sun, 25 Feb 2024 00:57:29 +0000 (17:57 -0700)]
loader/efi: Only include interpreter's linker script
For safety, only include the interpreter's linker script. Note that the
simple loader doesn't have one, but it's not an error to copy a ELF
section that does not exist. No functional change, however.
Warner Losh [Sun, 25 Feb 2024 00:57:21 +0000 (17:57 -0700)]
loader/ficl: Rename the ficl compile set to X4th_compile_set
And upcoming change will need this set to be named this. Since it's only
used in the efi Makefile, and inside if ficl itself, the change should
be a nop.
Warner Losh [Sun, 25 Feb 2024 00:57:13 +0000 (17:57 -0700)]
loader/efi: Use unique linker set for lua
After the linker set cleanup in ldscripts, there's now only one place we
need to know the linkerset name, so go ahead and change the lua
interpreter augmentation linker set to be uniquely named.
Warner Losh [Sun, 25 Feb 2024 00:57:04 +0000 (17:57 -0700)]
loader/efi: Linker sets not needed.
We don't need linker sets listed as sections. They are explicitly
included in the objcopy we use to create the .efi file. This practice
was added in 2002 by peter@ in a6d81d83a280 to make ia64 builds
self-hosted. However, it was added back to the objcopy in 2010 by rpaulo
in 8df7a05edd36 for i386 EFI support, though the ldscript file then
retained them needlessly. The gcc/binutils bug having been fixed in the
interim. We've not needed them since then, but the redundancy didn't
matter.
Warner Losh [Sun, 25 Feb 2024 00:56:56 +0000 (17:56 -0700)]
loader/kboot: simplify linker set inclusion a little
Linker set sections are included by default. No need to do so
explicitly. These were bogusly copied from the efi ldscripts. They were
there due to a workaround introduced in 2002 by peter@ for a gcc
upgrade, but whatever bugs necessitated it were filed by 2010 when
rpaulo@ imported the i386 support (though they were copied even though
the objcopy retained them correctly, the gcc bug having been
fixed). They've never been needed.