Mark Johnston [Sun, 3 Jan 2021 16:32:30 +0000 (11:32 -0500)]
Ensure that dirent's d_off field is initialized
We have the d_off field in struct dirent for providing the seek offset
of the next directory entry. Several filesystems were not initializing
the field, which ends up being copied out to userland.
Reported by: Syed Faraz Abrar <faraz@elttam.com>
Reviewed by: kib
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27792
Mark Johnston [Sun, 3 Jan 2021 16:31:00 +0000 (11:31 -0500)]
uma: Avoid unmapping direct-mapped slabs
startup_alloc() uses pmap_map() to map slabs used for bootstrapping the
VM. pmap_map() may ignore the hint address and simply return a range
from the direct map. In this case we must not unmap the range in
startup_free().
UMA uses bootstart and bootmem to track the range of KVA into which
slabs are mapped if the direct map is not used. Unmap a startup slab
only if it was mapped into that range.
Reported by: alc
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27885
Conrad Meyer [Sun, 3 Jan 2021 15:39:12 +0000 (07:39 -0800)]
zstd: Fix non-FreeBSD CI libzstd build
Fix non-FreeBSD CI build after v1.4.8. This definition was only used in
zstd(1), which isn't part of non-FreeBSD CI (I guess). The ifdef was
added in v1.4.5 import.
Upstream does not currently support shared-linked zstd(1), but I have
proposed https://github.com/facebook/zstd/pull/2450 . If that is
adopted, we can add -DZSTD_PROGRAMS_LINK_SHARED to our libzstd build and
drop some diffs.
Kirk McKusick [Sun, 3 Jan 2021 06:31:55 +0000 (22:31 -0800)]
The fsck_ffs program had previously only been able to expand the size
of its lost+found directory by allocating direct block pointers. The
effect was that it was limited to about 19,000 files. One of Peter Holm's
tests produced a filesystem with about 23,000 lost files which meant
that fsck_ffs was unable to recover it. This update allows lost+found
to be expanded into a single indirect block which allows it to store
up to about 6,573,000 lost files.
Alan Somers [Sat, 2 Jan 2021 23:34:20 +0000 (16:34 -0700)]
Add aio_writev and aio_readv
POSIX AIO is great, but it lacks vectored I/O functions. This commit
fixes that shortcoming by adding aio_writev and aio_readv. They aren't
part of the standard, but they're an obvious extension. They work just
like their synchronous equivalents pwritev and preadv.
It isn't yet possible to use vectored aiocbs with lio_listio, but that
could be added in the future.
Ryan Libby [Sun, 3 Jan 2021 02:09:37 +0000 (18:09 -0800)]
arm: fix atomic_testand{set,clear}_64 for ops on high bits
The fix in bd03acedb804add1e22178d50eb2bfb703974ddf worked for 32-bit
ops, and for 64-bit ops for bit arguments of 0 - 95, but then was broken
for operations on the high 32 bits after that.
Rick Macklem [Sun, 3 Jan 2021 01:21:21 +0000 (17:21 -0800)]
copy_file_range(2): add recommendation to use large "len"
PR#252358 reported a serious performance problem w.r.t.
cp(1) when copying large non-sparse files.
This problem appears to have been caused by cp(1)
calling copy_file_range(2) with a small "len" argument.
This patch adds a recommendation to use a large "len"
value where possible, for performance reasons.
Rick Macklem [Sun, 3 Jan 2021 00:58:43 +0000 (16:58 -0800)]
cp(1): fix performance issue for large non-sparse file copies
PR252358 reported a serious performance problem when
copying a large non-sparse file on a UFS file system.
This problem seems to have been caused by a large
number of SEEK_HOLE operations, with one done
for each copy_file_range(2) call.
This patch modifies cp(1) to use a large (SSIZE_MAX)
len argument, reducing the number of system calls
and resolving the performance issue.
While here, convert the type of the "rcount" from "int"
to "ssize_t" so that it is consistent with that returned
by both read(2) and copy_file_range(2).
Mitchell Horne [Sat, 2 Jan 2021 23:07:04 +0000 (19:07 -0400)]
Handle KDB_WHY_TRAP return on arm and mips
Upon exit from the debugger, checking the return code of kdb_trap()
allows one to retry the fatal page fault. This matches what is done on
all other architectures.
Ryan Libby [Sat, 2 Jan 2021 20:11:18 +0000 (12:11 -0800)]
arm64: fix mask in atomic_testand{set,clear}_64
These macros generate both the 32- and 64-bit ops, but the mask was hard
coded for 32-bit ops, causing the 64-bit ops always to affect only the
low 32 bits.
Toomas Soome [Mon, 21 Dec 2020 05:31:16 +0000 (07:31 +0200)]
loader: implement framebuffer console
Draw console on efi.
Add vbe framebuffer for BIOS loader (vbe off, vbe on, vbe list,
vbe set xxx).
autoload font (/boot/fonts) based on resolution and font size.
Add command loadfont (set font by file) and
variable screen.font (set font by size). Pass loaded font to kernel.
Michal Meloun [Sat, 2 Jan 2021 19:30:00 +0000 (20:30 +0100)]
arm: Fix atomic_testand{set,clear}_32(). According to atomic (9), the bit position argument should be a modulo operand size. While I'm in, add missing implementation of atomic_testandclear_64(). For more details see https://reviews.freebsd.org/D27886
Kyle Evans [Sat, 2 Jan 2021 05:59:21 +0000 (23:59 -0600)]
contrib: remove libgnuregex
This should have been a part of 47d1ad2413da, but it was overlooked. All of
the build bits have been previously removed, and nothing references this
anymore.
bootparamd: Fix several warnings and increase warn level to 6.
- Increase WARNS to 6.
- Except -Wcast-align and -Wincompatible-pointer-types-discards-qualifiers
checks.
- Use ANSI C prototype.
- Statically variables and functions.
- Add extern declaration for global variables.
- Rename local variables to resolve shadow warnings.
- Ignore malformed directory entries as created by Dropbox ("/").
(rev 1.24)
- Use libarchive 3.x interface: check result for archive_read_free()
and don't call archive_read_close manually. (rev 1.23)
- Always overwrite symlinks on extraction, ever if they're newer than
entries in archive.
- Use getline() rather than getdelim().
PR: 231827
Submitted by: ak
Reviewed by: mm
Obtained from: NetBSD
MFC after: 2 weeks
Rick Macklem [Fri, 1 Jan 2021 22:21:51 +0000 (14:21 -0800)]
nfsd: fix NFSv4.0 seqid handling for ERELOOKUP
Commit 774a36851e0e fixed the NFS server so that it could handle
ERELOOKUP returns from VOP calls by redoing the operation/RPC.
However, for NFSv4.0, redoing an Open would increment
the open_owner's seqid multiple times, breaking the protocol.
This patch sets a new flag called ND_ERELOOKUP on the RPC when
a redo is in progress. Then the code that increments the seqid
avoids the seqid increment/check when the flag is set, since
it indicates this has already been done for the Open.
Rick Macklem [Fri, 1 Jan 2021 21:55:51 +0000 (13:55 -0800)]
nfsd: fix NFS server for ERELOOKUP
r367672 modified UFS such that certain VOPs, such as
VOP_CREATE() will intermittently return ERELOOKUP.
When this happens, the entire system call, or NFS
operation in the case of the NFS server, must be redone.
This patch adds that support to the NFS server by rolling
back the state of the NFS request arguments and NFS
reply arguments mbuf lists to the condition they were
in before the operation and then redoing the operation.
scrmap is only used in the one compilation unit in all cases, make it static
rather than extern'ing it. There's little benefit, but it's easy to do.
It's unclear how this hasn't failed many builds before now, since it should
have cropped up sometime around deeper hierarchies getting a default WARNS.
Alan Somers [Tue, 29 Dec 2020 17:48:34 +0000 (10:48 -0700)]
fusefs: fix an expectation in one of the tests
An order-of-operations problem caused an expectation intended for
FUSE_READ to instead match FUSE_ACCESS. Surprisingly, only one test
case was affected.
MFC after: 2 weeks
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27818
Dimitry Andric [Fri, 1 Jan 2021 14:35:13 +0000 (15:35 +0100)]
Merge commit 4f568fbd2 from llvm git (by Nemanja Ivanovic):
[PowerPC] Do not emit HW loop when TLS var accessed in PHI of loop
exit
If any PHI nodes in loop exit blocks have incoming values from the
loop that are accesses of TLS variables with local dynamic or general
dynamic TLS model, the address will be computed inside the loop.
Since this includes a call to __tls_get_addr, this will in turn cause
the CTR loops verifier to complain. Disable CTR loops in such cases.
Fixes: https://bugs.llvm.org/show_bug.cgi?id=48527
This should fix building ceph 12.2.12 on powerpc64, powerpc, powerpcspe
and powerpc64le.
Kyle Evans [Fri, 1 Jan 2021 06:22:48 +0000 (00:22 -0600)]
stand: fix WITHOUT_FORTH/WITHOUT_LOADER_LUA build
Previously having ficl/liblua in LIB32LIST with their respective option
turned OFF would be relatively harmless, as we wouldn't act on it unless we
were building the non-32 variant. As of ac5f382a9d0a, however, these are
now used for dependencies in some cases and must reflect what's actually
going to be built.
The change to kern_jail_set that was supposed to "also properly clean
up when attachment fails" didn't fix a memory leak but actually caused
a double free. Back that part out, and leave the part that manages
allprison_lock state.
John Baldwin [Fri, 1 Jan 2021 00:01:35 +0000 (16:01 -0800)]
Use kdb_thr_* to iterate over threads consistently in DDB.
The "findstack", "show all trace", and "show active trace" commands
were iterating over allproc to enumerate threads. This missed threads
executing in exit1() after being removed from allproc.
John Baldwin [Fri, 1 Jan 2021 00:01:12 +0000 (16:01 -0800)]
Don't check P_INMEM in kdb_thr_*().
Not all debugger operations that enumerate threads require thread
stacks to be resident in memory to be useful. Instead, push P_INMEM
checks (if needed) into callers.
John Baldwin [Fri, 1 Jan 2021 00:00:54 +0000 (16:00 -0800)]
Enumerate processes via the pid hash table in kdb_thr_*().
Processes part way through exit1() are not included in allproc. Using
allproc to enumerate processes prevented getting the stack trace of a
thread in this part of exit1() via ddb.
John Baldwin [Fri, 1 Jan 2021 00:00:05 +0000 (16:00 -0800)]
ddb ps: Use the pidhash to enumerate processes not in allproc.
Exiting processes that have been removed from allproc but are still
executing are not yet marked PRS_ZOMBIE, so they were not listed (for
example, if a thread panics during exit1()). To detect these
processes, clear p_list.le_prev to NULL explicitly after removing a
process from the allproc list and check for this sentinel rather than
PRS_ZOMBIE when walking the pidhash.
While here, simplify the pidhash walk to use a single outer loop.
Kyle Evans [Thu, 31 Dec 2020 18:30:43 +0000 (12:30 -0600)]
libc: tests: add some tests for cpuset(2)
The cpuset(2) tests should be run as root (require.user properly set) with
>= 3 cpus for maximum coverage. All tests that want to modify the cpuset
don't assume any particular cpu layout (i.e. the first cpu may not be 0, the
last may not be first + count) and the following scenarios are tested:
1.) newset: basic execute cpuset() to grab a new cpuset, make sure the
assigned cpuset then has a different ID.
2.) transient: create a new cpuset then assign the process its original
cpuset, ensuring that the one we created is now gone.
3.) deadlk: test assigning an anonymous mask, then resetting the process
base affinity with 1-cpu overlap w.r.t. the anonymous mask and with
0-cpu overlap w.r.t. the anonymous mask.
4.) jail_attach_newbase: process attaches to a jail with its own
cpuset+mask (e.g. cpuset -c -l 1,2 jail -c path=/ command=/bin/sh)
5.) jail_attach_newbase_plain: process attaches to a jail with its own
cpuset (e.g. cpuset -c jail -c path=/ command=/bin/sh)
6.) jail_attach_prevbase: process attaches to a jail with the containing
jail's root cpuset (e.g. jail -c path=/ command=/bin/sh)
7.) jail_attach_plain: process attaches to a jail with the containing jail's
root cpuset+mask.
8.) badparent: creates a new cpuset and modifies the anonymous thread mask,
then setid's back to the original and checks that cpuset_getid() returns
the expected set.
Kyle Evans [Thu, 31 Dec 2020 18:26:01 +0000 (12:26 -0600)]
libc: tests: hook CPUSET(9) test up to the build
Add shims to map NetBSD's API to CPUSET(9). Obviously the invalid input
parts of these tests are relatively useless since we're just testing the
shims that aren't used elsewhere, there's still some amount of value in
the parts testing valid inputs.
Kyle Evans [Thu, 31 Dec 2020 17:15:45 +0000 (11:15 -0600)]
stand: properly declare subdir deps or .WAIT, do parallel build
buildworld already runs the stand build in parallel[1], so make it easier to
identify ordering issues by properly establishing dependencies or adding
.WAIT where needed.
Everything in stand/ relies on libsa, either directly or indirectly, because
libsa build is where the stand headers get installed and it gets linked in
most places.
Interpreters depend on their libs, machine dirs usually depend on top-level
libs that are getting built and at least one of the interpreter flavors.
For i386, order btx/libi386/libfirewire before everything else using a
big-ol-.WAIT hammer. btx is the most common dependency, but the others are
used sporadically. This seems to be where the race reporting on the mailing
list is- AFAICT, the following sequence is happening:
1.) One of the loaders gets built based on stale btx/btxldr
2.) btx/btxldr gets rebuilt
3.) installworld triggers loader rebuild because btx was rebuilt after
This seems like the most plausible explanation, as they've verified system
time and timestamps.
While we're here, let's switch stand/ over to a completely parallel build so
we can work out these kinds of issues in isolation rather than in the middle
of a larger build.
Kyle Evans [Thu, 31 Dec 2020 17:12:39 +0000 (11:12 -0600)]
arm: tune vmparam.h towards a little more modern
An 8MB max stack size is quite limiting in today's world, and in-fact is
the *default* stack size for almost every other arch (including mips).
Raise the default to 4MB (should be pretty reasonable) and the max to 64MB.
NetBSD made a similar move back in 2015 and raised MAXDSIZ to 1856 at the
same time, so let's just roll that in as well. They later lowered it, but
eventually raised it back to 1856 in order to build rust.
This was noticed while looking at qemu-bsd-user's default stack sizes and
growth behavior (or lack thereof).
Reviewed by: ian
Differential Revision: https://reviews.freebsd.org/D27218
Kyle Evans [Thu, 31 Dec 2020 16:50:43 +0000 (10:50 -0600)]
vt: restore tty when console is ungrabbed
When a break-to-debugger is triggered, kdb will grab the console and vt(4)
will generally switch back to ttyv0. If one issues a continue from the
debugger, then kdb will ungrab the console and the system rolls on.
This change adds a perhaps minor feature: when we're down to grab == 0 and
if vt actually switched away to ttyv0, switch back to the tty it was
previously on before the console was grabbed.
The justification behind this is that a typical flow is to work in
!ttyv0 to avoid console spam while occasionally dropping to ddb to inspect
system state before returning. This could easily enough be tossed behind
a sysctl or something if it's not generally appreciated, but I anticipate
indifference.
Reviewed by: ray
Differential Revision: https://reviews.freebsd.org/D27110
Kyle Evans [Thu, 31 Dec 2020 16:45:41 +0000 (10:45 -0600)]
vt: more carefully handle vt_allocate_keyboard grab work
vt_allocate_keyboard only needs to unwind the effects of keyboard-grabbing,
rather than any associated vt window action that may have also happened.
Split out the bits that do the keyboard work into *_noswitch equivalents,
and use those in keyboard allocation. This will be less error-prone when a
later change will offer up different window state behavior when the console
is ungrabbed.
Reviewed by: ray
Differential Revision: https://reviews.freebsd.org/D27110
lualoader: add loader_conf_dirs support (loader.conf.d)
loader_conf_dirs is the supporting mechanism for the included
/boot/loader.conf.d directory. When lualoader finishes processing all of
the loader_conf_files it finds after walking /boot/defaults/loader.conf,
it will now check any and all loader_conf_dirs and process files ending
in ".conf" as if they were a loader.conf.
Note that loader_conf_files may be specified in a loader.conf.d config
file, but loader_conf_dirs may *not*. It will only be processed as specified
in /boot/defaults/loader.conf and any loader_conf_files that were loaded
from there.
Ed Maste [Thu, 31 Dec 2020 15:29:08 +0000 (10:29 -0500)]
ldconfig: Retire a.out support
aout support in ldconfig hasn't been required since FreeBSD 2.x.
If someone needs to use FreeBSD 2 shared libraries they will be best
served by using a FreeBSD 2 ldconfig as well.
In aa5e1b42e6e3 we removed the ldconfig a.out invocation from rc.d but
left the support in ldconfig itself. Remove it now.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27481
John Baldwin [Wed, 30 Dec 2020 23:18:02 +0000 (15:18 -0800)]
rsu: Don't modify read-only firmware block.
The firmware header loaded into an rsu(4) device has to be customized
to reflect device settings. The driver was overwriting the header
from the shared firmware image before sending it to the device. If
two devices attached at the same time with different settings, one
device could potentially get a corrupted header. The recent changes
in a095390344fb1795c1b118a2f84da8f6a7f254ab exposed this bug in the
form of a panic as the firmware blobs are now marked read-only in
object files and mapped read-only by the kernel.
To avoid the bug, change the driver to allocate a copy of the firmware
header on the stack that is initialized before writing it to the
device.
projects / FreeBSD / FreeBSD.git / log