John Baldwin [Fri, 1 Jan 2021 00:01:35 +0000 (16:01 -0800)]
Use kdb_thr_* to iterate over threads consistently in DDB.
The "findstack", "show all trace", and "show active trace" commands
were iterating over allproc to enumerate threads. This missed threads
executing in exit1() after being removed from allproc.
John Baldwin [Fri, 1 Jan 2021 00:01:12 +0000 (16:01 -0800)]
Don't check P_INMEM in kdb_thr_*().
Not all debugger operations that enumerate threads require thread
stacks to be resident in memory to be useful. Instead, push P_INMEM
checks (if needed) into callers.
John Baldwin [Fri, 1 Jan 2021 00:00:54 +0000 (16:00 -0800)]
Enumerate processes via the pid hash table in kdb_thr_*().
Processes part way through exit1() are not included in allproc. Using
allproc to enumerate processes prevented getting the stack trace of a
thread in this part of exit1() via ddb.
John Baldwin [Fri, 1 Jan 2021 00:00:05 +0000 (16:00 -0800)]
ddb ps: Use the pidhash to enumerate processes not in allproc.
Exiting processes that have been removed from allproc but are still
executing are not yet marked PRS_ZOMBIE, so they were not listed (for
example, if a thread panics during exit1()). To detect these
processes, clear p_list.le_prev to NULL explicitly after removing a
process from the allproc list and check for this sentinel rather than
PRS_ZOMBIE when walking the pidhash.
While here, simplify the pidhash walk to use a single outer loop.
Kyle Evans [Thu, 31 Dec 2020 18:30:43 +0000 (12:30 -0600)]
libc: tests: add some tests for cpuset(2)
The cpuset(2) tests should be run as root (require.user properly set) with
>= 3 cpus for maximum coverage. All tests that want to modify the cpuset
don't assume any particular cpu layout (i.e. the first cpu may not be 0, the
last may not be first + count) and the following scenarios are tested:
1.) newset: basic execute cpuset() to grab a new cpuset, make sure the
assigned cpuset then has a different ID.
2.) transient: create a new cpuset then assign the process its original
cpuset, ensuring that the one we created is now gone.
3.) deadlk: test assigning an anonymous mask, then resetting the process
base affinity with 1-cpu overlap w.r.t. the anonymous mask and with
0-cpu overlap w.r.t. the anonymous mask.
4.) jail_attach_newbase: process attaches to a jail with its own
cpuset+mask (e.g. cpuset -c -l 1,2 jail -c path=/ command=/bin/sh)
5.) jail_attach_newbase_plain: process attaches to a jail with its own
cpuset (e.g. cpuset -c jail -c path=/ command=/bin/sh)
6.) jail_attach_prevbase: process attaches to a jail with the containing
jail's root cpuset (e.g. jail -c path=/ command=/bin/sh)
7.) jail_attach_plain: process attaches to a jail with the containing jail's
root cpuset+mask.
8.) badparent: creates a new cpuset and modifies the anonymous thread mask,
then setid's back to the original and checks that cpuset_getid() returns
the expected set.
Kyle Evans [Thu, 31 Dec 2020 18:26:01 +0000 (12:26 -0600)]
libc: tests: hook CPUSET(9) test up to the build
Add shims to map NetBSD's API to CPUSET(9). Obviously the invalid input
parts of these tests are relatively useless since we're just testing the
shims that aren't used elsewhere, there's still some amount of value in
the parts testing valid inputs.
Kyle Evans [Thu, 31 Dec 2020 17:15:45 +0000 (11:15 -0600)]
stand: properly declare subdir deps or .WAIT, do parallel build
buildworld already runs the stand build in parallel[1], so make it easier to
identify ordering issues by properly establishing dependencies or adding
.WAIT where needed.
Everything in stand/ relies on libsa, either directly or indirectly, because
libsa build is where the stand headers get installed and it gets linked in
most places.
Interpreters depend on their libs, machine dirs usually depend on top-level
libs that are getting built and at least one of the interpreter flavors.
For i386, order btx/libi386/libfirewire before everything else using a
big-ol-.WAIT hammer. btx is the most common dependency, but the others are
used sporadically. This seems to be where the race reporting on the mailing
list is- AFAICT, the following sequence is happening:
1.) One of the loaders gets built based on stale btx/btxldr
2.) btx/btxldr gets rebuilt
3.) installworld triggers loader rebuild because btx was rebuilt after
This seems like the most plausible explanation, as they've verified system
time and timestamps.
While we're here, let's switch stand/ over to a completely parallel build so
we can work out these kinds of issues in isolation rather than in the middle
of a larger build.
Kyle Evans [Thu, 31 Dec 2020 17:12:39 +0000 (11:12 -0600)]
arm: tune vmparam.h towards a little more modern
An 8MB max stack size is quite limiting in today's world, and in-fact is
the *default* stack size for almost every other arch (including mips).
Raise the default to 4MB (should be pretty reasonable) and the max to 64MB.
NetBSD made a similar move back in 2015 and raised MAXDSIZ to 1856 at the
same time, so let's just roll that in as well. They later lowered it, but
eventually raised it back to 1856 in order to build rust.
This was noticed while looking at qemu-bsd-user's default stack sizes and
growth behavior (or lack thereof).
Reviewed by: ian
Differential Revision: https://reviews.freebsd.org/D27218
Kyle Evans [Thu, 31 Dec 2020 16:50:43 +0000 (10:50 -0600)]
vt: restore tty when console is ungrabbed
When a break-to-debugger is triggered, kdb will grab the console and vt(4)
will generally switch back to ttyv0. If one issues a continue from the
debugger, then kdb will ungrab the console and the system rolls on.
This change adds a perhaps minor feature: when we're down to grab == 0 and
if vt actually switched away to ttyv0, switch back to the tty it was
previously on before the console was grabbed.
The justification behind this is that a typical flow is to work in
!ttyv0 to avoid console spam while occasionally dropping to ddb to inspect
system state before returning. This could easily enough be tossed behind
a sysctl or something if it's not generally appreciated, but I anticipate
indifference.
Reviewed by: ray
Differential Revision: https://reviews.freebsd.org/D27110
Kyle Evans [Thu, 31 Dec 2020 16:45:41 +0000 (10:45 -0600)]
vt: more carefully handle vt_allocate_keyboard grab work
vt_allocate_keyboard only needs to unwind the effects of keyboard-grabbing,
rather than any associated vt window action that may have also happened.
Split out the bits that do the keyboard work into *_noswitch equivalents,
and use those in keyboard allocation. This will be less error-prone when a
later change will offer up different window state behavior when the console
is ungrabbed.
Reviewed by: ray
Differential Revision: https://reviews.freebsd.org/D27110
lualoader: add loader_conf_dirs support (loader.conf.d)
loader_conf_dirs is the supporting mechanism for the included
/boot/loader.conf.d directory. When lualoader finishes processing all of
the loader_conf_files it finds after walking /boot/defaults/loader.conf,
it will now check any and all loader_conf_dirs and process files ending
in ".conf" as if they were a loader.conf.
Note that loader_conf_files may be specified in a loader.conf.d config
file, but loader_conf_dirs may *not*. It will only be processed as specified
in /boot/defaults/loader.conf and any loader_conf_files that were loaded
from there.
Ed Maste [Thu, 31 Dec 2020 15:29:08 +0000 (10:29 -0500)]
ldconfig: Retire a.out support
aout support in ldconfig hasn't been required since FreeBSD 2.x.
If someone needs to use FreeBSD 2 shared libraries they will be best
served by using a FreeBSD 2 ldconfig as well.
In aa5e1b42e6e3 we removed the ldconfig a.out invocation from rc.d but
left the support in ldconfig itself. Remove it now.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27481
John Baldwin [Wed, 30 Dec 2020 23:18:02 +0000 (15:18 -0800)]
rsu: Don't modify read-only firmware block.
The firmware header loaded into an rsu(4) device has to be customized
to reflect device settings. The driver was overwriting the header
from the shared firmware image before sending it to the device. If
two devices attached at the same time with different settings, one
device could potentially get a corrupted header. The recent changes
in a095390344fb1795c1b118a2f84da8f6a7f254ab exposed this bug in the
form of a panic as the firmware blobs are now marked read-only in
object files and mapped read-only by the kernel.
To avoid the bug, change the driver to allocate a copy of the firmware
header on the stack that is initialized before writing it to the
device.
Justin Hibbits [Tue, 8 Dec 2020 22:49:25 +0000 (16:49 -0600)]
powerpc: Fix copyin/copyout race condition
It's possible for a context switch, and CPU migration, to occur between
fetching the PCPU context and extracting the pc_curpcb. This can cause
the fault handler to be installed for the wrong thread, leading to a
panic in copyin()/copyout(). Since curthread is already in %r13, just
use that directly, as GPRs are migrated, so there is no migration race
risk.
Justin Hibbits [Thu, 4 Jun 2020 18:15:15 +0000 (13:15 -0500)]
powerpc: Optimize copyinstr() to avoid repeatedly mapping user strings
Currently copyinstr() uses fubyte() to read each byte from userspace.
However, this means that for each byte, it calls pmap_map_user_ptr() to
map the string into memory. This is needlessly wasteful, since the
string will rarely ever cross a segment boundary. Instead, map a
segment at a time, and copy as much from that segment as possible at a
time.
Measured with the HPT pmap on powerpc64, this saves roughly 8% time on
buildkernel, and 5% on buildworld, in wallclock time.
Alexander Motin [Wed, 30 Dec 2020 18:40:37 +0000 (13:40 -0500)]
Add "-n" flag to sockstat.
sockstat can "hang" on getpwuid() calls in situations when FreeBSD
is joined to a directory service (AD/LDAP etc) and the directory
service fail to answer in a timely manner when trying to resolve
numeric UIDs to user names.
Submitted by: Caleb St. John <caleb@ixsystems.com>
MFC after: 1 week
John Baldwin [Wed, 9 Dec 2020 22:36:01 +0000 (14:36 -0800)]
Don't try to adjust a TLS TOE socket that has been closed.
The handshake timer can race with another thread sending a FIN or RST
to close a TOE TLS socket. Just bail from the timer without
rescheduling if the connection is closed when the timer fires.
Reported by: Sony Arpita Das @ Chelsio QA
Reviewed by: np
Differential Revision: https://reviews.freebsd.org/D27583
Michal Meloun [Tue, 29 Dec 2020 13:49:41 +0000 (14:49 +0100)]
sys/tools: Add a tool for generating arm and arm64 kernel images.
This tool can generate kernel images without changing the offsets in
the final executable. It replaces the ELF header by properly sized zeroed
block then emits a relative jump to _start(for 'v7jump' or 'v8jump' option)
or the booti header (for 'v8booti' option) to the beginning of the converted file.
Submited by: ian
Roger Pau Monné [Wed, 25 Nov 2020 11:34:38 +0000 (12:34 +0100)]
xen: allow limiting the amount of duplicated pending xenstore watches
Xenstore watches received are queued in a list and processed in a
deferred thread. Such queuing was done without any checking, so a
guest could potentially trigger a resource starvation against the
FreeBSD kernel if such kernel is watching any user-controlled xenstore
path.
Allowing limiting the amount of pending events a watch can accumulate
to prevent a remote guest from triggering this resource starvation
issue.
For the PV device backends and frontends this limitation is only
applied to the other end /state node, which is limited to 1 pending
event, the rest of the watched paths can still have unlimited pending
watches because they are either local or controlled by a privileged
domain.
The xenstore user-space device gets special treatment as it's not
possible for the kernel to know whether the paths being watched by
user-space processes are controlled by a guest domain. For this reason
watches set by the xenstore user-space device are limited to 1000
pending events. Note this can be modified using the
max_pending_watch_events sysctl of the device.
This is XSA-349.
Sponsored by: Citrix Systems R&D
MFC after: 3 days
As pointed out, there are several problems with that commit:
1. The new semantics, while useful for clients where multiple
threads use separate contexts, breaks clients which correctly
share a single one
2. Change in semantics would require a library version bump
3. It doesn't build with GCC
Hiroki Sato [Tue, 29 Dec 2020 19:21:19 +0000 (04:21 +0900)]
Fix generation of colldef source files for non-UTF-8 locales
- Files for colldef were generated by duplicating UTF-8 collation files
for each language and included invalid characters in the non-UTF-8
encodings. localedef(1) does not allow those characters.
cldr2def.pl now checks if the characters are valid based on charmap files.
TODO: ja_JP.UTF-8 locale should not be generated solely from CLDR because
it was standardized in a document "UI-OSF Application Platform Profile for
Japanese Environment" which was incompatible with information in CLDR.
Most of commercial Unix vendors adopt this pre-Unicode-era document
as the reference even for UTF-8 locale. Newer versions of Solaris have
added a CLDR version as ja_JP.UTF-8@cldr, and IBM AIX has used
JA_JP.UTF-8 for the UI-OSF specification and ja_JP.UTF-8 for CLDR.
Note that this commit does not change generation of ja_JP.UTF-8.
Changes related to this issue will be committed separately later.
- Generate POSIX charamap UTF-32 as a reference. It was confusing that
charmap.xml used Unicode names defined in UnicodeData.txt though POSIX
charmap used slightly different names for the same code points.
cldr2def.pl now uses UTF-32.cm as single information source for Unicode
symbol names and code points. Charset.xml is also updated to use them.
- Fix a bug in get_encodings() in cldr2def.pl which did not understand
0x00+0x00 notation correctly in charmaps/ISCII-DEV.TXT.
- Do not regenerate posix/xx_Comm_C.UTF-8.src every time when doing
"make build".
Add two simple examples. In this case I opted to show a small portion of
the output since it helps to understand what the tool does. It shows the use
of the -t flag too.
Adrian Chadd [Tue, 29 Dec 2020 17:50:49 +0000 (09:50 -0800)]
[wlanwatch] fix compiler warnings-as-errors on gcc-6.4 mips
* argc/argv are currently unused
* msglen is currently unused
* "default" is a const buffer, but char *cp isn't, so
change default string to be a non-const global string variable
* Make 'cp' private to each context that's using it, which fixes
a "variable shadows previous declaration" warning and makes it
easier to track where it was being leaked between address family
sections
Each entry actually stores a native pointer, not a uint64_t quantity. While
we're here, go ahead and export the pointer as-is rather than converting it
to KVA. This may be more useful as consumers can map /dev/mem and observe
the entry.
For reference, see: sys/contrib/edk2/Include/Uefi/UefiSpec.h
Streamline the infiniband code according to the ethernet code.
Specifically implement the if_requestencap callback function for infiniband.
Most of the changes are simply a cut and paste of the equivalent ethernet part.
Need to update both link layer address and broadcast address when active link changes for IP over infiniband.
This is because the broadcast address contains the so-called P-key, which is interface dependent.
geom(4): make g_newprovider_event() return if G_P_WITHER is set
This fixes a failed assertion in scenario where the provider
disappears, disk_gone() gets called, and at the exact same
time something else closes the device node triggering a retaste.
Reviewed By: mav
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27330
libcam(3): make cam_getccb(3) zero the whole ccb, not just the header
Leaving zeroing to the clients leads to error-prone pointer
tricks (zeroing needs to preserve the CCB header), and this
code is not performance-critical, so there's really no reason
to not do it.
Reviewed By: imp, rpokala (manpages)
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27333
Ryan Libby [Mon, 28 Dec 2020 22:03:36 +0000 (14:03 -0800)]
kern.mk: drop flag only patched in-tree gcc understood
-mno-align-long-strings was a flag maintained by FreeBSD for the
now-deleted in-tree gcc. Upstream gcc has no such flag, so just drop
it.
The flag was originally submitted by bde and committed in 2002 (svn
r97911 & r104455). However, upstream gcc did address this same issue in
2004 (gcc svn r76694 / git 4137ba7ab7a), reducing long string alignment
in general, and to 1 with -Os.
Alan Somers [Thu, 24 Dec 2020 19:21:00 +0000 (19:21 +0000)]
fusefs: delete some dead code
The original fusefs GSoC project seems to have envisioned exchanging two
types of messages with FUSE servers. Perhaps vectored and non-vectored?
But in practice only one type has ever been used. Delete the other type.
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27770
Alan Somers [Thu, 24 Dec 2020 06:03:06 +0000 (06:03 +0000)]
fusefs: fix the tests for a wider range of maxphys
maxphys is now a tunable, ever since r368124. The default value is also
larger than it used to be. That broke several fusefs tests that made
assumptions about maxphys.
* WriteCluster.clustering used the MAXPHYS compile-time constant.
* WriteBackAsync.direct_io_partially_overlaps_cached_block implicitly
depended on the default value of maxphys. Fix it by making the
dependency explicit.
* Write.write_large implicitly assumed that maxphys would be no more
than twice maxbcachebuf. Fix it by explicitly setting m_max_write.
* WriteCluster.clustering and several others failed because the MockFS
module did not work for max_write > 128KB (which most tests would set
when maxphys > 256KB). Limit max_write accordingly. This is the same
as fusefs-libs's behavior.
* Bmap's tests were originally written for MAXPHYS=128KB. With larger
values, the simulated file size was too small.
Ed Maste [Mon, 28 Dec 2020 16:56:00 +0000 (11:56 -0500)]
Correct font.h comment describing vfnt font maps
Commit 41fb06651122 doubled the number of glyph maps in the vfnt format
from 2 to 4 to support double-width characters, but a comment describing
the maps was not updated to match.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Ryan Libby [Mon, 28 Dec 2020 08:44:25 +0000 (00:44 -0800)]
rtld-elf: link libcompiler_rt on all architectures
Statically link rtld-elf with libcompiler_rt on all architectures so
that we don't need to try to pick and choose the bits we need from it
for each architecture (we now leave that to the linker). Compilers may
emit calls to support functions in this library, but because of the use
of the linker flag -nostdlib for rtld's special needs, the library is
not linked as normal.
Previously we had two different solutions. On some architectures, we
were able to extract reimplementations of the necessary builtin
functions from our special build of libc. On ARM, we just linked
libcompiler_rt.
This is motivated by the same issue as D26199 and D27665, but should be
a simpler solution that will apply to all architectures.
Ryan Libby [Mon, 28 Dec 2020 08:44:25 +0000 (00:44 -0800)]
libcompiler_rt: stop building with stack smashing protection
libcompiler_rt implements certain functions that clang and gcc emit
calls to as part of their codegen (e.g. for extended width math). Build
it without stack smashing protection (SSP, -fstack-protector) in order
to support building binaries without SSP, especially the dynamic linker.
Besides, SSP is probably not very valuable in this library.
projects / FreeBSD / FreeBSD.git / log