Kyle Evans [Thu, 31 Dec 2020 17:15:45 +0000 (11:15 -0600)]
stand: properly declare subdir deps or .WAIT, do parallel build
buildworld already runs the stand build in parallel[1], so make it easier to
identify ordering issues by properly establishing dependencies or adding
.WAIT where needed.
Everything in stand/ relies on libsa, either directly or indirectly, because
libsa build is where the stand headers get installed and it gets linked in
most places.
Interpreters depend on their libs, machine dirs usually depend on top-level
libs that are getting built and at least one of the interpreter flavors.
For i386, order btx/libi386/libfirewire before everything else using a
big-ol-.WAIT hammer. btx is the most common dependency, but the others are
used sporadically. This seems to be where the race reporting on the mailing
list is- AFAICT, the following sequence is happening:
1.) One of the loaders gets built based on stale btx/btxldr
2.) btx/btxldr gets rebuilt
3.) installworld triggers loader rebuild because btx was rebuilt after
This seems like the most plausible explanation, as they've verified system
time and timestamps.
While we're here, let's switch stand/ over to a completely parallel build so
we can work out these kinds of issues in isolation rather than in the middle
of a larger build.
Kyle Evans [Thu, 31 Dec 2020 17:12:39 +0000 (11:12 -0600)]
arm: tune vmparam.h towards a little more modern
An 8MB max stack size is quite limiting in today's world, and in-fact is
the *default* stack size for almost every other arch (including mips).
Raise the default to 4MB (should be pretty reasonable) and the max to 64MB.
NetBSD made a similar move back in 2015 and raised MAXDSIZ to 1856 at the
same time, so let's just roll that in as well. They later lowered it, but
eventually raised it back to 1856 in order to build rust.
This was noticed while looking at qemu-bsd-user's default stack sizes and
growth behavior (or lack thereof).
Reviewed by: ian
Differential Revision: https://reviews.freebsd.org/D27218
Kyle Evans [Thu, 31 Dec 2020 16:50:43 +0000 (10:50 -0600)]
vt: restore tty when console is ungrabbed
When a break-to-debugger is triggered, kdb will grab the console and vt(4)
will generally switch back to ttyv0. If one issues a continue from the
debugger, then kdb will ungrab the console and the system rolls on.
This change adds a perhaps minor feature: when we're down to grab == 0 and
if vt actually switched away to ttyv0, switch back to the tty it was
previously on before the console was grabbed.
The justification behind this is that a typical flow is to work in
!ttyv0 to avoid console spam while occasionally dropping to ddb to inspect
system state before returning. This could easily enough be tossed behind
a sysctl or something if it's not generally appreciated, but I anticipate
indifference.
Reviewed by: ray
Differential Revision: https://reviews.freebsd.org/D27110
Kyle Evans [Thu, 31 Dec 2020 16:45:41 +0000 (10:45 -0600)]
vt: more carefully handle vt_allocate_keyboard grab work
vt_allocate_keyboard only needs to unwind the effects of keyboard-grabbing,
rather than any associated vt window action that may have also happened.
Split out the bits that do the keyboard work into *_noswitch equivalents,
and use those in keyboard allocation. This will be less error-prone when a
later change will offer up different window state behavior when the console
is ungrabbed.
Reviewed by: ray
Differential Revision: https://reviews.freebsd.org/D27110
lualoader: add loader_conf_dirs support (loader.conf.d)
loader_conf_dirs is the supporting mechanism for the included
/boot/loader.conf.d directory. When lualoader finishes processing all of
the loader_conf_files it finds after walking /boot/defaults/loader.conf,
it will now check any and all loader_conf_dirs and process files ending
in ".conf" as if they were a loader.conf.
Note that loader_conf_files may be specified in a loader.conf.d config
file, but loader_conf_dirs may *not*. It will only be processed as specified
in /boot/defaults/loader.conf and any loader_conf_files that were loaded
from there.
Ed Maste [Thu, 31 Dec 2020 15:29:08 +0000 (10:29 -0500)]
ldconfig: Retire a.out support
aout support in ldconfig hasn't been required since FreeBSD 2.x.
If someone needs to use FreeBSD 2 shared libraries they will be best
served by using a FreeBSD 2 ldconfig as well.
In aa5e1b42e6e3 we removed the ldconfig a.out invocation from rc.d but
left the support in ldconfig itself. Remove it now.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27481
John Baldwin [Wed, 30 Dec 2020 23:18:02 +0000 (15:18 -0800)]
rsu: Don't modify read-only firmware block.
The firmware header loaded into an rsu(4) device has to be customized
to reflect device settings. The driver was overwriting the header
from the shared firmware image before sending it to the device. If
two devices attached at the same time with different settings, one
device could potentially get a corrupted header. The recent changes
in a095390344fb1795c1b118a2f84da8f6a7f254ab exposed this bug in the
form of a panic as the firmware blobs are now marked read-only in
object files and mapped read-only by the kernel.
To avoid the bug, change the driver to allocate a copy of the firmware
header on the stack that is initialized before writing it to the
device.
Justin Hibbits [Tue, 8 Dec 2020 22:49:25 +0000 (16:49 -0600)]
powerpc: Fix copyin/copyout race condition
It's possible for a context switch, and CPU migration, to occur between
fetching the PCPU context and extracting the pc_curpcb. This can cause
the fault handler to be installed for the wrong thread, leading to a
panic in copyin()/copyout(). Since curthread is already in %r13, just
use that directly, as GPRs are migrated, so there is no migration race
risk.
Justin Hibbits [Thu, 4 Jun 2020 18:15:15 +0000 (13:15 -0500)]
powerpc: Optimize copyinstr() to avoid repeatedly mapping user strings
Currently copyinstr() uses fubyte() to read each byte from userspace.
However, this means that for each byte, it calls pmap_map_user_ptr() to
map the string into memory. This is needlessly wasteful, since the
string will rarely ever cross a segment boundary. Instead, map a
segment at a time, and copy as much from that segment as possible at a
time.
Measured with the HPT pmap on powerpc64, this saves roughly 8% time on
buildkernel, and 5% on buildworld, in wallclock time.
Alexander Motin [Wed, 30 Dec 2020 18:40:37 +0000 (13:40 -0500)]
Add "-n" flag to sockstat.
sockstat can "hang" on getpwuid() calls in situations when FreeBSD
is joined to a directory service (AD/LDAP etc) and the directory
service fail to answer in a timely manner when trying to resolve
numeric UIDs to user names.
Submitted by: Caleb St. John <caleb@ixsystems.com>
MFC after: 1 week
John Baldwin [Wed, 9 Dec 2020 22:36:01 +0000 (14:36 -0800)]
Don't try to adjust a TLS TOE socket that has been closed.
The handshake timer can race with another thread sending a FIN or RST
to close a TOE TLS socket. Just bail from the timer without
rescheduling if the connection is closed when the timer fires.
Reported by: Sony Arpita Das @ Chelsio QA
Reviewed by: np
Differential Revision: https://reviews.freebsd.org/D27583
Michal Meloun [Tue, 29 Dec 2020 13:49:41 +0000 (14:49 +0100)]
sys/tools: Add a tool for generating arm and arm64 kernel images.
This tool can generate kernel images without changing the offsets in
the final executable. It replaces the ELF header by properly sized zeroed
block then emits a relative jump to _start(for 'v7jump' or 'v8jump' option)
or the booti header (for 'v8booti' option) to the beginning of the converted file.
Submited by: ian
Roger Pau Monné [Wed, 25 Nov 2020 11:34:38 +0000 (12:34 +0100)]
xen: allow limiting the amount of duplicated pending xenstore watches
Xenstore watches received are queued in a list and processed in a
deferred thread. Such queuing was done without any checking, so a
guest could potentially trigger a resource starvation against the
FreeBSD kernel if such kernel is watching any user-controlled xenstore
path.
Allowing limiting the amount of pending events a watch can accumulate
to prevent a remote guest from triggering this resource starvation
issue.
For the PV device backends and frontends this limitation is only
applied to the other end /state node, which is limited to 1 pending
event, the rest of the watched paths can still have unlimited pending
watches because they are either local or controlled by a privileged
domain.
The xenstore user-space device gets special treatment as it's not
possible for the kernel to know whether the paths being watched by
user-space processes are controlled by a guest domain. For this reason
watches set by the xenstore user-space device are limited to 1000
pending events. Note this can be modified using the
max_pending_watch_events sysctl of the device.
This is XSA-349.
Sponsored by: Citrix Systems R&D
MFC after: 3 days
As pointed out, there are several problems with that commit:
1. The new semantics, while useful for clients where multiple
threads use separate contexts, breaks clients which correctly
share a single one
2. Change in semantics would require a library version bump
3. It doesn't build with GCC
Hiroki Sato [Tue, 29 Dec 2020 19:21:19 +0000 (04:21 +0900)]
Fix generation of colldef source files for non-UTF-8 locales
- Files for colldef were generated by duplicating UTF-8 collation files
for each language and included invalid characters in the non-UTF-8
encodings. localedef(1) does not allow those characters.
cldr2def.pl now checks if the characters are valid based on charmap files.
TODO: ja_JP.UTF-8 locale should not be generated solely from CLDR because
it was standardized in a document "UI-OSF Application Platform Profile for
Japanese Environment" which was incompatible with information in CLDR.
Most of commercial Unix vendors adopt this pre-Unicode-era document
as the reference even for UTF-8 locale. Newer versions of Solaris have
added a CLDR version as ja_JP.UTF-8@cldr, and IBM AIX has used
JA_JP.UTF-8 for the UI-OSF specification and ja_JP.UTF-8 for CLDR.
Note that this commit does not change generation of ja_JP.UTF-8.
Changes related to this issue will be committed separately later.
- Generate POSIX charamap UTF-32 as a reference. It was confusing that
charmap.xml used Unicode names defined in UnicodeData.txt though POSIX
charmap used slightly different names for the same code points.
cldr2def.pl now uses UTF-32.cm as single information source for Unicode
symbol names and code points. Charset.xml is also updated to use them.
- Fix a bug in get_encodings() in cldr2def.pl which did not understand
0x00+0x00 notation correctly in charmaps/ISCII-DEV.TXT.
- Do not regenerate posix/xx_Comm_C.UTF-8.src every time when doing
"make build".
Add two simple examples. In this case I opted to show a small portion of
the output since it helps to understand what the tool does. It shows the use
of the -t flag too.
Adrian Chadd [Tue, 29 Dec 2020 17:50:49 +0000 (09:50 -0800)]
[wlanwatch] fix compiler warnings-as-errors on gcc-6.4 mips
* argc/argv are currently unused
* msglen is currently unused
* "default" is a const buffer, but char *cp isn't, so
change default string to be a non-const global string variable
* Make 'cp' private to each context that's using it, which fixes
a "variable shadows previous declaration" warning and makes it
easier to track where it was being leaked between address family
sections
Each entry actually stores a native pointer, not a uint64_t quantity. While
we're here, go ahead and export the pointer as-is rather than converting it
to KVA. This may be more useful as consumers can map /dev/mem and observe
the entry.
For reference, see: sys/contrib/edk2/Include/Uefi/UefiSpec.h
Streamline the infiniband code according to the ethernet code.
Specifically implement the if_requestencap callback function for infiniband.
Most of the changes are simply a cut and paste of the equivalent ethernet part.
Need to update both link layer address and broadcast address when active link changes for IP over infiniband.
This is because the broadcast address contains the so-called P-key, which is interface dependent.
geom(4): make g_newprovider_event() return if G_P_WITHER is set
This fixes a failed assertion in scenario where the provider
disappears, disk_gone() gets called, and at the exact same
time something else closes the device node triggering a retaste.
Reviewed By: mav
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27330
libcam(3): make cam_getccb(3) zero the whole ccb, not just the header
Leaving zeroing to the clients leads to error-prone pointer
tricks (zeroing needs to preserve the CCB header), and this
code is not performance-critical, so there's really no reason
to not do it.
Reviewed By: imp, rpokala (manpages)
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27333
Ryan Libby [Mon, 28 Dec 2020 22:03:36 +0000 (14:03 -0800)]
kern.mk: drop flag only patched in-tree gcc understood
-mno-align-long-strings was a flag maintained by FreeBSD for the
now-deleted in-tree gcc. Upstream gcc has no such flag, so just drop
it.
The flag was originally submitted by bde and committed in 2002 (svn
r97911 & r104455). However, upstream gcc did address this same issue in
2004 (gcc svn r76694 / git 4137ba7ab7a), reducing long string alignment
in general, and to 1 with -Os.
Alan Somers [Thu, 24 Dec 2020 19:21:00 +0000 (19:21 +0000)]
fusefs: delete some dead code
The original fusefs GSoC project seems to have envisioned exchanging two
types of messages with FUSE servers. Perhaps vectored and non-vectored?
But in practice only one type has ever been used. Delete the other type.
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27770
Alan Somers [Thu, 24 Dec 2020 06:03:06 +0000 (06:03 +0000)]
fusefs: fix the tests for a wider range of maxphys
maxphys is now a tunable, ever since r368124. The default value is also
larger than it used to be. That broke several fusefs tests that made
assumptions about maxphys.
* WriteCluster.clustering used the MAXPHYS compile-time constant.
* WriteBackAsync.direct_io_partially_overlaps_cached_block implicitly
depended on the default value of maxphys. Fix it by making the
dependency explicit.
* Write.write_large implicitly assumed that maxphys would be no more
than twice maxbcachebuf. Fix it by explicitly setting m_max_write.
* WriteCluster.clustering and several others failed because the MockFS
module did not work for max_write > 128KB (which most tests would set
when maxphys > 256KB). Limit max_write accordingly. This is the same
as fusefs-libs's behavior.
* Bmap's tests were originally written for MAXPHYS=128KB. With larger
values, the simulated file size was too small.
Ed Maste [Mon, 28 Dec 2020 16:56:00 +0000 (11:56 -0500)]
Correct font.h comment describing vfnt font maps
Commit 41fb06651122 doubled the number of glyph maps in the vfnt format
from 2 to 4 to support double-width characters, but a comment describing
the maps was not updated to match.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Ryan Libby [Mon, 28 Dec 2020 08:44:25 +0000 (00:44 -0800)]
rtld-elf: link libcompiler_rt on all architectures
Statically link rtld-elf with libcompiler_rt on all architectures so
that we don't need to try to pick and choose the bits we need from it
for each architecture (we now leave that to the linker). Compilers may
emit calls to support functions in this library, but because of the use
of the linker flag -nostdlib for rtld's special needs, the library is
not linked as normal.
Previously we had two different solutions. On some architectures, we
were able to extract reimplementations of the necessary builtin
functions from our special build of libc. On ARM, we just linked
libcompiler_rt.
This is motivated by the same issue as D26199 and D27665, but should be
a simpler solution that will apply to all architectures.
Ryan Libby [Mon, 28 Dec 2020 08:44:25 +0000 (00:44 -0800)]
libcompiler_rt: stop building with stack smashing protection
libcompiler_rt implements certain functions that clang and gcc emit
calls to as part of their codegen (e.g. for extended width math). Build
it without stack smashing protection (SSP, -fstack-protector) in order
to support building binaries without SSP, especially the dynamic linker.
Besides, SSP is probably not very valuable in this library.
Kevin Lo [Mon, 28 Dec 2020 02:30:21 +0000 (10:30 +0800)]
Mention boot1.efi since it is an image of 800KB FAT filesystem stored as
/EFI/BOOT/BOOTX64.EFI.
Differential Revision: https://reviews.freebsd.org/D27782
Mateusz Guzik [Sun, 27 Dec 2020 23:33:04 +0000 (23:33 +0000)]
vfs: add FAILIFEXISTS flag
Both FreeBSD and Linux mkdir -p walk the tree up ignoring any EEXIST on
the way and both are used a lot when building respective kernels.
This poses a problem as spurious locking avoidably interferes with
concurrent operations like getdirentries on affected directories.
Work around the problem by adding FAILIFEXISTS flag. In case of lockless
lookup this manages to avoid any work to begin with, there is no speed
up for the locked case but perhaps this can be augmented later on.
For simplicity the only supported semantics are as used by mkdir.
Ryan Libby [Sun, 27 Dec 2020 22:33:13 +0000 (14:33 -0800)]
Merge commit d8a09b3a0 from openzfs git (by Ryan Libby):
lua: avoid gcc -Wreturn-local-addr bug
Avoid a bug with gcc's -Wreturn-local-addr warning with some
obfuscation. In buggy versions of gcc, if a return value is an
expression that involves the address of a local variable, and even if
that address is legally converted to a non-pointer type, a warning may
be emitted and the value of the address may be replaced with zero.
Howerver, buggy versions don't emit the warning or replace the value
when simply returning a local variable of non-pointer type.
Ryan Libby [Sun, 27 Dec 2020 22:33:13 +0000 (14:33 -0800)]
Merge commit 956f94010 from openzfs git (by Ryan Libby):
spa: avoid type narrowing warning
Building the spa module for i386 caused gcc to emit
-Wint-to-pointer-cast "cast to pointer from integer of different size"
because spa.spa_did was uint64_t but pthread_join (via thread_join in
spa_deactivate) takes a pointer (32-bit on i386). Define spa_did to be
pointer-size instead. For now spa_did is in fact never non-zero and the
thread_join could instead be ifdef'd out, but changing the size of
spa_did may be more useful for the future.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Libby <rlibby@FreeBSD.org>
Closes #11336
Ryan Libby [Sun, 27 Dec 2020 22:33:13 +0000 (14:33 -0800)]
Merge commit c7500ded3 from openzfs git (by Ryan Libby):
FreeBSD libzfs: gcc requires __thread after static
Building libzfs with gcc on FreeBSD failed because gcc is picky about
the order of keywords in declarations with __thread, whereas clang is
more relaxed.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Ryan Libby <rlibby@FreeBSD.org>
Closes #11331
This looks like it was once from the illumnos compat code.
FreeBSD doesn't have cmn_err as a compiler format attribute, so
it definitely errors out.
It doesn't show up on LLVM because it doesn't trigger at all.
Add in the format flags but keep them behind #if 0 for now;
there are too many format issues that trigger when one does
format checking in the shared code.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: adrian chadd <adrian@freebsd.org>
Closes #11068
Closes #11069
Ryan Libby [Sun, 27 Dec 2020 22:33:13 +0000 (14:33 -0800)]
openzfs: fix gcc kernel module builds
- Suppress -Wredundant-decls. Ultimately this warning is harmless in
any case, and it does not look like there is a simple way to avoid
redundant declarations in this case without a lot of header pollution
(e.g. having openzfs's shim param.h pulling in sys/kernel.h for hz).
- Suppress -Wnested-externs, which is useless anyway.
Unfortunately it was not sufficient just to modify OPENZFS_CFLAGS,
because the warning suppressions need to appear on the command line
after they are explicitly enabled by CWARNFLAGS from sys/conf/kern.mk,
but OPENZFS_CFLAGS get added before due to use of -I for the shims.
Mark Johnston [Sun, 27 Dec 2020 21:50:54 +0000 (16:50 -0500)]
vm: Fix some bugs in the page busying code
In vm_page_busy_acquire(), load the object pointer using
atomic_load_ptr() as we do elsewhere. Per the comment, the object
identity must be consistent across sleeps.
In vm_page_grab_sleep(), pass the correct pindex to
_vm_page_busy_sleep(). The pindex is used to re-check the page's
identity before going to sleep. In particular, vm_page_grab_sleep() is
used in unlocked grab, so the object lock is not necessarily held when
verifying the page's identity, and the pindex may change if the page is
moved, or freed and re-allocated. I believe this can result in spurious
VM_PAGER_FAILs from vm_page_grab_valid_unlocked() or early termination
of vm_page_grab_pages_unlocked().
In vm_page_grab_pages(), pass the correct pindex to
vm_page_grab_sleep(). Otherwise I believe vm_page_grab_pages() will
effectively spin when attempting to busy a busy page after the first
index in the range.
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27607