Roger Pau Monné [Wed, 25 Nov 2020 11:34:38 +0000 (12:34 +0100)]
xen: allow limiting the amount of duplicated pending xenstore watches
Xenstore watches received are queued in a list and processed in a
deferred thread. Such queuing was done without any checking, so a
guest could potentially trigger a resource starvation against the
FreeBSD kernel if such kernel is watching any user-controlled xenstore
path.
Allowing limiting the amount of pending events a watch can accumulate
to prevent a remote guest from triggering this resource starvation
issue.
For the PV device backends and frontends this limitation is only
applied to the other end /state node, which is limited to 1 pending
event, the rest of the watched paths can still have unlimited pending
watches because they are either local or controlled by a privileged
domain.
The xenstore user-space device gets special treatment as it's not
possible for the kernel to know whether the paths being watched by
user-space processes are controlled by a guest domain. For this reason
watches set by the xenstore user-space device are limited to 1000
pending events. Note this can be modified using the
max_pending_watch_events sysctl of the device.
This is XSA-349.
Sponsored by: Citrix Systems R&D
MFC after: 3 days
As pointed out, there are several problems with that commit:
1. The new semantics, while useful for clients where multiple
threads use separate contexts, breaks clients which correctly
share a single one
2. Change in semantics would require a library version bump
3. It doesn't build with GCC
Hiroki Sato [Tue, 29 Dec 2020 19:21:19 +0000 (04:21 +0900)]
Fix generation of colldef source files for non-UTF-8 locales
- Files for colldef were generated by duplicating UTF-8 collation files
for each language and included invalid characters in the non-UTF-8
encodings. localedef(1) does not allow those characters.
cldr2def.pl now checks if the characters are valid based on charmap files.
TODO: ja_JP.UTF-8 locale should not be generated solely from CLDR because
it was standardized in a document "UI-OSF Application Platform Profile for
Japanese Environment" which was incompatible with information in CLDR.
Most of commercial Unix vendors adopt this pre-Unicode-era document
as the reference even for UTF-8 locale. Newer versions of Solaris have
added a CLDR version as ja_JP.UTF-8@cldr, and IBM AIX has used
JA_JP.UTF-8 for the UI-OSF specification and ja_JP.UTF-8 for CLDR.
Note that this commit does not change generation of ja_JP.UTF-8.
Changes related to this issue will be committed separately later.
- Generate POSIX charamap UTF-32 as a reference. It was confusing that
charmap.xml used Unicode names defined in UnicodeData.txt though POSIX
charmap used slightly different names for the same code points.
cldr2def.pl now uses UTF-32.cm as single information source for Unicode
symbol names and code points. Charset.xml is also updated to use them.
- Fix a bug in get_encodings() in cldr2def.pl which did not understand
0x00+0x00 notation correctly in charmaps/ISCII-DEV.TXT.
- Do not regenerate posix/xx_Comm_C.UTF-8.src every time when doing
"make build".
Add two simple examples. In this case I opted to show a small portion of
the output since it helps to understand what the tool does. It shows the use
of the -t flag too.
Adrian Chadd [Tue, 29 Dec 2020 17:50:49 +0000 (09:50 -0800)]
[wlanwatch] fix compiler warnings-as-errors on gcc-6.4 mips
* argc/argv are currently unused
* msglen is currently unused
* "default" is a const buffer, but char *cp isn't, so
change default string to be a non-const global string variable
* Make 'cp' private to each context that's using it, which fixes
a "variable shadows previous declaration" warning and makes it
easier to track where it was being leaked between address family
sections
Each entry actually stores a native pointer, not a uint64_t quantity. While
we're here, go ahead and export the pointer as-is rather than converting it
to KVA. This may be more useful as consumers can map /dev/mem and observe
the entry.
For reference, see: sys/contrib/edk2/Include/Uefi/UefiSpec.h
Streamline the infiniband code according to the ethernet code.
Specifically implement the if_requestencap callback function for infiniband.
Most of the changes are simply a cut and paste of the equivalent ethernet part.
Need to update both link layer address and broadcast address when active link changes for IP over infiniband.
This is because the broadcast address contains the so-called P-key, which is interface dependent.
geom(4): make g_newprovider_event() return if G_P_WITHER is set
This fixes a failed assertion in scenario where the provider
disappears, disk_gone() gets called, and at the exact same
time something else closes the device node triggering a retaste.
Reviewed By: mav
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27330
libcam(3): make cam_getccb(3) zero the whole ccb, not just the header
Leaving zeroing to the clients leads to error-prone pointer
tricks (zeroing needs to preserve the CCB header), and this
code is not performance-critical, so there's really no reason
to not do it.
Reviewed By: imp, rpokala (manpages)
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27333
Ryan Libby [Mon, 28 Dec 2020 22:03:36 +0000 (14:03 -0800)]
kern.mk: drop flag only patched in-tree gcc understood
-mno-align-long-strings was a flag maintained by FreeBSD for the
now-deleted in-tree gcc. Upstream gcc has no such flag, so just drop
it.
The flag was originally submitted by bde and committed in 2002 (svn
r97911 & r104455). However, upstream gcc did address this same issue in
2004 (gcc svn r76694 / git 4137ba7ab7a), reducing long string alignment
in general, and to 1 with -Os.
Alan Somers [Thu, 24 Dec 2020 19:21:00 +0000 (19:21 +0000)]
fusefs: delete some dead code
The original fusefs GSoC project seems to have envisioned exchanging two
types of messages with FUSE servers. Perhaps vectored and non-vectored?
But in practice only one type has ever been used. Delete the other type.
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D27770
Alan Somers [Thu, 24 Dec 2020 06:03:06 +0000 (06:03 +0000)]
fusefs: fix the tests for a wider range of maxphys
maxphys is now a tunable, ever since r368124. The default value is also
larger than it used to be. That broke several fusefs tests that made
assumptions about maxphys.
* WriteCluster.clustering used the MAXPHYS compile-time constant.
* WriteBackAsync.direct_io_partially_overlaps_cached_block implicitly
depended on the default value of maxphys. Fix it by making the
dependency explicit.
* Write.write_large implicitly assumed that maxphys would be no more
than twice maxbcachebuf. Fix it by explicitly setting m_max_write.
* WriteCluster.clustering and several others failed because the MockFS
module did not work for max_write > 128KB (which most tests would set
when maxphys > 256KB). Limit max_write accordingly. This is the same
as fusefs-libs's behavior.
* Bmap's tests were originally written for MAXPHYS=128KB. With larger
values, the simulated file size was too small.
Ed Maste [Mon, 28 Dec 2020 16:56:00 +0000 (11:56 -0500)]
Correct font.h comment describing vfnt font maps
Commit 41fb06651122 doubled the number of glyph maps in the vfnt format
from 2 to 4 to support double-width characters, but a comment describing
the maps was not updated to match.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Ryan Libby [Mon, 28 Dec 2020 08:44:25 +0000 (00:44 -0800)]
rtld-elf: link libcompiler_rt on all architectures
Statically link rtld-elf with libcompiler_rt on all architectures so
that we don't need to try to pick and choose the bits we need from it
for each architecture (we now leave that to the linker). Compilers may
emit calls to support functions in this library, but because of the use
of the linker flag -nostdlib for rtld's special needs, the library is
not linked as normal.
Previously we had two different solutions. On some architectures, we
were able to extract reimplementations of the necessary builtin
functions from our special build of libc. On ARM, we just linked
libcompiler_rt.
This is motivated by the same issue as D26199 and D27665, but should be
a simpler solution that will apply to all architectures.
Ryan Libby [Mon, 28 Dec 2020 08:44:25 +0000 (00:44 -0800)]
libcompiler_rt: stop building with stack smashing protection
libcompiler_rt implements certain functions that clang and gcc emit
calls to as part of their codegen (e.g. for extended width math). Build
it without stack smashing protection (SSP, -fstack-protector) in order
to support building binaries without SSP, especially the dynamic linker.
Besides, SSP is probably not very valuable in this library.
Kevin Lo [Mon, 28 Dec 2020 02:30:21 +0000 (10:30 +0800)]
Mention boot1.efi since it is an image of 800KB FAT filesystem stored as
/EFI/BOOT/BOOTX64.EFI.
Differential Revision: https://reviews.freebsd.org/D27782
Mateusz Guzik [Sun, 27 Dec 2020 23:33:04 +0000 (23:33 +0000)]
vfs: add FAILIFEXISTS flag
Both FreeBSD and Linux mkdir -p walk the tree up ignoring any EEXIST on
the way and both are used a lot when building respective kernels.
This poses a problem as spurious locking avoidably interferes with
concurrent operations like getdirentries on affected directories.
Work around the problem by adding FAILIFEXISTS flag. In case of lockless
lookup this manages to avoid any work to begin with, there is no speed
up for the locked case but perhaps this can be augmented later on.
For simplicity the only supported semantics are as used by mkdir.
Ryan Libby [Sun, 27 Dec 2020 22:33:13 +0000 (14:33 -0800)]
Merge commit d8a09b3a0 from openzfs git (by Ryan Libby):
lua: avoid gcc -Wreturn-local-addr bug
Avoid a bug with gcc's -Wreturn-local-addr warning with some
obfuscation. In buggy versions of gcc, if a return value is an
expression that involves the address of a local variable, and even if
that address is legally converted to a non-pointer type, a warning may
be emitted and the value of the address may be replaced with zero.
Howerver, buggy versions don't emit the warning or replace the value
when simply returning a local variable of non-pointer type.
Ryan Libby [Sun, 27 Dec 2020 22:33:13 +0000 (14:33 -0800)]
Merge commit 956f94010 from openzfs git (by Ryan Libby):
spa: avoid type narrowing warning
Building the spa module for i386 caused gcc to emit
-Wint-to-pointer-cast "cast to pointer from integer of different size"
because spa.spa_did was uint64_t but pthread_join (via thread_join in
spa_deactivate) takes a pointer (32-bit on i386). Define spa_did to be
pointer-size instead. For now spa_did is in fact never non-zero and the
thread_join could instead be ifdef'd out, but changing the size of
spa_did may be more useful for the future.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Libby <rlibby@FreeBSD.org>
Closes #11336
Ryan Libby [Sun, 27 Dec 2020 22:33:13 +0000 (14:33 -0800)]
Merge commit c7500ded3 from openzfs git (by Ryan Libby):
FreeBSD libzfs: gcc requires __thread after static
Building libzfs with gcc on FreeBSD failed because gcc is picky about
the order of keywords in declarations with __thread, whereas clang is
more relaxed.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Ryan Libby <rlibby@FreeBSD.org>
Closes #11331
This looks like it was once from the illumnos compat code.
FreeBSD doesn't have cmn_err as a compiler format attribute, so
it definitely errors out.
It doesn't show up on LLVM because it doesn't trigger at all.
Add in the format flags but keep them behind #if 0 for now;
there are too many format issues that trigger when one does
format checking in the shared code.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: adrian chadd <adrian@freebsd.org>
Closes #11068
Closes #11069
Ryan Libby [Sun, 27 Dec 2020 22:33:13 +0000 (14:33 -0800)]
openzfs: fix gcc kernel module builds
- Suppress -Wredundant-decls. Ultimately this warning is harmless in
any case, and it does not look like there is a simple way to avoid
redundant declarations in this case without a lot of header pollution
(e.g. having openzfs's shim param.h pulling in sys/kernel.h for hz).
- Suppress -Wnested-externs, which is useless anyway.
Unfortunately it was not sufficient just to modify OPENZFS_CFLAGS,
because the warning suppressions need to appear on the command line
after they are explicitly enabled by CWARNFLAGS from sys/conf/kern.mk,
but OPENZFS_CFLAGS get added before due to use of -I for the shims.
Mark Johnston [Sun, 27 Dec 2020 21:50:54 +0000 (16:50 -0500)]
vm: Fix some bugs in the page busying code
In vm_page_busy_acquire(), load the object pointer using
atomic_load_ptr() as we do elsewhere. Per the comment, the object
identity must be consistent across sleeps.
In vm_page_grab_sleep(), pass the correct pindex to
_vm_page_busy_sleep(). The pindex is used to re-check the page's
identity before going to sleep. In particular, vm_page_grab_sleep() is
used in unlocked grab, so the object lock is not necessarily held when
verifying the page's identity, and the pindex may change if the page is
moved, or freed and re-allocated. I believe this can result in spurious
VM_PAGER_FAILs from vm_page_grab_valid_unlocked() or early termination
of vm_page_grab_pages_unlocked().
In vm_page_grab_pages(), pass the correct pindex to
vm_page_grab_sleep(). Otherwise I believe vm_page_grab_pages() will
effectively spin when attempting to busy a busy page after the first
index in the range.
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27607
Stefan Eßer [Sat, 26 Dec 2020 21:21:49 +0000 (22:21 +0100)]
Replace sscanf() by strtoul()
This change has been motivated by a mail from bde sent in 2015 in
which he mentioned inappropriate use of sscanf() in 3 programs in
/bin.
This change removes the potential mismatch of the types of the return
values and the variable width specified in the scan pattern. While
there was no issue with the patterns and types used, the new code is
simpler and more efficient.
Stefan Eßer [Sat, 26 Dec 2020 21:10:54 +0000 (22:10 +0100)]
Simplify LS_COLWIDTHS processing
The previous version normalized the width list (replaced empty fields
with "0") just to be able to use sscanf() on the string.
It is much simpler to just parse the string as-is.
The clearing of f_notabs is preserved for the case that less than 9
width values have been defined, but I do not understand the rationale
for this particular condition. E.g., LS_COLWIDTHS="::::::::" will be
counted as 9 defined fields (may clear f_notabs) but is no different
fron LS_COLWIDTHS="" with regard to the field width (and that does not
clear f_notabs, since there are less than 9 fields).
Ed Maste [Sun, 27 Dec 2020 00:34:24 +0000 (19:34 -0500)]
Perform kernel linker ifunc test only for builds
dvl reported that "make installkernel" failed with "amd64/arm64/i386
kernel requires linker ifunc support." This test should apply to builds
only; the linker is not used at install time.
I think the same (ifunc-supporting) linker used to build the kernel
should be detected at install time in usual cases (and so not trigger
this error). However, there is no reason to disallow the install, if
for some reason the expected linker isn't the one tested at install
time.
PR: 251580
Reported by: dvl
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
These handlers could interrupt code which has interrupts disabled,
and if a spurious page fault occurs during exception handler run,
we get clobbered %cr2 in higher level stack.
This is mostly a speculation, but it is based on hints from good sources.
MFC after: 1 week
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27772
Expose eventfd in the native API/ABI using a new __specialfd syscall
eventfd is a Linux system call that produces special file descriptors
for event notification. When porting Linux software, it is currently
usually emulated by epoll-shim on top of kqueues. Unfortunately, kqueues
are not passable between processes. And, as noted by the author of
epoll-shim, even if they were, the library state would also have to be
passed somehow. This came up when debugging strange HW video decode
failures in Firefox. A native implementation would avoid these problems
and help with porting Linux software.
Since we now already have an eventfd implementation in the kernel (for
the Linuxulator), it's pretty easy to expose it natively, which is what
this patch does.
Mark Johnston [Sat, 26 Dec 2020 21:07:40 +0000 (16:07 -0500)]
sendfile: Ensure that sfio->npages is initialized
We initialize sfio->npages only when some I/O is required to satisfy the
request. However, sendfile_iodone() contains an INVARIANTS-only check
that references sfio->npages, and this check is executed even if no I/O
is performed, so the check may use an uninitialized value.
Fix the problem by initializing sfio->npages earlier. Note that
sendfile_swapin() always initializes the page array. In some rare cases
we need to trim the page array so ensure that sfio->npages gets updated
accordingly.
Reported by: syzkaller (with KASAN)
Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27726
Andrew Turner [Wed, 23 Dec 2020 18:56:09 +0000 (18:56 +0000)]
Use the base address for early arm64 page tables
Use the kernel physical base rather than the ttbr0 base when building
the kernel identity map. The latter is correct with current assumptions
but may not always be the case.
projects / FreeBSD / FreeBSD.git / log