Alex Richardson [Wed, 6 Jan 2021 17:52:02 +0000 (17:52 +0000)]
Stop passing MK_WARNS=no for bootstrap tools
I got annoyed by the number of warnings that the CheriBSD build was
emitting. It turns out that we are emitting lots of warnings during
bootstrap because bootstrap tools are built with the default compiler
flags and ignore the warnings flags that are set in bsd.sys.mk.
Looking at git blame, it appears that MK_WARNS=no has been passed since
rS112869, replacing the -DNO_WERROR option that was added in rS87775.
This commit changes MK_WARNS=no back to -DNO_WERROR. We need to pass
-DNO_WERROR, since the system compiler might have new warnings that we
don't know about yet, and we shouldn't fail the build in that case.
Alex Richardson [Wed, 6 Jan 2021 17:46:36 +0000 (17:46 +0000)]
include: Use printf(1) instead of shell loops for header symlinks
Using a shell for loop means we have to spawn a separate install(1)
process for each header that is symlinked. This patch uses printf(1) to
generate an argument list that has been prefixed with the correct number
of ../ and then uses a single install(1) invocation.
This reduces the number of execve() calls during make includes from 2442
(with D27622) to 1382. Running `make symlinks` in include/ now spawns 214
processes instead of 1276 without this patch.
Alex Richardson [Wed, 6 Jan 2021 17:08:52 +0000 (17:08 +0000)]
Add mkimg to the list of bootstrap tools
Having this as part of the bootstrap tools is useful to build disk images
to boot in QEMU (especially when building on a Linux/macOS host where
mkimg is not available). We have been bootstrapping mkimg in CheriBSD for
a long time (using LOCAL_XTOOL_DIRS) but I believe this is also useful
upstream.
Alex Richardson [Wed, 6 Jan 2021 15:39:10 +0000 (15:39 +0000)]
Install dtrace.h and dependencies
This makes the minimum amount of changes to allow inclusion of dtrace.h
without all the solaris compatibility headers. Installing dtrace.h allows
compiling consumers of libdtrace (e.g. https://github.com/tmetsch/python-dtrace)
without requiring a copy of the source tree.
For python-dtrace I worked around this in https://github.com/tmetsch/python-dtrace/commit/58019c9a12022203a9ffda286dd8b41f1a5ace42
but being able to build the library without installed sources would be
extremely useful.
Support for NS_MOREFRAG is broken, as NS_MOREFRAG is copied from
the TX slot to the RX slot rather than the other way around.
Also, the NS_MOREFRAG must be copied also in case of packet
copy (no zerocopy).
Reported by: rajesh1.kumar_amd.com
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D27980
Kristof Provost [Wed, 6 Jan 2021 21:07:25 +0000 (22:07 +0100)]
pfctl: Fix NOCLEAN build
We've created a new pf_ruleset.c file for pfctl and no longer use the
kernel vrsion, but the build system doesn't handle this dependency
change correctly. Delete the dependency file if it contains the kernel
version of the file.
Toomas Soome [Wed, 6 Jan 2021 11:46:34 +0000 (13:46 +0200)]
loader: instead of hw.vga.textmode, use screen.textmode
hw.vga.textmode is directing VT VGA backend to use text mode.
The default screen mode for BIOS loader is text, and default
screen mode for VT VGA backend is graphics (unless we are running on
hypervisor or hw.vga.textmode is set to 1). Using hw.vga.textmode
for loader does remove possibility to have graphical mode VT VGA with
text mode loader.
screen.textmode can have possible values "0" to disable text mode,
and "1" to set text mode.
Toomas Soome [Tue, 5 Jan 2021 22:09:30 +0000 (00:09 +0200)]
loader: rewrite vidc_install_font
Instead of trying to set reasonable register values, save significant
register values, then prepare for font upload and then restore
registers from saved data.
This seems to fix text mode for most cases where text mode breakage
was reported.
Mateusz Guzik [Wed, 6 Jan 2021 06:11:15 +0000 (07:11 +0100)]
seqc: add seqc_read_notmodify
The routine can be used when the caller does not expect to ever have to
wait for anything. Checking later with seqc_consistent retains all the
guarantees.
Kristof Provost [Thu, 24 Dec 2020 15:02:04 +0000 (16:02 +0100)]
pfctl: Stop sharing pf_ruleset.c with the kernel
Now that we've split up the datastructures used by the kernel and
userspace there's essentually no more overlap between the pf_ruleset.c
code used by userspace and kernelspace.
Copy the userspace bits to the pfctl directory and stop using the kernel
file.
Reviewed by: philip
MFC after: 2 weeks
Sponsored by: Orange Business Services
Differential Revision: https://reviews.freebsd.org/D27764
Kristof Provost [Sun, 13 Dec 2020 16:20:02 +0000 (17:20 +0100)]
pf: Convert pfi_kkif to use counter_u64
Improve caching behaviour by using counter_u64 rather than variables
shared between cores.
The result of converting all counters to counter(9) (i.e. this full
patch series) is a significant improvement in throughput. As tested by
olivier@, on Intel Xeon E5-2697Av4 (16Cores, 32 threads) hardware with
Mellanox ConnectX-4 MCX416A-CCAT (100GBase-SR4) nics we see:
x FreeBSD 20201223: inet packets-per-second
+ FreeBSD 20201223 with pf patches: inet packets-per-second
+--------------------------------------------------------------------------+
| + |
| xx + |
|xxx +++|
||A| |
| |A||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 5 9216962952635693439029371057.6 116720.36
+ 5 19427190196984001950292219546509 109084.92
Difference at 95.0% confidence
1.01755e+07 +/- 164756
108.584% +/- 2.9359%
(Student's t, pooled s = 112967)
Reviewed by: philip
MFC after: 2 weeks
Sponsored by: Orange Business Services
Differential Revision: https://reviews.freebsd.org/D27763
Kristof Provost [Sun, 13 Dec 2020 10:36:54 +0000 (11:36 +0100)]
pf: Allocate and free pfi_kkif in separate functions
Factor out allocating and freeing pfi_kkif structures. This will be
useful when we change the counters to be counter_u64, so we don't have
to deal with that complexity in the multiple locations where we allocate
pfi_kkif structures.
Kristof Provost [Sat, 5 Dec 2020 13:38:12 +0000 (14:38 +0100)]
pf: Remove unused fields from pf_krule
The u_* counters are used only to communicate with userspace, as
userspace cannot use counter_u64. As pf_krule is not passed to userspace
these fields are now obsolete.
Emmanuel Vadot [Tue, 5 Jan 2021 19:50:43 +0000 (20:50 +0100)]
arm64: genet: Add bcm2711-genet-v5 compatible
Later dtb from the RPI Foundation removed the brcm,genet-v5 compatible
for some unknown reason, add brcm,bcm2711-genet-v5 to the list of compatible
device for if_genet
- Split synopsis into two parts. The first explains how to record
sessions, while the second one explains how to replay (some of)
the recorded sessions.
- Fix the -width argument of the environment variables list.
ncurses: only keep the version with widechar support
Only keep the widechar version of ncurses as libncursesw.so.9
Keep the old name to avoid breaking the ABI compatibility (the non
widechar version libncurses.so.9 is not binary compatible with
libncursesw.so.9) since all ports and base are already only linking
against the widechar version we can simply remove libncurses.so.9
Since the .9 version only lived in the dev branch and never ended in a
release, it is simply removed and not added to any binary compat
package.
Add symlinks to keep build time compatibility for anyone linking against
-lncurses
In libcasper, the first argument to the function is a structure that
represents a connection to Casper. On systems without Casper, macros
are used to interpose the Casper functions to standard libc ones.
This may cause errors/warnings that the variable is not used.
With the inline function, there is no such problem.
Emmanuel Vadot [Mon, 14 Dec 2020 17:56:56 +0000 (18:56 +0100)]
include: Use INCSGROUPS for a few files
Instead of using install directly use INCSGROUPS for them.
All those files are the ones we installed when the directory have more .h
files that we don't want to install so they aren't using the LSUBDIR stuff.
Emmanuel Vadot [Mon, 14 Dec 2020 17:53:34 +0000 (18:53 +0100)]
bsd.incs.mk: Add support to override tags for a specific include group
With this patch if a Makefile is using the INCSGROUPS mechanisms it can
override the default package for specific includes files using
GROUPPACKAGE= mynewpackage
While here add a few comments after endif/endfor so it's easier to read.
Mark Johnston [Mon, 4 Jan 2021 13:22:21 +0000 (08:22 -0500)]
mvneta: Fix 64-bit MIB reads
It appears we must read MIB values as 2 4-byte words, lower address
first. A single 8-byte MIB read returns the value with the lower 4
bytes copied into the upper 4 bytes, resulting in bogus byte counter
values.
Mark Johnston [Mon, 4 Jan 2021 13:21:57 +0000 (08:21 -0500)]
md: Fix a race in mdstart_swap()
Release a grabbed page's busy state only after marking it as referenced.
Otherwise there exists a narrow window where the page could be freed
before the update. Before r356902 this was not a problem since the
object lock was held.
Discussed with: kib
Sponsored by: The FreeBSD Foundation
Jilles Tjoelker [Sun, 3 Jan 2021 20:27:50 +0000 (21:27 +0100)]
sh/tests: Add a second kind of binary scripts without #!
One of the reasons for git commit e0f5c1387df23c8c4811f5b24a7ef6ecac51a71a was to make "actually portable
executables" work. Add a test that is more like those.
Mark Johnston [Sun, 3 Jan 2021 16:35:42 +0000 (11:35 -0500)]
sendfile: Clear page pointers when handling a pager error
When INVARIANTS is configred, the sendfile_iodone() callback verifies
that pages attached to the sendfile header are wired, but we unwire all
such pages after a synchronous pager error, before calling
sendfile_iodone().
Reported by: pho
Tested by: pho
Sponsored by: The FreeBSD Foundation
Mark Johnston [Sun, 3 Jan 2021 16:32:30 +0000 (11:32 -0500)]
Ensure that dirent's d_off field is initialized
We have the d_off field in struct dirent for providing the seek offset
of the next directory entry. Several filesystems were not initializing
the field, which ends up being copied out to userland.
Reported by: Syed Faraz Abrar <faraz@elttam.com>
Reviewed by: kib
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27792
Mark Johnston [Sun, 3 Jan 2021 16:31:00 +0000 (11:31 -0500)]
uma: Avoid unmapping direct-mapped slabs
startup_alloc() uses pmap_map() to map slabs used for bootstrapping the
VM. pmap_map() may ignore the hint address and simply return a range
from the direct map. In this case we must not unmap the range in
startup_free().
UMA uses bootstart and bootmem to track the range of KVA into which
slabs are mapped if the direct map is not used. Unmap a startup slab
only if it was mapped into that range.
Reported by: alc
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27885