Conrad Meyer [Fri, 30 Oct 2020 19:02:59 +0000 (19:02 +0000)]
linux(4): Quiesce warning about madvise(..., -1)
This API misuse is intended to produce an error value to detect certain
bogus stub implementations of MADV_WIPEONFORK. We don't need to log a
warning about it.
Conrad Meyer [Fri, 30 Oct 2020 19:00:42 +0000 (19:00 +0000)]
UFS2: Fix DoS due to corrupted extattrfile
Prior versions of FreeBSD (11.x) may have produced a corrupt extattr file.
(Specifically, r312416 accidentally fixed this defect by removing a strcpy.)
CURRENT FreeBSD supports disk images from those prior versions of FreeBSD.
Validate the internal structure as soon as we read it in from disk, to
prevent these extattr files from causing invariants violations and DoS.
Attempting to access the extattr portion of these files results in
EINTEGRITY. At this time, the only way to repair files damaged in this way
is to copy the contents to another file and move it over the original.
PR: 244089
Reported by: Andrea Venturoli <ml AT netfence.it>
Reviewed by: kib
Discussed with: mckusick (earlier draft)
Security: no
Differential Revision: https://reviews.freebsd.org/D27010
Stefan Eßer [Fri, 30 Oct 2020 18:48:09 +0000 (18:48 +0000)]
Add read only sysctl variable user.localbase
The value is provided by the C library as for other sysctl variables in
the user tree. It is compiled in and returns the value of _PATH_LOCALBASE
defined in paths.h.
Mitchell Horne [Fri, 30 Oct 2020 18:22:46 +0000 (18:22 +0000)]
arm64: set the correct partition type in make-memstick.sh
We create a UFS root filesystem using makefs(8), and later pass it to
mkimg(1) when creating the final image. The correct partition type is
freebsd-ufs; the freebsd parition type is for partitions containing a
BSD disklabel.
Reviewed by: emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26987
Mitchell Horne [Fri, 30 Oct 2020 18:20:52 +0000 (18:20 +0000)]
arm64: convert virtual machine images to GPT
These images were switched to MBR in r281876 as a way to cope with a
hard-coded partition GUID in QEMU's default EFI firmware. Enough time
has passed that this is no longer a problem; QEMU versions >= 4.0
include a copy of edk2 EFI firmware that can detect the root filesystem
properly. Alternatively, sysutils/u-boot-qemu-arm64 can be used.
Switch back to building these images with a GPT partition table, and
re-enable the swap partition.
Reviewed by: gjb, emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26986
Mitchell Horne [Fri, 30 Oct 2020 18:18:25 +0000 (18:18 +0000)]
vmimage.subr: noisier failure for unsupported targets
The return code of write_partition_layout() doesn't bubble up, so an
invocation of make vm-release for an incorrect/unsupported target will
appear to succeed while make vm-install will fail due to missing
files. This isn't a common point of failure, but is worth handling
properly.
Upgrade this case to print a message to stderr, and exit in place. This
is okay to do since at this point in the execution of mk-vmimage.sh,
cleanup() has already been run.
Stefan Eßer [Fri, 30 Oct 2020 10:44:46 +0000 (10:44 +0000)]
Re-implement comment parsing missing in the internal pre-processor
The internal pre-processor ignored lines that did not parse a calendar
entries, but did not support multi-line comments in the way the external
cpp did.
The calendar files distributed with the base system (now in a port) do
use comments, though.
Implement comment processing for single-line (//) and multi-line comments
(/* */) with same semantics as in a standard C pre-processor.
All tests pass with this version, but there are no tests that specifically
verify comment processing.
Reported by: jhs@berklix.com (Julian H. Stacey)
MFC after: 3 days
beinstall.sh: Use bectl instead of beadm by default
This patch also introduces an environment variable BE_UTILITY,
which can be used to specify the utility to use for managing
ZFS boot environments (which can be either bectl or beadm).
While here, fix some typos in the manual page and
remove beadm from section "SEE ALSO".
Olivier Cochard [Fri, 30 Oct 2020 00:03:59 +0000 (00:03 +0000)]
bhyve currently reports each of "smbios.system.maker" and
"smbios.system.family" as " ".
This presents challenges for both humans and tools when trying to parse output
that uses those results.
The new values reported are now:
smbios.system.family="Virtual Machine"
smbios.system.maker="FreeBSD"
John Baldwin [Thu, 29 Oct 2020 22:22:27 +0000 (22:22 +0000)]
Use public interfaces to manage the nested rate limit send tag.
Each TLS send tag in mlx5 contains a nested rate limit send tag.
Previously, the driver was calling internal functions to manage the
nested tag. Calling free methods directly instead of m_snd_tag_rele()
leaked send tag references and references on the ifp. Changes to use
the ifp methods for the nested tag for other methods are more cosmetic
but do simplify the code.
John Baldwin [Thu, 29 Oct 2020 22:16:59 +0000 (22:16 +0000)]
Remove an extra if_ref().
In r348254, if_snd_tag_alloc() routines were changed to bump the ifp
refcount via m_snd_tag_init(). This function wasn't in the tree at
the time and wasn't updated for the new semantics, so was still doing
a separate bump after if_snd_tag_alloc() returned.
Add defines for Linux errno values and use them to make linux_errtbl[]
more readable. While here, add linux_check_errtbl() function to make
sure we don't leave holes.
No objections: emaste (earlier version)
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26972
Stefan Eßer [Thu, 29 Oct 2020 08:26:38 +0000 (08:26 +0000)]
Fix calendar -a processing of files included in the user's home directory
The existing code performed a chdir() into the home directory, but the
parser fell back to using the invoking user's home directory as the base
directory for the search for an include file.
Since use of the -a option is limited to UID==0, the directory searched
was typically ~root/.calendar, not the .calendar directory of the user
whose file is being processed.
Ed Maste [Thu, 29 Oct 2020 02:02:30 +0000 (02:02 +0000)]
CI: switch to qemu42 package
It appears that booting FreeBSD from qemu's synthesized FAT filesystem
broke somehow in a recent qemu-devel update. qemu42 works so switch to
it for now.
John Baldwin [Thu, 29 Oct 2020 00:23:16 +0000 (00:23 +0000)]
Support hardware rate limiting (pacing) with TLS offload.
- Add a new send tag type for a send tag that supports both rate
limiting (packet pacing) and TLS offload (mostly similar to D22669
but adds a separate structure when allocating the new tag type).
- When allocating a send tag for TLS offload, check to see if the
connection already has a pacing rate. If so, allocate a tag that
supports both rate limiting and TLS offload rather than a plain TLS
offload tag.
- When setting an initial rate on an existing ifnet KTLS connection,
set the rate in the TCP control block inp and then reset the TLS
send tag (via ktls_output_eagain) to reallocate a TLS + ratelimit
send tag. This allocates the TLS send tag asynchronously from a
task queue, so the TLS rate limit tag alloc is always sleepable.
- When modifying a rate on a connection using KTLS, look for a TLS
send tag. If the send tag is only a plain TLS send tag, assume we
failed to allocate a TLS ratelimit tag (either during the
TCP_TXTLS_ENABLE socket option, or during the send tag reset
triggered by ktls_output_eagain) and ignore the new rate. If the
send tag is a ratelimit TLS send tag, change the rate on the TLS tag
and leave the inp tag alone.
- Lock the inp lock when setting sb_tls_info for a socket send buffer
so that the routines in tcp_ratelimit can safely dereference the
pointer without needing to grab the socket buffer lock.
- Add an IFCAP_TXTLS_RTLMT capability flag and associated
administrative controls in ifconfig(8). TLS rate limit tags are
only allocated if this capability is enabled. Note that TLS offload
(whether unlimited or rate limited) always requires IFCAP_TXTLS[46].
Brooks Davis [Wed, 28 Oct 2020 23:10:54 +0000 (23:10 +0000)]
Disable ssp raw test without ASAN
r366981 disabled ASAN when it might not be reliable (with an external
compiler), but this test is broken without ASAN so disable it completely
in that case.
Check for process group change in tty_wait_background().
The calling process's process group can change between PROC_UNLOCK(p)
and PGRP_LOCK(pg) in tty_wait_background(), e.g. by a setpgid() call
from another process. If that happens, the signal is not sent to the
calling process, even if the prior checks determine that one should be
sent. Re-check that the process group hasn't changed after acquiring
the pgrp lock, and if it has, redo the checks.
Warner Losh [Wed, 28 Oct 2020 21:18:04 +0000 (21:18 +0000)]
loader: add aarch64 support for zstd
We don't have NEON available in the boot loader, so we have to disable
it. OpenZFS included ZSTD which used the wrong symbol to bring in neon
support. Change to use the code that's been submitted upstream as a
pull request to both.
__ARM_NEON is the proper symbol, defined in ARM C Language Extensions
Release 2.1 (https://developer.arm.com/documentation/ihi0053/d/). Some
sources suggest __ARM_NEON__, but that's the obsolete spelling from
prior versions of the standard.
Warner Losh [Wed, 28 Oct 2020 21:09:56 +0000 (21:09 +0000)]
Note that sys/systm.h is special too
If you need / want to includerd sys/systm.h, it has to be just after
param.h/types.h. Document this existing practice. Not all kernel files
include systm.h, but when you do, it should be done out of order.
Code was supposed to call callout_reset_sbt_on() rather than
callout_reset_sbt(). This resulted into passing a "cpu" value
to a "flag" argument. A recipe for subtle errors.
Alexander Motin [Wed, 28 Oct 2020 17:28:48 +0000 (17:28 +0000)]
Use proper variable for device path.
It seems *-passthru commands were broken from the day one, since the
device path is fetched into opt.dev variable and not left in argv[optind].
The other three wrong argv[optind] instances are just in error messages.
Stefan Eßer [Wed, 28 Oct 2020 14:48:58 +0000 (14:48 +0000)]
Add support for nested conditionals
The previous behavior was to support nested #ifdef and #ifndef, but to
return to unconditional parsing after the next #endif, independently of
the number of previously parsed conditions.
E.g. after "#ifdef A / #ifdef B / #endif" the following lines were
unconditially parsed again, independently of A and/or B being defined.
The new behavior is to count the level of false conditions and to only
restart parsing of calendar entries when the corresponding number of
#endif tokens have been seen.
In addition to the above, an #else directive has been added, to toggle
between parsing and ignoring of the following lines.
No validation of the correct use of the condition directives is made.
#endif without prior #define or #ifndef is ignored and #else toggles
between parsing and skipping of entries.
The MFC period has been set to 1 month to allow for a review of the
changes and for a discussion, whether these modifications should not
be merged at all.
No correct input file is parsed differently than before, but if calendar
data files are published that use these new features, those data files
will not parse correctly on prior versions of this program.
Drop "All rights reserved" from all my stuff. This includes
Foundation copyrights, approved by emaste@. It does not include
files which carry other people's copyrights; if you're one
of those people, feel free to make similar change.
Stefan Eßer [Wed, 28 Oct 2020 13:30:24 +0000 (13:30 +0000)]
Fix off-by-one error in processing of #ifdef lines
The convention in this program is to parse the line immediately starting
after the token (e.g. #defineA and #ifdefA define respectively look-up "A"),
and this commit restores this behavior instead of skipping an assumed
white-space character following #ifdef.
Stefan Eßer [Wed, 28 Oct 2020 13:06:39 +0000 (13:06 +0000)]
Fix parsing of #ifdef in calendar files
There was code to process an #ifndef tokens, but none for #ifdef.
The #ifdef token was mentioned as unsupported in the BUGS section,
but no reason was given and I do not see why it should stay omitted.
Misleading information in The BUGS section of the man-page regarding
the maximum number of #define and #include statements supported has
been removed. These limits might have applied to a prior version of
this program, but do not seem to apply to the current implementation.
I have not tried to test for the existence of the limits, but the
include file processing just recursively calls the parser (without
counting the recursion depth) and the stringlist functions do not
impose a limit on the number of entries.
Alex Richardson [Wed, 28 Oct 2020 11:54:09 +0000 (11:54 +0000)]
Speed up Cirrus CI by using CROSS_TOOLCHAIN
Installing the llvm11 package instead of bootstrapping it from the source
tree reduces the build time by about 20 minutes.
The last freebsd/freebsd build that was tested (r366629) took 1h 21m 22s,
whereas my GitHub fork with this .cirrus.yml took 58m 6s.
We could probably further reduce time by using images that have LLVM
pre-installed: the pkg install step took 4 minutes 30s.
Since the bootstrap toolchain is still tested by Jenkins, this should not
reduce test coverage of the CI testing.
Alex Richardson [Wed, 28 Oct 2020 11:54:04 +0000 (11:54 +0000)]
Significantly reduce compile time for googletest internal tests
Clang's optimizer spends a really long time on these tests at -O2, so we now
use -O0 instead. This reduces the -j32 time for lib/googletest/test from 131s
to 29s. Using -O0 also reduces the disk usage from 144MB (at -O2) / 92MB (at
-O1) to 82MB.
Reviewed By: ngie, dim
Differential Revision: https://reviews.freebsd.org/D26751
The way netmap TX is handled in iflib when TX interrupts are not
used (IFC_NETMAP_TX_IRQ not set) has some issues:
- The netmap_tx_irq() function gets called by iflib_timer(), which
gets scheduled with tick granularity (hz). This is not frequent
enough for 10Gbps NICs and beyond (e.g., ixgbe or ixl). The end
result is that the transmitting netmap application is not woken
up fast enough to saturate the link with small packets.
- The iflib_timer() functions also calls isc_txd_credits_update()
to ask for more TX completion updates. However, this violates
the netmap requirement that only txsync can access the TX queue
for datapath operations. Only netmap_tx_irq() may be called out
of the txsync context.
This change introduces per-tx-queue netmap timers, using microsecond
granularity to ensure that netmap_tx_irq() can be called often enough
to allow for maximum packet rate. The timer routine simply calls
netmap_tx_irq() to wake up the netmap application. The latter will
wake up and call txsync to collect TX completion updates.
This change brings back line rate speed with small packets for ixgbe.
For the time being, timer expiration is hardcoded to 90 microseconds,
in order to avoid introducing a new sysctl.
We may eventually implement an adaptive expiration period or use another
deferred work mechanism in place of timers.
Also, fix the timers usage to make sure that each queue is serviced
by a different CPU.
Mateusz Guzik [Tue, 27 Oct 2020 18:12:07 +0000 (18:12 +0000)]
vfs: fix vnode reclaim races against getnwevnode
All vnodes allocated by UMA are present on the global list used by
vnlru. getnewvnode modifies the state of the vnode (most notably
altering v_holdcnt) but never locks it. Moreover filesystems also
modify it in arbitrary manners sometimes before taking the vnode
lock or adding any other indicator that the vnode can be used.
Picking up such a vnode by vnlru would be problematic.
To that end there are 2 fixes:
- vlrureclaim, not recycling v_holdcnt == 0 vnodes, takes the
interlock and verifies that v_mount has been set. It is an
invariant that the vnode lock is held by that point, providing
the necessary serialisation against locking after vhold.
- vnlru_free_locked, only wanting to free v_holdcnt == 0 vnodes,
now makes sure to only transition the count 0->1 and newly allocated
vnodes start with v_holdcnt == VHOLD_NO_SMR. getnewvnode will only
transition VHOLD_NO_SMR->1 once more making the hold fail
Ruslan Bukin [Tue, 27 Oct 2020 15:29:53 +0000 (15:29 +0000)]
o Add the domain member to the struct bus_dma_tag_common as required by
busdma_iommu.c.
o Add tag_set_domain() pointer to the struct bus_dma_impl as well.
Mark Johnston [Tue, 27 Oct 2020 13:27:47 +0000 (13:27 +0000)]
arm64: Remove a racy KASSERT from pmap_remove_pages()
PCPU_GET(curpmap) expands to multiple instructions on arm64, and if the
current thread is migrated in between execution of those instructions, a
stale value may be used in the assertion condition.
There was a question raised in freebsd-questions@ mail list[1] about the
"who" parameters in this man page. It seems OpenBSD[2] amd NetBSD[3]
both have more legible descriptions so I borrowed some of their ideas to try
and make this page clearer.
Stefan Eßer [Tue, 27 Oct 2020 11:29:11 +0000 (11:29 +0000)]
Replace literal uses of /usr/local in C sources with _PATH_LOCALBASE
Literal references to /usr/local exist in a large number of files in
the FreeBSD base system. Many are in contributed software, in configuration
files, or in the documentation, but 19 uses have been identified in C
source files or headers outside the contrib and sys/contrib directories.
This commit makes it possible to set _PATH_LOCALBASE in paths.h to use
a different prefix for locally installed software.
In order to avoid changes to openssh source files, LOCALBASE is passed to
the build via Makefiles under src/secure. While _PATH_LOCALBASE could have
been used here, there is precedent in the construction of the path used to
a xauth program which depends on the LOCALBASE value passed on the compiler
command line to select a non-default directory.
This could be changed in a later commit to make the openssh build
consistently use _PATH_LOCALBASE. It is considered out-of-scope for this
commit.
- Sort flags
- Stylize incr|+ and decr|- properly
- Add a missing period at the end of the description
- Use the standard layout for the EXAMPLES section (remove the list macro
and add indentation to the code block)
It turns out that without /dev/null beinstall is not able to complete and
instead exits with messages similar to these:
--------------------------------------------------------------
>>> Installing kernel GENERIC completed on Sun Oct 25 17:47:37 CET 2020
--------------------------------------------------------------
/tmp/beinstall.JleGoP/mnt: Inspecting dirs /usr/src /usr/obj/usr/src/amd64.amd64
--- installworld ---
make[1]: "/usr/obj/usr/src/amd64.amd64/toolchain-metadata.mk" line 1: Using cached toolchain metadata from build at t480 on Sun Oct 25 15:53:28 CET 2020
make[2]: "/dev/null" line 2: Need an operator
make[2]: Fatal errors encountered -- cannot continuemake[1]: "/usr/src/Makefile.inc1" line 593: CPUTYPE global should be set with ?=.
Cleaning up ...
umount -f /tmp/beinstall.JleGoP/mnt/usr/src /tmp/beinstall.JleGoP/mnt/usr/obj/usr/src/amd64.amd64 /tmp/beinstall.JleGoP/mnt
Destroyed successfully
error: Installworld failed!
Upon a bit of debugging, it turns out that /dev/null inside the chroot
environment is full random bytes, which cause "make -f /dev/null" to
misbehave. Mounting a proper devfs inside the chroot seems to be the most
appropriate way to fix it.
will@ also noted that this change requires that whatever is needed in devfs
must exist in the old kernel.
Approved by: will
MFC after: 2 week
Differential Revision: https://reviews.freebsd.org/D26944
Warner Losh [Tue, 27 Oct 2020 06:43:24 +0000 (06:43 +0000)]
Remove frontstuff
Nothing implements this in the tree. Remove the ioctl and the
conversion to the geom atttribute stuff.
This was introduced in r94287 in 2002 and was retired in r113390
2003. It appeared in FreeBSD 5.0, but no other releases. This is a
vestige that was missed at the time and overlooked until now. No
compat is provided for this reason. And there's no implementation of
it today. And it was never part of a release from a stable branch.
Mitchell Horne [Mon, 26 Oct 2020 19:13:22 +0000 (19:13 +0000)]
riscv: make use of SBI legacy replacement extensions
Version 0.2 of the SBI specification [1] marked the existing SBI
functions as "legacy" in order to move to a newer calling convention. It
also introduced a set of replacement extensions for some of the legacy
functionality. In particular, the TIME, IPI, and RFENCE extensions
implement and extend the semantics of their legacy counterparts, while
conforming to the newer version of the spec.
Update our SBI code to use the new replacement extensions when
available, and fall back to the legacy ones. These will eventually be
dropped, when support for version 0.2 is ubiquitous.
Mitchell Horne [Mon, 26 Oct 2020 19:06:30 +0000 (19:06 +0000)]
riscv: remove sbi_clear_ipi()
S-mode software has write access to the SIP.SSIP bit, so instead of
making a second round-trip through the SBI we can clear it ourselves.
The SBI spec has deprecated this function for this exactly this reason.
Brooks Davis [Mon, 26 Oct 2020 17:52:28 +0000 (17:52 +0000)]
Key decleration of union semun on src version
__FreeBSD__ is defined by the compiler derived from the triple. When
building FreeBSD 11 on a FreeBSD 12 with a CROSS_TOOLCHAIN=llvm10,
__FreeBSD__ was set to 12 when building lib32 (for some reason no triple
is being passed which seems to mean that we're taking default values
from the build system). This in turn meant we end up with a double
decleration of union semun which is a build error.
Reviewed by: gshapiro, dim
Differential Revision: https://reviews.freebsd.org/D26902
Eric van Gyzen [Mon, 26 Oct 2020 16:42:53 +0000 (16:42 +0000)]
db_search_symbol: prevent pollution from bogus symbols
The kernel will never map the first page, so any symbols in that
range cannot refer to addresses. Some third-party assembly files
define internal constants which appear in their symbol table.
Avoiding the lookup for those symbols avoids replacing small offsets
with those symbols during disassembly.
Reported by: Anton Rang <rang%acm.org>
Reviewed by: Anton Rang <rang%acm.org>, markj
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D26895
projects / FreeBSD / FreeBSD.git / log