Stefan Eßer [Wed, 28 Oct 2020 14:48:58 +0000 (14:48 +0000)]
Add support for nested conditionals
The previous behavior was to support nested #ifdef and #ifndef, but to
return to unconditional parsing after the next #endif, independently of
the number of previously parsed conditions.
E.g. after "#ifdef A / #ifdef B / #endif" the following lines were
unconditially parsed again, independently of A and/or B being defined.
The new behavior is to count the level of false conditions and to only
restart parsing of calendar entries when the corresponding number of
#endif tokens have been seen.
In addition to the above, an #else directive has been added, to toggle
between parsing and ignoring of the following lines.
No validation of the correct use of the condition directives is made.
#endif without prior #define or #ifndef is ignored and #else toggles
between parsing and skipping of entries.
The MFC period has been set to 1 month to allow for a review of the
changes and for a discussion, whether these modifications should not
be merged at all.
No correct input file is parsed differently than before, but if calendar
data files are published that use these new features, those data files
will not parse correctly on prior versions of this program.
Drop "All rights reserved" from all my stuff. This includes
Foundation copyrights, approved by emaste@. It does not include
files which carry other people's copyrights; if you're one
of those people, feel free to make similar change.
Stefan Eßer [Wed, 28 Oct 2020 13:30:24 +0000 (13:30 +0000)]
Fix off-by-one error in processing of #ifdef lines
The convention in this program is to parse the line immediately starting
after the token (e.g. #defineA and #ifdefA define respectively look-up "A"),
and this commit restores this behavior instead of skipping an assumed
white-space character following #ifdef.
Stefan Eßer [Wed, 28 Oct 2020 13:06:39 +0000 (13:06 +0000)]
Fix parsing of #ifdef in calendar files
There was code to process an #ifndef tokens, but none for #ifdef.
The #ifdef token was mentioned as unsupported in the BUGS section,
but no reason was given and I do not see why it should stay omitted.
Misleading information in The BUGS section of the man-page regarding
the maximum number of #define and #include statements supported has
been removed. These limits might have applied to a prior version of
this program, but do not seem to apply to the current implementation.
I have not tried to test for the existence of the limits, but the
include file processing just recursively calls the parser (without
counting the recursion depth) and the stringlist functions do not
impose a limit on the number of entries.
Alex Richardson [Wed, 28 Oct 2020 11:54:09 +0000 (11:54 +0000)]
Speed up Cirrus CI by using CROSS_TOOLCHAIN
Installing the llvm11 package instead of bootstrapping it from the source
tree reduces the build time by about 20 minutes.
The last freebsd/freebsd build that was tested (r366629) took 1h 21m 22s,
whereas my GitHub fork with this .cirrus.yml took 58m 6s.
We could probably further reduce time by using images that have LLVM
pre-installed: the pkg install step took 4 minutes 30s.
Since the bootstrap toolchain is still tested by Jenkins, this should not
reduce test coverage of the CI testing.
Alex Richardson [Wed, 28 Oct 2020 11:54:04 +0000 (11:54 +0000)]
Significantly reduce compile time for googletest internal tests
Clang's optimizer spends a really long time on these tests at -O2, so we now
use -O0 instead. This reduces the -j32 time for lib/googletest/test from 131s
to 29s. Using -O0 also reduces the disk usage from 144MB (at -O2) / 92MB (at
-O1) to 82MB.
Reviewed By: ngie, dim
Differential Revision: https://reviews.freebsd.org/D26751
The way netmap TX is handled in iflib when TX interrupts are not
used (IFC_NETMAP_TX_IRQ not set) has some issues:
- The netmap_tx_irq() function gets called by iflib_timer(), which
gets scheduled with tick granularity (hz). This is not frequent
enough for 10Gbps NICs and beyond (e.g., ixgbe or ixl). The end
result is that the transmitting netmap application is not woken
up fast enough to saturate the link with small packets.
- The iflib_timer() functions also calls isc_txd_credits_update()
to ask for more TX completion updates. However, this violates
the netmap requirement that only txsync can access the TX queue
for datapath operations. Only netmap_tx_irq() may be called out
of the txsync context.
This change introduces per-tx-queue netmap timers, using microsecond
granularity to ensure that netmap_tx_irq() can be called often enough
to allow for maximum packet rate. The timer routine simply calls
netmap_tx_irq() to wake up the netmap application. The latter will
wake up and call txsync to collect TX completion updates.
This change brings back line rate speed with small packets for ixgbe.
For the time being, timer expiration is hardcoded to 90 microseconds,
in order to avoid introducing a new sysctl.
We may eventually implement an adaptive expiration period or use another
deferred work mechanism in place of timers.
Also, fix the timers usage to make sure that each queue is serviced
by a different CPU.
Mateusz Guzik [Tue, 27 Oct 2020 18:12:07 +0000 (18:12 +0000)]
vfs: fix vnode reclaim races against getnwevnode
All vnodes allocated by UMA are present on the global list used by
vnlru. getnewvnode modifies the state of the vnode (most notably
altering v_holdcnt) but never locks it. Moreover filesystems also
modify it in arbitrary manners sometimes before taking the vnode
lock or adding any other indicator that the vnode can be used.
Picking up such a vnode by vnlru would be problematic.
To that end there are 2 fixes:
- vlrureclaim, not recycling v_holdcnt == 0 vnodes, takes the
interlock and verifies that v_mount has been set. It is an
invariant that the vnode lock is held by that point, providing
the necessary serialisation against locking after vhold.
- vnlru_free_locked, only wanting to free v_holdcnt == 0 vnodes,
now makes sure to only transition the count 0->1 and newly allocated
vnodes start with v_holdcnt == VHOLD_NO_SMR. getnewvnode will only
transition VHOLD_NO_SMR->1 once more making the hold fail
Ruslan Bukin [Tue, 27 Oct 2020 15:29:53 +0000 (15:29 +0000)]
o Add the domain member to the struct bus_dma_tag_common as required by
busdma_iommu.c.
o Add tag_set_domain() pointer to the struct bus_dma_impl as well.
Mark Johnston [Tue, 27 Oct 2020 13:27:47 +0000 (13:27 +0000)]
arm64: Remove a racy KASSERT from pmap_remove_pages()
PCPU_GET(curpmap) expands to multiple instructions on arm64, and if the
current thread is migrated in between execution of those instructions, a
stale value may be used in the assertion condition.
There was a question raised in freebsd-questions@ mail list[1] about the
"who" parameters in this man page. It seems OpenBSD[2] amd NetBSD[3]
both have more legible descriptions so I borrowed some of their ideas to try
and make this page clearer.
Stefan Eßer [Tue, 27 Oct 2020 11:29:11 +0000 (11:29 +0000)]
Replace literal uses of /usr/local in C sources with _PATH_LOCALBASE
Literal references to /usr/local exist in a large number of files in
the FreeBSD base system. Many are in contributed software, in configuration
files, or in the documentation, but 19 uses have been identified in C
source files or headers outside the contrib and sys/contrib directories.
This commit makes it possible to set _PATH_LOCALBASE in paths.h to use
a different prefix for locally installed software.
In order to avoid changes to openssh source files, LOCALBASE is passed to
the build via Makefiles under src/secure. While _PATH_LOCALBASE could have
been used here, there is precedent in the construction of the path used to
a xauth program which depends on the LOCALBASE value passed on the compiler
command line to select a non-default directory.
This could be changed in a later commit to make the openssh build
consistently use _PATH_LOCALBASE. It is considered out-of-scope for this
commit.
- Sort flags
- Stylize incr|+ and decr|- properly
- Add a missing period at the end of the description
- Use the standard layout for the EXAMPLES section (remove the list macro
and add indentation to the code block)
It turns out that without /dev/null beinstall is not able to complete and
instead exits with messages similar to these:
--------------------------------------------------------------
>>> Installing kernel GENERIC completed on Sun Oct 25 17:47:37 CET 2020
--------------------------------------------------------------
/tmp/beinstall.JleGoP/mnt: Inspecting dirs /usr/src /usr/obj/usr/src/amd64.amd64
--- installworld ---
make[1]: "/usr/obj/usr/src/amd64.amd64/toolchain-metadata.mk" line 1: Using cached toolchain metadata from build at t480 on Sun Oct 25 15:53:28 CET 2020
make[2]: "/dev/null" line 2: Need an operator
make[2]: Fatal errors encountered -- cannot continuemake[1]: "/usr/src/Makefile.inc1" line 593: CPUTYPE global should be set with ?=.
Cleaning up ...
umount -f /tmp/beinstall.JleGoP/mnt/usr/src /tmp/beinstall.JleGoP/mnt/usr/obj/usr/src/amd64.amd64 /tmp/beinstall.JleGoP/mnt
Destroyed successfully
error: Installworld failed!
Upon a bit of debugging, it turns out that /dev/null inside the chroot
environment is full random bytes, which cause "make -f /dev/null" to
misbehave. Mounting a proper devfs inside the chroot seems to be the most
appropriate way to fix it.
will@ also noted that this change requires that whatever is needed in devfs
must exist in the old kernel.
Approved by: will
MFC after: 2 week
Differential Revision: https://reviews.freebsd.org/D26944
Warner Losh [Tue, 27 Oct 2020 06:43:24 +0000 (06:43 +0000)]
Remove frontstuff
Nothing implements this in the tree. Remove the ioctl and the
conversion to the geom atttribute stuff.
This was introduced in r94287 in 2002 and was retired in r113390
2003. It appeared in FreeBSD 5.0, but no other releases. This is a
vestige that was missed at the time and overlooked until now. No
compat is provided for this reason. And there's no implementation of
it today. And it was never part of a release from a stable branch.
Mitchell Horne [Mon, 26 Oct 2020 19:13:22 +0000 (19:13 +0000)]
riscv: make use of SBI legacy replacement extensions
Version 0.2 of the SBI specification [1] marked the existing SBI
functions as "legacy" in order to move to a newer calling convention. It
also introduced a set of replacement extensions for some of the legacy
functionality. In particular, the TIME, IPI, and RFENCE extensions
implement and extend the semantics of their legacy counterparts, while
conforming to the newer version of the spec.
Update our SBI code to use the new replacement extensions when
available, and fall back to the legacy ones. These will eventually be
dropped, when support for version 0.2 is ubiquitous.
Mitchell Horne [Mon, 26 Oct 2020 19:06:30 +0000 (19:06 +0000)]
riscv: remove sbi_clear_ipi()
S-mode software has write access to the SIP.SSIP bit, so instead of
making a second round-trip through the SBI we can clear it ourselves.
The SBI spec has deprecated this function for this exactly this reason.
Brooks Davis [Mon, 26 Oct 2020 17:52:28 +0000 (17:52 +0000)]
Key decleration of union semun on src version
__FreeBSD__ is defined by the compiler derived from the triple. When
building FreeBSD 11 on a FreeBSD 12 with a CROSS_TOOLCHAIN=llvm10,
__FreeBSD__ was set to 12 when building lib32 (for some reason no triple
is being passed which seems to mean that we're taking default values
from the build system). This in turn meant we end up with a double
decleration of union semun which is a build error.
Reviewed by: gshapiro, dim
Differential Revision: https://reviews.freebsd.org/D26902
Eric van Gyzen [Mon, 26 Oct 2020 16:42:53 +0000 (16:42 +0000)]
db_search_symbol: prevent pollution from bogus symbols
The kernel will never map the first page, so any symbols in that
range cannot refer to addresses. Some third-party assembly files
define internal constants which appear in their symbol table.
Avoiding the lookup for those symbols avoids replacing small offsets
with those symbols during disassembly.
Reported by: Anton Rang <rang%acm.org>
Reviewed by: Anton Rang <rang%acm.org>, markj
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D26895
Alexander Motin [Mon, 26 Oct 2020 04:04:06 +0000 (04:04 +0000)]
Enable bioq 'car limit' added at r335066 at 128 bios.
Without the 'car limit' enabled (before this), running sequential ZFS scrub
on HDD without command queuing support, I've measured latency on concurrent
random reads reaching 4 seconds (surprised that not more). Enabling this
reduced the latency to 65 milliseconds, while scrub still doing ~180MB/s.
For disks with command queuing this does not make much difference (if any),
since most time all the requests are queued down to the disk or HBA, leaving
nothing in the queue to sort. And even if something does not fit, staying on
the queue, it is likely not for long. To not limit sorting in such bursty
scenarios I've added batched counter zeroing when the queue is getting empty.
The internal scheduler of the SAS HDD I was testing seems to be even more
loyal to random I/O, reducing the scrub speed to ~120MB/s. So in case
somebody worried this is limit is too strict -- it actually looks relaxed.
Warner Losh [Mon, 26 Oct 2020 03:26:18 +0000 (03:26 +0000)]
calendar: remove all datafiles except freebsd one
Move all the data files for the calendar(1) program, except
calendar.freebsd to the calendar-data package. When a file
can't be found, and /usr/local/share/calendar doesn't exist
provide a helpful hint to install this package.
Kirk McKusick [Sun, 25 Oct 2020 21:04:07 +0000 (21:04 +0000)]
Use proper type (ino_t) for inode numbers to avoid improper sign extention
in the Pass 5 checks. The manifestation was fsck_ffs exiting with this error:
** Phase 5 - Check Cyl groups
fsck_ffs: inoinfo: inumber 18446744071562087424 out of range
The error only manifests itself for filesystems bigger than about 100Tb.
Alexander Motin [Sun, 25 Oct 2020 19:34:02 +0000 (19:34 +0000)]
Introduce support of SCSI Command Priority.
SAM-3 specification introduced concept of Task Priority, that was renamed
to Command Priority in SAM-4, and supported by all modern SCSI transports.
It provides 15 levels of relative priorities: 1 - highest, 15 - lowest and
0 - default. SAT specification for SATA devices translates priorities 1-3
into NCQ high priority.
This change adds new "priority" field into empty spots of struct ccb_scsiio
and struct ccb_accept_tio of CAM and struct ctl_scsiio of CTL. Respective
support is added into iscsi(4), isp(4), mpr(4), mps(4) and ocs_fc(4) drivers
for both initiator and where applicable target roles. Minimal support was
added to CTL to receive the priority value from different frontends, pass it
between HA controllers and report in few places.
This patch does not add consumers of this functionality, so nothing should
really change yet, since the field is still set to 0 (default) on initiator
and not actively used on target. Those are to be implemented separately.
I've confirmed priority working on WD Red SATA disks connected via mpr(4)
and properly transferred to CTL target via iscsi(4), isp(4) and ocs_fc(4).
While there, added missing tag_action support to ocs_fc(4) initiator role.
MFC after: 1 month
Relnotes: yes
Sponsored by: iXsystems, Inc.
Alexander Motin [Sun, 25 Oct 2020 16:58:48 +0000 (16:58 +0000)]
Fix incorrect constants of target tag action.
ocs_scsi_recv_cmd() receives the flags after ocs_get_flags_fcp_cmd(),
which translates them from FCP_TASK_ATTR_* to OCS_SCSI_CMD_*. As result
non-SIMPLE requests turned into HEAD or ORDERED depending on direction.
Kirk McKusick [Sun, 25 Oct 2020 01:36:33 +0000 (01:36 +0000)]
Filesystem utilities that modify the filesystem (growfs(8), tunefs(8),
and fsirand(8)) should check the filesystem status and require that
fsck(8) be run if it is unclean. This requirement is not imposed on
fsdb(8) or clri(8) since they may be used to clean up a filesystem.
Kirk McKusick [Sun, 25 Oct 2020 00:43:48 +0000 (00:43 +0000)]
Various new check-hash checks have been added to the UFS filesystem
over various major releases. Superblock check hashes were added for
the 12 release and cylinder-group and inode check hashes will appear
in the 13 release.
When a disk with a UFS filesystem is writably mounted, the kernel
clears the feature flags for anything that it does not support. For
example, if a UFS disk from a 12-stable kernel is mounted on an
11-stable system, the 11-stable kernel will clear the flag in the
filesystem superblock that indicates that superblock check-hashs
are being maintained. Thus if the disk is later moved back to a
12-stable system, the 12-stable system will know to ignore its
incorrect check-hash.
If the only filesystem modification done on the earlier kernel is
to run a utility such as growfs(8) that modifies the superblock but
neither updates the check-hash nor clears the feature flag indicating
that it does not support the check-hash, the disk will fail to mount
if it is moved back to its original newer kernel.
This patch moves the code that clears the filesystem feature flags
from the mount code (ffs_mountfs()) to the code that reads the
superblock (ffs_sbget()). As ffs_sbget() is used by the kernel mount
code and is imported into libufs(3), all the filesystem utilities
will now also clear these flags when they make modifications to the
filesystem.
As suggested by John Baldwin, fsck_ffs(8) has been changed to accept
and repair bad superblock check-hashes rather than refusing to run.
This change allows fsck to recover filesystems that have been impacted
by utilities older than those created after this change and is a
sensible thing to do in any event.
Warner Losh [Sat, 24 Oct 2020 23:21:31 +0000 (23:21 +0000)]
cdefs.h: remove intel_compiler support
The age of the intel compiler support is so old as to be
uninteresting. No recent recports of intel compiler support have been
received. Remove all the special case workarounds for the Intel
compiler. Should there be interest in supporting the compiler, contact
me and I'll work with people to make it happen, though I suspect these
instances are more likely to be in the way than to be helpful.
Reviewed by: cem, emaste, vangyzen, dim
Differential Revision: https://reviews.freebsd.org/D26817
Warner Losh [Sat, 24 Oct 2020 23:21:27 +0000 (23:21 +0000)]
Remove support for intel compiler from i386 in_cksum
We no longer support building the kernel with the old intel
compiler. Remove support for it from in_cksum. Should there be
interest in reviving it, this is as likely to get in the way as to
help anyway.
Warner Losh [Sat, 24 Oct 2020 23:21:22 +0000 (23:21 +0000)]
Remove intel compiler support from math.h
The intel compiler support has badly decayed over the years. Stop
pretending that we support it. Note, I've stopped short of requiring
gcc builtin support with this commit since other compilers may be used
to build non-base software and we need to support those so more
investigation is needed before simplifying further.
Warner Losh [Sat, 24 Oct 2020 23:21:18 +0000 (23:21 +0000)]
Remove support for ancient compilers
We don't support building the kernel from such old compilers, nor with
the Intel Compiler specifically. Remove support for this old construct
that was copied from stdbool.h and not relevant here.
Warner Losh [Sat, 24 Oct 2020 23:21:06 +0000 (23:21 +0000)]
Remove obsolete check for GCC < 3 and support for Intel Compiler
We no longer support old versions of GCC. Remove this check by
assuming it's false. That will make the entire expression false. Also
remove support for Intel compiler, it's badly bitrotted. Technically,
this removes support for C89 and K&R from compilers that don't define
_Bool in those compilation environments as well. I'm unaware of any
working compiler today for which that would be relevant (pcc has it
and tcc sadly isn't working for other reasons), though if one
pops up in ports, I'll work to resolve the issue.
Rick Macklem [Sat, 24 Oct 2020 22:48:28 +0000 (22:48 +0000)]
Add "-R" option to tell mountd to not register with rpcbind.
rpcbind is now considered a security risk for some sites.
Since an NFSv4 only NFS server does not need rpcbind,
it makes sense to have an option that implements this.
This patch adds a "-R" option that disables the Mount
protocol (not used by NFSv4) and does not register
with rpcbind.
Changes are required to /etc/rc.d/mountd and /etc/rc.d/nfsd.
Those will be in a separate commit.
Alexander Motin [Sat, 24 Oct 2020 21:07:10 +0000 (21:07 +0000)]
Fix asymmetry in devstat(9) calls by GEOM.
Before this GEOM passed bio pointer to transaction start, but not end.
It was irrelevant until devstat(9) got DTrace hooks, that appeared to
provide bio pointer on I/O completion, but not on submission.
Make use of IP_VLAN_PCP setsockopt in ping and ping6.
In order to validate the proper marking and use of a different
ethernet priority class, add the new session-specific PCP
feature to the ping/ping6 utilities.
Mitchell Horne [Sat, 24 Oct 2020 20:57:13 +0000 (20:57 +0000)]
riscv: improve exception code naming
The existing names were inherited from arm64, but we should prefer
RISC-V terminology. Change the prefix to SCAUSE, and further change the
names to better match the RISC-V spec and be more consistent with one
another. Also, remove two codes that are not defined for S-mode (machine
and hypervisor ecall).
While here, apply style(9) to some condition checks.
Ruslan Bukin [Sat, 24 Oct 2020 20:09:27 +0000 (20:09 +0000)]
o Add iommu de-initialization method for MSI interface.
o Add iommu_unmap_msi() to release the msi GAS entry.
o Provide default implementations for iommu init/deinit methods.
Ryan Moeller [Sat, 24 Oct 2020 17:08:59 +0000 (17:08 +0000)]
flua: Add a libjail module
libjail is pretty small, so it makes for a good proof of concept demonstrating
how a system library can be wrapped to create a loadable Lua module for flua.
* Introduce 3lua section for man pages
* Add libjail module
Ryan Moeller [Sat, 24 Oct 2020 16:05:37 +0000 (16:05 +0000)]
sysctl: Misc code cleanup
* Use defined SYS_SYSCTL names
* Use memcmp instead of explicit loop
* Use NULL instead of 0 for pointer value
* Use __FBSDID
* Reformat, improve comments in parse()
Kyle Evans [Sat, 24 Oct 2020 15:38:04 +0000 (15:38 +0000)]
backlight(9): compile with COMPAT_LINUXKPI as well
This would be more accurately expressed as COMPAT_LINUXKPI implying or
requiring backlight, but config(8) doesn't really have a way to express
that. This fixes the build with COMPAT_LINUXKPI specified in one's kernel
config.
Kyle Evans [Sat, 24 Oct 2020 14:39:17 +0000 (14:39 +0000)]
audit: correct reporting of *execve(2) success
r326145 corrected do_execve() to return EJUSTRETURN upon success so that
important registers are not clobbered. This had the side effect of tapping
out 'failures' for all *execve(2) audit records, which is less than useful
for auditing purposes.
Audit exec returns earlier, where we can know for sure that EJUSTRETURN
translates to success. Note that this unsets TDP_AUDITREC as we commit the
audit record, so the usual audit in the syscall return path will do nothing.
Alan Somers [Sat, 24 Oct 2020 05:52:29 +0000 (05:52 +0000)]
nfsstat: delete unused fields
Ever since r192762 nfsstat has included a few fields whose values were
always 0. They were copied from OpenBSD, but have never been used on
FreeBSD. Don't display them.
Warner Losh [Sat, 24 Oct 2020 01:59:01 +0000 (01:59 +0000)]
nvme: Remove compat code for older kernels
Remove code that supported pre-2011 kernels. CTLTYPE_S64 was defined
in rev 217616. All supported branches have it, so remove its compat
definition as OBE.