kevans [Tue, 4 Aug 2020 02:14:51 +0000 (02:14 +0000)]
libregex: Implement a subset of the GNU extensions
The entire patch-set is not yet mature enough for commit, but this usable
subset is generally enough for googletest to be happy with and mostly map to
some existing concepts, so they're not as invasive.
The specific changes included here are:
- Branching in BREs with \|
- \w and \W for [[:alnum:]] and [^[:alnum:]] respectively
- \s and \S for [[:space:]] and [^[:space:]] respectively
- Additional quantifiers in BREs, \? and \+ (self-explanatory)
There's some #ifdef'd out work for allowing empty branches as a match-all.
This is a feature that's under assessment... future work will determine
how standard this behavior is and act accordingly.
The tests compare the command output (including of error cases) with the
expected output and exit code.
Not all tests are executed, since some expect to have a known good bc and
dc binary installed and compare results of large amounts of generated data
being processed by both versions to test for regressions.
This version omits the printing of a copyright header in interactive mode
and the dc command now exits after execution of the commands passed via -e
or -f instead of switching to interactive mode. To pass further commands
via STDIN when dc has been invoked with -e or -f, add "-f -" to the
parameter list.
This version omits the printing of a copyright header in interactive mode
and the dc command now exits after execution of the commands passed via -e
or -f instead of switching to interactive mode. To pass further commands
via STDIN when dc has been invoked with -e or -f, add "-f -" to the
parameter list.
This version makes dc exit after processing all commands passed via -e or -f
instead of waiting for more input on STDIN (add "-f -" to the command line
to emulate the behavior of versionm 3.1.3 and earlier, if desired).
The version and copyright message are no longer printed for interactive
sessions as was the case with the prior implementation in the FreeBSD base
system.
arichardson [Mon, 3 Aug 2020 18:08:10 +0000 (18:08 +0000)]
Allow bootstrapping mtree on Linux systems
Linux glibc has a dummy lchmod that always fails and emitting a linker
warning when used. Don't fail the build due to that warning when
bootstrapping by setting LD_FATAL_WARNINGS=no.
arichardson [Mon, 3 Aug 2020 18:08:04 +0000 (18:08 +0000)]
Allow building setmode.c on Linux/macOS
We bootstrap this file to allow compiling FreeBSD on Linux systems since
some boostrap tools use setmode(). Unfortunately, glibc's sys/stat.h
declares a non-static getumask() function (which is unimplemented!) and
that conflicts with the local getumask() function. To work around this
simply use a different name here.
jhb [Mon, 3 Aug 2020 17:53:15 +0000 (17:53 +0000)]
Pass the full CFLAGS to cpp for MKlib_gen.sh.
GCC's cpp was exiting immediately when it failed to find requested
includes (<ncurses_cfg.h> and <ncurses_defs.h>). clang-cpp emitted an
error for the missing header files but continued processing the file
(thus not honoring any macros defined in the missing headers).
Arguably, the awk script is buggy since it doesn't check the return
value of the command it executes.
andrew [Mon, 3 Aug 2020 16:43:40 +0000 (16:43 +0000)]
Allow the Raspberry Pi firmware driver to be a bus
There are child nodes in the device tree, e.g. the Raspberry Pi firmware
GPIO device. Add support for this to be a bus so we can attach these
children.
Reviewed by: manu
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D25848
andrew [Mon, 3 Aug 2020 16:26:10 +0000 (16:26 +0000)]
Allow child classes of simplebus to call attach directly
Reduce code duplication when a bus is subclassed from simplebus by allowing
them to call simplebus_attach directly. This is useful when the child bus
will just implement the same calls.
As not all children will expect to have a ranges property, e.g. the
Raspberry Pi firmware, allow this property to be missing.
Reviewed by: manu
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D25925
libc: Provide sub fp(s|g)etmask() implementations for RISC-V
RISC-V doesn't support floating-point exceptions.
RISC-V Instruction Set Manual: Volume I: User-Level ISA, 11.2 Floating-Point
Control and Status Register: "As allowed by the standard, we do not support
traps on floating-point exceptions in the base ISA, but instead require
explicit checks of the flags in software. We considered adding branches
controlled directly by the contents of the floating-point accrued exception
flags, but ultimately chose to omit these instructions to keep the ISA simple."
We still need these functions, because some applications (notably Perl) call
them, but we cannot provide a meaningful implementation.
andrew [Mon, 3 Aug 2020 10:19:50 +0000 (10:19 +0000)]
Handle Raspberry Pi 4 xhci firmware loading.
The newer hardware revisions of the Raspberry Pi 4 removed the ability of
the VIA VL805 xhci controller to load its own firmware. Instead the
firmware must be installed at the appropriate time by the VideoCore
coprocessor.
Submitted by: Robert Crowston <crowston_protonmail.com>
Differential Revision: https://reviews.freebsd.org/D25261
jah [Sun, 2 Aug 2020 20:18:37 +0000 (20:18 +0000)]
vt(4): CONS_HISTORY/CONS_CLRHIST should operate on issuing terminal
Currently the CONS_HISTORY and CONS_CLRHIST ioctls modify the state of the
active terminal instead of the terminal against which the ioctl was issued.
Because of the way vidcontrol(1) works, these are the same in most cases.
But a poorly-timed window switch can make them differ. This is reproducible
by issuing e.g. 'vidcontrol -s 2 && vidcontrol -C' to switch from vty 1 to
vty 2; teken will reset the cursor position on vty 1 but vt(4) will clear
the history buffer of vty 2, producing an interesting state of affairs.
mjg [Sun, 2 Aug 2020 20:02:06 +0000 (20:02 +0000)]
vfs: store precomputed namecache hash in the vnode
This significantly speeds up path lookup, Cascade Lake doing access(2) on ufs
on /usr/obj/usr/src/amd64.amd64/sys/GENERIC/vnode_if.c, ops/s:
before: 2535298
after: 2797621
Over +10%.
The reversed order of computation here does not seem to matter for hash
distribution.
0mp [Sun, 2 Aug 2020 16:59:14 +0000 (16:59 +0000)]
core(5) appeared in Version 1 AT&T UNIX
Based on the scans of manual pages available at
https://www.bell-labs.com/usr/dmr/www/man51.pdf,
which are a part of the following collection:
https://www.bell-labs.com/usr/dmr/www/1stEdman.html.
cem [Sun, 2 Aug 2020 16:34:27 +0000 (16:34 +0000)]
Unlocked getblk: Fix new false-positive assertion
A free buf's lock may be held (temporarily) due to unlocked lookup, so
buf_alloc() must acquire it without LK_NOWAIT. The unlocked getblk path
should unlock it promptly once it realizes the identity does not match
the buffer it was searching for.
The mvneta device requires MVNETA_TX_CMD_L4_CHECKSUM_NONE bit to be set in the tx descriptor is checksum not required. However, mvneta_tx_set_csumflag() is not setting this flag currently, causing the hardware to randomly corrupt IP header during transmission.
This affects injected IPv4 packets that skips kernel IP stack processing (e.g. DHCP), as well as all IPv6 packets, since the driver currently does not offload csum for IPv6.
The fix is to remove all the early return paths from mvneta_tx_set_csumflag() which do not set the MVNETA_TX_CMD_L4_CHECKSUM_NONE flag.
PR: 248306
Submitted by: Mike Cui <cuicui@gmail.com>
Reported by: Mike Cui <cuicui@gmail.com>
Add optional support for ext_pgs mbufs to the NFS server's read, readlink
and getxattr operations.
This patch optionally enables generation of read, readlink and getxattr replies
in ext_pgs mbufs. Since neither of ND_EXTPG or ND_TLS are currently ever set,
there is no change in semantics at this time.
It also corrects the message in a couple of panic()s that should never occur.
This is another in the series of commits that add support to the NFS client
and server for building RPC messages in ext_pgs mbufs with anonymous pages.
This is useful so that the entire mbuf list does not need to be
copied before calling sosend() when NFS over TLS is enabled.
Use of ext_pgs mbufs will not be enabled until the kernel RPC is updated
to handle TLS.
Fix typo in interp_include() invocation (missing 'p')
Remove setting tftpip, as servip is used by the tftp code in libsa. There
is no separate tftpip global variable any more.
ng_iface(4): Set the current VNET before calling netisr_dispatch().
This is normally handled by a netgraph thread, but netgraph messages may
be dispatched directly to a node, in which case no VNET is set before
ng_iface calls into the network stack. Netgraph could probably handle
this more generally, but for now just be sure to set the current VNET in
ng_iface.
PR: 242406
Tested by: Michael Muenz <m.muenz@gmail.com>
Reviewed by: Lutz Donnerhacke
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25788
For libc regcomp, this will be a nop. libregex will take this to mean that
it needs to turn off GNU extensions, effectively switching it back to the
POSIX-compliant libc implementation at runtime.
The recent changes to move the ref count increment
back from the end of the function created an issue.
If one of the routines returns NULL during setup
we have inp's with extra references (which is why
the increment was at the end).
Also the stack switch return code was being ignored
and actually has meaning if the stack cannot take over
it should return NULL.
Fix both of these situation by being sure to test the
return code and of course in any case of return NULL (there
are 3) make sure we properly reduce the ref count.
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D25903
UPDATING: Add a note about running installworld twice
Some folks seem to be hitting issues with build orchestration; presumably
some of our .WAIT-removal optimizations are going awry, and they're ending
up with applications linked against new libc being installed before the new
libc.
Letting installworld complete the first time should ensure that the new libc
is installed by the end of it, then the second installworld will ensure
consistency as everything should succeed.
cem [Fri, 31 Jul 2020 00:07:01 +0000 (00:07 +0000)]
getblk: Avoid sleeping on wrong buf in lockless path
If the buffer identity changed during lookup, sleeping could introduce a
lock order reversal. Since we do not know if the identity changed until we
get the lock, we must try-lock (LK_NOWAIT) only. EINTR and ERESTART error
handling becomes irrelevant, as we no longer sleep.
ip6_output(): Check the return value of in6_getlinkifnet().
If the destination address has an embedded scope ID, make sure that it
corresponds to a valid ifnet before proceeding. Otherwise a sendto()
with a bogus link-local address can trigger a NULL pointer dereference.
Reported by: syzkaller
Reviewed by: ae
Fixes: r358572
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25887
Almost all consumers use the NDF_ONLY_PNBUF macro, making them avoidably branch
a lot in the NDFREE routine. Also note most of them should not need to call
any cleanup anyway as they don't request HASBUF.
Re-implement clocks for these SoC by using now standard extres/clk framework.
This is necessary for future expansion of these. The new implementation
is (due to the size of the patch) only the initial (minimum) version.
It will be updated/expanded with a subsequent set of particular patches.
This patch is also not tested on OMAP4 based boards (BeagleBone),
so all possible issues should be (and will be) fixed by ASAP once
identified.
Submited by: Oskar Holmlund (oskar.holmlund@ohdata.se)
Differential Revision: https://reviews.freebsd.org/D25118
r363679 is in-fact the future change referenced by the comment, helpfully
left and forgotten by kevans. Instead of just silently not matching, we
should now be erroring out with vigor.
It's currently unclear to me how this could have worked previously; \n here
is not a literal newline but actual '\' 'n', and was getting passed to the
underlying regex engine as such. regex(3) does not translate this to a
newline, and this became an error because we don't really allow escaping
of arbitrary ordinary characters anymore.
Run the pattern strings through printf to make sure we're dealing with real
newlines before passing them through to atf_check, which ultimately feeds
them directly to regcomp(3).
This fix is different than that will be needed for sed, in that this is the
proper way to inject newlines into search strings as long as regex(3)
won't combine \ + n as folks might expect.
Prevent framebuffer mmio space from being allocated to other devices on HyperV.
On Gen2 VMs, Hyper-V provides mmio space for framebuffer.
This mmio address range is not useable for other PCI devices.
Currently only efifb driver is using this range without reserving
it from system.
Therefore, vmbus driver reserves it before any other PCI device
drivers start to request mmio addresses.
Properly handle a closed TLS socket with pending receive data.
If the remote end closes a TLS socket and the socket buffer still
contains not-yet-decrypted TLS records but no decrypted TLS records,
soreceive needs to block or fail with EWOULDBLOCK. Previously it was
trying to return data and dereferencing a NULL pointer.
regex(3): Interpret many escaped ordinary characters as EESCAPE
In IEEE 1003.1-2008 [1] and earlier revisions, BRE/ERE grammar allows for
any character to be escaped, but "ORD_CHAR preceded by an unescaped
<backslash> character [gives undefined results]".
Historically, we've interpreted an escaped ordinary character as the
ordinary character itself. This becomes problematic when some extensions
give special meanings to an otherwise ordinary character
(e.g. GNU's \b, \s, \w), meaning we may have two different valid
interpretations of the same sequence.
To make this easier to deal with and given that the standard calls this
undefined, we should throw an error (EESCAPE) if we run into this scenario
to ease transition into a state where some escaped ordinaries are blessed
with a special meaning -- it will either error out or have extended
behavior, rather than have two entirely different versions of undefined
behavior that leave the consumer of regex(3) guessing as to what behavior
will be used or leaving them with false impressions.
This change bumps the symbol version of regcomp to FBSD_1.6 and provides the
old escape semantics for legacy applications, just in case one has an older
application that would immediately turn into a pumpkin because of an
extraneous escape that's embedded or otherwise critical to its operation.
This is the final piece needed before enhancing libregex with GNU extensions
and flipping the switch on bsdgrep.
Add support for ext_pgs mbufs to nfsrvd_readdir() and nfsrvd_readdirplus().
This patch code that optionally (based on ND_TLS, never set yet) generates
readdir replies in ext_pgs mbufs.
To trim the list back, a new function that is ext_pgs aware called
nfsm_trimtrailing() replaces newnfs_trimtrailing().
newnfs_trimtrailing() is no longer used, but will be removed in a future
commit, since its removal does modify the internal kpi between the NFS
modules.
This is another in the series of commits that add support to the NFS client
and server for building RPC messages in ext_pgs mbufs with anonymous pages.
This is useful so that the entire mbuf list does not need to be
copied before calling sosend() when NFS over TLS is enabled.
Use of ext_pgs mbufs will not be enabled until the kernel RPC is updated
to handle TLS.
Use atomic(9) to load the lock state. Some places were doing this
already, so it was inconsistent. In initialization code, the lock state
is still initialized with plain stores.
Reviewed by: alc, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25861
o Move iommu_set_buswide_ctx, iommu_is_buswide_ctx to
the generic iommu busdma backend;
o Move bus_dma_iommu_set_buswide, bus_dma_iommu_load_ident
prototypes to iommu.h.
blacklistd has been renamed to blocklistd upstream, and a future
import into FreeBSD will follow that change. Support the new name
as an alias in config files.
Reviewed by: bz, delphij
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25865
Before switching to nvlists CTL merged previous and new options, so
any options not passed just kept previous value. Now CTL completely
replaces them, so we must pass everything still relevant.
Fixup some incorrect information and some comments. These changes
were cherry picked up the upstream OpenBSD repository. At some point we
will look at doing another import, but the diffs are substantial and will
require some careful testing.
vm_page_xbusy_claim(): Use atomics to update busy lock state.
vm_page_xbusy_claim() could clobber the waiter bit. For its original
use, kernel memory pages, this was not a problem since nothing would
ever block on the busy lock for such pages. r363607 introduced a new
use where this could in principle be a problem.
Fix the problem by using atomic_cmpset to update the lock owner. Since
this macro is defined only for INVARIANTS kernels the extra overhead
doesn't seem prohibitive.
Reported by: vangyzen
Reviewed by: alc, kib, vangyzen
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25859
Add further clarification on si_addr and si_trapno.
- In the initial description of si_addr, do not claim that it is
always the faulting instruction.
- For si_addr, document that it is generally set to the PC for
synchronous signals, but that it can be set to the the address of
the faulting memory reference for some signals including SIGSEGV and
SIGBUS. In particular, while SIGSEGV generally sets si_addr to the
faulting memory reference, SIGBUS can vary. On some platforms, some
SIGBUS signals set si_addr to the PC and other SIGBUS signals set
si_addr to the faulting address depending on the specific hardware
exception.
- For si_trapno, synchronous signals should set this to some value.
ps(1): Fix formatting of the "command" field for kernel threads.
When -H is specified, for kernel threads the command is formatted as
"<proc name>/<td name>" and truncated to MAXCOMLEN. But each of the
proc name and td name may be up to MAXCOMLEN bytes in length.
Also handle the ki_moretdname field to ensure that the full thread name
gets printed. This is already handled correctly when formatting for
"-o tdname".
andrew [Tue, 28 Jul 2020 11:32:45 +0000 (11:32 +0000)]
Add a workaround for a bug when setting the Raspberry GIO config and state
The Raspberry Pi GPIO config and state messages incorrectly return with
the tag length set to 0. We then check this value to have the response
flag set. Work around this by setting the response flag when setting the
GPIO config or state and this value is zero.
PowerPC support was fixed in r357596 by changing PCI bustag to BE as
part of the solution, but this caused regression on mips. This change
implements byte swapping of virtio PCI config area in the driver,
leaving lower layer untouched.
Submittnd by: Fernando Valle <fernando.valle@eldorado.org.br>
Reported by: arichardson
Reviewed by: alfredo, arichardson
Sponsored by: Eldorado Research Institute (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D25416