Kyle Evans [Mon, 8 Feb 2021 18:31:17 +0000 (12:31 -0600)]
grep: fix -A handling in conjunction with -m match limitation
The basic issue here is that grep, when given -m 1, would stop all
line processing once it hit the match count and exit immediately. The
problem with exiting immediately is that -A processing only happens when
subsequent lines are processed and do not match.
The fix here is relatively easy; when bsdgrep matches a line, it resets
the 'tail' of the matching context to the value supplied to -A and
dumps anything that's been queued up for -B. After the current line has
been printed and tail is reset, we check our mcount and do what's
needed. Therefore, at the time that we decide we're doing nothing, we
know that 'tail' of the context is correct and we can simply continue
on if there's still more to pick up.
With this change, we still bail out immediately if there's been no -A
flag. If -A was supplied, we signal that we should continue on. However,
subsequent lines will not even bothere to try and process the line. We
have reached the match count, so even if the next line would match then
we must process it if it hadn't. Thus, the loop in procfile() can
short-circuit and just process the line as a non-match until
procmatches() indicates that it's safe to stop.
A test has been added to reflect both that we should be picking up the
next line and that the next line should be considered a non-match even
if it should have been.
Kyle Evans [Thu, 4 Feb 2021 21:26:45 +0000 (15:26 -0600)]
grep: fix null pattern and empty pattern file behavior
The null pattern semantics were terrible because I tried to match gnugrep,
but I got it wrong. Let's unwind that:
- The null pattern should match every line if neither -w nor -x.
- The null pattern should match empty lines if -x.
- The null pattern should not match any lines if -w.
The first two will stop processing (shortcut) even if additional patterns
are specified. In any other case, we will continue processing other
patterns. If no other patterns are specified beside a null pattern, then
we match if neither -w nor -x or set and do not match if either of those
are specified.
The justification for -w is that it should match on a whole word, but the
null pattern deos not have a whole word to match on.
Empty pattern files should never match anything, and more importantly, -v
should cause everything to be written.
Eric van Gyzen [Thu, 1 Oct 2020 21:48:22 +0000 (21:48 +0000)]
zgrep: fix exit status with multiple files
zgrep should exit with success when given multiple files and the
pattern is found in at least one file. Prior to this change,
it would exit with success only if the pattern was found in _every_ file.
Warner Losh [Mon, 8 Feb 2021 21:43:25 +0000 (14:43 -0700)]
acpi: limit the AMDI0020/AMDI0010 workaround to an option
It appears that production versions of EPYC firmware get the _STA method right
for these nodes. In fact, this workaround breaks on production hardware by
including too many uart nodes. This work around was for pre-release hardware
that wound up not having a large deployment. Move this work around to a kernel
option since the machines that needed it have been powered off and are difficult
to resurrect. Should there be a more significant deployment than is understood,
we can restrict it based on smbios strings.
Discussed with: mmacy@, seanc@, jhb@
MFC After: 3 days
Conrad Meyer [Wed, 2 Oct 2019 19:13:35 +0000 (19:13 +0000)]
nvdimm: Fix error path mis-free
(diff applied manually due to refactoring)
Regression introduced in r343629 when malloc result was renamed from spa to
spa_mapping and the 'spa' name was instead used to iterate a table, but the
free() target was not updated.
Indentation cleanup resulting from the cleanup of #ifdefs.
The conscious decision was made not to perform any indentation or
whitespace cleanup while cleaning out old redunant #ifdefs. The
reason for this was to avoid confusing future readers of history and
diffs with cosmetic changes, making bisection of any possible bugs
introduced more difficult. This commit cleans up the whitespace
detritus left behind from the previous #ifdef cleanup commits.
In the old days when K&R C and STD C were each in use a workaround
(read hack) was required to allow the same code to work on each
without modification. All C compilers support STD C. We can finally
put the __P prototype to rest.
ukbd(4): Push LED events in ioctl handler rather than in xfer callback
If LED state is set through evdev interface, than asynchronous nature
of USB transfer callback can lead to change of order of events echoed
back to userland as it causes LED events to be echoed with some lag.
Fix that with echoing of LED events synchronously in ioctl handler.
Unlike AT keyboards, HID devices are able to send all pc105 key
states within a single report. Let evdev to transmit all key state
changes within a single report too.
Ed Maste [Tue, 2 Feb 2021 21:55:51 +0000 (16:55 -0500)]
Correct description for kern.proc.proc_td
kern.proc.proc_td returns the process table with an entry for each
thread. Previously the description included "no threads", presumably
a cut-and-pasteo in 2648efa621748.
Description suggested by PauAmma.
PR: 253146
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Andrew Gierth [Wed, 5 Feb 2020 04:32:49 +0000 (04:32 +0000)]
service(8): set the environment of the "daemon" class before invoking
As mentioned in r357562, this gives the user a single place to configure
environment variables that need to be used for various services -- the
"daemon" class -- for, e.g., configuring a system-wide HTTP proxy.
Andrew Gierth [Wed, 5 Feb 2020 04:27:44 +0000 (04:27 +0000)]
init(8): set environment variables from the "daemon" class as well
Specifically, when running /etc/rc. This allows one to specify via
login.conf(5) an environment that should be used when running services to
ease, e.g., setting up env vars for an HTTP proxy consistently across cron
and services alike.
Future changes will extend cron(8)/service(8) to use environment vars
pecified in login.conf(5) as well to promote a more cohesive experience.
Alexander Motin [Thu, 21 Jan 2021 02:33:14 +0000 (21:33 -0500)]
Remove FirstBurstLength limit for software iSCSI.
For hardware offload solicited data may potentially be handled more
efficiently than unsolicited due to direct data placement. Or there
can be some unsolicited write buffering limitations. It may create
situations where FirstBurstLength limit is really useful.
Software driver though has no those factors, having to do memcopy()
any way and having no so hard limit on the temporary storage. Same
time more active use of unsolicited transfers allows to avoid some
of Ready To Transfer (R2T) PDU round-trip times and processing.
This change effectively doubles from 64KB to 128KB the maximum size
of write command that can be transferred within one link RTT. Tests
of (64KB, 128KB] QD1 writes mixed with simultaneous QD8 reads over
the same connection, increasing RTT, shows almost double write speed
and half latency, while we should be able to afford few megabytes of
RAM for additional buffering on a target these days.
Warner Losh [Mon, 25 Jan 2021 19:53:31 +0000 (12:53 -0700)]
newvers: tweak uname to be more useful
The current uname is branch-cXXXX-gHASH
Three changes to make uname more useful.
1. Move from using git rev-list --count to git rev-lis --count --first-parent
since that gives a better, incrementing number.
2. Report this count as 'nXXXXX' rather than 'cXXXXX' because c is part of
a hash and we've changed the sematnics of XXXXX
3. Remove g to make HASH cut and pastable.
Durting review, #1 & #3 had the largest consensus. There was a diversity of
opinion on #2, but on the whole it was positive so I'll acknowledge the dissent,
but move forward with something seems to have support since the dissent was all
about what letter to use where I chose 'n'.
Mark Johnston [Wed, 20 Jan 2021 01:34:35 +0000 (20:34 -0500)]
opencrypto: Fix assignment of crypto completions to worker threads
Since r336439 we simply take the session pointer value mod the number of
worker threads (ncpu by default). On small systems this ends up
funneling all completion work through a single thread, which becomes a
bottleneck when processing IPSec traffic using hardware crypto drivers.
(Software drivers such as aesni(4) are unaffected since they invoke
completion handlers synchonously.)
Instead, maintain an incrementing counter with a unique value per
session, and use that to distribute work to completion threads.
Kyle Evans [Sun, 31 Jan 2021 15:51:39 +0000 (09:51 -0600)]
lualoader: position hyphens at the beginning of character classes
According to the Lua 5.4 manual section 6.4.1 ("Patterns"), the interaction
between ranges and classes is not defined and hyphens must be specified at
either the beginning or the end of a set if they are not escaped.
Kyle Evans [Sun, 24 Jan 2021 01:32:38 +0000 (19:32 -0600)]
stand: lua: enhance lfs.dir() to speed up kernels_autodetect
This eliminates a lot of stat() calls that happen when lualoader renders the
menu with the default settings, and greatly speeds up rendering on my
laptop.
ftype is nil if loader/loader.efi hasn't been updated yet, falling back to
lfs.attributes() to test.
This is technically incompatible with lfs, but not in a particularly
terrible way.
Alexander Motin [Sun, 31 Jan 2021 17:46:57 +0000 (12:46 -0500)]
cxgb(4): Remove assumption of physically contiguous mbufs.
Investigation of iSCSI target data corruption reports brought me to
discovery that cxgb(4) expects mbufs to be physically contiguous, that
is not true after I've started using m_extaddref() in software iSCSI
for large zero-copy transmissions. In case of fragmented memory the
driver transmitted garbage from pages following the first one due to
simple use of pmap_kextract() for the first pointer instead of proper
bus_dmamap_load_mbuf_sg(). Seems like it was done as some optimization
many years ago, and at very least it is wrong in a world of IOMMUs.
This patch just removes that optimization, plus limits packet coalescing
for mbufs crossing page boundary, also depending on assumption of one
segment per packet.
Conrad Meyer [Sun, 22 Dec 2019 03:19:17 +0000 (03:19 +0000)]
fstyp(8): Show exFAT volume labels with -l flag
exfat is fundamentally the same design as fat32. The superblock differs
marginally, and there are some additional optional features irrelevant to
fstype(8); the structure of dirents has changed slightly to enable, among
other things, larger files; the directory entries are no longer DOS 8.3
ASCII or local 8-bit encoding, but instead explicitly UCS-2-LE.
(As a result, this change uses iconv to convert a found exfat volume label
to the user's locale.)
Locating the volume label is identical to FAT32: locate the root directory
and walk through dirents until you find a volume label. Like FAT32, follow
the FAT chain between root directory clusters as necessary.
PR: 242225
Reported by: Victor Sudakov <vas AT sibptus.ru>
ixl: Permit 802.1ad frames to pass though the chip
This patch is a quick hack to change the internal Ethertype used
within the chip. All frames with this type are dropped silently.
This patch allows you to overwrite the factory default 0x88a8, which
is used by IEEE 802.1ad VLAN stacking.
Marcin Wojtas [Tue, 26 Jan 2021 10:49:35 +0000 (11:49 +0100)]
marvell: ap806_clock: add missing frequency modes
In the driver init routine the CPU clock frequency
value is obtained from a dedicated register. Until now
only part of the values were handled by the mv_ap806_clock
driver. Fix that by adding missing cases.
ng_tag(4) operate on arbitrary data of mbuf_tags(9). Those structures
are padded to the next multiple of the alignment by the compiler.
Hence a valid argument has be at most as long as the data received.
The NS_MOREFRAG flag can be set in a netmap slot to represent a
multi-fragment packet. Only the last fragment of a packet does
not have the flag set. On TX rings, the flag may be set by the
userspace application. The kernel will look at the flag and use it
to properly set up the NIC TX descriptors.
On RX rings, the kernel may set the flag if the packet received
was split across multiple netmap buffers. The userspace application
should look at the flag to know when the packet is complete.
All C compilers in 2021 support standard C and architectures that did
not were retired long ago. Simplify by removing now redundant
pre-standard C code.
Kyle Evans [Sun, 24 Jan 2021 19:25:34 +0000 (13:25 -0600)]
lualoader: improve loader.conf var processing
lualoader was previously not processing \ as escapes; this commit fixes
that and does better error checking on the value as well.
Additionally, loader.conf had some odd restrictions on values that make
little sense. Previously, lines like:
kernel=foo
Would simply be discarded with a malformed line complaint you might not
see unless you disable beastie.
lualoader tries to process these as well as it can and manipulates the
environment, while forthloader did minimal processing and constructed a
`set` command to do the heavy lifting instead. The lua approach was
re-envisioned from building a `set` command so that we can appropriately
reset the environment when, for example, boot environments change.
Lift the previous restrictions to allow unquoted values on the right hand
side of an expression. Note that an unquoted value is effectively:
[A-Za-z0-9-][A-Za-z0-9-_.]*
This commit also stops trying to weirdly limit what it can handle in a
quoted value. Previously it only allowed spaces, alphanumeric, and
punctuation, which is kind of weird. Change it here to grab as much as it
can between two sets of quotes, then let processEnvVar() do the needful and
complain if it finds something malformed looking.
My extremely sophisticated test suite is as follows:
Future work may provide a stub loader module in userland so that we can
formally test the loader scripts rather than sketchy setups like the above
in conjunction with the lua-* tools in ^/tools/boot.
Kyle Evans [Wed, 27 Jan 2021 18:54:07 +0000 (12:54 -0600)]
stand: ensure that the efi directory's dependencies are correct
efi, like the various ${MACHINE} directories, should have a dependency on
the enabled interpreters.
The general rule here is that any top-level directory that has a program at
any depth within that includes loader.mk should add ${INTERP_DEPENDS} added
to its dependencies so that the appropriate ficl/lua bits are ready before
they begin.
Note that the only directories in-tree that require it but will not get it
in a more appropriate manner are i386 (on amd64), efi, and userboot. i386
and userboot are handled explicitly in Makefile.amd64 where they are added
to S.yes.
Jilles Tjoelker [Sun, 3 Jan 2021 20:27:50 +0000 (21:27 +0100)]
sh/tests: Add a second kind of binary scripts without #!
One of the reasons for git commit e0f5c1387df23c8c4811f5b24a7ef6ecac51a71a was to make "actually portable
executables" work. Add a test that is more like those.
Jilles Tjoelker [Sat, 30 May 2020 16:00:49 +0000 (16:00 +0000)]
sh: Allow more scripts without #!
Austin Group bugs #1226 and #1250 changed the requirements for shell scripts
without #! (POSIX does not specify #!; this is about the shell execution
when execve(2) returns an [ENOEXEC] error).
POSIX says we shall allow execution if the initial part intended to be
parsed by the shell consists of characters and does not contain the NUL
character. This allows concatenating a shell script (ending with exec or
exit) and a binary payload.
In order to reject common binary files such as PNG images, check that there
is a lowercase letter or expansion before the last newline before the NUL
character, in addition to the check for the newline character suggested by
POSIX.
Jilles Tjoelker [Mon, 30 Dec 2019 21:32:55 +0000 (21:32 +0000)]
sh: Test that executing various binary files is rejected
If executing a file fails with an [ENOEXEC] error, the shell executes the
file as a shell script, except that this execution may instead result in an
error message if the file is binary.
Per a recent Austin Group interpretation, we will need to change this to
allow a concatenation of a shell script and a binary payload. See
Austin Group bugs #1226 and #1250.
Ed Maste [Wed, 13 Jan 2021 18:08:31 +0000 (13:08 -0500)]
elfctl: prefix disable flags with "no"
Some ELF feature flags indicate a request to opt-out of some feature,
for example NT_FREEBSD_FCTL_ASLR_DISABLE indicates that ASLR should be
disabled for the tagged binary. Using "aslr" as the short name for the
flag is confusing as it seems to indicate a request for ASLR to be
enabled. Rename "noaslr", and make a similar change for other opt-out
flags.
Reviewed by: bapt, manu, markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D28139
Ed Maste [Fri, 22 Jan 2021 17:22:35 +0000 (12:22 -0500)]
elfctl: allow features to be specified by value
This will allow elfctl on older releases to set bits that are not yet
known there, so that the binary will have the correct settings applied
if run on a later FreeBSD version.
PR: 252629 (related)
Suggested by: kib
Reviewed by: gbe (manpage, earlier), kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D28284
Target value for val has uint32_t type, not uint, adjust used constant.
Change val type to unsigned so that left and right sides of comparision
operator do not expose different signed types of same range [*].
Switch to unsigned long long and strtoll(3) so that 0x80000000 is
accepted by conversion function [**].
Reported by: kargl [*]
Noted by: emaste [**]
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D28301
Ed Maste [Wed, 13 Jan 2021 19:21:38 +0000 (14:21 -0500)]
elfctl: add backwards compatibility for "no" prefixes
I am going to prefix opt-out ELF feature flag names with "no" to make
their meaning more clear (review D28139), but there are some uses of the
existing names already (e.g., the PR referenced below).
For now accept the older, unprefixed name as well, and emit a warning.
We can revert this after FreeBSD 13 branches.
For rate-based resources that support throttling (e.g.
readiops/writeips), this fixes a divide-by-zero panic when rctl(8)
passes 0 as the throttle value. For these resources, treat
zero-throttle requests as requests to suspend forward progress as long
as possible using the duration specified in
kern.racct.rctl.throttle_max.
Mark Johnston [Mon, 18 Jan 2021 22:07:56 +0000 (17:07 -0500)]
safexcel: Simplify request allocation
Rather than preallocating a set of requests and moving them between
queues during state transitions, maintain a shadow of the command
descriptor ring to track the driver context of each request. This is
simpler and requires less synchronization between safexcel_process() and
the ring interrupt handler.
Rather than returning a hard error in this case, return ERESTART so that
upper layers get a chance to retry the request (or drop it, depending on
the desired policy).
This case is hard to hit due to the somewhat low bound on queued
requests, but that will no longer be true after an upcoming change.
Michael Tuexen [Wed, 13 Jan 2021 21:48:17 +0000 (22:48 +0100)]
tcp: add sysctl to tolerate TCP segments missing timestamps
When timestamp support has been negotiated, TCP segements received
without a timestamp should be discarded. However, there are broken
TCP implementations (for example, stacks used by Omniswitch 63xx and
64xx models), which send TCP segments without timestamps although
they negotiated timestamp support.
This patch adds a sysctl variable which tolerates such TCP segments
and allows to interoperate with broken stacks.
iflib: add assert to prevent out-of-bounds array access
The iflib_queues_alloc() allocates isc_nrxqs iflib_dma_info structs
for each rxqset, and links each struct to a different free list.
As a result, it must be isc_nrxqs >= isc_nfl (plus the completion
queue, if present).
Add an assertion to make this constraint explicit.
Kyle Evans [Thu, 31 Dec 2020 17:15:45 +0000 (11:15 -0600)]
stand: properly declare subdir deps or .WAIT, do parallel build
buildworld already runs the stand build in parallel[1], so make it easier to
identify ordering issues by properly establishing dependencies or adding
.WAIT where needed.
Everything in stand/ relies on libsa, either directly or indirectly, because
libsa build is where the stand headers get installed and it gets linked in
most places.
Interpreters depend on their libs, machine dirs usually depend on top-level
libs that are getting built and at least one of the interpreter flavors.
For i386, order btx/libi386/libfirewire before everything else using a
big-ol-.WAIT hammer. btx is the most common dependency, but the others are
used sporadically. This seems to be where the race reporting on the mailing
list is- AFAICT, the following sequence is happening:
1.) One of the loaders gets built based on stale btx/btxldr
2.) btx/btxldr gets rebuilt
3.) installworld triggers loader rebuild because btx was rebuilt after
This seems like the most plausible explanation, as they've verified system
time and timestamps.
While we're here, let's switch stand/ over to a completely parallel build so
we can work out these kinds of issues in isolation rather than in the middle
of a larger build.
Brandon Bergren [Thu, 2 Jan 2020 04:34:22 +0000 (04:34 +0000)]
Move stand/ofw/libofw to stand/libofw.
Since rS330365, there has been no particular reason for libofw to be in a
subdirectory of ofw. Move libofw up a level to make it fit in better with
the other top level libraries.
Also add a LIBOFWSRC to stand/defs.mk to match what all the other
libraries are doing.
refcount_load() does not yet exist on this branch, and the path to MFC'ing
it is slightly non-trivial. Back out the part that uses it -- it's a ddb
command anyways, so the cost of getting it wrong is ~low.
Kyle Evans [Tue, 5 Jan 2021 21:49:46 +0000 (15:49 -0600)]
du: tests: make H_flag tests more strict about output requirements
The current version of this test will effectively pass as long as one of the
specified paths is in the output, and it could even be a subset of one of
the paths.
Strengthen up the test a little bit:
* Specify beginning/end anchors for each path
* Add egrep -v checks to make sure we don't have any *additional* paths
* Ratchet down paths2 to exactly the two paths we expect to appear
Kyle Evans [Tue, 5 Jan 2021 21:33:06 +0000 (15:33 -0600)]
du: tests: fix the H_flag test (primarily grep usage)
This test attempts to use \t (tab intended) in a grep expression. With the
former /usr/bin/grep (i.e. gnugrep), this was interpreted as a literal 't'.
The expression would work anyways because the tr(1) usage would ultimately
replace all of the spaces with a single newline, and they would match the
paths whether they were correctly fromatted or not.
Current /usr/bin/grep (i.e. bsdgrep) is less-tolerant of ordinary-escapes, a
property of the underlying regex(3) engine, to make it easier to identify
when stuff like this happens. In-fact, this expression broke after the
switch happened.
This revision does the bare basics to fix the usage by using a printf to get
a literal tab character to insert into the expression. It also swaps out the
manual insertion of the line prefix into the grep expression by pulling
that part out of $sep and reusing it for the leading path.
The secondary issue was the tr(1) usage, since tr would only replace the
first character of string1 with the first character of string2. This has
instead been replaced by a sed expression, which similary understands \n to
be a newline on all supported versions of FreeBSD. Each path now gets
prefixed with the appropriate context that should be there (i.e. numeric
sequence followed by a tab).
Kyle Evans [Thu, 31 Dec 2020 18:30:43 +0000 (12:30 -0600)]
libc: tests: add some tests for cpuset(2)
The cpuset(2) tests should be run as root (require.user properly set) with
>= 3 cpus for maximum coverage. All tests that want to modify the cpuset
don't assume any particular cpu layout (i.e. the first cpu may not be 0, the
last may not be first + count) and the following scenarios are tested:
1.) newset: basic execute cpuset() to grab a new cpuset, make sure the
assigned cpuset then has a different ID.
2.) transient: create a new cpuset then assign the process its original
cpuset, ensuring that the one we created is now gone.
3.) deadlk: test assigning an anonymous mask, then resetting the process
base affinity with 1-cpu overlap w.r.t. the anonymous mask and with
0-cpu overlap w.r.t. the anonymous mask.
4.) jail_attach_newbase: process attaches to a jail with its own
cpuset+mask (e.g. cpuset -c -l 1,2 jail -c path=/ command=/bin/sh)
5.) jail_attach_newbase_plain: process attaches to a jail with its own
cpuset (e.g. cpuset -c jail -c path=/ command=/bin/sh)
6.) jail_attach_prevbase: process attaches to a jail with the containing
jail's root cpuset (e.g. jail -c path=/ command=/bin/sh)
7.) jail_attach_plain: process attaches to a jail with the containing jail's
root cpuset+mask.
8.) badparent: creates a new cpuset and modifies the anonymous thread mask,
then setid's back to the original and checks that cpuset_getid() returns
the expected set.
Kyle Evans [Thu, 31 Dec 2020 18:26:01 +0000 (12:26 -0600)]
libc: tests: hook CPUSET(9) test up to the build
Add shims to map NetBSD's API to CPUSET(9). Obviously the invalid input
parts of these tests are relatively useless since we're just testing the
shims that aren't used elsewhere, there's still some amount of value in
the parts testing valid inputs.
Kyle Evans [Fri, 8 Jan 2021 19:57:32 +0000 (13:57 -0600)]
libregex: re-enable `make check`
The tests are generally expected to pass, uncomment the annotation that
lets `make check` work. Note that `make check` currently requires kyua
from ports or an appropriate symlink into /usr/local/bin.
Alex Richardson [Tue, 25 Aug 2020 13:30:34 +0000 (13:30 +0000)]
Fix -Wundef warnings when building liblua
We need to define the LUA_FLOAT_INT64 macro even if we don't use it (copied
from stand/luaconf.h). While touching luaconf.h.dist also sync it with the
the 5.3.5 release version (matches the one in lib/liblua).
projects / FreeBSD / FreeBSD.git / log