rmacklem [Fri, 25 Aug 2017 22:58:54 +0000 (22:58 +0000)]
MFC: r321689
Add a new "-N" option to umount(8), that does a forced dismount of an NFS mount
point.
The new "-N" option does a forced dismount of an NFS mount point, but avoids
doing any checking of the mounted-on path, so that it will not get hung
when a vnode lock is held by another hung process on the mounted-on vnode.
The most common case of this is a "umount" with the "-f" option.
Other than avoiding checking the mounted-on path, it performs the same
forced dismount as a successful "umount -f" would do.
This commit includes a content change to the man page.
rmacklem [Fri, 25 Aug 2017 22:52:40 +0000 (22:52 +0000)]
MFC: r321688
Add kernel support for the NFS client forced dismount "umount -N" option.
When an NFS mount is hung against an unresponsive NFS server, the "umount -f"
option can be used to dismount the mount. Unfortunately, "umount -f" gets
hung as well if a "umount" without "-f" has already been done. Usually,
this is because of a vnode lock being held by the "umount" for the mounted-on
vnode.
This patch adds kernel code so that a new "-N" option can be added to "umount",
allowing it to avoid getting hung for this case.
It adds two flags. One indicates that a forced dismount is about to happen
and the other is used, along with setting mnt_data == NULL, to handshake
with the nfs_unmount() VFS call.
It includes a slight change to the interface used between the client and
common NFS modules, so I bumped __FreeBSD_version to ensure both modules are
rebuilt.
rmacklem [Fri, 25 Aug 2017 22:44:55 +0000 (22:44 +0000)]
MFC: r321675
Fix possible crash for the NFSv4.1 pNFS client.
If the nfsrpc_createlayoutrpc() call in nfsrpc_getcreatelayout() fails,
the code used nfhpp when it might be set NULL. This patch checks for
the error cases (laystat != 0) and avoids using nfhpp for the failure case.
This would only affect NFSv4.1 mounts with the "pnfs" option.
Found while testing the "umount -N" patch not yet in head.
rmacklem [Fri, 25 Aug 2017 22:39:49 +0000 (22:39 +0000)]
MFC: r321628
Replace the checks for MNTK_UNMOUNTF with a macro that does the same thing.
This patch defines a macro that checks for MNTK_UNMOUNTF and replaces
explicit checks with this macro. It has no effect on semantics, but
prepares the code for a future patch where there will also be a
NFS specific flag for "forced dismount about to occur".
asomers [Fri, 25 Aug 2017 14:32:03 +0000 (14:32 +0000)]
MFC r320974-r320975, r321001, r321206
r320974:
Use ATF cleanup routines in aio_test.c
Remove aio_test's legacy timeout handling and cleanup routines. Instead,
use ATF's builtin capabilities. ATF automatically cleans up newly created
files, too, so we don't have to explicitly unlink them. The only tests than
need a cleanup routine are the md(4) tests, which must destroy their md
device.
marius [Thu, 24 Aug 2017 20:51:16 +0000 (20:51 +0000)]
MFC: r322726
Bring back the much more readable unified format for differences in
/etc/{group,master.passwd}. This was originally turned on for all of
/etc/{aliases,group,master.passwd} in r55196, but then backed out
only for the latter two in r56697, as the adaption of the sed(1)ing
done in r56308 was incorrect. This left us with inconsistent diff(1)
formats in the daily output of periodic(8) ever since, despite in
r56697 having been promised to be revisited. So properly adapt the
password hash filtering to the unified format and turn the later on
again for /etc/{group,master.passwd}, too.
davidcs [Thu, 24 Aug 2017 18:51:55 +0000 (18:51 +0000)]
MFC r322408
Performance enhancements to reduce CPU utililization for large number of
TCP connections (order of tens of thousands), with predominantly Transmits.
gjb [Thu, 24 Aug 2017 13:37:22 +0000 (13:37 +0000)]
MFC r322770, r322796:
r322770:
Apply changes from bin/chmod/tests/chmod_test.sh, adding
atf_expect_fail() before chflags(8) is invoked if the filesystem
is ZFS, which does not support UF_IMMUTABLE.
r322796:
Revert part of r322770 in usr.sbin/chown/tests/chown_test.sh,
which incorrectly adds atf_expect_fail() where there is no
failure case.
ae [Wed, 23 Aug 2017 08:56:18 +0000 (08:56 +0000)]
MFC r322310:
Add to if_enc(4) ability to capture packets via BPF after pfil processing.
New flag 0x4 can be configured in net.enc.[in|out].ipsec_bpf_mask.
When it is set, if_enc(4) additionally captures a packet via BPF after
invoking pfil hook. This may be useful for debugging.
mckusick [Wed, 23 Aug 2017 04:35:03 +0000 (04:35 +0000)]
MFC of 322200, 322201, 322271, and 322297
322200: Remove (broken) search for alternate superblocks
322201: Show differences when alternate superblock fails to match
322271: Cleanup for 322200.
322297: Restore fsck_ffs ability to find alternate superblocks
mckusick [Tue, 22 Aug 2017 15:20:48 +0000 (15:20 +0000)]
MFC of 322179, 322463, and 322464:
322179: Correct ordering of bio's in gjournal queue
322463: Eliminate a variable that is set-only in g_journal.c
322464: Correct check for reads in gjournal
Submitted by: Dr. Andreas Longwitz <longwitz@incore.de>
Discussed with: kib
ken [Tue, 22 Aug 2017 13:59:50 +0000 (13:59 +0000)]
MFC r322410:
------------------------------------------------------------------------
r322410 | ken | 2017-08-11 12:43:52 -0600 (Fri, 11 Aug 2017) | 16 lines
Add historical notes on QIC tape drives and fix a couple of issues in mt(1).
o Density code 0x5 is also known as QIC-11, and should have a footnote
reference.
o Add notes on QIC tape drives from the bug report. These may help anyone
trying to use a QIC drive.
o Take out a "more more" instance found by igor.
o Bump the man page date.
The PR is 14 years old, so it's past time to retire it.
kevans [Tue, 22 Aug 2017 02:03:01 +0000 (02:03 +0000)]
MFC r321450: bsdgrep(1): Don't exit before processing every file
Given an empty pattern (i.e. grep "" A B), bsdgrep(1) would previously
exit() with the appropriate exit code upon encountering an empty file.
Likely intended as an optimization, but this behavior is technically
incorrect since an empty pattern should match every line.
jhb [Mon, 21 Aug 2017 17:35:04 +0000 (17:35 +0000)]
MFC 322437: Reliably enable debug exceptions on all CPUs.
Previously, debug exceptions were only enabled on the boot CPU if
DDB was enabled in the dbg_monitor_init() function. APs also called
this function, but since mp_machdep.c doesn't include opt_ddb.h, the
APs ended up calling an empty stub defined in <machine/debug_monitor.h>
instead of the real function. Also, if DDB was not enabled in the kernel,
the boot CPU would not enable debug exceptions.
Fix this by adding a new dbg_init() function that always clears the OS
lock to enable debug exceptions which the boot CPU and the APs call.
This function also calls dbg_monitor_init() to enable hardware breakpoints
from DDB on all CPUs if DDB is enabled. Eventually base support for
hardware breakpoints/watchpoints will need to move out of the DDB-only
debug_monitor.c for use by userland debuggers.
jhb [Mon, 21 Aug 2017 17:29:37 +0000 (17:29 +0000)]
MFC 322436: Don't panic for PT_GETFPREGS.
Only fetch the VFP state from the CPU if the thread whose registers are
being requested is the current thread. If a stopped thread's registers
are being fetched by a debugger, the saved state in the PCB is already
valid.
ae [Mon, 21 Aug 2017 09:03:20 +0000 (09:03 +0000)]
MFC r321779:
Add inpcb pointer to struct ipsec_ctx_data and pass it to the pfil hook
from enc_hhook().
This should solve the problem when pf is used with if_enc(4) interface,
and outbound packet with existing PCB checked by pf, and this leads to
deadlock due to pf does its own PCB lookup and tries to take rlock when
wlock is already held.
Now we pass PCB pointer if it is known to the pfil hook, this helps to
avoid extra PCB lookup and thus rlock acquiring is not needed.
For inbound packets it is safe to pass NULL, because we do not held any
PCB locks yet.
Move libcasper tests from regression/capsicum/libcasper/ to
lib/libcasper/service/${service_name}/tests.
r305629 (by jkim):
Add new directories added in r305626 to fix "make installworld".
r307863 (by emaste):
Set SHLIBDIR before .including src.opts.mk in libcapser services
bsd.own.mk (included from src.opts.mk) sets SHLIBDIR?=${LIBDIR}, so
SHLIBDIR must be set before including either one of them.
MFC with: 305626
r322447:
Fix result printing
- Flushing stdout prevents the buffer from being printed twice, fixing
issues with stdout printing out the testplan, etc, twice.
- Don't print out raw source/line numbers; hide them behind comments.
r322448:
Make root-privileges a requirement for the test
Some of the testcases try to manipulate sysctls that require root privileges,
e.g., "kern.sync_on_panic". Make root-privileges a hard requirement so the
tests don't raise false positives due to privilege issues when calling
sysctlbyname(3) on writable sysctls.
r322449:
Use hardcoded IPv4/IPv6 addresses for google-public-dns-a.google.com instead
of freefall.freebsd.org to unbreak the DNS tests
The address allocations for freefall.freebsd.org have changed in the past 4 years.
Use a more stable set of hardcoded addresses for now to make the tests succeed
reliably.
The hostname should be resolved dynamically instead of hardcoding the addresses in
the future. This is just a bandaid.
r322450:
Integrate the tests moved in r305626 in to the FreeBSD test suite
The reachover Kyuafiles were never added, and thus the tests were installed
as standalone tests, and not integrated into the full suite.
From the linux tune2fs(8) manpage:
"Allow the kernel to initialize bitmaps and inode tables and keep a high
watermark for the unused inodes in a filesystem, to reduce e2fsck(8) time.
This first e2fsck run after enabling this feature will take the full time,
but subsequent e2fsck runs will take only a fraction of the original time,
depending on how full the file system is."
ext2fs: add read-write support for Extended Attributes and linux ACLs.
Extended attributes and their particular implementation in linux are
different from FreeBSD so in this case we have started diverging from
the UFS EA implementation, which would be the natural reference.
ngie [Sat, 19 Aug 2017 01:40:30 +0000 (01:40 +0000)]
MFC r321954:
Delete comment above "__DEFAULT_DEPENDENT_OPTIONS" related to "meta mode options"
src.conf(5) should document which knobs are which and the dependency between each;
remove the comment so the variable can apply to non-"meta mode options".
ngie [Sat, 19 Aug 2017 01:26:26 +0000 (01:26 +0000)]
MFC r321080:
Expose the ILP32/LP64 programming environments based on
__ILP32__/__LP64__ instead of by architecture.
The list was incomplete (previous commits purged invalid architectures,
like __alpha__, but failed to add new ones). It's best to base the symbol
presence on whether or not the architecture is ILP32 / LP64 capable, per
the compiler.
This fixes the ILP32/LP64 program environments on some architectures like
arm64, and by proxy fixes the tests on those architectures.
Split the interrupt setup code into two parts: allocation and configuration.
Do the allocation before requesting the IOCFacts message. This triggers
the LSI firmware to recognize the multiqueue should be enabled if available.
Multiqueue isn't used by the driver yet, but this also fixes a problem with
the cached IOCFacts not matching latter checks, leading to potential problems
with error recovery.
As a side-effect, fetch the driver tunables as early as possible.
Change from using underbar function names to normal function names for
the informational print functions. Collapse the debug API a bit to be
more generic and not require as much code duplication. While here, fix
a bug in MPS that was already fixed in MPR.
Fix a logic bug in the split PCI interrupt code that slipped through
Reported by: Harry Schmalzbauer
------------------------------------------------------------------------
r322364 | ken | 2017-08-10 08:59:17 -0600 (Thu, 10 Aug 2017) | 39 lines
Changes to make mps(4) and mpr(4) handle reinit with reallocation.
When the mps(4) and mpr(4) drivers need to reinitialize the
firmware, they sometimes need to reallocate all of the memory
allocated by the driver. The reallocation happens whenever the IOC
Facts change. That should only happen after a firmware upgrade.
If the reinitialization happens as a result of a timed out command
sent to the card, the command that timed out and triggered the
reinit may have been freed if iocfacts_allocate() reallocated all
memory. If the caller attempts to access the command after that,
the kernel will panic because the caller will be dereferencing
freed memory.
The solution is to set a flag in the softc when we reallocate,
and avoid dereferencing the command strucure if we've reallocated.
The changes are largely the same in both drivers, since mpr(4) is a
derivative of mps(4).
o In iocfacts_allocate(), if the IOC Facts have changed and we
need to reallocate, set the REALLOCATED flag in the softc.
o Change wait_command() to take a struct mps_command ** instead of
a struct mps_command *. This allows us to NULL out the caller's
command pointer if we have to reinit the controller and the data
structures get reallocated. (The REALLOCATED flag will be set
in the softc if that has happened.)
o In every place that calls wait_command(), make sure we handle
the case where the command is NULL after the call.
o The mpr(4) driver has mpr_request_polled() which can also
reinitialize the card. Also check for reallocation there.
kevans [Thu, 17 Aug 2017 17:17:28 +0000 (17:17 +0000)]
bsdgrep: bump version number to 2.6.0 and update copyright information
MFC r319132: bsdgrep: bump version number and add Kyle Evans copyright
The following changes have been made over the last couple of months:
Features:
- With bsdgrep -r, the working directory is implied if no directory is
specified
- bsdgrep will now behave as bsdgrep -r does when it's named rgrep
- bsdgrep now understands -z/--null-data to use \0 as EOL
- GNU regex compatibility is now indicated with a "GNU compatible" in
the version string
Fixes:
- --mmap no longer hangs when coming across an EOF without an
accompanying EOL
- -o/--color matching generally improved, now produces earliest /
longest matches
- Context output now more closely aligns with GNU grep
- Zero-length matches no longer exhibit broken behavior
- Every output line now honors -b/-H/-n flags
Tests have been added for previous regressions as well as other
previously untested behaviors.
Various other fixes have been commited, and refactoring for further /
later improvements has taken place.
(The original submission changed the version string to 2.5.2, but I
decided to use 2.6.0 to reflect the addition of new features.)
MFC r320754: Update copyright e-mail address to @FreeBSD.org address
kevans [Thu, 17 Aug 2017 04:30:31 +0000 (04:30 +0000)]
MFC r318574: bsdgrep: Correct per-line line metadata printing
Metadata printing with -b, -H, or -n flags suffered from a few flaws:
1) -b/offset printing was broken when used in conjunction with -o
2) With -o, bsdgrep did not print metadata for every match/line, just
the first match of a line
3) There were no tests for this
Address these issues by outputting this data per-match if the -o flag is
specified, and prior to outputting any matches if -o but not --color,
since --color alone will not generate a new line of output for every
iteration over the matches.
To correct -b output, fudge the line offset as we're printing matches.
While here, make sure we're using grep_printline in -A context. Context
printing should *never* look at the parsing context, just the line.
The tests included do not pass with gnugrep in base due to it exhibiting
similar quirky behavior that bsdgrep previously exhibited.
kevans [Thu, 17 Aug 2017 04:26:04 +0000 (04:26 +0000)]
MFC r318571: bsdgrep: emit more than MAX_LINE_MATCHES per line
We should not set an arbitrary cap on the number of matches on a line,
and in any case MAX_LINE_MATCHES of 32 is much too low. Instead, if we
match more than MAX_LINE_MATCHES, keep processing and matching from the
last match until all are found.
For the regression test, we produce 4096 matches (larger than we expect
we'll ever set MAX_LINE_MATCHES) and make sure we actually get 4096
lines of output with the -o flag.
We'll also make sure that every distinct line is getting its own line
number to detect line metadata not being printed as appropriate along
the way.
kevans [Thu, 17 Aug 2017 04:18:31 +0000 (04:18 +0000)]
bsdgrep: fix segfault with --mmap and add relevant test
MFC r318565: bsdgrep: fix segfault with --mmap
r313948 partially fixed --mmap behavior but was incomplete. This commit
generally reverts it and does it the more correct way- by just consuming
the rest of the buffer and moving on.
MFC r318908: bsdgrep: add --mmap tests
Basic sanity tests as well as coverage for the bug fixed in r318565.
Previously, when given a negative -A/-B/-C argument bsdgrep would
overflow the respective context flag(s) and exhibited surprising
behavior.
Fix this by removing unsignedness of Aflag/Bflag and erroring out if
we're given a value < 0. Also adjust the type used to track 'tail'
context in procfile() so that it accurately reflects the Aflag value
rather than overflowing and losing trailing context.
This also fixes an inconsistency previously existing between -n and
-C "n" behavior. They are now both limited to LLONG_MAX, to be
consistent.
Add some test cases to make sure grep errors out properly for both
negative context values as well as non-numeric context values rather
than giving bogus matches.
MFC r318317: bsdgrep: add more tests for different binary flags
The existing 'binary' test in netbsd-tests/ does a basic check of the
default treatment for binary behavior, but not much more than that.
Given some opportunity for breakage recently that did not trigger any
failures, add some tests to cover the three different binary file
behaviors (a, -I, -U) and their --binary-files= equivalent values.
sephe [Thu, 17 Aug 2017 03:48:50 +0000 (03:48 +0000)]
MFC 322483,322485-322487
322483
hyperv/hn: Update VF's ibytes properly under transparent VF mode.
While, I'm here add comment about why updating VF's imcast stat is
not necessary.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D11948
322485
hyperv/hn: Fix/enhance receiving path when VF is activated.
- Update hn(4)'s stats properly for non-transparent mode VF.
- Allow BPF tapping to hn(4) for non-transparent mode VF.
- Don't setup mbuf hash, if 'options RSS' is set.
In Azure, when VF is activated, TCP SYN and SYN|ACK go through hn(4)
while the rest of segments and ACKs belonging to the same TCP 4-tuple
go through the VF. So don't setup mbuf hash, if a VF is activated
and 'options RSS' is not enabled. hn(4) and the VF may use neither
the same RSS hash key nor the same RSS hash function, so the hash
value for packets belonging to the same flow could be different!
- Disable LRO.
hn(4) will only receive broadcast packets, multicast packets, TCP
SYN and SYN|ACK (in Azure), LRO is useless for these packet types.
For non-transparent, we definitely _cannot_ enable LRO at all, since
the LRO flush will use hn(4) as the receiving interface; i.e.
hn_ifp->if_input(hn_ifp, m).
While I'm here, remove unapplied comment and minor style change.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D11978
322486
hyperv/hn: Minor cleanup
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D11979
322487
hyperv/hn: Re-set datapath after synthetic parts reattached.
Do this even for non-transparent mode VF. Better safe than sorry.
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D11981
kp [Wed, 16 Aug 2017 19:52:31 +0000 (19:52 +0000)]
MFC r322280:
pf_get_sport(): Prevent possible endless loop when searching for an unused nat port
This is an import of Alexander Bluhm's OpenBSD commit r1.60,
the first chunk had to be modified because on OpenBSD the
'cut' declaration is located elsewhere.
Upstream report by Jingmin Zhou:
https://marc.info/?l=openbsd-pf&m=150020133510896&w=2
OpenBSD commit message:
Use a 32 bit variable to detect integer overflow when searching for
an unused nat port. Prevents a possible endless loop if high port
is 65535 or low port is 0.
report and analysis Jingmin Zhou; OK sashan@ visa@
Quoted from: https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/net/pf_lb.c
kevans [Wed, 16 Aug 2017 18:00:32 +0000 (18:00 +0000)]
bsdgrep: fix -w flag matching with an empty pattern
MFC r317703: bsdgrep: fix -w flag matching with an empty pattern
-w flag matching with an empty pattern was generally 'broken', allowing
matches to occur on any line whether or not it actually matches -w
criteria.
This fix required a good amount of refactoring to address. procline()
is altered to *only* process the line and return whether it was a match
or not, necessary to be able to short-circuit the whole function in case
of this matchall flag. -m flag handling is moved out as well because it
suffers from the same fate as context handling if we bypass any actual
pattern matching.
The matching context (matches, mostly) didn't previously exist outside
of procline(), so we go ahead and create context object for file
processing bits to pass around. grep_printline() was created due to
this, for the scenarios where the matches don't actually matter and we
just want to print a line or two, a la flushing the context queue and
no -o or --color specified.
Damage from this broken behavior would have been mitigated by the fact
that it is unlikely users would invoke grep -w with an empty pattern.
This was identified while checking PR 105221 for problems it this may
cause in BSD grep, but PR 105221 is *not* a report of this behavior.
MFC r317741: bsdgrep: correct uninitialized variable introduced in r317703
MFC r317842: bsdgrep: don't ouptut matches with -c, -l, -L
Refactoring done in r317703 broke -c, -l, and -L flags implying
suppression of match printing. Fortunately this is just a matter of not
doing any printing of the resulting matches and context printing was not
broken in this refactoring.
Add some regression tests since this area may still see further
refactoring, include different context flags as well even though they
were not broken in this case.
kevans [Wed, 16 Aug 2017 17:54:29 +0000 (17:54 +0000)]
bsdgrep: fix escape map building when using TRE (BSD_GREP_FASTMATCH)
MFC r317700: bsdgrep: use calloc where appropriate in grep's tre-fastmatch
Also apply style(9) to a related NULL check.
MFC r317701: bsdgrep: correct test sense from r317700
Kyle's change in review D10098 was correct. I introduced the error when
extracting a portion of that change.
MFC r317704: bsdgrep: fix escape map building for multibyte strings
In BSD grep, fix escape map building in the regex parser. It was
previously using memory not explicitly initialized, and the MBS escape
map was being built based on a version of the pattern with escapes
already parsed out.
This is Kyle's change, but I restored the broken style that already
exists in this file.
kevans [Wed, 16 Aug 2017 17:42:39 +0000 (17:42 +0000)]
MFC r317665: bsdgrep: fix -w -v matching improperly with certain patterns
-w and -v flag matching was mostly functional but had some minor
problems:
1. -w flag processing only allowed one iteration through pattern
matching on a line. This was problematic if one pattern could match
more than once, or if there were multiple patterns and the earliest/
longest match was not the most ideal, and
2. Previous work "fixed" things to not further process a line if the
first iteration through patterns produced no matches. This is clearly
wrong if we're dealing with the more restrictive -w matching.
#2 breakage could have also occurred before recent broad rewrites, but
it would be more arbitrary based on input patterns as to whether or not
it actually affected things.
Fix both of these by forcing a retry of the patterns after advancing
just past the start of the first match if we're doing more restrictive
-w matching and we didn't get any hits to start with. Also move -v flag
processing outside of the loop so that we have a greater change to match
in the more restrictive cases. This wasn't strictly wrong, but it could
be a little more error prone.
While here, introduce some regressions tests for this behavior and fix
some excessive wrapping nearby that hindered readability. GNU grep
passes these new tests.
kevans [Wed, 16 Aug 2017 17:38:37 +0000 (17:38 +0000)]
MFC r317254: bsdgrep: add BSD_GREP_FASTMATCH knob for built-in fastmatch
Bugs have been found in the fastmatch implementation as used in bsdgrep.
Some have been fixed (r316495) while fixes for others are in review
(D10098).
In comparison with the fastmatch implementation, Kyle Evans found that:
- regex(3)'s performance with literal expressions offers a speed
improvement over fastmatch
- regex(3)'s performance, both with simple BREs and EREs, seems to be
comparable
The regex implementation was imported in r226035, and the commit message
reports:
This is a temporary solution until the whole regex library is
not replaced so that BSD grep development can continue and the
backported code gets some review and testing. This change only
improves scalability slightly, there is no big performance boost
yet but several minor bugs have been found and fixed.
Introduce a WITH_/WITHOUT_BSD_GREP_FASTMATCH knob to support testing
of both approaches.
kevans [Wed, 16 Aug 2017 13:06:26 +0000 (13:06 +0000)]
MFC r303444 (ed): Call basename() in a portable way.
Pull a copy of the filename string before calling basename(). Change the
loop to not return on its own, so we can put a free() statement at the
bottom.
ae [Wed, 16 Aug 2017 12:01:22 +0000 (12:01 +0000)]
MFC r322328:
Make user supplied data checks a bit stricter.
key_msg2sp() is used for parsing data from setsockopt(IP[V6]_IPSEC_POLICY)
call. This socket option is usually used to configure IPsec bypass for
socket. Only privileged user can set this socket option.
The message syntax is described here
http://www.kame.net/newsletter/20021210/
and our libipsec is usually used to create the correct request.
Add additional checks:
* that sadb_x_ipsecrequest_len is not out of bounds of user supplied buffer
* that src/dst's sa_len is the same
* that 2*sa_len is not out of bounds of user supplied buffer
* that 2*sa_len fits into bounds of sadb_x_ipsecrequest
truckman [Wed, 16 Aug 2017 07:59:57 +0000 (07:59 +0000)]
MFC r321899
Lower the amd64 shared page, which contains the signal trampoline,
from the top of user memory to one page lower on machines with the
Ryzen (AMD Family 17h) CPU. This pushes ps_strings and the stack
down by one page as well. On Ryzen there is some sort of interaction
between code running at the top of user memory address space and
interrupts that can cause FreeBSD to either hang or silently reset.
This sounds similar to the problem found with DragonFly BSD that
was fixed with this commit:
https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/b48dd28447fc8ef62fbc963accd301557fd9ac20
but our signal trampoline location was already lower than the address
that DragonFly moved their signal trampoline to. It also does not
appear to be related to SMT as described here:
https://www.phoronix.com/forums/forum/hardware/processors-memory/955368-some-ryzen-linux-users-are-facing-issues-with-heavy-compilation-loads?p=955498#post955498
"Hi, Matt Dillon here. Yes, I did find what I believe to be a
hardware issue with Ryzen related to concurrent operations. In a
nutshell, for any given hyperthread pair, if one hyperthread is
in a cpu-bound loop of any kind (can be in user mode), and the
other hyperthread is returning from an interrupt via IRETQ, the
hyperthread issuing the IRETQ can stall indefinitely until the
other hyperthread with the cpu-bound loop pauses (aka HLT until
next interrupt). After this situation occurs, the system appears
to destabilize. The situation does not occur if the cpu-bound
loop is on a different core than the core doing the IRETQ. The
%rip the IRETQ returns to (e.g. userland %rip address) matters a
*LOT*. The problem occurs more often with high %rip addresses
such as near the top of the user stack, which is where DragonFly's
signal trampoline traditionally resides. So a user program taking
a signal on one thread while another thread is cpu-bound can cause
this behavior. Changing the location of the signal trampoline
makes it more difficult to reproduce the problem. I have not
been because the able to completely mitigate it. When a cpu-thread
stalls in this manner it appears to stall INSIDE the microcode
for IRETQ. It doesn't make it to the return pc, and the cpu thread
cannot take any IPIs or other hardware interrupts while in this
state."
since the system instability has been observed on FreeBSD with SMT
disabled. Interrupts to appear to play a factor since running a
signal-intensive process on the first CPU core, which handles most
of the interrupts on my machine, is far more likely to trigger the
problem than running such a process on any other core.
Also lower sv_maxuser to prevent a malicious user from using mmap()
to load and execute code in the top page of user memory that was made
available when the shared page was moved down.
Make the same changes to the 64-bit Linux emulator.
kevans [Wed, 16 Aug 2017 01:45:53 +0000 (01:45 +0000)]
bsdgrep: Use implied working directory for -r if no directories are passed
MFC r317050: bsdgrep: for -r, use the working directory if none specified
This is more sensible than the previous behaviour of grepping stdin,
and matches newer GNU grep behaviour.
MFC r317300 (ngie): Only expect :grep_r_implied to pass with bsdgrep(1)
The test fails with gnu grep from base and ports.
MFC r319002 (ngie): :rgrep : use atf-check to check the exit code/save the
output of grep -r instead of calling grep -r without it, and saving the
output to a file
This ensures that any errors thrown via grep -r are caught, not lost, and
uses existing atf-sh idioms for saving files.
kevans [Wed, 16 Aug 2017 01:03:04 +0000 (01:03 +0000)]
MFC r317051: bsdgrep: remove output separators between overlapping segments
Make bsdgrep more sensitive to context overlaps. If it's printing
context that either overlaps or is immediately adjacent to another bit
of context, don't print a separator.
- Non-overlapping segments no longer have two separators between them
- Overlapping segments no longer have separators between them with
overlapping sections repeated
kevans [Wed, 16 Aug 2017 00:55:56 +0000 (00:55 +0000)]
bsdgrep: Revise tests based on recent fixes and future changes
MFC r317297: Remove the expected failures for :context and :context2 with
bsdgrep(1)
They're no longer needed after recent fixes made to bsdgrep(1).
MFC r317299: Add more sanity tests for grep, egrep, and fgrep
The test suite currently lacks basic sanity checks to ensure that egrep,
fgrep, and grep are actually matching the right expression types, i.e.
passing the right flags to regcomp(3). Amend the test suite to make sure
that not only are the individual versions doing the right thing, but also
that we don't have some kind of frankenregex situation happening where egrep
is accepting a BRE or grep an ERE.
I've chosen to not expand the 'basic' test but to add the 'grep_sanity'
checks to their own test case since this is testing for more than just
'grep matches things', but actual expression types.
MFC r317694: bsdgrep: revise test case which will soon become a failure
Work in progress (D10315) is going to make egrep_empty_invalid an
actually invalid regex, to be consistent with the equivalent BRE "{"
behavior, when using regex(3).
Any non-0 exit value is acceptable, depending on how the installed grep
interprets the expression. GNU grep interprets it as non-matching, and
in the future BSD grep will interpret it is an error.
kevans [Wed, 16 Aug 2017 00:47:53 +0000 (00:47 +0000)]
bsdgrep: add -z/--null-data support and update NLS catalogs accordingly
MFC r317049: bsdgrep: add -z/--null-data support
-z treats input and output data as sequences of lines terminated by a
zero byte instead of a newline. This brings it more in line with GNU grep
and brings us closer to passing the current tests with BSD grep.
MFC r317679: bsdgrep: correct nls usage data after r317049
r317049 added -z/--null-data to BSD grep but missed the update to nls
catalogs.
kevans [Wed, 16 Aug 2017 00:40:13 +0000 (00:40 +0000)]
MFC r316495: bsdgrep(1): Fix errors with invalid expressions
Invalid expressions with an ultimate compiled pattern length of 0 (e.g.,
"grep -E {") were not taken into account and caused a segfault while trying
to fill in the good suffix table.
projects / FreeBSD / FreeBSD.git / log