Ian Lepore [Sun, 22 Jul 2018 16:42:22 +0000 (16:42 +0000)]
Remove the .if ${MK_FOO} wrappers around the user/group ID checks. These
names are referenced in mtree files without any conditional logic, so the
users/groups must exist even if the corresponding tool(s) are disabled.
Ben Woods [Sun, 22 Jul 2018 13:40:52 +0000 (13:40 +0000)]
geli attach: Fix exit codes and errors not being printed after r335673
Now that multiple providers can be attached at once, exit codes and
error messages must be handled correctly if there are failures in on
any of the providers.
Reported by: asomers (Kyua test failures via continuous integration)
Reviewed by: asomers
Approved by: allanjude
Differential Revision: https://reviews.freebsd.org/D16386
release: Add arm_install_boot to install the commit boot bits
This reduce the per-board arm_install_uboot to just install u-boot.
While here remove the installation of rpi.dtb and rpi2.dtb as we load
them from the UFS partition via ubldr.
Michael Tuexen [Sun, 22 Jul 2018 10:50:59 +0000 (10:50 +0000)]
Improve TCP related tests for dtrace.
Ensure that the TCP connections are terminated gracefully as expected
by the test. Use appropriate numbers for sent/received packets.
In addition, enable tst.localtcpstate.ksh, which should pass, but
doesn't until https://reviews.freebsd.org/D16369 is committed.
Reviewed by: markj@
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D16288
Matt Macy [Sun, 22 Jul 2018 05:37:58 +0000 (05:37 +0000)]
NULL out cc_data in pluggable TCP {cc}_cb_destroy
When ABE was added (rS331214) to NewReno and leak fixed (rS333699) , it now has
a destructor (newreno_cb_destroy) for per connection state. Other congestion
controls may allocate and free cc_data on entry and exit, but the field is
never explicitly NULLed if moving back to NewReno which only internally
allocates stateful data (no entry contstructor) resulting in a situation where
newreno_cb_destory might be called on a junk pointer.
- NULL out cc_data in the framework after calling {cc}_cb_destroy
- free(9) checks for NULL so there is no need to perform not NULL checks
before calling free.
- Improve a comment about NewReno in tcp_ccalgounload
This is the result of a debugging session from Jason Wolfe, Jason Eggleston,
and mmacy@ and very helpful insight from lstewart@.
Alan Somers [Sat, 21 Jul 2018 23:54:40 +0000 (23:54 +0000)]
Fix tmpfs detection in the sys/fs/tmpfs tests
This code was originally written for NetBSD. r306031 tried to adapt it to
FreeBSD, but didn't correctly handle the case that tmpfs was available, but
not already loaded. Fix the logic to load the module if necessary. The
tmpfs tests shouldn't be skipped anymore.
Also, fix a comment that was dislocated by r306031.
Colin Percival [Sat, 21 Jul 2018 22:54:43 +0000 (22:54 +0000)]
Use svn or svnlite, or ${SVN_CMD} if set, for extracting the SVN branch
and revision number announced in SNS notifications about new EC2 AMIs.
While I'm here, incorporate that information into the AMI "description"
fields, since it's more useful than simply echoing the information
already provided via the AMI "name".
Alexander Motin [Sat, 21 Jul 2018 21:34:10 +0000 (21:34 +0000)]
Stop further SCSI recovery attempts after one has failed.
We've got a set of probably damaged hard disks, reporting 0x04,0x02
("Logical unit not ready, initializing command required") in response
to READ CAPACITY(16), where attempts to use START STOP UNIT for recovery
results in 0x44,0x00 ("Internal target failure") after ~1 second delay.
As result of all recovery retries, device open attempt took ~3 seconds
before finally reporting to GEOM that device is opened, but has no media.
If the open was for writing and since it hasn't formally failed, following
close triggered GEOM retaste, opening device few more times with respective
delays.
This change reduces whole time of this cycle from ~12 seconds to ~3 by
giving up on recovery after the first failure.
Reviewed by: ken
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Alan Somers [Sat, 21 Jul 2018 20:14:01 +0000 (20:14 +0000)]
Fix sys.fs.tmpfs.mknod_test.{char, block} by reverting r321967
In r321967 ngie "fixed" these tests by changing their expectations to match
the device numbers produced by the new ino64 code. But it wasn't the tests
that were broken, it was the kernel. bde fixed the kernel in r335053.
Reported by: Jenkins
MFC after: Never (only applies to >= 12)
Alan Somers [Sat, 21 Jul 2018 19:48:31 +0000 (19:48 +0000)]
tftpd(8): when completing an WRQ, flush the file before acknowleding receipt
tftpd(8) should flush a newly written file to disk before ACKing the final DATA
packet. Otherwise there is a narrow race window when a subsequent read may not
see the file. This is somewhat related to r330710, but the race window is much
smaller. Hopefully this will fix the intermittent tests in Jenkins.
Use the correct DOT language syntax for an ID with a leading digit (0mp).
It turns out that my username (0mp) is not a valid ID in the DOT language
because it starts with a digit. The easiest way to fix it is to put my
username in double quotes.
Alan Somers [Sat, 21 Jul 2018 17:24:14 +0000 (17:24 +0000)]
makefs(8): add test case for PR 229929
Fix two failing makefs test cases by adding "-M 1m", which was already used
for every other FFS test case. Add a new test case for the underlying
issue: with no -M, -m, or -s options, makefs can underestimate image size.
Alan Somers [Sat, 21 Jul 2018 16:26:00 +0000 (16:26 +0000)]
audit(4): add more test cases for auditon(2)
auditon(2) is an ioctl-like syscall with several different variants, each of
which has a distinct audit event. This commit tests the remaining variants
that weren't tested in r336564.
Rick Macklem [Sat, 21 Jul 2018 01:33:07 +0000 (01:33 +0000)]
Set CLSET_TIMEOUT on TCP connections to pNFS DSs.
Use CLSET_TIMEOUT to set the timeout for connections to DSs instead of
specifying a timeout on each RPC. This is done so that SO_SNDTIMEO
is set on the TCP socket as well as specifying a time limit when
waiting for an RPC reply. Useful if the send queue for the TCP
connection has become constipated, due to a failed DS.
The choice of lease_duration / 4 is fairly arbitrary, but seems to work
ok, with a lower bound of 10sec.
For client connections to a DS, set the retry limit to vfs.nfsd.dsretries,
which is 2 by default.
This patch should only affect pNFS connections to DSs.
This patch requires r336542.
- Use "Fl -" instead of "Cm --" for long options.
- Sort options alphabetically.
- Pet "mandoc -Tlint".
- Clean up the description of the "--interpreter" option.
- Clean up the description of the first example in the examples section.
- Use ".Bd -literal -offset indent" for all example code blocks for consistency.
- Use "Nm" instead of "Cm binmiscctl".
- Indent all examples for consistency.
Reviewed by: allanjude
Approved by: mat (mentor)
Differential Revision: https://reviews.freebsd.org/D15589
Alan Somers [Fri, 20 Jul 2018 18:59:48 +0000 (18:59 +0000)]
Separate the audit(4) tests for auditon(2)'s individual commands
auditon(2) is an ioctl-like syscall with several different variants, each of
which has a distinct audit event. Write separate audit(4) tests for each
variant.
Recommit r336497: Fix powl, cpow, cpowf, and cpowl imports from OpenBSD
This is a follow-up to r336299.
* lib/msun/Makefile:
. Remove polevll.c
* lib/msun/ld80/e_powl.c:
. Copy contents of polevll.c to here. This is the only consumer of
these functions. Make functions 'static inline'.
. Make reducl a 'static inline' function.
* lib/msun/man/exp.3:
. Remove BUGS section that no longer applies.
* lib/msun/src/math_private.h:
. Remove prototypes of __p1evll() and __polevll()
* lib/msun/src/s_cpow.c:
* lib/msun/src/s_cpowf.c:
* lib/msun/src/s_cpowl.c
. Include math_private.h.
. Use the CMPLX macro from either C99 or math_private.h (depends on
compiler support) instead of the problematic use of complex I.
Stephen Hurd [Fri, 20 Jul 2018 17:45:26 +0000 (17:45 +0000)]
Add knob to control tx ring abdication.
r323954 changed the mp ring behaviour when 64-bit atomics were
available to abdicate the TX ring rather than having one become a
consumer thereby running to completion on TX. The consumer of the mp
ring was then triggered in the tx task rather than blocking the TX call.
While this significantly lowered the number of RX drops in small-packet
forwarding, it also negatively impacts TX performance.
With this change, the default behaviour is reverted, causing one TX ring
to become a consumer during the enqueue call. A new sysctl,
dev.X.Y.iflib.tx_abdicate is added to control this behaviour.
Stephen Hurd [Fri, 20 Jul 2018 17:24:45 +0000 (17:24 +0000)]
Improve netmap TX handling when TX IRQs are not used/supported
Use the timer to poll for TX completions when there are
outstanding TX slots. Track when the last driver timer was called
to prevent overcalling it. Also clean up some kring vs NIC ring
usage.
vt/ofwfb: Check that /chosen/stdout is valid before using it in initialization
The FDT implementation of OF_instance_to_package() backend checks the
cross-reference to get the node. On failure, this returns the input handle
unchanged. In the case of ofwfb attachment, if /chosen/stdout property does not
exist, sc->sc_handle is either garbage or 0, which then gets propagated to node.
This will prevent "screen" from being used, resulting in not properly attaching.
Correct this by matching the code in ofwfb_probe().
Ian Lepore [Fri, 20 Jul 2018 16:06:44 +0000 (16:06 +0000)]
Apply some late-arriving markup suggestions from the phab review, and add a
paragraph that mentions the possibility of starting ntpd as a non-root user
rather than starting it as root and using its '-u' option to drop root privs
after startup.
Michael Tuexen [Fri, 20 Jul 2018 15:32:20 +0000 (15:32 +0000)]
Add missing dtrace probes for received UDP packets.
Fire UDP receive probes when a packet is received and there is no
endpoint consuming it. Fire the probe also if the TTL of the
received packet is smaller than the minimum required by the endpoint.
Clarify also in the man page, when the probe fires.
Ian Lepore [Fri, 20 Jul 2018 14:23:01 +0000 (14:23 +0000)]
Support installing manpages for multiple or alternate architecture(s).
Some section-4 manpages are architecture-specific, and the build process
currently generates only the pages for the MACHINE_CPUARCH being built.
man(1) supports a '-m' option to find manpages belonging to an arbitrary
architecture other than the MACHINE_[CPU]ARCH, but we have no way to
generate and install alternate-arch pages right now.
This change adds a new make.conf variable, MAN_ARCH, which can be a list of
one or more MACHINE_ARCH or MACHINE_CPUARCH values. All arch-specific
manpages that exist for the named arches will be installed. If unset, it
continues the behavior of installing just the MACHINE_CPUARCH being built.
Ian Lepore [Fri, 20 Jul 2018 13:59:29 +0000 (13:59 +0000)]
Automatically run ntpd as non-root when possible.
Ntpd needs only a subset of full root privileges to do its job. Specifically
it needs the ability to manipulate system time, and to re-bind to a
privileged UDP port after interface changes. The mac_ntpd(4) policy module
(see r336525) can grant these privs.
These changes detect the availability of mac_ntpd(4). If enabled, and if the
ntpd configuration is fairly vanilla, it automatically runs ntpd as the
non-root user 'ntpd' (uid 123). "Vanilla" means the config doesn't include
command line or ntp.conf options changing the location of files or using any
files/dirs likely to be inaccessible to user ntpd. Ntpd can still run as
non-root when using such options, but the admin must ensure all required
files and dirs are accessible, and then set ntpd_user=ntpd in rc.conf.
Note that these changes also address PR 199127 by using the command_args
technique suggested in the patch. They also tangentially address PR 113552,
which is primarily about inconsistent filenames in documentation, but some
of the inconsistancy was caused by old code in rc.d/ntpd which is leftover
from the intial import from netbsd. There was code to do chroot setup which
required the use of the netbsd clockctl(4) device; that code never had any
effect on freebsd, because we lack that device and don't build ntpd with the
options that would allow using it.
Bruce Evans [Fri, 20 Jul 2018 12:42:24 +0000 (12:42 +0000)]
Centralize the complications for special efficient rounding to integers.
This was open-coded in range reduction for trig and exp functions. Now
there are 3 static inline functions rnint[fl]() that replace open-coded
expressions, and type-generic irint() and i64rint() macros that hide the
complications for efficiently using non-generic irint() and irintl()
functions and casts.
Special details:
ld128/e_rem_pio2l.h needs to use i64rint() since it needs a 46-bit integer
result. Everything else only needs a (less than) 32-bit integer result so
uses irint().
Float and double cases now use float_t and double_t locally instead of
STRICT_ASSIGN() to avoid bugs in extra precision.
On amd64, inline asm is now only used for irint() on long doubles. The SSE
asm for irint() on amd64 only existed because the ifdef tangles made the
correct method of simply casting to int for this case non-obvious.
Rick Macklem [Fri, 20 Jul 2018 12:03:16 +0000 (12:03 +0000)]
Set SO_SNDTIMEO in the client side krpc when CLSET_TIMEOUT is done.
During testing of the pNFS client, it was observed that an RPC could get
stuck in sosend() for a very long time if the network connection to a DS
had failed. This is fixed by setting SO_SNDTIMEO on the TCP socket.
This is only done when CLSET_TIMEOUT is done and this is not done by any
use of the krpc currently in the source tree, so there should be no effect
on extant uses.
A future patch will use CLSET_TIMEOUT for TCP connections to DSs.
Warner Losh [Fri, 20 Jul 2018 05:34:02 +0000 (05:34 +0000)]
Bump version to 1200075 for zfsloader being folded into loader.
As an aside: 1200074 should be used as the last version with big
endian arm support, should that be needed. it was actually removed
a day later, but no bump was made until now.
Warner Losh [Fri, 20 Jul 2018 05:17:37 +0000 (05:17 +0000)]
Collapse zfsloader functionality back down into loader.
We no longer really need a separate zfsloader. It was useful when we
were first supporting ZFS and had limited ability to properly boot off
of ZFS without the special boot loader. Now that the boot loader has
matured, go the way loader.efi pioneered and just build one
binary. Change the name of the loader to load in the secondary boot
blocks to be just /boot/loader. Provide a symbolic link from zfsloader
to loader so people who have not upgraded their boot blocks are not
affected. This has the happy benefit of making coexistence easier as
well (fewer binaries in the matrix).
cryptocheck: Correct mismatch between OpenSSL use and OCF Blake2
This corrects a mistake introduced to the cryptocheck tool in r331418.
Our CRYPTO_BLAKE2B and CRYPTO_BLAKE2S algorithms refer to either the plain,
unkeyed hashes (specified with cri_klen = 0), or a Blake2-specific keyed MAC
(when a cri_key is provided).
In contrast, OpenSSL's Blake2 algorithms only provide the plain hash.
Cryptocheck's T_HMAC corresponds to OpenSSL's HMAC() routine, which is the
ordinary HMAC construction applied to any plain, unkeyed hash. We don't
have any HMAC-Blake2 cipher modes in OCF, so fix the test to only test
Blake2 as a plain hash.
(Ideally we would test keyed Blake2 as well, but that is left as future
work.)
Ian Lepore [Thu, 19 Jul 2018 23:55:29 +0000 (23:55 +0000)]
Make it possible to run ntpd as a non-root user, add ntpd uid and gid.
Code analysis and runtime analysis using truss(8) indicate that the only
privileged operations performed by ntpd are adjusting system time, and
(re-)binding to privileged UDP port 123. These changes add a new mac(4)
policy module, mac_ntpd(4), which grants just those privileges to any
process running with uid 123.
This also adds a new user and group, ntpd:ntpd, (uid:gid 123:123), and makes
them the owner of the /var/db/ntp directory, so that it can be used as a
location where the non-privileged daemon can write files such as the
driftfile, and any optional logfile or stats files.
Because there are so many ways to configure ntpd, the question of how to
configure it to run without root privs can be a bit complex, so that will be
addressed in a separate commit. These changes are just what's required to
grant the limited subset of privs to ntpd, and the small change to ntpd to
prevent it from exiting with an error if running as non-root.
I had naively assumed that building kernel would be sufficient to test that
the header is sane. However, it turns out this now needs -fms-extensions to
build. Rather than sprinkling -fms-extensions all over the place, revert
for now, and revisit with a better fix.
Merge the md_page structs for AIM and Book-E into a single unioned struct
Summary:
Ports like sysutils/lsof troll through kernel structures, and
therefore include kernel headers and all the dirty secrets involved. struct
vm_page includes the struct md_page inline, which currently is only defined
if AIM or BOOKE is defined. Thus, by default, sysutils/lsof cannot build,
due to the struct md_page having an incomplete type. Fix this by merging
the two struct definitions into an anonymous struct-union.
A similar change could be made to unify the pmap structures as well.
Mark Johnston [Thu, 19 Jul 2018 20:00:28 +0000 (20:00 +0000)]
Have preload_delete_name() free pages backing preloaded data.
On i386 and amd64, add a vm_phys segment for physical memory used to
store the kernel binary and other preloaded data. This makes it
possible to free such memory back to the system once it is no longer
needed, e.g., when a preloaded kernel module is unloaded. Previously,
it would have remained unused.
Reviewed by: kib, royger
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16330
Revert r336497 for now, as it breaks on architectures using gcc, with:
cc1: warnings being treated as errors
/usr/src/lib/msun/src/s_cpow.c: In function 'cpow':
/usr/src/lib/msun/src/s_cpow.c:63: warning: implicit declaration of function 'CMPLX'
Fix powl, cpow, cpowf, and cpowl imports from OpenBSD
This is a follow-up to r336299.
* lib/msun/Makefile:
. Remove polevll.c
* lib/msun/ld80/e_powl.c:
. Copy contents of polevll.c to here. This is the only consumer of
these functions. Make functions 'static inline'.
. Make reducl a 'static inline' function.
* lib/msun/man/exp.3:
. Remove BUGS section that no longer applies.
* lib/msun/src/math_private.h:
. Remove prototypes of __p1evll() and __polevll()
* lib/msun/src/s_cpow.c:
* lib/msun/src/s_cpowf.c:
* lib/msun/src/s_cpowl.c
. Use the CMPLX macro from either C99 or math_private.h (depends of
compiler support) instead of the problematic use of complex I.
Alan Cox [Thu, 19 Jul 2018 17:01:10 +0000 (17:01 +0000)]
Revert r329254. The underlying cause for the copy-on-write problem in
multithreaded programs that was addressed by r329254 was in the
implementation of pmap_enter() on some architectures, notably, amd64.
kib@, markj@ and I have audited all of the pmap_enter() implementations,
and fixed the broken ones, specifically, amd64 (r335784, r335971), i386
(r336092), mips (r336248), and riscv (r336294).
To be clear, the reason to address the problem within pmap_enter() and
revert r329254 is not just a matter of principle. An effect of r329254
was that a copy-on-write fault actually entailed two page faults, not
one, even for single-threaded programs. Now, in the expected case for
either single- or multithreaded programs, we are back to a single page
fault to complete a copy-on-write operation. (In extremely rare
circumstances, a multithreaded program could suffer two page faults.)
projects / FreeBSD / FreeBSD.git / log