Rick Macklem [Mon, 3 Jun 2019 22:58:51 +0000 (22:58 +0000)]
Modify mountd so that it incrementally updates the kernel exports upon a reload.
Without this patch, mountd would delete/load all exports from the exports
file(s) when it receives a SIGHUP. This works fine for small exports file(s),
but can take several seconds to do when there are large numbers (10000+) of
exported file systems. Most of this time is spent doing the system calls
that delete/export each of these file systems. When the "-S" option
has been specified (the default these days), the nfsd threads are suspended
for several seconds while the reload is done.
This patch changes mountd so that it only does system calls for file systems
where the exports have been changed/added/deleted as compared to the exports
done for the previous load/reload of the exports file(s).
Basically, when SIGHUP is posted to mountd, it saves the exportlist structures
from the previous load and creates a new set of structures from the current
exports file(s). Then it compares the current with the previous and only does
system calls for cases that have been changed/added/deleted.
The nfsd threads do not need to be suspended until the comparison step is
being done. This results in a suspension period of milliseconds for a server
with 10000+ exported file systems.
There is some code using a LOGDEBUG() macro that allow runtime debugging
output via syslog(LOG_DEBUG,...) that can be enabled by creating a file
called /var/log/mountd.debug. This code is expected to be replaced with
code that uses dtrace by cy@ in the near future, once issues w.r.t. dtrace
in stable/12 have been resolved.
The patch should not change the usage of the exports file(s), but improves
the performance of reloading large exports file(s) where there are only a
small number of changes done to the file(s).
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Pavel Zakharov <pavel.zakharov@delphix.com>
This is irrelevant to FreeBSD, just to reduce divergence.
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: George Wilson <george.wilson@delphix.com>
Reviewed by: Jason King <jason.king@joyent.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Approved by: Joshua M. Clulow <josh@sysmgr.org>
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Warner Losh [Mon, 3 Jun 2019 19:10:46 +0000 (19:10 +0000)]
[zfsboot] Fix boot env back compat (#190)
* Fix boot env back compat
zfsboot must try zfsloader before loader in order to remain compatible
with boot environments created prior to zfs functionality being rolled
into loader proper.
* Improve comments in zfsboot
Explain the significance of the load path order, and put the comment
about looping through the paths in the appropriate scope.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Allan Jude <allanjude@freebsd.org>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Andrew Stormont <astormont@racktopsystems.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Alexander Motin <mav@FreeBSD.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Allan Jude <allanjude@freebsd.org>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
John Baldwin [Mon, 3 Jun 2019 15:41:54 +0000 (15:41 +0000)]
Add 'device cxgbe' explicitly in the synopsis.
ccr depends on symbols exported by the cxgbe driver as well as having
a runtime dependency. While the runtime depenency was noted in the
manpage already, the compile-time dependency wasn't as clear.
PR: 238265
MFC after: 3 days
Sponsored by: Chelsio Communications
amd64 ef_rt_arch_call: Preserve %rflags around call into EFI RT service.
If service code faulted, we might end up unwinding with interrupts
disabled. Top-level kernel code should have interrupts enabled, which
is enforced by checks.
Save %rflags before entering EFI, and restore to the known good value
on return. This handles situation with disabled interrupts on fault
and perhaps other potential bugs, e.g. invalid value for PSL_D.
Reported and tested by: Jan Martin Mikkelsen <janm@transactionware.com>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Andrew Stormont <astormont@racktopsystems.com>
Maxim Sobolev [Mon, 3 Jun 2019 15:12:44 +0000 (15:12 +0000)]
Leave mtree hardcoded for now. Reverting partially 348521 and also
the followup stopgap change, because I don't think it's a correct. I still
need to figure out where to stick it in. In cannot be in Makefile.inc1
and it cannot be in etc/Makefile from the looks of it to avoid
chicken-and-egg problem.
psm(4): Add natural scrolling support to sysmouse protocol
This change enables natural scrolling with two finger scroll enabled
and when user is using a trackpad (mouse and trackpoint are not affected).
Depending on trackpad model it can be activated with setting of
hw.psm.synaptics.natural_scroll or hw.psm.elantech.natural_scroll sysctl
values to 1.
Evdev protocol is not affected by this change too. Tune userland client
e.g. libinput to enable natural scrolling in that case.
Warner Losh [Mon, 3 Jun 2019 05:25:22 +0000 (05:25 +0000)]
Another partial revert of r301289.
In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.
Warner Losh [Mon, 3 Jun 2019 05:25:16 +0000 (05:25 +0000)]
Fix mismerge that crept into r301289.
The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.
Alan Cox [Mon, 3 Jun 2019 05:15:36 +0000 (05:15 +0000)]
Retire vm_reserv_extend_{contig,page}(). These functions were introduced
as part of a false start toward fine-grained reservation locking. In the
end, they were not needed, so eliminate them.
Order the parameters to vm_reserv_alloc_{contig,page}() consistently with
the vm_page functions that call them.
Update the comments about the locking requirements for
vm_reserv_alloc_{contig,page}(). They no longer require a free page
queues lock.
Wrap several lines that became too long after the "req" and "domain"
parameters were added to vm_reserv_alloc_{contig,page}().
Warner Losh [Sun, 2 Jun 2019 16:25:07 +0000 (16:25 +0000)]
Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git
Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).
Kyle Evans [Sun, 2 Jun 2019 14:03:56 +0000 (14:03 +0000)]
jail_getid(3): add special-case immediate return for jid 0
As depicted in the comment: jid 0 always exists, but the lookup will fail as
it does not appear in the kernel's alljails list being a special jail. Some
callers will expect/rely on this, and we have no reason to lie because it
does always exist.
Warner Losh [Sun, 2 Jun 2019 04:23:56 +0000 (04:23 +0000)]
Import latest one-true-awk from upstream
Import git hash 4189ef5d from https://github.com/onetrueawk/awk.git as
there's not been a release in a while.
Upstream one-true-awk woke-up! Time to catch up. This may also revert
FreeBSD changes that we'd placed in the vendor branch in anticipation
of their inclusion in upstream. That's not yet the case, and these
will be resolved in the merge.
See FIXES for a complete list of bugs fixed (starting with the Jun 7,
2018 entry).
Kyle Evans [Sun, 2 Jun 2019 04:04:21 +0000 (04:04 +0000)]
llvm-symbolizer: Move out of CLANG_EXTRAS, into CLANG
ASAN reports become a lot more useful with llvm-symbolizer in $PATH, and the
build is not much more time-consuming. The added benefit is that the
resulting reports will actually include symbol information; without, thread
trace information includes a bunch of addresses that immediately resolve to
an inline function in
^/contrib/compiler-rt/lib/sanitizer_common/sanitizer_common.h and take a
little more effort to examine.
Kyle Evans [Sun, 2 Jun 2019 02:38:44 +0000 (02:38 +0000)]
grep: Move lone 'r'grep case into the adjacent switch
This 'r' case should have belonged to the switch in the first place, but
I had somehow missed the switch when initially adding the rgrep link. The
zgrep script later came along and faithfully left this case standing alone,
so we will now go ahead and join it.
Nearby comment also adjusted a tad bit for wording and style.
Ed Maste [Sat, 1 Jun 2019 18:26:07 +0000 (18:26 +0000)]
libatf: remove workaround not required after atf >= 0.18 update
lib/atf/libatf-c/tests/Makefile added the -Wno-duplicate-decl-specifier
due to an issue with an old version of ATF. ATF has long since been
updated to a version with the fix so the workaround is no longer
necessary.
Found during review for PR 236889.
PR: 236889
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Warner Losh [Sat, 1 Jun 2019 15:37:44 +0000 (15:37 +0000)]
Since a fatal trap can happen at aribtrary times, don't panic when the
completions are not in a consistent state. Cope with the different
places the normal I/O completion polling thread can be interrupted and
then re-entered during a kernel panic + dump.
Bjoern A. Zeeb [Sat, 1 Jun 2019 14:57:42 +0000 (14:57 +0000)]
After parts of the locking fixes in r346595, syzkaller found
another one in udp_output(). This one is a race condition.
We do check on the laddr and lport without holding a lock in
order to determine whether we want a read or a write lock
(this is in the "sendto/sendmsg" cases where addr (sin) is given).
Instrumenting the kernel showed that after taking the lock, we
had bound to a local port from a parallel thread on the same socket.
If we find that case, unlock, and retry again. Taking the write
lock would not be a problem in first place (apart from killing some
parallelism). However the retry is needed as later on based on
similar condition checks we do acquire the pcbinfo lock and if the
conditions have changed, we might find ourselves with a lock
inconsistency, hence at the end of the function when trying to
unlock, hitting the KASSERT.
Navdeep Parhar [Sat, 1 Jun 2019 03:03:48 +0000 (03:03 +0000)]
cxgbe/t4_tom: adjust the hardware receive window to match changes to the
receive sockbuf's high water mark.
Calculate rx credits on the spot instead of tracking sbused/sb_cc and
rx_credits in the toepcb. The previous method worked when the high
water mark changed due to SB_AUTOSIZE but not when it was adjusted
directly (for example, by the soreserve in nfsrvd_addsock).
This fixes a connection hang while running iozone over an NFS mounted
share where nfsd's TCP sockets are being handled by t4_tom.
MFC after: 3 days
Sponsored by: Chelsio Communications
Justin Hibbits [Sat, 1 Jun 2019 01:40:14 +0000 (01:40 +0000)]
powerpc/moea: Fix moea64 native VA invalidation
Summary:
moea64_insert_pteg_native()'s invalidation only works by happenstance.
The purpose of the shifts and XORs is to extract the VSID in order to
reverse-engineer the lower bits of the VPN. Currently a segment size is 256MB
(2**28), and ADDR_API_SHFT64 is 16, so ADDR_PIDX_SHIFT is equivalent. However,
it's semantically incorrect, in that we don't want to shift by the page shift
size, we want to shift to get to the VSID.
Conrad Meyer [Sat, 1 Jun 2019 01:22:21 +0000 (01:22 +0000)]
random(4): Fix RANDOM_LOADABLE build
I introduced an obvious compiler error in r346282, so this change fixes
that.
Unfortunately, RANDOM_LOADABLE isn't covered by our existing tinderbox, and
it seems like there were existing latent linking problems. I believe these
were introduced on accident in r338324 during reduction of the boolean
expression(s) adjacent to randomdev.c and hash.c. It seems the
RANDOM_LOADABLE build breakage has gone unnoticed for nine months.
This change correctly annotates randomdev.c and hash.c with !random_loadable
to match the pre-r338324 logic; and additionally updates the HWRNG drivers
in MD 'files.*', which depend on random_device symbols, with
!random_loadable (it is invalid for the kernel to depend on symbols from a
module).
(The expression for both randomdev.c and hash.c was the same, prior to
r338324: "optional random random_yarrow | random !random_yarrow
!random_loadable". I.e., "random && (yarrow || !loadable)." When Yarrow
was removed ("yarrow := False"), the expression was incorrectly reduced to
"optional random" when it should have retained "random && !loadable".)
Additionally, I discovered that virtio_random was missing a MODULE_DEPEND on
random_device, which breaks kld load/link of the driver on RANDOM_LOADABLE
kernels. Address that issue as well.
Warner Losh [Fri, 31 May 2019 22:57:20 +0000 (22:57 +0000)]
Defer evaluation of modified until after it's set
With the reorg r348175, we now look at modified before it is
set. Rearrange things so that we can set include_metadata to either
yes, no or if-modified. This should fix the -R flag that was broken in
r348175, which broke WITH_REPRODUCIBLE_BUILD for kernels.
Doug Moore [Fri, 31 May 2019 21:02:42 +0000 (21:02 +0000)]
The function vm_phys_free_contig invokes vm_phys_free_pages for every
power-of-two page block it frees, launching an unsuccessful search for
a buddy to pair up with each time. The only possible buddy-up mergers
are across the boundaries of the freed region, so change
vm_phys_free_contig simply to enqueue the freed interior blocks, via a
new function vm_phys_enqueue_contig, and then call vm_phys_free_pages
on the bounding blocks to create as big a cross-boundary block as
possible after buddy-merging.
The only callers of vm_phys_free_contig at the moment call it in
situations where merging blocks across the boundary is clearly
impossible, so just call vm_phys_enqueue_contig in those places and
avoid trying to buddy-up at all.
One beneficiary of this change is in breaking reservations. For the
case where memory is freed in breaking a reservation with only the
first and last pages allocated, the number of cycles consumed by the
operation drops about 11% with this change.
Simplify flow of pmap_demote_pde_locked() and add more comprehensive
debugging checks.
In particular,
- Move the code to handle failure to allocate page table page into
a helper.
- After the previous item is done, it is possible to distinguish !PG_A
case and case of missed page, in the control flow.
- Make the variable to indicate that in-kernel mapping is demoted.
- Assert that missed page table page can only happen for in-kernel
mapping when demoting direct map.
- If DIAGNOSTIC is enabled, and the page table page should be already
filled, check all ptes instead of only first one.
Glen Barber [Fri, 31 May 2019 18:40:19 +0000 (18:40 +0000)]
Revert r348438.
The fix to override the default python version when building
the sysutils/py-google-compute-engine did not work, and there
are still issues that need to be addressed in the port itself.
See bugzilla 238267 for additional details.
MFC after: 6 days
MFC with: r348438
MFC note: no-op to appease the merge tracker
Sponsored by: The FreeBSD Foundation
Mark Johnston [Fri, 31 May 2019 18:29:12 +0000 (18:29 +0000)]
netdump: Buffer pages to avoid calling netdump_send() on each 4KB write.
netdump waits for acknowledgement from the server for each write. When
dumping page table pages, we perform many small writes, limiting
throughput. Use the netdump client's buffer to buffer small contiguous
writes before calling netdump_send() to flush the MAXDUMPPGS-sized
buffer. This results in a significant reduction in the time taken to
complete a netdump.
Submitted by: Sam Gwydir <sam@samgwydir.com>
Reviewed by: cem
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D20317
Li-Wen Hsu [Fri, 31 May 2019 04:29:29 +0000 (04:29 +0000)]
Remove tests for the deprecated algorithms in r348206
The tests are failing because the return value and output have changed, but
before test code structure adjusted, removing these test cases help people
be able to focus on more important cases.
Discussed with: emaste
MFC with: r348206
Sponsored by: The FreeBSD Foundation
Rick Macklem [Fri, 31 May 2019 01:28:48 +0000 (01:28 +0000)]
Replace a single linked list with a hash table of lists.
mountd.c uses a single linked list of "struct exportlist" structures,
where there is one of these for each exported file system on the NFS server.
This list gets long if there are a large number of file systems exported and
the list must be searched for each line in the exports file(s) when
SIGHUP causes the exports file(s) to be reloaded.
A simple benchmark that traverses SLIST() elements and compares two 32bit
fields in the structure for equal (which is what the search is)
appears to take a couple of nsec. So, for a server with 72000 exported file
systems, this can take about 5sec during reload of the exports file(s).
By replacing the single linked list with a hash table with a target of
10 elements per list, the time should be reduced to less than 1msec.
Peter Errikson (who has a server with 72000+ exported file systems) ran
a test program using 5 hashes to see how they worked.
fnv_32_buf(fsid,..., 0)
fnv_32_buf(fsid,..., FNV1_32_INIT)
hash32_buf(fsid,..., 0)
hash32_buf(fsid,..., HASHINIT)
- plus simply using the low order bits of fsid.val[0].
The first three behaved about equally well, with the first one being
slightly better than the others.
It has an average variation of about 4.5% about the target list length
and that is what this patch uses.
Peter Errikson also tested this hash table version and found that the
performance wasn't measurably improved by a larger hash table, so a
load factor of 10 appears adequate.
projects / FreeBSD / FreeBSD.git / log