jhb [Mon, 3 Jun 2019 15:41:54 +0000 (15:41 +0000)]
Add 'device cxgbe' explicitly in the synopsis.
ccr depends on symbols exported by the cxgbe driver as well as having
a runtime dependency. While the runtime depenency was noted in the
manpage already, the compile-time dependency wasn't as clear.
PR: 238265
MFC after: 3 days
Sponsored by: Chelsio Communications
kib [Mon, 3 Jun 2019 15:32:42 +0000 (15:32 +0000)]
amd64 ef_rt_arch_call: Preserve %rflags around call into EFI RT service.
If service code faulted, we might end up unwinding with interrupts
disabled. Top-level kernel code should have interrupts enabled, which
is enforced by checks.
Save %rflags before entering EFI, and restore to the known good value
on return. This handles situation with disabled interrupts on fault
and perhaps other potential bugs, e.g. invalid value for PSL_D.
Reported and tested by: Jan Martin Mikkelsen <janm@transactionware.com>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
sobomax [Mon, 3 Jun 2019 15:12:44 +0000 (15:12 +0000)]
Leave mtree hardcoded for now. Reverting partially 348521 and also
the followup stopgap change, because I don't think it's a correct. I still
need to figure out where to stick it in. In cannot be in Makefile.inc1
and it cannot be in etc/Makefile from the looks of it to avoid
chicken-and-egg problem.
wulf [Mon, 3 Jun 2019 10:04:34 +0000 (10:04 +0000)]
psm(4): Add natural scrolling support to sysmouse protocol
This change enables natural scrolling with two finger scroll enabled
and when user is using a trackpad (mouse and trackpoint are not affected).
Depending on trackpad model it can be activated with setting of
hw.psm.synaptics.natural_scroll or hw.psm.elantech.natural_scroll sysctl
values to 1.
Evdev protocol is not affected by this change too. Tune userland client
e.g. libinput to enable natural scrolling in that case.
imp [Mon, 3 Jun 2019 05:25:22 +0000 (05:25 +0000)]
Another partial revert of r301289.
In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.
imp [Mon, 3 Jun 2019 05:25:16 +0000 (05:25 +0000)]
Fix mismerge that crept into r301289.
The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.
alc [Mon, 3 Jun 2019 05:15:36 +0000 (05:15 +0000)]
Retire vm_reserv_extend_{contig,page}(). These functions were introduced
as part of a false start toward fine-grained reservation locking. In the
end, they were not needed, so eliminate them.
Order the parameters to vm_reserv_alloc_{contig,page}() consistently with
the vm_page functions that call them.
Update the comments about the locking requirements for
vm_reserv_alloc_{contig,page}(). They no longer require a free page
queues lock.
Wrap several lines that became too long after the "req" and "domain"
parameters were added to vm_reserv_alloc_{contig,page}().
imp [Sun, 2 Jun 2019 16:25:07 +0000 (16:25 +0000)]
Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git
Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).
kevans [Sun, 2 Jun 2019 14:03:56 +0000 (14:03 +0000)]
jail_getid(3): add special-case immediate return for jid 0
As depicted in the comment: jid 0 always exists, but the lookup will fail as
it does not appear in the kernel's alljails list being a special jail. Some
callers will expect/rely on this, and we have no reason to lie because it
does always exist.
imp [Sun, 2 Jun 2019 04:23:56 +0000 (04:23 +0000)]
Import latest one-true-awk from upstream
Import git hash 4189ef5d from https://github.com/onetrueawk/awk.git as
there's not been a release in a while.
Upstream one-true-awk woke-up! Time to catch up. This may also revert
FreeBSD changes that we'd placed in the vendor branch in anticipation
of their inclusion in upstream. That's not yet the case, and these
will be resolved in the merge.
See FIXES for a complete list of bugs fixed (starting with the Jun 7,
2018 entry).
kevans [Sun, 2 Jun 2019 04:04:21 +0000 (04:04 +0000)]
llvm-symbolizer: Move out of CLANG_EXTRAS, into CLANG
ASAN reports become a lot more useful with llvm-symbolizer in $PATH, and the
build is not much more time-consuming. The added benefit is that the
resulting reports will actually include symbol information; without, thread
trace information includes a bunch of addresses that immediately resolve to
an inline function in
^/contrib/compiler-rt/lib/sanitizer_common/sanitizer_common.h and take a
little more effort to examine.
kevans [Sun, 2 Jun 2019 02:38:44 +0000 (02:38 +0000)]
grep: Move lone 'r'grep case into the adjacent switch
This 'r' case should have belonged to the switch in the first place, but
I had somehow missed the switch when initially adding the rgrep link. The
zgrep script later came along and faithfully left this case standing alone,
so we will now go ahead and join it.
Nearby comment also adjusted a tad bit for wording and style.
emaste [Sat, 1 Jun 2019 18:26:07 +0000 (18:26 +0000)]
libatf: remove workaround not required after atf >= 0.18 update
lib/atf/libatf-c/tests/Makefile added the -Wno-duplicate-decl-specifier
due to an issue with an old version of ATF. ATF has long since been
updated to a version with the fix so the workaround is no longer
necessary.
Found during review for PR 236889.
PR: 236889
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
imp [Sat, 1 Jun 2019 15:37:44 +0000 (15:37 +0000)]
Since a fatal trap can happen at aribtrary times, don't panic when the
completions are not in a consistent state. Cope with the different
places the normal I/O completion polling thread can be interrupted and
then re-entered during a kernel panic + dump.
After parts of the locking fixes in r346595, syzkaller found
another one in udp_output(). This one is a race condition.
We do check on the laddr and lport without holding a lock in
order to determine whether we want a read or a write lock
(this is in the "sendto/sendmsg" cases where addr (sin) is given).
Instrumenting the kernel showed that after taking the lock, we
had bound to a local port from a parallel thread on the same socket.
If we find that case, unlock, and retry again. Taking the write
lock would not be a problem in first place (apart from killing some
parallelism). However the retry is needed as later on based on
similar condition checks we do acquire the pcbinfo lock and if the
conditions have changed, we might find ourselves with a lock
inconsistency, hence at the end of the function when trying to
unlock, hitting the KASSERT.
cxgbe/t4_tom: adjust the hardware receive window to match changes to the
receive sockbuf's high water mark.
Calculate rx credits on the spot instead of tracking sbused/sb_cc and
rx_credits in the toepcb. The previous method worked when the high
water mark changed due to SB_AUTOSIZE but not when it was adjusted
directly (for example, by the soreserve in nfsrvd_addsock).
This fixes a connection hang while running iozone over an NFS mounted
share where nfsd's TCP sockets are being handled by t4_tom.
MFC after: 3 days
Sponsored by: Chelsio Communications
jhibbits [Sat, 1 Jun 2019 01:40:14 +0000 (01:40 +0000)]
powerpc/moea: Fix moea64 native VA invalidation
Summary:
moea64_insert_pteg_native()'s invalidation only works by happenstance.
The purpose of the shifts and XORs is to extract the VSID in order to
reverse-engineer the lower bits of the VPN. Currently a segment size is 256MB
(2**28), and ADDR_API_SHFT64 is 16, so ADDR_PIDX_SHIFT is equivalent. However,
it's semantically incorrect, in that we don't want to shift by the page shift
size, we want to shift to get to the VSID.
cem [Sat, 1 Jun 2019 01:22:21 +0000 (01:22 +0000)]
random(4): Fix RANDOM_LOADABLE build
I introduced an obvious compiler error in r346282, so this change fixes
that.
Unfortunately, RANDOM_LOADABLE isn't covered by our existing tinderbox, and
it seems like there were existing latent linking problems. I believe these
were introduced on accident in r338324 during reduction of the boolean
expression(s) adjacent to randomdev.c and hash.c. It seems the
RANDOM_LOADABLE build breakage has gone unnoticed for nine months.
This change correctly annotates randomdev.c and hash.c with !random_loadable
to match the pre-r338324 logic; and additionally updates the HWRNG drivers
in MD 'files.*', which depend on random_device symbols, with
!random_loadable (it is invalid for the kernel to depend on symbols from a
module).
(The expression for both randomdev.c and hash.c was the same, prior to
r338324: "optional random random_yarrow | random !random_yarrow
!random_loadable". I.e., "random && (yarrow || !loadable)." When Yarrow
was removed ("yarrow := False"), the expression was incorrectly reduced to
"optional random" when it should have retained "random && !loadable".)
Additionally, I discovered that virtio_random was missing a MODULE_DEPEND on
random_device, which breaks kld load/link of the driver on RANDOM_LOADABLE
kernels. Address that issue as well.
imp [Fri, 31 May 2019 22:57:20 +0000 (22:57 +0000)]
Defer evaluation of modified until after it's set
With the reorg r348175, we now look at modified before it is
set. Rearrange things so that we can set include_metadata to either
yes, no or if-modified. This should fix the -R flag that was broken in
r348175, which broke WITH_REPRODUCIBLE_BUILD for kernels.
dougm [Fri, 31 May 2019 21:02:42 +0000 (21:02 +0000)]
The function vm_phys_free_contig invokes vm_phys_free_pages for every
power-of-two page block it frees, launching an unsuccessful search for
a buddy to pair up with each time. The only possible buddy-up mergers
are across the boundaries of the freed region, so change
vm_phys_free_contig simply to enqueue the freed interior blocks, via a
new function vm_phys_enqueue_contig, and then call vm_phys_free_pages
on the bounding blocks to create as big a cross-boundary block as
possible after buddy-merging.
The only callers of vm_phys_free_contig at the moment call it in
situations where merging blocks across the boundary is clearly
impossible, so just call vm_phys_enqueue_contig in those places and
avoid trying to buddy-up at all.
One beneficiary of this change is in breaking reservations. For the
case where memory is freed in breaking a reservation with only the
first and last pages allocated, the number of cycles consumed by the
operation drops about 11% with this change.
kib [Fri, 31 May 2019 18:53:04 +0000 (18:53 +0000)]
Simplify flow of pmap_demote_pde_locked() and add more comprehensive
debugging checks.
In particular,
- Move the code to handle failure to allocate page table page into
a helper.
- After the previous item is done, it is possible to distinguish !PG_A
case and case of missed page, in the control flow.
- Make the variable to indicate that in-kernel mapping is demoted.
- Assert that missed page table page can only happen for in-kernel
mapping when demoting direct map.
- If DIAGNOSTIC is enabled, and the page table page should be already
filled, check all ptes instead of only first one.
gjb [Fri, 31 May 2019 18:40:19 +0000 (18:40 +0000)]
Revert r348438.
The fix to override the default python version when building
the sysutils/py-google-compute-engine did not work, and there
are still issues that need to be addressed in the port itself.
See bugzilla 238267 for additional details.
MFC after: 6 days
MFC with: r348438
MFC note: no-op to appease the merge tracker
Sponsored by: The FreeBSD Foundation
markj [Fri, 31 May 2019 18:29:12 +0000 (18:29 +0000)]
netdump: Buffer pages to avoid calling netdump_send() on each 4KB write.
netdump waits for acknowledgement from the server for each write. When
dumping page table pages, we perform many small writes, limiting
throughput. Use the netdump client's buffer to buffer small contiguous
writes before calling netdump_send() to flush the MAXDUMPPGS-sized
buffer. This results in a significant reduction in the time taken to
complete a netdump.
Submitted by: Sam Gwydir <sam@samgwydir.com>
Reviewed by: cem
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D20317
lwhsu [Fri, 31 May 2019 04:29:29 +0000 (04:29 +0000)]
Remove tests for the deprecated algorithms in r348206
The tests are failing because the return value and output have changed, but
before test code structure adjusted, removing these test cases help people
be able to focus on more important cases.
Discussed with: emaste
MFC with: r348206
Sponsored by: The FreeBSD Foundation
rmacklem [Fri, 31 May 2019 01:28:48 +0000 (01:28 +0000)]
Replace a single linked list with a hash table of lists.
mountd.c uses a single linked list of "struct exportlist" structures,
where there is one of these for each exported file system on the NFS server.
This list gets long if there are a large number of file systems exported and
the list must be searched for each line in the exports file(s) when
SIGHUP causes the exports file(s) to be reloaded.
A simple benchmark that traverses SLIST() elements and compares two 32bit
fields in the structure for equal (which is what the search is)
appears to take a couple of nsec. So, for a server with 72000 exported file
systems, this can take about 5sec during reload of the exports file(s).
By replacing the single linked list with a hash table with a target of
10 elements per list, the time should be reduced to less than 1msec.
Peter Errikson (who has a server with 72000+ exported file systems) ran
a test program using 5 hashes to see how they worked.
fnv_32_buf(fsid,..., 0)
fnv_32_buf(fsid,..., FNV1_32_INIT)
hash32_buf(fsid,..., 0)
hash32_buf(fsid,..., HASHINIT)
- plus simply using the low order bits of fsid.val[0].
The first three behaved about equally well, with the first one being
slightly better than the others.
It has an average variation of about 4.5% about the target list length
and that is what this patch uses.
Peter Errikson also tested this hash table version and found that the
performance wasn't measurably improved by a larger hash table, so a
load factor of 10 appears adequate.
rmacklem [Fri, 31 May 2019 00:56:31 +0000 (00:56 +0000)]
Clean up silly code case.
This silly code segment has existed in the sources since it was brought
into FreeBSD 10 years ago. I honestly have no idea why this was done.
It was possible that I thought that it might have been better to not
set B_ASYNC for the "else" case, but I can't remember.
Anyhow, this patch gets rid of the if/else that does the same thing
either way, since it looks silly and upsets a static analyser.
This will have no semantic effect on the NFS client.
gjb [Thu, 30 May 2019 16:49:40 +0000 (16:49 +0000)]
Fix GCE virtual machine startup.
The ports/head branch recently switched to python3 as the default,
which breaks the sysutils/py-google-compute-engine startup scripts,
as lang/python installs lang/python3{,.x} where lang/python2{,.x}
are needed.
Set DEFAULT_VERSIONS in release/tools/gce.conf to python=2.7, and
remove the lang/python3 inclusion in VM_EXTRA_PACKAGES.
Additionally, unset DEFAULT_VERSIONS in release/tools/vmimage.subr
to prevent persistence of DEFAULT_VERSIONS=python=2.7 in subsequent
VM/cloud image builds.
Note: at present, this affects only 13-CURRENT and 12-STABLE, as
the stable/11 branch had already switched to using the 2019Q2 branch
at the start of the 11.3-RELEASE cycle, so this does not immediately
affect 11.3-BETA, hence the 1-week merge timeout. This had been
manually tested on 13-CURRENT.
Reported by: ler (privately)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
markj [Thu, 30 May 2019 15:28:48 +0000 (15:28 +0000)]
elfcopy: Optimize for insertions at the end of the section list.
This is the common case when strip(1) is creating the output file.
The change provides a significant speedup when running on ELF files with
many sections.
PR: 234949
Reviewed by: emaste
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20444
mav [Thu, 30 May 2019 15:07:39 +0000 (15:07 +0000)]
Pass data pointers to the driver in way in expects.
Probably due to historical reasons the driver uses In/Out arguments in
odd way. While this tool still never uses Out arguments to see that,
make the code to not trigger EINVAL in possible future uses.
kib [Thu, 30 May 2019 15:04:09 +0000 (15:04 +0000)]
Silence witness warning about duplicated mutex type.
The order is correct, it is nullfs vnode interlock -> lower vnode
interlock. vop_stdadd_writecount() is called from nullfs
VOP_ADD_WRITECOUNT() and both take interlocks.
Requested by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
mw [Thu, 30 May 2019 13:45:41 +0000 (13:45 +0000)]
Improve ENA reset handling
For easier debugging, the reset is being triggered and the reset reason is
being set only in case it is done for the first time. Such approach will
ensure that the first reset reason is not going to be overwritten and
will make it easier for debugging.
Also, add a reset trigger upon invalid Tx requested ID.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:41:39 +0000 (13:41 +0000)]
Unify new line characters in the ENA driver
Some messages were missing new line character and traces were not having
unified behavior. To fix that, each trace and printout should add new
line character at the end of each string - that should improve
readability.
Submitted by: Rafal Kozik <rk@semihalf.com>
Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:40:51 +0000 (13:40 +0000)]
Fix Tx offloads for fragmented pkt headers in ENA
If the headers of the packets are split into multiple segments of the
mbuf chain, the previous version of ena_tx_csum which was assuming,
that all segments will lay in the first mbuf, will eventually fail to
map the headers properties to meta descriptor.
That will cause Tx checksum offload to do not work and was leading to
memory corruption. It could even cause the crash of the system.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:37:15 +0000 (13:37 +0000)]
Use bitfield for storing global ENA device states
As the ENA can have multiple states turned on/off, it is more convenient
to store them in single bitfield instead of multiple boolean variables.
The bitset FreeBSD API was used for the bitfield implementation, as it
provides flexible structure together with API which also supports atomic
bitfield operations.
For better readability basic macros from API were wrapped into custom
ENA_FLAG_* macros, which are filling up common parameters for all calls.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:33:31 +0000 (13:33 +0000)]
Add additional doorbells on ENA Tx path
The new ENA HAL is introducing API, which can determine on Tx path if
the doorbell is needed.
That way, it can tell the driver, that it should call an doorbell.
The old threshold value wasn't removed, as not all HW is supporting this
feature - so it was reworked to also work with the new API.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:30:52 +0000 (13:30 +0000)]
Add support for the LLQv2 and WC in ENA
LLQ (Low Latency Queue) is the feature, that allows pushing header
directly to the device through PCI before even DMA is triggered.
It reduces latency, because device can start preparing packet before
payload is sent through DMA.
To speed up sending data through PCI, the Write Combining is enabled,
which allows hardware to buffer data before sending them on the PCI - it
allows to reduce number of PCI IO operations.
ENAv2 is using special descriptor for the negotiation of the LLQ.
Currently, only the default configuration is supported.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:24:47 +0000 (13:24 +0000)]
Change attach order to prevent crash upon failure in ENA
The if_detach was causing crash if the MSI-x configuration in the attach
failed. To prevent this issue, the ifnet is being configured at the end
of the attach function.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:22:12 +0000 (13:22 +0000)]
Check for number of MSI-x upon partial allocation in ENA
The ENA driver needs at least 2 MSI-x - one for admin queue, and one for
IO queues pair. If there were not enough resources to allocate more than
one MSI-x, the device should not be attached.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:20:42 +0000 (13:20 +0000)]
Set error value when allocation of IO irq fails in ENA
bus_alloc_resource_any() is not returning error value in case of an
error.
If the function call fails, the error value was not passed to the
ena_up() function.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:18:23 +0000 (13:18 +0000)]
Fix DMA synchronization in the ENA driver Tx and Rx paths
The DMA in FreeBSD requires explicit synchronization. ENA driver was
only doing PREREAD and PREWRITE synchronizations. Missing
bus_dmamap_sync() calls were added.
It is also required to synchronize DMA engine before unloading DMA map.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
mw [Thu, 30 May 2019 13:16:56 +0000 (13:16 +0000)]
Check for missing MSI-x and Tx completions in ENA
If the first MSI-x won't be executed, then the timer service will detect
that and trigger device reset.
The checking for missing Tx completion was reworked, so it will also
check for missing interrupts. Checking number of missing Tx completions
can be performed after loop, instead of checking it every iteration.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
projects / FreeBSD / FreeBSD.git / log