]> CyberLeo.Net >> Repos - FreeBSD/FreeBSD.git/log
FreeBSD/FreeBSD.git
4 years agoloader: Respect loader_color=YES for serial consoles
kevans [Fri, 20 Sep 2019 19:43:40 +0000 (19:43 +0000)]
loader: Respect loader_color=YES for serial consoles

It's not uncommon these days for the terminals attached to serial consoles
to support ANSI escape sequences. However, we assume escape sequences may
break some serial consoles and default to not using them when boot_serial or
boot_multicons (or if console contains "comconsole" in the forth loader) for
broader compatibility. We also have loader_color which can be explicitly set
to "NO" to disable the use of ANSI escape sequences.

The problem is that loader_color=YES gets ignored when boot_serial=YES or
boot_multicons=YES (or when console contains "comconsole" in the forth
loader).

To fix, the existing default behavior remains unchanged when loader_color is
unset, loader_color=NO explicitly disables the use of ANSI escape sequences
still, and the change is that loader_color=YES can now be used to explicitly
allow ANSI escapes when a serial console is enabled.

Submitted by: Ryan Moeller <ryan@ixsystems.com>
Reviewed by: tsoome (forth), kevans (lua)
MFC after: 1 week
Sponsored by: iXsystems, Inc. (Ryan)
Differential Revision: https://reviews.freebsd.org/D21732

4 years agotop(1): support multibyte characters in command names (ARGV array)
daichi [Fri, 20 Sep 2019 17:37:23 +0000 (17:37 +0000)]
top(1): support multibyte characters in command names (ARGV array)
depending on locale.

 - add setlocale()
 - remove printable() function
 - add VIS_OCTAL and VIS_SAFE to the flag of strvisx() to display
   non-printable characters that do not use C-style backslash sequences
   in three digit octal sequence, or remove it

This change allows multibyte characters to be displayed according to
locale. If it is recognized as a non-display character according to the
locale, it is displayed in three digit octal sequence.

Reference:
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165751.html
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165766.html
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165833.html
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165846.html
https://www.mail-archive.com/svn-src-all@freebsd.org/msg165891.html

Submitted by: hrs
Differential Revision: https://reviews.freebsd.org/D16204

4 years agopowerpc/loader: Install ubldr without stripping
jhibbits [Fri, 20 Sep 2019 13:35:28 +0000 (13:35 +0000)]
powerpc/loader: Install ubldr without stripping

Summary:
Install's strip capability, by way of strip(1), doesn't seem to work
correctly on msdosfs, and instead ends up truncating the resulting
binary to 0-length.  As a workaround, don't strip ubldr(8).  This
fixes installworld on Book-E ubldr-based platforms, which prior to this
would need to manually install ubldr separately after installworld, in
order to have a functional ubldr.

The same thing could be done on PowerNV platforms that use msdosfs /boot
volumes, since loader and loader.kboot, etc, all get truncated to 0 on
install.  However, PowerNV does not use loader, instead loading from
petitboot, so it's not really necessary at this time.

Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D21725

4 years agoAdd quirk for XHCI(4) controllers to support USB control transfers
hselasky [Fri, 20 Sep 2019 11:28:45 +0000 (11:28 +0000)]
Add quirk for XHCI(4) controllers to support USB control transfers
above 1Kbyte.  It might look like some XHCI(4) controllers do not
support when the USB control transfer is split using a link TRB. The
next NORMAL TRB after the link TRB is simply failing with XHCI error
code 4. The quirk ensures we allocate a 64Kbyte buffer so that the
data stage TRB is not broken with a link TRB.

Found at: EuroBSDcon 2019
MFC after: 1 week
Sponsored by: Mellanox Technologies

4 years agoIncrease the maximum user-space buffer size from 256kBytes to 32MBytes for
hselasky [Fri, 20 Sep 2019 11:00:02 +0000 (11:00 +0000)]
Increase the maximum user-space buffer size from 256kBytes to 32MBytes for
libusb. This is useful for speeding up large data transfers while reducing
the interrupt rate.

Found at: EuroBSDcon 2019
MFC after: 1 week
Sponsored by: Mellanox Technologies

4 years agoThe maximum TD size is 31 and not 15.
hselasky [Fri, 20 Sep 2019 10:56:13 +0000 (10:56 +0000)]
The maximum TD size is 31 and not 15.

Found at: EuroBSDcon 2019
MFC after: 1 week
Sponsored by: Mellanox Technologies

4 years agoEnsure libthr is always built before libprivatezstd when building the
bapt [Fri, 20 Sep 2019 09:45:38 +0000 (09:45 +0000)]
Ensure libthr is always built before libprivatezstd when building the
startup libs

Reported by: "Galazka, Krzysztof" <krzysztof.galazka@intel.com>

4 years agoremove redundant "ktls" in KTLS thr name
gallatin [Fri, 20 Sep 2019 09:36:07 +0000 (09:36 +0000)]
remove redundant "ktls" in  KTLS thr name

This reducesthe string width of the ktls thread name
and improves "ps" output.

Glanced at by: jhb
Event: EuroBSDCon hackathon
Sponsored by: Netflix

4 years agoelf_common: add ELF note names
emaste [Fri, 20 Sep 2019 09:04:52 +0000 (09:04 +0000)]
elf_common: add ELF note names

r348628 added a definition of NT_GNU_BUILD_ID.  Some software (Valgrind)
also expects a #define for the note name (ELF_NOTE_GNU) in the case that
NT_GNU_BUILD_ID is defined.

PR: 239669
Reported by: Yuichiro NAITO
Sponsored by: The FreeBSD Foundation
Event: EuroBSDCon FreeBSD DevSummit 2019

4 years agoFix the handling of invalid parameters in ASCONF chunks.
tuexen [Fri, 20 Sep 2019 08:20:20 +0000 (08:20 +0000)]
Fix the handling of invalid parameters in ASCONF chunks.
Thanks to Mark Wodrich from Google for reproting the issue in
https://github.com/sctplab/usrsctp/issues/376
for the userland stack.

MFC after: 3 days

4 years agoloader: fix typo in zalloc.
tsoome [Fri, 20 Sep 2019 05:22:34 +0000 (05:22 +0000)]
loader: fix typo in zalloc.

4 years agoImprove ioat(4) NUMA-awareness.
mav [Thu, 19 Sep 2019 22:15:57 +0000 (22:15 +0000)]
Improve ioat(4) NUMA-awareness.

Allocate ioat->ring memory from the device domain.
Schedule ioat->poll_timer to the first CPU of the device domain.

According to pcm-numa tool from intel-pcm port, this reduces number of
remote DRAM accesses while copying data by 75%.  And unless it is a noise,
I've noticed some speed improvement when copying data to other domain.

MFC after: 1 week
Sponsored by: iXsystems, Inc.

4 years agovfs: group fields used for per-cpu ops in one cacheline
mjg [Thu, 19 Sep 2019 21:23:14 +0000 (21:23 +0000)]
vfs: group fields used for per-cpu ops in one cacheline

Sponsored by: The FreeBSD Foundation

4 years agoFix src component detection
grembo [Thu, 19 Sep 2019 21:13:51 +0000 (21:13 +0000)]
Fix src component detection

Reviewed by: emaste
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D21579

4 years agoFollow up on r352304 which disabled default mlockall() at startup.
cy [Thu, 19 Sep 2019 20:16:51 +0000 (20:16 +0000)]
Follow up on r352304 which disabled default mlockall() at startup.
Unfortunately though the original tarball supports this in ./configure
(for Linux), to fully support disabling of mlockall() by default requires
a little extra help otherwise the following is logged in syslog:

Cannot set RLIMIT_MEMLOCK: Operation not permitted

MFC after: 2 weeks
X-MFC with: r352304

4 years agoApply r346792 (cperciva) from stable/12 to head. The original commit
gjb [Thu, 19 Sep 2019 16:43:12 +0000 (16:43 +0000)]
Apply r346792 (cperciva) from stable/12 to head.  The original commit
message:

 On non-x86 systems, use "quarterly" packages.

 x86 architectures have "latest" package builds on stable/*, so keep using
 those (they'll get switched over to "quarterly" during releases).

The original commit was a direct commit to stable/12, as at the time it
was presumed it would not be necessary for head.  However, when it is time
to create a releng branch or switch from PRERELEASE/STABLE to BETA/RC, the
pkg(7) Makefile needs further adjusting.  This commit includes those
further adjustments, evaluating the BRANCH variable from release/Makefile
to determine the pkg(7) repository to use.

MFC after: immediate (if possible)
Sponsored by: Rubicon Communications, LLC (Netgate)

4 years agoReduce calls to close(2) at startup through the use of closefrom(2).
cy [Thu, 19 Sep 2019 14:45:04 +0000 (14:45 +0000)]
Reduce calls to close(2) at startup through the use of closefrom(2).

Submitted by: pawel.biernacki@gmail.com
Reviewed by: mjg, cy
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D21715

4 years agoWhitespace cleanup, no functional change
lwhsu [Thu, 19 Sep 2019 13:25:19 +0000 (13:25 +0000)]
Whitespace cleanup, no functional change

Sponsored by: The FreeBSD Foundation

4 years agoTemporarily add test_write_filter_zstd BROKEN_TESTS as it always fails in CI
lwhsu [Thu, 19 Sep 2019 13:23:25 +0000 (13:23 +0000)]
Temporarily add test_write_filter_zstd BROKEN_TESTS as it always fails in CI

There is no trivial way to mark single libarchive test skip currently so just
add it to BROKEN_TESTS for now.

PR: 240683
Sponsored by: The FreeBSD Foundation

4 years agofreebsd-update: make usage output consistent
emaste [Thu, 19 Sep 2019 11:46:43 +0000 (11:46 +0000)]
freebsd-update: make usage output consistent

Drop trailing . which appeared only on description of IDS.

Submitted by: grembo
Event: EuroBSDCon Norway FreeBSD DevSummit

4 years agofreebsd-update.8: appease igor
emaste [Thu, 19 Sep 2019 11:34:35 +0000 (11:34 +0000)]
freebsd-update.8: appease igor

igor follows American style guides in the belief that abbreviations i.e.
and e.g. are always followed by a comma.  Make that change now so that
future updates to freebsd-update.8 do not complain about this.

Submitted by: grembo
Event: EuroBSDCon Norway FreeBSD DevSummit

4 years agoWhen the RACK stack computes the space for user data in a TCP segment,
tuexen [Thu, 19 Sep 2019 10:27:47 +0000 (10:27 +0000)]
When the RACK stack computes the space for user data in a TCP segment,
it wasn't taking the IP level options into account. This patch fixes this.
In addition, it also corrects a KASSERT and adds protection code to assure
that the IP header chain and the TCP head fit in the first fragment as
required by RFC 7112.

Reviewed by: rrs@
MFC after: 3 days
Sponsored by: Nertflix, Inc.
Differential Revision: https://reviews.freebsd.org/D21666

4 years agoWhen processing an incoming IPv6 packet over the loopback interface which
tuexen [Thu, 19 Sep 2019 10:22:29 +0000 (10:22 +0000)]
When processing an incoming IPv6 packet over the loopback interface which
contains Hop-by-Hop options, the mbuf chain is potentially changed in
ip6_hopopts_input(), called by ip6_input_hbh().
This can happen, because of the the use of IP6_EXTHDR_CHECK, which might
call m_pullup().
So provide the updated pointer back to the called of ip6_input_hbh() to
avoid using a freed mbuf chain in`ip6_input()`.

Reviewed by: markj@
MFC after: 3 days
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D21664

4 years agoupdate zfs send usage help with r352447
avg [Thu, 19 Sep 2019 09:48:01 +0000 (09:48 +0000)]
update zfs send usage help with r352447

MFC after: 3 days

4 years agofix dsl_scan_ds_clone_swapped logic
avg [Thu, 19 Sep 2019 09:43:56 +0000 (09:43 +0000)]
fix dsl_scan_ds_clone_swapped logic

It was incorrect with respect to swapping dataset IDs both in the
on-disk ZAP object and the in-memory queue.

In both cases, if only ds1 was already present, then it would be first
replaced with ds2 and then ds2 would be replaced back with ds1.  Also,
both cases did not properly handle a situation where both ds1 and ds2
are already queued.  A duplicate insertion would be attempted and its
failure would result in a panic.

This change has also been submitted to ZoL as zfsonlinux/zfs@dd262c9

PR: 239566
Reported by: pascal.guitierrez@gmail.com
MFC after: 4 days
Sponsored by: CyberSecure

4 years agovt: fix problems with trying to switch to a closed VT
avg [Thu, 19 Sep 2019 09:22:45 +0000 (09:22 +0000)]
vt: fix problems with trying to switch to a closed VT

If there is an attempt to switch from a process-owned VT to a closed VT,
then vt(4) first requests the process to release its VT and only then
realizes that the target VT is closed and, so, the switch is not
possible.  So, the driver does not actually do any switch, but at the
same time the owning process is not notified about that and it does not
re-acquire the VT.

This change adds an early check for the target VT state, so that the
switch can be refused before the process coordination dance.
On top of that, the code now checks for a failure of vt_window_switch()
and calls vt_window_postswitch() for the current VT if it is in the
process mode.

Test Plan:
- configure VT1 - VT8 (ttyv0 - ttyv7) to be text consoles (run getty)
- configure VT9 (ttyv8) to rn X server
- make sure that the X server configuration allows VT switching
- leave VT10 - VT12 unconfigured
- while in the X server press Ctrl+Alt+F10
- without the patch, observe strange screen content and problems with
  keyboard input
- with the patch, observe that nothing happens

The problem has been observed and the fix has been tested with an nVidia
graphics card and the proprietary nvidia driver.
Not sure if that matters.

Reviewed by: ray
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D21704

4 years agosys/vm/vm_glue.c: Incorrect function name in panic string
allanjude [Thu, 19 Sep 2019 07:28:24 +0000 (07:28 +0000)]
sys/vm/vm_glue.c: Incorrect function name in panic string

Use __func__ to avoid this issue in the future.

Submitted by: Wuyang Chung <wuyang.chung1@gmail.com>
Reviewed by: markj, emaste
Obtained from: https://github.com/freebsd/freebsd/pull/410

4 years agoAdd some tests for page fault signals and codes
jilles [Wed, 18 Sep 2019 21:00:32 +0000 (21:00 +0000)]
Add some tests for page fault signals and codes

It is useful to have some tests for page fault signals.

More tests would be useful but creating the conditions (such as various
kinds of running out of memory and I/O errors) is more complicated.

The tests page_fault_signal__bus_objerr_1 and
page_fault_signal__bus_objerr_2 depend on https://reviews.freebsd.org/D21566
before they can pass.

PR: 211924
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D21624

4 years agoFix typo, setting hidden flag instead of reparse.
mav [Wed, 18 Sep 2019 19:33:08 +0000 (19:33 +0000)]
Fix typo, setting hidden flag instead of reparse.

Submitted by: Ryan Moeller <ryan@ixsystems.com>
MFC after: 3 days
Sponsored by: iXsystems, Inc.

4 years agotruss: decode sysctl names.
kib [Wed, 18 Sep 2019 16:15:05 +0000 (16:15 +0000)]
truss: decode sysctl names.

Submitted by: Pawel Biernacki
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D21688

4 years agoAdd support for BERI statcounters.
br [Wed, 18 Sep 2019 16:13:50 +0000 (16:13 +0000)]
Add support for BERI statcounters.

BERI stands for Bluespec Extensible RISC Implementation, based on MIPS.

BERI has not implemented standard MIPS perfomance monitoring counters,
instead it provides statistical counters.

BERI statcounters have a several limitations:
- They can't be written
- They don't support start/stop operation
- None of hardware interrupt is provided on a counter overflow.

So make it separate to hwpmc_mips module and support process/system
counting mode only.

Sponsored by: DARPA, AFRL

4 years agosysctl: use names instead of magic numbers.
kib [Wed, 18 Sep 2019 16:13:10 +0000 (16:13 +0000)]
sysctl: use names instead of magic numbers.

Replace magic numbers with symbols for internal sysctl operations.
Convert in-kernel and libc consumers.

Submitted by: Pawel Biernacki
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D21693

4 years agoAdd the missing bits for LIBADD to properly function now that
bapt [Wed, 18 Sep 2019 08:02:03 +0000 (08:02 +0000)]
Add the missing bits for LIBADD to properly function now that
libarchive is linked to libzstd

Pointy hat: bapt
Reported by: antoine

4 years agoAdd native support for zstd to libarchive
bapt [Wed, 18 Sep 2019 07:57:56 +0000 (07:57 +0000)]
Add native support for zstd to libarchive

Note that old pkg will failed to build after this. A recent ports tree (one
providing pkg 1.12+) is required to build. Older already built pkg, should
continue working as expected

PR: 238797
Exp run by: antoine
Reviewed by: cem
Approved by: cem
Differential Revision: https://reviews.freebsd.org/D20752

4 years agosrc.conf(5): regenerate after r352465, r352466
kevans [Wed, 18 Sep 2019 02:04:41 +0000 (02:04 +0000)]
src.conf(5): regenerate after r352465, r352466

These changed the defaults for the GOOGLETEST knob and added a description
for WITH_GOOGLETEST.

4 years agoAdd description for WITH_GOOGLETEST
kevans [Wed, 18 Sep 2019 02:03:39 +0000 (02:03 +0000)]
Add description for WITH_GOOGLETEST

This is the logical negation of WITHOUT_GOOGLETEST, and helpful to have as
we now have different per-arch defaults for this option.

4 years agogoogletest: default-disable on all of MIPS for now
kevans [Wed, 18 Sep 2019 01:58:56 +0000 (01:58 +0000)]
googletest: default-disable on all of MIPS for now

Parts of the fusefs tests trigger a bug in current versions of llvm: IR
representation of some routine for the MIPS targets is a function with a
large number of arguments. This then leads the compiler on an hour+ long
goose chase, which is OK if you build the current tree but less-so if you're
trying external toolchain or doing a universe build involving mips when it
eventually gets switched over to LLVM.

Better, accurate details can be found in LLVM PR43263.

4 years agomips: ubldr: use truncated load address for mips32
kevans [Wed, 18 Sep 2019 01:33:17 +0000 (01:33 +0000)]
mips: ubldr: use truncated load address for mips32

BFD appears to silently truncate 0xffffffff80800000 when it processes the
ldscript for 32-bit mips, but LLD chokes on it as the linker script tries to
place elements above 32-bit range. It's unclear to me if silent truncation
is kosher or not and whether this patch is really what we want to do, but it
is one approach at least.

Reviewed by: imp, mizhka
Differential Revision: https://reviews.freebsd.org/D21487

4 years agoTemporarily skip sys.netpfil.common.tos.pf_tos on i386 CI as it always fails
lwhsu [Tue, 17 Sep 2019 22:09:14 +0000 (22:09 +0000)]
Temporarily skip sys.netpfil.common.tos.pf_tos on i386 CI as it always fails

PR: 240086
Sponsored by: The FreeBSD Foundation

4 years agoTemporarily skip sys.netpfil.common.forward.pf_v4 on i386 CI as it always fails
lwhsu [Tue, 17 Sep 2019 22:08:16 +0000 (22:08 +0000)]
Temporarily skip sys.netpfil.common.forward.pf_v4 on i386 CI as it always fails

PR: 240085
Sponsored by: The FreeBSD Foundation

4 years agoUse correct filename in newsyslog.conf
swills [Tue, 17 Sep 2019 20:05:06 +0000 (20:05 +0000)]
Use correct filename in newsyslog.conf

Approved by: bapt (implicit)
Differential Revision: https://reviews.freebsd.org/D21561

4 years agolog daemon.info to /var/log/daemon.log by default
swills [Tue, 17 Sep 2019 20:03:20 +0000 (20:03 +0000)]
log daemon.info to /var/log/daemon.log by default

log daemon facility now that daemon(8) has syslog support which defaults to
daemon facility, info priority

Reviewed by: bapt
Approved by: bapt
Differential Revision: https://reviews.freebsd.org/D21561

4 years agoifconfig: add report of the string from SIOCGIFDOWNREASON.
kib [Tue, 17 Sep 2019 18:51:10 +0000 (18:51 +0000)]
ifconfig: add report of the string from SIOCGIFDOWNREASON.

Sample output:
# ifconfig mce0
mce0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=3ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,TXRTLMT,HWRXTSTMP>
        ether e4:1d:2d:e7:10:0a
        media: Ethernet autoselect <full-duplex,rxpause,txpause>
        status: no carrier (Negotiation failure)
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Reviewed by: hselasky, rrs
Sponsored by: Mellanox Technologies
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D21527

4 years agoAdd SIOCGIFDOWNREASON.
kib [Tue, 17 Sep 2019 18:49:13 +0000 (18:49 +0000)]
Add SIOCGIFDOWNREASON.

The ioctl(2) is intended to provide more details about the cause of
the down for the link.

Eventually we might define a comprehensive list of codes for the
situations.  But interface also allows the driver to provide free-form
null-terminated ASCII string to provide arbitrary non-formalized
information.  Sample implementation exists for mlx5(4), where the
string is fetched from firmware controlling the port.

Reviewed by: hselasky, rrs
Sponsored by: Mellanox Technologies
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D21527

4 years agoFurther refine r352393, only call vnode_pager_setsize() outside the
kib [Tue, 17 Sep 2019 18:41:39 +0000 (18:41 +0000)]
Further refine r352393, only call vnode_pager_setsize() outside the
node lock when shrinking.

This is similar to r252528, applied to the above commit.

Apparently there is a race which makes necessary at least to keep the
n_size and pager size consistent when extending.  Current suspect is
that iod threads perform vnode_pager_setsize() without taking the
vnode lock, which corrupts the file content.

Reported and tested by: Masachika ISHIZUKA <ish@amail.plala.or.jp>
Discussed with: rmacklem (related issues)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

4 years agorealloc(x, 0) should not return NULL.
kib [Tue, 17 Sep 2019 18:36:29 +0000 (18:36 +0000)]
realloc(x, 0) should not return NULL.

See http://www.open-std.org/jtc1/sc22/wg14/www/docs/summary.htm#dr_400.
Upstream jemalloc issue is opened by emaste at
https://github.com/jemalloc/jemalloc/issues/1629.

Reviewed by: emaste
PR: 240456
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
DIfferential revision: https://reviews.freebsd.org/D21632

4 years agoReturn EISDIR when directory is opened with O_CREAT without O_DIRECTORY.
kib [Tue, 17 Sep 2019 18:32:18 +0000 (18:32 +0000)]
Return EISDIR when directory is opened with O_CREAT without O_DIRECTORY.

Reviewed by: bcr (man page), emaste (previous version)
PR: 240452
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
DIfferential revision: https://reviews.freebsd.org/D21634

4 years agoloader: provide u> and xemit words if needed
tsoome [Tue, 17 Sep 2019 18:05:33 +0000 (18:05 +0000)]
loader: provide u> and xemit words if needed

We have external interpreter (userboot.so) which may be lagging behind
with updates and may be missing u> xemit words.

4 years agoThe VFS-level clustering code collects together sequential blocks
mckusick [Tue, 17 Sep 2019 17:44:50 +0000 (17:44 +0000)]
The VFS-level clustering code collects together sequential blocks
by issuing delayed-writes (bdwrite()) until a non-sequential block
is written or the maximum cluster size is reached. At that point
it collects the delayed buffers together (using bread()) to write
them in a single operation. The assumption was that since we just
looked at them they will still be in memory so there is no need to
check for a read error from bread(). Very occationally (apparently
every 10-hours or so when being pounded by Peter Holm's tests)
this assumption is wrong.

The fix is to check for errors from bread() and fail the cluster
write thus falling back to the default individual flushing of any
still dirty buffers.

Reported by: Peter Holm and Chuck Silvers
Reviewed by: kib
MFC after:   3 days

4 years agoloader: revert r352421
tsoome [Tue, 17 Sep 2019 16:16:46 +0000 (16:16 +0000)]
loader: revert r352421

As insisted by kib, malloc(0) is quite legal.

4 years agovfs: apply r352437 to the fast path as well
mjg [Tue, 17 Sep 2019 15:53:40 +0000 (15:53 +0000)]
vfs: apply r352437 to the fast path as well

This one is very hard to run into. If the filesystem is being unmounted or
the mount point is freed the vfs_op_thread_enter will fail. For it to
succeed the mount point itself would have to be reallocated in the time
window between the initial read and the attempt to enter.

Sponsored by: The FreeBSD Foundation

4 years agofixup up fallout from r352447 in libbe
avg [Tue, 17 Sep 2019 14:15:48 +0000 (14:15 +0000)]
fixup up fallout from r352447 in libbe

I totally forgot that we now have another in-tree consumer of libzfs.

MFC after: 3 days
X-MFC with: r352447

4 years agoTemporarily skip flakey test case lib.libc.sys.stat_test.stat_socket
lwhsu [Tue, 17 Sep 2019 14:08:09 +0000 (14:08 +0000)]
Temporarily skip flakey test case lib.libc.sys.stat_test.stat_socket

PR: 240621
Sponsored by: The FreeBSD Foundation

4 years agoMFZoL: Add -vnP support to 'zfs send' for bookmarks
avg [Tue, 17 Sep 2019 13:58:15 +0000 (13:58 +0000)]
MFZoL: Add -vnP support to 'zfs send' for bookmarks

zfsonlinux/zfs@835db58592d7d947e5818eb7281882e2a46073e0

We have long supported estimating a size of an incremental stream from a
snapshot.  We should do the same for bookmarks as well.

Obtained from: ZoL
Author: loli10K <ezomori.nozomu@gmail.com>
MFC after: 3 days

4 years agoloader.efi: efipart needs to use ioalign
tsoome [Tue, 17 Sep 2019 13:50:25 +0000 (13:50 +0000)]
loader.efi: efipart needs to use ioalign

UEFI specification 2.7A, EFI_BLOCK_IO_PROTOCOL, page 566.

The ioalign property does define the alignment of data buffer.

If the alignment is required and our buffer is not aligned, or if
the data buffer is not multiple of Blocksize, we need to use bounce buffer
to perform the block IO. This is much like with BIOS version, except
there the INT13 needs buffer to be located in low memory.

Additionally, we need to handle disk writes properly.

4 years agoloader: add memalign() to libsa
tsoome [Tue, 17 Sep 2019 13:15:27 +0000 (13:15 +0000)]
loader: add memalign() to libsa

Implement memalign(size_t alignment, size_t size) to allocate aligned memory.

4 years agoloader: stand.h should define reallocf as Reallocf
tsoome [Tue, 17 Sep 2019 13:07:02 +0000 (13:07 +0000)]
loader: stand.h should define reallocf as Reallocf

Use the same approach as other zalloc functions.

4 years agoloader: cstyle cleanup libsa zalloc sources
tsoome [Tue, 17 Sep 2019 11:35:53 +0000 (11:35 +0000)]
loader: cstyle cleanup libsa zalloc sources

Clean up libstand zalloc* sources. Note that it is not 100% whitespace cleanup.
I also reduced block in znalloc and zfree as those were obvious simplifications
and did help to save one level of indent.

4 years agoloader: factor out guard location setup from Realloc
tsoome [Tue, 17 Sep 2019 09:47:35 +0000 (09:47 +0000)]
loader: factor out guard location setup from Realloc

To simplify and make it easier to read, use ga_Bytes field from guard
structure.

4 years agoOnly allow a SCTP-AUTH shared key to be updated by the application
tuexen [Tue, 17 Sep 2019 09:46:42 +0000 (09:46 +0000)]
Only allow a SCTP-AUTH shared key to be updated by the application
if it is not deactivated and not used.
This avoids a use-after-free problem.

Reported by: da_cheng_shao@yeah.net
MFC after: 3 days

4 years agovfs: fix braino resulting in NULL pointer deref in r352424
mjg [Tue, 17 Sep 2019 08:09:39 +0000 (08:09 +0000)]
vfs: fix braino resulting in NULL pointer deref in r352424

The breakage was added after all the testing and the testing which followed
was not sufficient to find it.

Reported by: pho
Sponsored by: The FreeBSD Foundation

4 years agomips: move support for temporary mappings above KSEG0 to per-CPU data
jah [Tue, 17 Sep 2019 03:39:31 +0000 (03:39 +0000)]
mips: move support for temporary mappings above KSEG0 to per-CPU data

This is derived from similar work done in r310481 for i386 and r312610 for
armv6/armv7. Additionally, use a critical section to keep the thread
pinned for per-CPU operations instead of completely disabling local interrupts.

No objections from: adrian, jmallett, imp
Differential Revision:  https://reviews.freebsd.org/D18593

4 years agoRemove dead code from vm_map_unlink_entry made dead by r351476, and also
dougm [Tue, 17 Sep 2019 02:53:59 +0000 (02:53 +0000)]
Remove dead code from vm_map_unlink_entry made dead by r351476, and also
a no-longer-used enumerant.

Reviewed by: alc
Approved by: markj (mentor, implicit)
Tested by: pho (as part of a larger change)
Differential Revision: https://reviews.freebsd.org/D21668

4 years agoAmend r352422, add missing 'n'
dmgk [Tue, 17 Sep 2019 00:22:20 +0000 (00:22 +0000)]
Amend r352422, add missing 'n'

Reported by: yuripv
Approved by: araujo (mentor)
Differential Revision: https://reviews.freebsd.org/D21679

4 years agofix the article to be correct...
jmg [Mon, 16 Sep 2019 22:48:40 +0000 (22:48 +0000)]
fix the article to be correct...

4 years agoRISC-V: Support EARLY_AP_STARTUP
mhorne [Mon, 16 Sep 2019 22:17:16 +0000 (22:17 +0000)]
RISC-V: Support EARLY_AP_STARTUP

The EARLY_AP_STARTUP option initializes non-boot processors
much sooner during startup. This adds support for this option
on RISC-V and enables it by default for GENERIC.

Reviewed by: jhb, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21661

4 years agovfs: convert struct mount counters to per-cpu
mjg [Mon, 16 Sep 2019 21:37:47 +0000 (21:37 +0000)]
vfs: convert struct mount counters to per-cpu

There are 3 counters modified all the time in this structure - one for
keeping the structure alive, one for preventing unmount and one for
tracking active writers. Exact values of these counters are very rarely
needed, which makes them a prime candidate for conversion to a per-cpu
scheme, resulting in much better performance.

Sample benchmark performing fstatfs (modifying 2 out of 3 counters) on
a 104-way 2 socket Skylake system:
before:   852393 ops/s
after:  76682077 ops/s

Reviewed by: kib, jeff
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21637

4 years agovfs: manage mnt_writeopcount with atomics
mjg [Mon, 16 Sep 2019 21:33:16 +0000 (21:33 +0000)]
vfs: manage mnt_writeopcount with atomics

See r352424.

Reviewed by: kib, jeff
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21575

4 years agovfs: manage mnt_lockref with atomics
mjg [Mon, 16 Sep 2019 21:32:21 +0000 (21:32 +0000)]
vfs: manage mnt_lockref with atomics

See r352424.

Reviewed by: kib, jeff
Tested by: pho
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21574

4 years agovfs: manage mnt_ref with atomics
mjg [Mon, 16 Sep 2019 21:31:02 +0000 (21:31 +0000)]
vfs: manage mnt_ref with atomics

New primitive is introduced to denote sections can operate locklessly
on aspects of struct mount, but which can also be disabled if necessary.
This provides an opportunity to start scaling common case modifications
while providing stable state of the struct when facing unmount, write
suspendion or other events.

mnt_ref is the first counter to start being managed in this manner with
the intent to make it per-cpu.

Reviewed by: kib, jeff
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21425

4 years agoAdd myself (dmgk) to calendar.freebsd
dmgk [Mon, 16 Sep 2019 20:43:20 +0000 (20:43 +0000)]
Add myself (dmgk) to calendar.freebsd

Approved by: tz (mentor)
Differential Revision: https://reviews.freebsd.org/D21675

4 years agoAdd myself (dmgk) as a ports committer
dmgk [Mon, 16 Sep 2019 20:41:37 +0000 (20:41 +0000)]
Add myself (dmgk) as a ports committer

Approved by: tz (mentor)
Differential Revision: https://reviews.freebsd.org/D21672

4 years agoloader: Malloc(0) should return NULL.
tsoome [Mon, 16 Sep 2019 20:28:08 +0000 (20:28 +0000)]
loader: Malloc(0) should return NULL.

We really should not allocate anything with size 0.

4 years agoloader_4th: scan_buffer can leave empty string on stack
tsoome [Mon, 16 Sep 2019 20:26:53 +0000 (20:26 +0000)]
loader_4th: scan_buffer can leave empty string on stack

When the file processing is done, we will have string with lenght 0 in stack and we will attempt to
allocate 0 bytes.

4 years agoFix an off-by-one error from r351961
asomers [Mon, 16 Sep 2019 16:41:01 +0000 (16:41 +0000)]
Fix an off-by-one error from r351961

That revision addressed a Coverity CID that could lead to a buffer overflow,
but it had an off-by-one error in the buffer size check.

Reported by: Coverity
Coverity CID: 1405530
MFC after: 3 days
MFC-With: 351961
Sponsored by: The FreeBSD Foundation

4 years agofusefs: initialize C++ classes the Coverity way
asomers [Mon, 16 Sep 2019 15:56:21 +0000 (15:56 +0000)]
fusefs: initialize C++ classes the Coverity way

Coverity complained that I wasn't initializing some class members until the
SetUp method.  Do it in the constructor instead.

Reported by: Coverity
Coverity CIDs: 14043521404378
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

4 years agofusefs: fix some minor Coverity CIDs in the tests
asomers [Mon, 16 Sep 2019 15:44:59 +0000 (15:44 +0000)]
fusefs: fix some minor Coverity CIDs in the tests

Where open(2) is expected to fail, the tests should assert or expect that
its return value is -1.  These tests all accepted too much but happened to
pass anyway.

Reported by: Coverity
Coverity CID: 1404512140437814045041404483
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation

4 years agoAssert that the refcount value is not VPRC_BLOCKED in vm_page_drop().
markj [Mon, 16 Sep 2019 15:16:48 +0000 (15:16 +0000)]
Assert that the refcount value is not VPRC_BLOCKED in vm_page_drop().

VPRC_BLOCKED is a refcount flag used to indicate that a thread is
tearing down mappings of a page.  When set, it causes attempts to wire a
page via a pmap lookup to fail.  It should never represent the last
reference to a page, so assert this.

Suggested by: kib
Reviewed by: alc, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21639

4 years agoFix a race in vm_page_dequeue_deferred_free() after r352110.
markj [Mon, 16 Sep 2019 15:12:49 +0000 (15:12 +0000)]
Fix a race in vm_page_dequeue_deferred_free() after r352110.

This function loaded the page's queue index before setting PGA_DEQUEUE.
In this window the page daemon may have deactivated the page, updating
its queue index.  Make the operation atomic using vm_page_pqstate_cmpset();
the page daemon will not modify the page once it observes that PGA_DEQUEUE
is set.

Reported and tested by: pho
Reviewed by: alc, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21639

4 years agoFix a page leak in vm_page_reclaim_run().
markj [Mon, 16 Sep 2019 15:09:31 +0000 (15:09 +0000)]
Fix a page leak in vm_page_reclaim_run().

After r352110 the attempt to remove mappings of the page being replaced
may fail if the page is wired.  In this case we must free the replacement
page.

Reviewed by: alc, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21639

4 years agoFix a couple of nits in r352110.
markj [Mon, 16 Sep 2019 15:06:19 +0000 (15:06 +0000)]
Fix a couple of nits in r352110.

- Remove a dead variable from the amd64 pmap_extract_and_hold().
- Fix grammar in the vm_page_wire man page.

Reported by: alc
Reviewed by: alc, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21639

4 years agoRevert r352406, which contained changes I didn't intend to commit.
markj [Mon, 16 Sep 2019 15:04:45 +0000 (15:04 +0000)]
Revert r352406, which contained changes I didn't intend to commit.

4 years agoFix a couple of nits in r352110.
markj [Mon, 16 Sep 2019 15:03:12 +0000 (15:03 +0000)]
Fix a couple of nits in r352110.

- Remove a dead variable from the amd64 pmap_extract_and_hold().
- Fix grammar in the vm_page_wire man page.

Reported by: alc
Reviewed by: alc, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21639

4 years agofusefs: fix some minor issues with fuse_vnode_setparent
asomers [Mon, 16 Sep 2019 14:51:49 +0000 (14:51 +0000)]
fusefs: fix some minor issues with fuse_vnode_setparent

* When unparenting a vnode, actually clear the flag. AFAIK this is basically
  a no-op because we only unparent a vnode when reclaiming it or when
  unlinking.

* There's no need to call fuse_vnode_setparent during reclaim, because we're
  about to free the vnode data anyway.

Reviewed by: emaste
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D21630

4 years agonfscl_loadattrcache: fix rest of the cases to not call
kib [Mon, 16 Sep 2019 13:26:27 +0000 (13:26 +0000)]
nfscl_loadattrcache: fix rest of the cases to not call
vnode_pager_setsize() under the node mutex.

r248567 moved some calls of vnode_pager_setsize() after the node lock
is unlocked, do the rest now.

Reported and tested by: peterj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

4 years agosbuf(9): fix sbuf_drain_func typedef markup
yuripv [Mon, 16 Sep 2019 13:10:03 +0000 (13:10 +0000)]
sbuf(9): fix sbuf_drain_func typedef markup

Reviewed by: 0mp (previous version)
Differential Revision: https://reviews.freebsd.org/D21569

4 years agopkgbase: Move cap_mkdb from runtime to utilities POST-INSTALL
manu [Mon, 16 Sep 2019 12:51:30 +0000 (12:51 +0000)]
pkgbase: Move cap_mkdb from runtime to utilities POST-INSTALL

Since login and login.conf moved to the utilities packages move also
the post-install related commands.

Reported by: mj-mailinglist@gmx.de
Reviewed by: bapt

4 years agoFix 20190507 UPDATING entry
kevans [Mon, 16 Sep 2019 12:44:44 +0000 (12:44 +0000)]
Fix 20190507 UPDATING entry

The rc mechanism for loading kernel modules is actually called 'kld_list',
not 'kld_load'

Reported by: yuripv

4 years agoDon't write to memory outside of the allocated array for SACK blocks.
tuexen [Mon, 16 Sep 2019 08:18:05 +0000 (08:18 +0000)]
Don't write to memory outside of the allocated array for SACK blocks.

Obtained from: rrs@
MFC after: 3 days
Sponsored by: Netflix, Inc.

4 years agoDo not use our custom completion function, it is not needed anymore
bapt [Mon, 16 Sep 2019 07:31:59 +0000 (07:31 +0000)]
Do not use our custom completion function, it is not needed anymore

4 years agoIncrease the size of the send and receive buffers for YP client rpc
kib [Mon, 16 Sep 2019 06:42:01 +0000 (06:42 +0000)]
Increase the size of the send and receive buffers for YP client rpc
calls to max allowed UDP datagram size.

Since max allowed size both for keys and values where increased, the
old sizes of around 1K cause ypmatch(3) failures, while plain maps
fetches work.

The buffers were reduced in r34146 from default UDP rpcclient values
to 1024/2304 due to the key and value size being 1K.

Reviewed by: slavash
Sponsored by: Mellanox Technologies
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D21586

4 years agoDocument logic for __DEFAULT_DEPENDENT_OPTIONS
sjg [Mon, 16 Sep 2019 00:32:23 +0000 (00:32 +0000)]
Document logic for __DEFAULT_DEPENDENT_OPTIONS

Reviewed by: stevek
Differential Revision: https://reviews.freebsd.org/D21640

4 years agoRelax TX draining in ns8250_bus_transmit().
mav [Sun, 15 Sep 2019 23:56:39 +0000 (23:56 +0000)]
Relax TX draining in ns8250_bus_transmit().

Since TX interrupt is generated when THRE is set, wait for TEMT set means
wait for full character transmission time.  At low speeds that may take
awhile, burning CPU time while holding sc_hwmtx lock, also congested.

This is partial revert of r317659.

PR: 240121
MFC after: 2 weeks

4 years agoAvoid mixing cluster numbers and sector numbers. Makes code more readable.
delphij [Sun, 15 Sep 2019 19:41:54 +0000 (19:41 +0000)]
Avoid mixing cluster numbers and sector numbers. Makes code more readable.

Obtained from: NetBSD
MFC after: 2 weeks

4 years agoApply a runtime patch to the FDT data for imx6 to fix iomuxc problems.
ian [Sun, 15 Sep 2019 19:38:15 +0000 (19:38 +0000)]
Apply a runtime patch to the FDT data for imx6 to fix iomuxc problems.

The latest imported FDT data defines a node for an iomuxc-gpr device,
which we don't support (or need, right now) in addition to the usual
iomuxc device.  Unfortunately, the dts improperly assigns overlapping
ranges of mmio space to both devices.  The -gpr device is also a syscon
and simple_mfd device.

At runtime the simple_mfd driver attaches for the iomuxc-gpr node, then
when the real iomuxc driver comes along later, it fails to attach because
it tries to allocate its register space, and it's already partially in
use by the bogus instance of simple_mfd.

This change works around the problem by simply disabling the node for
the iomuxc-gpr device, since we don't need it for anything.

4 years agoWhen the IP layer calls back into the SCTP layer to perform the SCTP
tuexen [Sun, 15 Sep 2019 18:29:45 +0000 (18:29 +0000)]
When the IP layer calls back into the SCTP layer to perform the SCTP
checksum computation, do not assume that the IP header chain and the
SCTP common header are in contiguous memory although the SCTP lays
out the mbuf chains that way. If there are IP-level options inserted
by the IP layer, the constraint is not fulfilled anymore.

This issues was found by running syzkaller. Thanks to markj@ who is
running an instance which also provides kernel dumps. This allowed me
to find this issue.

MFC after: 3 days

4 years agorangelock: add rangelock_cookie_assert
kevans [Sun, 15 Sep 2019 02:59:53 +0000 (02:59 +0000)]
rangelock: add rangelock_cookie_assert

A future change to posixshm to add file sealing (in DIFF_21391[0] and child)
will move locking out of shm_dotruncate as kern_shm_open() will require the
lock to be held across the dotruncate until the seal is actually applied.
For this, the cookie is passed into shm_dotruncate_locked which asserts
RCA_WLOCKED.

[0] Name changed to protect the innocent, hopefully, from getting autoclosed
due to this reference...

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D21628

4 years agoMake the ti_sysc device quiet. It's an internal utility pseudo-device
ian [Sun, 15 Sep 2019 01:02:01 +0000 (01:02 +0000)]
Make the ti_sysc device quiet.  It's an internal utility pseudo-device
that makes the upstream FDT data work right, so we don't need to see a
couple dozen instances of it spam the dmesg at boot time unless it's a
verbose boot.

4 years agoFix arm and aarch64 builds of libedit after r352275
dim [Sat, 14 Sep 2019 21:49:42 +0000 (21:49 +0000)]
Fix arm and aarch64 builds of libedit after r352275

On arm and arm64, where chars are unsigned by default, buildworld dies
with:

--- terminal.o ---
/usr/src/contrib/libedit/terminal.c:569:41: error: comparison of
integers of different signs: 'wint_t' (aka 'int') and 'wchar_t' (aka
'unsigned int') [-Werror,-Wsign-compare]
                                     el->el_cursor.v][where & 0370] !=
                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/usr/src/contrib/libedit/terminal.c:659:28: error: comparison of
integers of different signs: 'wint_t' (aka 'int') and 'wchar_t' (aka
'unsigned int') [-Werror,-Wsign-compare]
                                     [el->el_cursor.h] == MB_FILL_CHAR)
                                     ~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~

Fix this by making MB_FILL_CHAR a wint_t, so no casting is needed.

Note that in https://reviews.freebsd.org/D21584 this was also proposed
by Yuichiro Naito <naito.yuichiro_gmail.com>.

Reviewed by: bapt
Subscribers: naito.yuichiro_gmail.com, ml_vishwin.info
MFC after: 3 weeks
X-MFC-With: r352275
Differential Revision: https://reviews.freebsd.org/D21657

4 years agoFix aux_info corruption in rtld direct execution mode.
bdragon [Sat, 14 Sep 2019 21:18:10 +0000 (21:18 +0000)]
Fix aux_info corruption in rtld direct execution mode.

After the aux vector is moved, it is necessary to re-digest aux_info so the
pointers are updated to the new locations.

This was causing thread creation to fail on powerpc64 when using direct
execution due to a nonsense value being read for aux_info[AT_STACKPROT].

Approved by: jhibbits (mentor)
Differential Revision: https://reviews.freebsd.org/D21656