]> CyberLeo.Net >> Repos - FreeBSD/FreeBSD.git/log
FreeBSD/FreeBSD.git
5 years agoMark the send queue ready so ALTQ is available.
Patrick Kelsey [Sat, 4 Aug 2018 01:45:17 +0000 (01:45 +0000)]
Mark the send queue ready so ALTQ is available.

5 years agoNow that after r335979 the kernel addresses in API structures are
Gleb Smirnoff [Sat, 4 Aug 2018 00:03:21 +0000 (00:03 +0000)]
Now that after r335979 the kernel addresses in API structures are
fixed size, there is no reason left for the unions.

Discussed with: brooks

5 years agoUse if_tunnel_check_nesting() for ng_iface(4).
Gleb Smirnoff [Fri, 3 Aug 2018 22:55:58 +0000 (22:55 +0000)]
Use if_tunnel_check_nesting() for ng_iface(4).

5 years agoarm: Remove ALLWINNER_UP kernel config
Emmanuel Vadot [Fri, 3 Aug 2018 22:15:58 +0000 (22:15 +0000)]
arm: Remove ALLWINNER_UP kernel config

This was needed when we GENERIC couldn't boot on UP system.

5 years agodtb: rpi: Only compile and copy the DTSO
Emmanuel Vadot [Fri, 3 Aug 2018 22:06:15 +0000 (22:06 +0000)]
dtb: rpi: Only compile and copy the DTSO

The DTB is now loaded via the firmware, passed to u-boot then to loader.efi
Only compile and copy the dts overlays.

5 years agodtb: am335x: Remove links and add more dts
Emmanuel Vadot [Fri, 3 Aug 2018 22:04:00 +0000 (22:04 +0000)]
dtb: am335x: Remove links and add more dts

The links were to cope with the switch to upstream dts.
We don't need them anymore.
While here add the rest of the beaglebone family dts as u-boot is common
on all those boards and load the dtb based on the product name.
This just miss the pocketbeagle variant as it's not yet in sys/gnu/dts but
will be with the Linux 4.18 dts import.

5 years agonvme(4): Add bus_dmamap_sync() at the end of the request path
Justin Hibbits [Fri, 3 Aug 2018 20:04:06 +0000 (20:04 +0000)]
nvme(4): Add bus_dmamap_sync() at the end of the request path

Summary:
Some architectures, in this case powerpc64, need explicit synchronization
barriers vs device accesses.

Prior to this change, when running 'make buildworld -j72' on a 18-core
(72-thread) POWER9, I would see controller resets often.  With this change, I
don't see these resets messages, though another tester still does, for yet to be
determined reasons, so this may not be a complete fix.  Additionally, I see a
~5-10% speed up in buildworld times, likely due to not needing to reset the
controller.

Reviewed By: jimharris
Differential Revision: https://reviews.freebsd.org/D16570

5 years agoFix some filemon path logging issues.
Bryan Drewery [Fri, 3 Aug 2018 19:24:04 +0000 (19:24 +0000)]
Fix some filemon path logging issues.

- Properly handle snprintf return value for truncation and avoid
  overflowing the later write with the bogus length.
- Increase the msgbufr size to handle a rename of 2 full files.

The larger allocation causes a slight performance hit which will be mitigated
in the future.  A rewrite with sbufs will likely be done as well.

Reported by: Ilja Van Sprundel <ivansprundel@ioactive.com>
MFC after: 2 weeks
Approved by: so (gtetlow)
Reviewed by: kib
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D16098

5 years agoSome drives report a geometry that is inconsisetent with the total
Cy Schubert [Fri, 3 Aug 2018 19:11:00 +0000 (19:11 +0000)]
Some drives report a geometry that is inconsisetent with the total
number of sectors reported through the BIOS. Cylinders * heads *
sectors may not necessarily be equal to the total number of sectors
reported through int13h function 48h.

An example of this is when a Mediasonic HD3-U2B PATA to USB enclosure
with a 80 GB disk is attached. Loader hangs at line 506 of
stand/i386/libi386/biosdisk.c while attempting to read sectors beyond
the end of the disk, sector 156906855. I discovered that the Mediasonic
enclosure was reporting the disk with 9767 cylinders, 255 heads, 63
sectors/track. That's 156906855 sectors. However camcontrol and
Windows 10 both report report the disk having 156301488 sectors, not
the calculated value. At line 280 biosdisk.c sets the sectors to the
higher of either bd->bd_sectors or the total calculated at line 276
(156906855) instead of the lower and correct value of 156301488 reported
by int 13h 48h.

This was tested on all three of my Mediasonic HD3-U2B PATA to USB
enclosures.

Instead of using the higher of bd_sectors (returned by int13h) or the
calculated value, this patch uses the lower and safer of the values.

Reviewed by: tsoome@
Differential Revision: https://reviews.freebsd.org/D16577

5 years agoInstall the 32-bit compat sanitizer libraries.
John Baldwin [Fri, 3 Aug 2018 18:52:51 +0000 (18:52 +0000)]
Install the 32-bit compat sanitizer libraries.

The lib32 build was already building the i386 version of
the clang sanitizers (libclang_rt) but they were not being
installed.  This enables the installation.

MK_TOOLCHAIN=no was originally added to the install make
environment to disable includes so that NO_INCS could be
removed.  The MK_TOOLCHAIN in bsd.incs.mk was subsequently
renamed to MK_INCLUDES, but bsd.lib.mk doesn't even include
bsd.incs.mk when LIBRARIES_ONLY is defined which the install
make environment for compat libs now defines.  However,
setting MK_TOOLCHAIN=no forced MK_CLANG=no which disabled
libclang_rt during the install32 phase.  Remove MK_TOOLCHAIN=no
since LIBRARIES_ONLY is now sufficient.

Since the libcompat environment overrides both LIBDIR and
SHLIBDIR, libclang_rt/Makefile.inc has to set both variables
to force the libraries to be installed to the location
expected by the compiler.

Reviewed by: bdrewery, dim
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16574

5 years agoRequire write access when mmapping BAR.
Konstantin Belousov [Fri, 3 Aug 2018 18:35:20 +0000 (18:35 +0000)]
Require write access when mmapping BAR.

This actually makes the rights requirements for accessing PCI config
space and BARs using /dev/pci same.  Since unchanged /dev/pci mode
only allows write open for root, default configuration de-facto limits
the BAR read to root only.  In particular, state-changing reads of the
registers are limited to root.

Discussed with: se
Suggested and reviewed by: jhb (kernel part)
Sponsored by: The FreeBSD Foundation
MFC after: 12 days
Differential revision: https://reviews.freebsd.org/D16580

5 years agoFix word-size dumping.
Konstantin Belousov [Fri, 3 Aug 2018 18:30:02 +0000 (18:30 +0000)]
Fix word-size dumping.

Sponsored by: The FreeBSD Foundation
MFC after: 12 days

5 years agoReturn ENAMETOOLONG if the latest copied character
Ruslan Bukin [Fri, 3 Aug 2018 16:44:56 +0000 (16:44 +0000)]
Return ENAMETOOLONG if the latest copied character
is not null terminator.

Sponsored by: DARPA, AFRL

5 years agoAdd the required page accounting to kmem_bootstrap_free().
Mark Johnston [Fri, 3 Aug 2018 16:35:37 +0000 (16:35 +0000)]
Add the required page accounting to kmem_bootstrap_free().

Reviewed by: alc, kib
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16581

5 years agoMAXLOGNAME changed to 33 in r243023.
Ruslan Bukin [Fri, 3 Aug 2018 16:05:03 +0000 (16:05 +0000)]
MAXLOGNAME changed to 33 in r243023.
Update man pages.

Sponsored by: DARPA, AFRL

5 years agoFix typo in copyinstr_smap, resulting in mis-handling of too long strings.
Konstantin Belousov [Fri, 3 Aug 2018 15:35:29 +0000 (15:35 +0000)]
Fix typo in copyinstr_smap, resulting in mis-handling of too long strings.

Reported and tested by: pho
PR: 230286
Sponsored by: The FreeBSD Foundation

5 years agosafer wait-free iteration of shared interrupt handlers
Andriy Gapon [Fri, 3 Aug 2018 14:27:28 +0000 (14:27 +0000)]
safer wait-free iteration of shared interrupt handlers

The code that iterates a list of interrupt handlers for a (shared)
interrupt, whether in the ISR context or in the context of an interrupt
thread, does so in a lock-free fashion.   Thus, the routines that modify
the list need to take special steps to ensure that the iterating code
has a consistent view of the list.  Previously, those routines tried to
play nice only with the code running in the ithread context.  The
iteration in the ISR context was left to a chance.

After commit r336635 atomic operations and memory fences are used to
ensure that ie_handlers list is always safe to navigate with respect to
inserting and removal of list elements.

There is still a question of when it is safe to actually free a removed
element.

The idea of this change is somewhat similar to the idea of the epoch
based reclamation.  There are some simplifications comparing to the
general epoch based reclamation.  All writers are serialized using a
mutex, so we do not need to worry about concurrent modifications.  Also,
all read accesses from the open context are serialized too.

So, we can get away just two epochs / phases.  When a thread removes an
element it switches the global phase from the current phase to the other
and then drains the previous phase.  Only after the draining the removed
element gets actually freed. The code that iterates the list in the ISR
context takes a snapshot of the global phase and then increments the use
count of that phase before iterating the list.  The use count (in the
same phase) is decremented after the iteration.  This should ensure that
there should be no iteration over the removed element when its gets
freed.

This commit also simplifies the coordination with the interrupt thread
context.  Now we always schedule the interrupt thread when removing one
of handlers for its interrupt.  This makes the code both simpler and
safer as the interrupt thread masks the interrupt thus ensuring that
there is no interaction with the ISR context.

P.S.  This change matters only for shared interrupts and I realize that
those are becoming a thing of the past (and quickly).  I also understand
that the problem that I am trying to solve is extremely rare.

PR: 229106
Reviewed by: cem
Discussed with: Samy Al Bahra
MFC after: 5 weeks
Differential Revision: https://reviews.freebsd.org/D15905

5 years agoReplace __riscv__ with __riscv.
Ruslan Bukin [Fri, 3 Aug 2018 12:47:54 +0000 (12:47 +0000)]
Replace __riscv__ with __riscv.

__riscv__ is not pre-defined anymore by latest version of GNU compiler.

Sponsored by: DARPA, AFRL

5 years agoIncrease timeout for nop_test:stripesize.
Ruslan Bukin [Fri, 3 Aug 2018 12:16:02 +0000 (12:16 +0000)]
Increase timeout for nop_test:stripesize.

It takes 49s to complete this test in QEMU/RISC-V.

Sponsored by: DARPA, AFRL

5 years agoSome updates to vm_map(9).
Konstantin Belousov [Fri, 3 Aug 2018 12:14:29 +0000 (12:14 +0000)]
Some updates to vm_map(9).

Remove min_offset, max_offset.  Mention the busy flag.

Prodded by: https://reviews.freebsd.org/D16105.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week

5 years agoRemove pointless comment.
Edward Tomasz Napierala [Fri, 3 Aug 2018 10:59:05 +0000 (10:59 +0000)]
Remove pointless comment.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL

5 years ago.Xr pam_securetty(8) from ttys(5), to hint the users as to what actually
Edward Tomasz Napierala [Fri, 3 Aug 2018 10:58:13 +0000 (10:58 +0000)]
.Xr pam_securetty(8) from ttys(5), to hint the users as to what actually
enforces the 'secure' tty flag.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL

5 years agoImplement ktime_add_ms() and ktime_before() in the LinuxKPI.
Hans Petter Selasky [Fri, 3 Aug 2018 09:02:57 +0000 (09:02 +0000)]
Implement ktime_add_ms() and ktime_before() in the LinuxKPI.

Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies

5 years agoloader.efi: clean up misleading noise from missing block devices
Toomas Soome [Fri, 3 Aug 2018 07:59:29 +0000 (07:59 +0000)]
loader.efi: clean up misleading noise from missing block devices

If there are no block devices, there is no need to printout
error (ENOENT).

In case of netboot, our image path has no block device, no need to make
noise about it.

5 years agoVerify that each frame pointer lies within the thread's kstack.
Mark Johnston [Fri, 3 Aug 2018 02:51:37 +0000 (02:51 +0000)]
Verify that each frame pointer lies within the thread's kstack.

Previously, this check was omitted for the first frame pointer.

Reported by: pho
Reviewed by: kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16572

5 years agoReduce taskq and context-switch cost of zio pipe
Alexander Motin [Fri, 3 Aug 2018 02:16:45 +0000 (02:16 +0000)]
Reduce taskq and context-switch cost of zio pipe

When doing a read from disk, ZFS creates 3 ZIO's: a zio_null(), the
logical zio_read(), and then a physical zio. Currently, each of these
results in a separate taskq_dispatch(zio_execute).

On high-read-iops workloads, this causes a significant performance
impact. By processing all 3 ZIO's in a single taskq entry, we reduce the
overhead on taskq locking and context switching.  We accomplish this by
allowing zio_done() to return a "next zio to execute" to zio_execute().

This results in a ~12% performance increase for random reads, from
96,000 iops to 108,000 iops (with recordsize=8k, on SSD's).

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: George Wilson <george.wilson@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
External-issue: DLPX-59292
Closes #7736

zfsonlinux/zfs@62840030a7dceaee013ddbcc1eebcfc7922edf7c

5 years agoMFV r337223:
Alexander Motin [Fri, 3 Aug 2018 01:52:25 +0000 (01:52 +0000)]
MFV r337223:
9580 Add a hash-table on top of nvlist to speed-up operations

illumos/illumos-gate@2ec7644aab2a726a64681fa66c6db8731b160de1

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Serapheim Dimitropoulos <serapheim@delphix.com>

5 years ago9580 Add a hash-table on top of nvlist to speed-up operations
Alexander Motin [Fri, 3 Aug 2018 01:37:14 +0000 (01:37 +0000)]
9580 Add a hash-table on top of nvlist to speed-up operations

illumos/illumos-gate@2ec7644aab2a726a64681fa66c6db8731b160de1

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Serapheim Dimitropoulos <serapheim@delphix.com>

5 years agoFix LOCAL_PEERCRED with socketpair(2)
Alan Somers [Fri, 3 Aug 2018 01:37:00 +0000 (01:37 +0000)]
Fix LOCAL_PEERCRED with socketpair(2)

Enable the LOCAL_PEERCRED socket option for unix domain stream sockets
created with socketpair(2). Previously, it only worked with unix domain
stream sockets created with socket(2)/listen(2)/connect(2)/accept(2).

PR: 176419
Reported by: Nicholas Wilson <nicholas@nicholaswilson.me.uk>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16350

5 years agoMFV r337220: 8375 Kernel memory leak in nvpair code
Alexander Motin [Fri, 3 Aug 2018 01:30:03 +0000 (01:30 +0000)]
MFV r337220: 8375 Kernel memory leak in nvpair code

illumos/illumos-gate@843c2111b160463f014d325560ad4b051711928e

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

5 years ago8375 Kernel memory leak in nvpair code
Alexander Motin [Fri, 3 Aug 2018 01:28:11 +0000 (01:28 +0000)]
8375 Kernel memory leak in nvpair code

illumos/illumos-gate@843c2111b160463f014d325560ad4b051711928e

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

5 years agoMFV r337218: 7261 nvlist code should enforce name length limit
Alexander Motin [Fri, 3 Aug 2018 01:26:07 +0000 (01:26 +0000)]
MFV r337218: 7261 nvlist code should enforce name length limit

illumos/illumos-gate@48dd5e630c9b1773b7b10d08a3b90b6c9062d713

Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

5 years ago7261 nvlist code should enforce name length limit
Alexander Motin [Fri, 3 Aug 2018 01:17:14 +0000 (01:17 +0000)]
7261 nvlist code should enforce name length limit

illumos/illumos-gate@48dd5e630c9b1773b7b10d08a3b90b6c9062d713

Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

5 years agoMFV r337216: 7263 deeply nested nvlist can overflow stack
Alexander Motin [Fri, 3 Aug 2018 01:09:12 +0000 (01:09 +0000)]
MFV r337216: 7263 deeply nested nvlist can overflow stack

illumos/illumos-gate@9ca527c3d3dfa7c8f304b34a9e03b5eddace838f

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

5 years ago7263 deeply nested nvlist can overflow stack
Alexander Motin [Fri, 3 Aug 2018 00:47:24 +0000 (00:47 +0000)]
7263 deeply nested nvlist can overflow stack

illumos/illumos-gate@9ca527c3d3dfa7c8f304b34a9e03b5eddace838f

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

5 years agoMFV 337214:
Alexander Motin [Fri, 3 Aug 2018 00:24:27 +0000 (00:24 +0000)]
MFV 337214:
9621 Make createtxg and guid properties public

illumos/illumos-gate@e8d4a73c868afb740396041be80ed2b141065e76

Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Yuri Pankov <yuripv@yuripv.net>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Josh Paetzel <josh@tcbug.org>

5 years ago9621 Make createtxg and guid properties public
Alexander Motin [Fri, 3 Aug 2018 00:19:59 +0000 (00:19 +0000)]
9621 Make createtxg and guid properties public

illumos/illumos-gate@e8d4a73c868afb740396041be80ed2b141065e76

Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Yuri Pankov <yuripv@yuripv.net>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Josh Paetzel <josh@tcbug.org>

5 years agoMFV r337212:
Alexander Motin [Fri, 3 Aug 2018 00:14:36 +0000 (00:14 +0000)]
MFV r337212:
9465 ARC check for 'anon_size > arc_c/2' can stall the system

illumos/illumos-gate@abe1fd01ce5a83718c5a840daeab4abdaec1c104

Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Don Brady <don.brady@delphix.com>

5 years ago9465 ARC check for 'anon_size > arc_c/2' can stall the system
Alexander Motin [Fri, 3 Aug 2018 00:11:06 +0000 (00:11 +0000)]
9465 ARC check for 'anon_size > arc_c/2' can stall the system

illumos/illumos-gate@abe1fd01ce5a83718c5a840daeab4abdaec1c104

Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Don Brady <don.brady@delphix.com>

5 years agoMFV r337210: 9577 remove zfs_dbuf_evict_key tsd
Alexander Motin [Fri, 3 Aug 2018 00:01:48 +0000 (00:01 +0000)]
MFV r337210: 9577 remove zfs_dbuf_evict_key tsd

The zfs_dbuf_evict_key TSD (thread-specific data) is not necessary - we can
instead pass a flag down in a few places to prevent recursive dbuf eviction.
Making this change has 3 benefits:

1. The code semantics are easier to understand.
2. On Linux, performance is improved, because creating/removing TSD values
(by setting to NULL vs non-NULL) is expensive, and we do it very often.
3. According to Nexenta, the current semantics can cause a deadlock when
concurrently calling dmu_objset_evict_dbufs() (which is rare today, but they
are working on a "parallel unmount" change that triggers this more easily)

illumos/illumos-gate@c2919acbea007fa95c709b60d073db9a24526e01

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years ago9577 remove zfs_dbuf_evict_key tsd
Alexander Motin [Thu, 2 Aug 2018 23:59:52 +0000 (23:59 +0000)]
9577 remove zfs_dbuf_evict_key tsd

The zfs_dbuf_evict_key TSD (thread-specific data) is not necessary - we can
instead pass a flag down in a few places to prevent recursive dbuf eviction.
Making this change has 3 benefits:

1. The code semantics are easier to understand.
2. On Linux, performance is improved, because creating/removing TSD values
(by setting to NULL vs non-NULL) is expensive, and we do it very often.
3. According to Nexenta, the current semantics can cause a deadlock when
concurrently calling dmu_objset_evict_dbufs() (which is rare today, but they
are working on a "parallel unmount" change that triggers this more easily)

illumos/illumos-gate@c2919acbea007fa95c709b60d073db9a24526e01

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years agoMFV r337208: 9591 ms_shift can be incorrectly changed in MOS config for
Alexander Motin [Thu, 2 Aug 2018 23:56:07 +0000 (23:56 +0000)]
MFV r337208: 9591 ms_shift can be incorrectly changed in MOS config for
indirect vdevs that have been historically expanded

illumos/illumos-gate@11f6a9680e013a7c9c57dc0b64d3e91e2eee1a6b

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <gwilson@zfsmail.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Tim Chase <tim@chase2k.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>

5 years ago9591 ms_shift can be incorrectly changed in MOS config for indirect vdevs that have...
Alexander Motin [Thu, 2 Aug 2018 23:52:29 +0000 (23:52 +0000)]
9591 ms_shift can be incorrectly changed in MOS config for indirect vdevs that have been historically expanded

illumos/illumos-gate@11f6a9680e013a7c9c57dc0b64d3e91e2eee1a6b

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <gwilson@zfsmail.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Tim Chase <tim@chase2k.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>

5 years agoMFV r337206: 9338 moved dnode has incorrect dn_next_type
Alexander Motin [Thu, 2 Aug 2018 23:50:03 +0000 (23:50 +0000)]
MFV r337206: 9338 moved dnode has incorrect dn_next_type

illumos/illumos-gate@c7fbe46df966ea665df63b6e6071808987e839d1

Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years ago9338 moved dnode has incorrect dn_next_type
Alexander Motin [Thu, 2 Aug 2018 23:48:19 +0000 (23:48 +0000)]
9338 moved dnode has incorrect dn_next_type

illumos/illumos-gate@c7fbe46df966ea665df63b6e6071808987e839d1

Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years agoMFV r337204: 9439 ZFS double-free due to failure to dirty indirect block
Alexander Motin [Thu, 2 Aug 2018 23:46:30 +0000 (23:46 +0000)]
MFV r337204: 9439 ZFS double-free due to failure to dirty indirect block

illumos/illumos-gate@99a19144e82244f3426f055cc73af8a937c0135c

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years ago9439 ZFS double-free due to failure to dirty indirect block
Alexander Motin [Thu, 2 Aug 2018 23:45:24 +0000 (23:45 +0000)]
9439 ZFS double-free due to failure to dirty indirect block

illumos/illumos-gate@99a19144e82244f3426f055cc73af8a937c0135c

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years agowc(1): Fix 'wc -L'
Conrad Meyer [Thu, 2 Aug 2018 23:45:14 +0000 (23:45 +0000)]
wc(1): Fix 'wc -L'

I inadvertently broke 'wc -L' in r326736.  We must skip the fast path if -L
was specified, in addition to the existing check for the -l option.

Document long-standing -L behavior (count varies depending on whether wc(1)
is run with the -m option or not) in wc.1.  That behavior dates back to the
introduction of the -L option, but was not documented.

PR: 230300
Reported by: <amstrnad+bugzilla AT gmail.com>
Sponsored by: Dell EMC Isilon

5 years agoMFV r337200:
Alexander Motin [Thu, 2 Aug 2018 23:43:01 +0000 (23:43 +0000)]
MFV r337200:
9438 Holes can lose birth time info if a block has a mix of birth times

Ultimately, the problem here is that when you truncate and write a file in
the same transaction group, the dbuf for the indirect block will be zeroed
out to deal with the truncation, and then written for the write. During
this process, we will lose hole birth time information for any holes in the
range. In the case where a dnode is being freed, we need to determine
whether the block should be converted to a higher-level hole in the zio
pipeline, and if so do it when the dnode is being synced out.

illumos/illumos-gate@738e2a3ce3b2579222d6855e7fe75b5bcfcddf8d

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Paul Dagnelie <pcd@delphix.com>

5 years agoFix build after r337196 mismerge.
Alexander Motin [Thu, 2 Aug 2018 23:40:28 +0000 (23:40 +0000)]
Fix build after r337196 mismerge.

5 years ago9438 Holes can lose birth time info if a block has a mix of birth times
Alexander Motin [Thu, 2 Aug 2018 23:37:26 +0000 (23:37 +0000)]
9438 Holes can lose birth time info if a block has a mix of birth times

Ultimately, the problem here is that when you truncate and write a file in
the same transaction group, the dbuf for the indirect block will be zeroed
out to deal with the truncation, and then written for the write. During
this process, we will lose hole birth time information for any holes in the
range. In the case where a dnode is being freed, we need to determine
whether the block should be converted to a higher-level hole in the zio
pipeline, and if so do it when the dnode is being synced out.

illumos/illumos-gate@738e2a3ce3b2579222d6855e7fe75b5bcfcddf8d

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Paul Dagnelie <pcd@delphix.com>

5 years ago8906 uts: illumos rootfs should support salted cksum
Alexander Motin [Thu, 2 Aug 2018 23:28:47 +0000 (23:28 +0000)]
8906 uts: illumos rootfs should support salted cksum

illumos/illumos-gate@0dd498c0020c4a0dd34d350b07f97399a475d102

Reviewed by: C Fraire <cfraire@me.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Yuri Pankov <yuripv@yuripv.net>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Toomas Soome <tsoome@me.com>

5 years agoMFV r337197: 9456 ztest failure in zil_commit_waiter_timeout
Alexander Motin [Thu, 2 Aug 2018 23:25:49 +0000 (23:25 +0000)]
MFV r337197: 9456 ztest failure in zil_commit_waiter_timeout

illumos/illumos-gate@b6031810da58df96413bf76e068638fcab1f228a

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author:     Prakash Surya <prakash.surya@delphix.com>

5 years ago9456 ztest failure in zil_commit_waiter_timeout
Alexander Motin [Thu, 2 Aug 2018 23:24:40 +0000 (23:24 +0000)]
9456 ztest failure in zil_commit_waiter_timeout

illumos/illumos-gate@b6031810da58df96413bf76e068638fcab1f228a

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author:     Prakash Surya <prakash.surya@delphix.com>

5 years agoMFV r337195: 9454 ::zfs_blkstats should count embedded blocks
Alexander Motin [Thu, 2 Aug 2018 23:23:10 +0000 (23:23 +0000)]
MFV r337195: 9454 ::zfs_blkstats should count embedded blocks

illumos/illumos-gate@dec267e7ea9828898b1c64462daa6636c4ef5e29

Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years ago9454 ::zfs_blkstats should count embedded blocks
Alexander Motin [Thu, 2 Aug 2018 23:17:10 +0000 (23:17 +0000)]
9454 ::zfs_blkstats should count embedded blocks

illumos/illumos-gate@dec267e7ea9828898b1c64462daa6636c4ef5e29

Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years agoMFV r337193:
Alexander Motin [Thu, 2 Aug 2018 23:15:10 +0000 (23:15 +0000)]
MFV r337193:
9424 ztest failure: "unprotected error in call to Lua API (Invalid value type 'f
unction' for key 'error')"

illumos/illumos-gate@fe3ba4d1227d8746116ece7240682b13595c3142

Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Don Brady <don.brady@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years ago9424 ztest failure: "unprotected error in call to Lua API (Invalid value type 'functi...
Alexander Motin [Thu, 2 Aug 2018 23:13:40 +0000 (23:13 +0000)]
9424 ztest failure: "unprotected error in call to Lua API (Invalid value type 'function' for key 'error')"

illumos/illumos-gate@fe3ba4d1227d8746116ece7240682b13595c3142

Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Don Brady <don.brady@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years agocxgbe(4): Improvements in TID management.
Navdeep Parhar [Thu, 2 Aug 2018 22:52:05 +0000 (22:52 +0000)]
cxgbe(4): Improvements in TID management.

- Ignore any type of TID where the start/end values are not in the
  correct order.  There are situations where the firmware isn't able to
  reserve room for the number requested in the config file but doesn't
  report a failure during configuration and instead sets end <= start.

- Track start/end in tid_tab and remove some redundant copies from
  adapter->params.

- Move all the start/end and other read-only parameters to a quiet part
  of tid_tab, away from the tid locks.

MFC after: 1 week
Sponsored by: Chelsio Communications

5 years agoMFV r337190: 9486 reduce memory used by device removal on fragmented pools
Alexander Motin [Thu, 2 Aug 2018 21:59:46 +0000 (21:59 +0000)]
MFV r337190: 9486 reduce memory used by device removal on fragmented pools

In the most fragmented real-world cases, this reduces memory used by the
mapping from ~1GB to ~50MB of RAM per 1TB of storage removed. Less
fragmented cases will typically also see around 50-100MB of RAM per 1TB
of storage.

illumos/illumos-gate@cfd63e1b1bcf7ba4bf72f55ddbd87ce008d2986d

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Tim Chase <tim@chase2k.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years ago9486 reduce memory used by device removal on fragmented pools
Alexander Motin [Thu, 2 Aug 2018 21:57:59 +0000 (21:57 +0000)]
9486 reduce memory used by device removal on fragmented pools

In the most fragmented real-world cases, this reduces memory used by the
mapping from ~1GB to ~50MB of RAM per 1TB of storage removed. Less
fragmented cases will typically also see around 50-100MB of RAM per 1TB
of storage.

illumos/illumos-gate@cfd63e1b1bcf7ba4bf72f55ddbd87ce008d2986d

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Tim Chase <tim@chase2k.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years agobhyve: set title before entering capability mode
Mariusz Zaborski [Thu, 2 Aug 2018 21:54:34 +0000 (21:54 +0000)]
bhyve: set title before entering capability mode

PR: 230082
Submitted by: Yuichiro NAITO <naito.yuichiro@gmail.com>

5 years ago9479 fix wrong format specifier for vdev_id
Alexander Motin [Thu, 2 Aug 2018 21:52:47 +0000 (21:52 +0000)]
9479 fix wrong format specifier for vdev_id

illumos/illumos-gate@c7a7b2fa084facd68c3d1e860dfabe20c713877b

Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Andriy Gapon <avg@FreeBSD.org>

5 years agoUpdate dirdeps.mk et al to latest
Simon J. Gerraty [Thu, 2 Aug 2018 21:33:45 +0000 (21:33 +0000)]
Update dirdeps.mk et al to latest

dirdeps.mk and meta.autodep.mk will now look for
Makefile.depend.options
to handle optional dependencies, the work is all done by
dirdeps-options.mk

Also update to latest meta.stage.mk and gendirdeps.mk

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D15701

5 years agoMFV r337184: 9457 libzfs_import.c:add_config() has a memory leak
Alexander Motin [Thu, 2 Aug 2018 21:25:32 +0000 (21:25 +0000)]
MFV r337184: 9457 libzfs_import.c:add_config() has a memory leak

A memory leak occurs on lines 209 and 213 because the config is not freed
in the error case.  The interface to add_config() seems less than ideal -
it would be better if it copied any data necessary from the config and the
caller freed it.

illumos/illumos-gate@ddfe901b12348d31c500fb57f9174e88860a4061

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     sara hartse <sara.hartse@delphix.com>

5 years ago9457 libzfs_import.c:add_config() has a memory leak
Alexander Motin [Thu, 2 Aug 2018 21:24:04 +0000 (21:24 +0000)]
9457 libzfs_import.c:add_config() has a memory leak

A memory leak occurs on lines 209 and 213 because the config is not freed
in the error case.  The interface to add_config() seems less than ideal -
it would be better if it copied any data necessary from the config and the
caller freed it.

illumos/illumos-gate@ddfe901b12348d31c500fb57f9174e88860a4061

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     sara hartse <sara.hartse@delphix.com>

5 years agoMFV r337182: 9330 stack overflow when creating a deeply nested dataset
Alexander Motin [Thu, 2 Aug 2018 21:19:35 +0000 (21:19 +0000)]
MFV r337182: 9330 stack overflow when creating a deeply nested dataset

Datasets that are deeply nested (~100 levels) are impractical. We just put
a limit of 50 levels to newly created datasets. Existing datasets should
work without a problem.

illumos/illumos-gate@5ac95da7d61660aa299c287a39277cb0372be959

Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author:     Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>

5 years ago9330 stack overflow when creating a deeply nested dataset
Alexander Motin [Thu, 2 Aug 2018 21:12:52 +0000 (21:12 +0000)]
9330 stack overflow when creating a deeply nested dataset

Datasets that are deeply nested (~100 levels) are impractical. We just put
a limit of 50 levels to newly created datasets. Existing datasets should
work without a problem.

illumos/illumos-gate@5ac95da7d61660aa299c287a39277cb0372be959

Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author:     Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>

5 years ago9539 Make zvol operations use _by_dnode routines
Alexander Motin [Thu, 2 Aug 2018 21:07:04 +0000 (21:07 +0000)]
9539 Make zvol operations use _by_dnode routines

Continues what was started in 7801 add more by-dnode routines by fully
converting zvols to avoid unnecessary dnode_hold() calls. This saves a
small amount of CPU time and slightly improves latencies of operations
on zvols.

illumos/illumos-gate@8dfe5547fbf0979fc1065a8b6fddc1e940a7cf4f

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Richard Yao <richard.yao@prophetstor.com>

5 years ago9539 Make zvol operations use _by_dnode routines
Alexander Motin [Thu, 2 Aug 2018 20:49:08 +0000 (20:49 +0000)]
9539 Make zvol operations use _by_dnode routines

Continues what was started in 7801 add more by-dnode routines by fully
converting zvols to avoid unnecessary dnode_hold() calls. This saves a
small amount of CPU time and slightly improves latencies of operations
on zvols.

illumos/illumos-gate@8dfe5547fbf0979fc1065a8b6fddc1e940a7cf4f

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Richard Yao <richard.yao@prophetstor.com>

5 years ago9523 Large alloc in zdb can cause trouble
Alexander Motin [Thu, 2 Aug 2018 20:44:07 +0000 (20:44 +0000)]
9523 Large alloc in zdb can cause trouble

16MB alloc in zdb_embedded_block() can cause cores in certain situations
(clang, gcc55).

OsX commit: https://github.com/openzfsonosx/zfs/commit/ced236a5da6e72ea7bf6d2919fe14e17cffe10f1
FreeBSD commit: https://svnweb.freebsd.org/base?view=revision&revision=326150
illumos/illumos-gate@03a4c2f4bfaca30115963b76445279b36468a614

Reviewed by: Igor Kozhukhov <igor@dilos.org>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Jorgen Lundman <lundman@lundman.net>

This is an update for r326150 (by avg), where this change comes from.

5 years ago9523 Large alloc in zdb can cause trouble
Alexander Motin [Thu, 2 Aug 2018 20:37:53 +0000 (20:37 +0000)]
9523 Large alloc in zdb can cause trouble

16MB alloc in zdb_embedded_block() can cause cores in certain situations
(clang, gcc55).

OsX commit: https://github.com/openzfsonosx/zfs/commit/ced236a5da6e72ea7bf6d2919fe14e17cffe10f1
FreeBSD commit: https://svnweb.freebsd.org/base?view=revision&revision=326150
illumos/illumos-gate@03a4c2f4bfaca30115963b76445279b36468a614

Reviewed by: Igor Kozhukhov <igor@dilos.org>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Jorgen Lundman <lundman@lundman.net>

5 years agoMFV r337175: 9487 Free objects when receiving full stream as clone
Alexander Motin [Thu, 2 Aug 2018 20:33:13 +0000 (20:33 +0000)]
MFV r337175: 9487 Free objects when receiving full stream as clone

All objects after the last written or freed object are not supposed to
exist after receiving the stream. We should free them accordingly, as if
a freeobjects record for them had been included in the stream.

zfsonlinux/zfs@48fbb9ddbf2281911560dfbc2821aa8b74127315
illumos/illumos-gate@7864b8192b8d30471fa2240466d516292e5765b8

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Paul Dagnelie <pcd@delphix.com>

5 years ago9487 Free objects when receiving full stream as clone
Alexander Motin [Thu, 2 Aug 2018 20:29:58 +0000 (20:29 +0000)]
9487 Free objects when receiving full stream as clone

All objects after the last written or freed object are not supposed to
exist after receiving the stream. We should free them accordingly, as if
a freeobjects record for them had been included in the stream.

zfsonlinux/zfs@48fbb9ddbf2281911560dfbc2821aa8b74127315
illumos/illumos-gate@7864b8192b8d30471fa2240466d516292e5765b8

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Paul Dagnelie <pcd@delphix.com>

5 years agoMFV r337171:
Alexander Motin [Thu, 2 Aug 2018 20:18:49 +0000 (20:18 +0000)]
MFV r337171:
9464 txg_kick() fails to see that we are quiescing, forcing transactions
to their next stages without leaving them accumulate changes

Ideally we would like txg_kick() to get triggered only when we are sure
that we are not syncing AND not quiescing any txg. This way we can kick
an open TXG to the quiescing state when we are sure that there is nothing
going on and we would benefit from the different states running
concurrently.

illumos/illumos-gate@fa41d87de9ec9000964c605eb01d6dc19e4a1abe

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>

5 years ago9464 txg_kick() fails to see that we are quiescing, forcing transactions
Alexander Motin [Thu, 2 Aug 2018 20:17:37 +0000 (20:17 +0000)]
9464 txg_kick() fails to see that we are quiescing, forcing transactions
to their next stages without leaving them accumulate changes

Ideally we would like txg_kick() to get triggered only when we are sure
that we are not syncing AND not quiescing any txg. This way we can kick
an open TXG to the quiescing state when we are sure that there is nothing
going on and we would benefit from the different states running
concurrently.

illumos/illumos-gate@fa41d87de9ec9000964c605eb01d6dc19e4a1abe

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>

5 years agoSilence newer gcc warnings.
Rick Macklem [Thu, 2 Aug 2018 20:10:59 +0000 (20:10 +0000)]
Silence newer gcc warnings.

Newer versions of gcc generate "might not be initialized" warnings for
several variables in nfsrpc_doiods().  I have checked and all of these
variables are assigned values before they are used.
In the one case of "tdrpc", it could have passed garbage as an argument
to nfscl_dofflayoutio() when mirrorcnt is one. However nfscl_dofflayoutio() only
uses the argument when mirrorcnt > 1, so it wasn't actually broken.
This patch initializes "tdrpc" to avoid confusion and initializes the rest
to make the compiler happy.

Requested by: mmacy

5 years agoMFV r337167: 9442 decrease indirect block size of spacemaps
Alexander Motin [Thu, 2 Aug 2018 20:06:46 +0000 (20:06 +0000)]
MFV r337167: 9442 decrease indirect block size of spacemaps

Updates to indirect blocks of spacemaps can contribute significantly to
write inflation.  Therefore we want to reduce the indirect block size of
spacemaps from 128K to 16K.

illumos/illumos-gate@221813c13b43ef48330b03725e00edee85108cf1

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Albert Lee <trisk@forkgnu.org>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years agocxgbe(4): Use the tx credit limit for ethofld rather than TOE when
Navdeep Parhar [Thu, 2 Aug 2018 19:50:12 +0000 (19:50 +0000)]
cxgbe(4): Use the tx credit limit for ethofld rather than TOE when
initializing the softc for a per-flow rate limiter.  The limit happens
to be the same for both and the existing code worked by accident for
common configurations.

Reported by: gallatin@
Sponsored by: Chelsio Communications

5 years ago9442 decrease indirect block size of spacemaps
Alexander Motin [Thu, 2 Aug 2018 19:37:13 +0000 (19:37 +0000)]
9442 decrease indirect block size of spacemaps

Updates to indirect blocks of spacemaps can contribute significantly to
write inflation.  Therefore we want to reduce the indirect block size of
spacemaps from 128K to 16K.

illumos/illumos-gate@221813c13b43ef48330b03725e00edee85108cf1

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Albert Lee <trisk@forkgnu.org>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>

5 years agoFUSE: Bump maximum IO size to enable more performant operation
Conrad Meyer [Thu, 2 Aug 2018 19:25:43 +0000 (19:25 +0000)]
FUSE: Bump maximum IO size to enable more performant operation

Various components restrict size of IO passed up to the userspace filesystem
based on the mount's f_iosize value.  The previous default of PAGE_SIZE
is anemic, even for normal filesystems, but especially considering every
FUSE operation involves a kernel <-> userspace IPC upcall.

Bump to DFLTPHYS (currently 64kB) to match other FUSE implementations.

Anecdotally, Jakub reports IO read performance increased from 600 MB/s ->
2700 MB/s with a basic RAM-backed FUSE filesystem.

PR: 230260
Reported by: Peter (MooseFS) <freebsd AT moosefs.com>
Tested by: Jakub Kruszona-Zawadzki <acid AT moosefs.com>
MFC after: 3 days

5 years agoMFV r337161: 9512 zfs remap poolname@snapname coredumps
Alexander Motin [Thu, 2 Aug 2018 19:13:45 +0000 (19:13 +0000)]
MFV r337161: 9512 zfs remap poolname@snapname coredumps

Only filesystems and volumes are valid "zfs remap" parameters: when passed
a snapshot name zfs_remap_indirects() does not handle the EINVAL returned
from libzfs_core, which results in failing an assertion and consequently
crashing.

illumos/illumos-gate@0b2e8253986c5c761129b58cfdac46d204903de1

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Wren Kennedy <john.kennedy@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author:     loli10K <ezomori.nozomu@gmail.com>

5 years ago9512 zfs remap poolname@snapname coredumps
Alexander Motin [Thu, 2 Aug 2018 19:09:13 +0000 (19:09 +0000)]
9512 zfs remap poolname@snapname coredumps

Only filesystems and volumes are valid "zfs remap" parameters: when passed
a snapshot name zfs_remap_indirects() does not handle the EINVAL returned
from libzfs_core, which results in failing an assertion and consequently
crashing.

illumos/illumos-gate@0b2e8253986c5c761129b58cfdac46d204903de1

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Wren Kennedy <john.kennedy@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author:     loli10K <ezomori.nozomu@gmail.com>

5 years agoDo not blindly include illumos kernel headers instead of user-space.
Alexander Motin [Thu, 2 Aug 2018 18:55:55 +0000 (18:55 +0000)]
Do not blindly include illumos kernel headers instead of user-space.
It is not needed now, and I doubt it much helped at all, creating more
confusions then good.

5 years agoFix the ftp-stage target for arm embedded builds.
Glen Barber [Thu, 2 Aug 2018 18:51:44 +0000 (18:51 +0000)]
Fix the ftp-stage target for arm embedded builds.

The images were renamed from KERNCONF to BOARDNAME when
specified, which would result in an image name of:

 12.0-CURRENT-arm-armv7-GENERIC.img

which would then be renamed to use the BOARDNAME for the
SoC the image is targeted to use.  BOARDNAME was specified
for all images as of r336994, which now causes the ftp-stage
target to fail, as the rename is no longer necessary.

Sponsored by: The FreeBSD Foundation

5 years agoRemove spuriously added svn properties
Ben Widawsky [Thu, 2 Aug 2018 18:37:02 +0000 (18:37 +0000)]
Remove spuriously added svn properties

5 years agotools: Add a git-svn bootstrap script
Ben Widawsky [Thu, 2 Aug 2018 18:28:02 +0000 (18:28 +0000)]
tools: Add a git-svn bootstrap script

codified version of https://wiki.freebsd.org/GitWorkflow/GitSvn#Using_git-svn

Approved by: emaste (mentor)
Suggested by: Warner Losh (imp)
Differential Revision:  https://reviews.freebsd.org/D16528

5 years agoAdd a rudimentary test for procstat kstack.
Mark Johnston [Thu, 2 Aug 2018 16:03:47 +0000 (16:03 +0000)]
Add a rudimentary test for procstat kstack.

MFC after: 1 week

5 years agoloader.efi: efipart_inithandles() failure should not prevent devsw initialization
Toomas Soome [Thu, 2 Aug 2018 12:43:40 +0000 (12:43 +0000)]
loader.efi: efipart_inithandles() failure should not prevent devsw initialization

The efipart_inithandles() can return error code when there are no block devices,
however, we still should walk and initialize devsw.

5 years agoRegen src.conf.5 after r337057, LLD_BOOTSTRAP on armv7
Ed Maste [Thu, 2 Aug 2018 12:39:23 +0000 (12:39 +0000)]
Regen src.conf.5 after r337057, LLD_BOOTSTRAP on armv7

5 years agoImplement pt_fpreg_to_ucontext(), pt_ucontext_to_fpreg().
Ruslan Bukin [Thu, 2 Aug 2018 12:24:34 +0000 (12:24 +0000)]
Implement pt_fpreg_to_ucontext(), pt_ucontext_to_fpreg().

Sponsored by: DARPA, AFRL

5 years agoSave and restore floating-point arguments.
Ruslan Bukin [Thu, 2 Aug 2018 12:21:39 +0000 (12:21 +0000)]
Save and restore floating-point arguments.

Sponsored by: DARPA, AFRL

5 years agoDon't overwrite tp in set_mcontext().
Ruslan Bukin [Thu, 2 Aug 2018 12:13:52 +0000 (12:13 +0000)]
Don't overwrite tp in set_mcontext().

This makes libthr/swapcontext_test:swapcontext1 happy.

Sponsored by: DARPA, AFRL

5 years agoo Don't overwrite tp in fork_trampoline().
Ruslan Bukin [Thu, 2 Aug 2018 12:12:13 +0000 (12:12 +0000)]
o Don't overwrite tp in fork_trampoline().
o Save and restore tp in cpu_switch().
o Restore tp in cpu_throw().
o Save tp in savectx().

This makes libthr tests happy. In particular fork_test:fork.

Sponsored by: DARPA, AFRL

5 years agoo Correctly set user tls base: consider TP_OFFSET.
Ruslan Bukin [Thu, 2 Aug 2018 12:08:52 +0000 (12:08 +0000)]
o Correctly set user tls base: consider TP_OFFSET.
o Ensure tp (thread pointer) saved before copying the pcb.

Sponsored by: DARPA, AFRL

5 years agoFix regression tests broken by r337067.
Edward Tomasz Napierala [Thu, 2 Aug 2018 11:55:16 +0000 (11:55 +0000)]
Fix regression tests broken by r337067.

MFC after: 2 weeks
Sponsored by: DARPA, AFRL

5 years agofix a typo resulting in a wrong variable in kern_syscall_deregister
Andriy Gapon [Thu, 2 Aug 2018 09:41:55 +0000 (09:41 +0000)]
fix a typo resulting in a wrong variable in kern_syscall_deregister

The difference is between sysent, a global, and sysents, a function
parameter.

5 years agoMake sure the rtld(1) error messages go to stderr, not stdout.
Edward Tomasz Napierala [Thu, 2 Aug 2018 07:43:28 +0000 (07:43 +0000)]
Make sure the rtld(1) error messages go to stderr, not stdout.
While here fix capitalization of a few nearby strings, add the
rtld's file name prefix so it's obvious where the message come
from, and return zero when "-h" is used.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D16530

5 years agolibsa: dereferencing type-punned pointer in cd9660
Toomas Soome [Thu, 2 Aug 2018 06:22:10 +0000 (06:22 +0000)]
libsa: dereferencing type-punned pointer in cd9660

The warning is given by gcc build, but it is good to fix anyhow.
use bcopy instead of direct assignment.

5 years agoMFV r316926:
Alexander Motin [Wed, 1 Aug 2018 21:51:49 +0000 (21:51 +0000)]
MFV r316926:
7955 libshare needs to initialize only those datasets being modified by the consumer

illumos/illumos-gate@8a981c3356b194b3b5c0ae9276a9cc31cd2f93a3
https://github.com/illumos/illumos-gate/commit/8a981c3356b194b3b5c0ae9276a9cc31cd2f93a3

https://www.illumos.org/issues/7955
  Libshare currently initializes all available filesystems when doing any
  libshare operation. This requires iterating through all the filesystem
  multiple times, which is a huge performance problem for sharing and
  unsharing operations.

Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
Approved by: Gordon Ross <gordon.w.ross@gmail.com>
Author: Daniel Hoffman <dj.hoffman@delphix.com>

For FreeBSD this is practically a NOP, just a diff reduction.