Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Don Brady <don.brady@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Don Brady <don.brady@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Navdeep Parhar [Thu, 2 Aug 2018 22:52:05 +0000 (22:52 +0000)]
cxgbe(4): Improvements in TID management.
- Ignore any type of TID where the start/end values are not in the
correct order. There are situations where the firmware isn't able to
reserve room for the number requested in the config file but doesn't
report a failure during configuration and instead sets end <= start.
- Track start/end in tid_tab and remove some redundant copies from
adapter->params.
- Move all the start/end and other read-only parameters to a quiet part
of tid_tab, away from the tid locks.
Alexander Motin [Thu, 2 Aug 2018 21:59:46 +0000 (21:59 +0000)]
MFV r337190: 9486 reduce memory used by device removal on fragmented pools
In the most fragmented real-world cases, this reduces memory used by the
mapping from ~1GB to ~50MB of RAM per 1TB of storage removed. Less
fragmented cases will typically also see around 50-100MB of RAM per 1TB
of storage.
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Tim Chase <tim@chase2k.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Alexander Motin [Thu, 2 Aug 2018 21:57:59 +0000 (21:57 +0000)]
9486 reduce memory used by device removal on fragmented pools
In the most fragmented real-world cases, this reduces memory used by the
mapping from ~1GB to ~50MB of RAM per 1TB of storage removed. Less
fragmented cases will typically also see around 50-100MB of RAM per 1TB
of storage.
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Tim Chase <tim@chase2k.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Alexander Motin [Thu, 2 Aug 2018 21:25:32 +0000 (21:25 +0000)]
MFV r337184: 9457 libzfs_import.c:add_config() has a memory leak
A memory leak occurs on lines 209 and 213 because the config is not freed
in the error case. The interface to add_config() seems less than ideal -
it would be better if it copied any data necessary from the config and the
caller freed it.
Alexander Motin [Thu, 2 Aug 2018 21:24:04 +0000 (21:24 +0000)]
9457 libzfs_import.c:add_config() has a memory leak
A memory leak occurs on lines 209 and 213 because the config is not freed
in the error case. The interface to add_config() seems less than ideal -
it would be better if it copied any data necessary from the config and the
caller freed it.
Alexander Motin [Thu, 2 Aug 2018 21:19:35 +0000 (21:19 +0000)]
MFV r337182: 9330 stack overflow when creating a deeply nested dataset
Datasets that are deeply nested (~100 levels) are impractical. We just put
a limit of 50 levels to newly created datasets. Existing datasets should
work without a problem.
Alexander Motin [Thu, 2 Aug 2018 21:12:52 +0000 (21:12 +0000)]
9330 stack overflow when creating a deeply nested dataset
Datasets that are deeply nested (~100 levels) are impractical. We just put
a limit of 50 levels to newly created datasets. Existing datasets should
work without a problem.
Alexander Motin [Thu, 2 Aug 2018 21:07:04 +0000 (21:07 +0000)]
9539 Make zvol operations use _by_dnode routines
Continues what was started in 7801 add more by-dnode routines by fully
converting zvols to avoid unnecessary dnode_hold() calls. This saves a
small amount of CPU time and slightly improves latencies of operations
on zvols.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Richard Yao <richard.yao@prophetstor.com>
Alexander Motin [Thu, 2 Aug 2018 20:49:08 +0000 (20:49 +0000)]
9539 Make zvol operations use _by_dnode routines
Continues what was started in 7801 add more by-dnode routines by fully
converting zvols to avoid unnecessary dnode_hold() calls. This saves a
small amount of CPU time and slightly improves latencies of operations
on zvols.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Richard Yao <richard.yao@prophetstor.com>
Alexander Motin [Thu, 2 Aug 2018 20:33:13 +0000 (20:33 +0000)]
MFV r337175: 9487 Free objects when receiving full stream as clone
All objects after the last written or freed object are not supposed to
exist after receiving the stream. We should free them accordingly, as if
a freeobjects record for them had been included in the stream.
Alexander Motin [Thu, 2 Aug 2018 20:29:58 +0000 (20:29 +0000)]
9487 Free objects when receiving full stream as clone
All objects after the last written or freed object are not supposed to
exist after receiving the stream. We should free them accordingly, as if
a freeobjects record for them had been included in the stream.
Alexander Motin [Thu, 2 Aug 2018 20:18:49 +0000 (20:18 +0000)]
MFV r337171:
9464 txg_kick() fails to see that we are quiescing, forcing transactions
to their next stages without leaving them accumulate changes
Ideally we would like txg_kick() to get triggered only when we are sure
that we are not syncing AND not quiescing any txg. This way we can kick
an open TXG to the quiescing state when we are sure that there is nothing
going on and we would benefit from the different states running
concurrently.
Alexander Motin [Thu, 2 Aug 2018 20:17:37 +0000 (20:17 +0000)]
9464 txg_kick() fails to see that we are quiescing, forcing transactions
to their next stages without leaving them accumulate changes
Ideally we would like txg_kick() to get triggered only when we are sure
that we are not syncing AND not quiescing any txg. This way we can kick
an open TXG to the quiescing state when we are sure that there is nothing
going on and we would benefit from the different states running
concurrently.
Rick Macklem [Thu, 2 Aug 2018 20:10:59 +0000 (20:10 +0000)]
Silence newer gcc warnings.
Newer versions of gcc generate "might not be initialized" warnings for
several variables in nfsrpc_doiods(). I have checked and all of these
variables are assigned values before they are used.
In the one case of "tdrpc", it could have passed garbage as an argument
to nfscl_dofflayoutio() when mirrorcnt is one. However nfscl_dofflayoutio() only
uses the argument when mirrorcnt > 1, so it wasn't actually broken.
This patch initializes "tdrpc" to avoid confusion and initializes the rest
to make the compiler happy.
Alexander Motin [Thu, 2 Aug 2018 20:06:46 +0000 (20:06 +0000)]
MFV r337167: 9442 decrease indirect block size of spacemaps
Updates to indirect blocks of spacemaps can contribute significantly to
write inflation. Therefore we want to reduce the indirect block size of
spacemaps from 128K to 16K.
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Albert Lee <trisk@forkgnu.org>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Navdeep Parhar [Thu, 2 Aug 2018 19:50:12 +0000 (19:50 +0000)]
cxgbe(4): Use the tx credit limit for ethofld rather than TOE when
initializing the softc for a per-flow rate limiter. The limit happens
to be the same for both and the existing code worked by accident for
common configurations.
Alexander Motin [Thu, 2 Aug 2018 19:37:13 +0000 (19:37 +0000)]
9442 decrease indirect block size of spacemaps
Updates to indirect blocks of spacemaps can contribute significantly to
write inflation. Therefore we want to reduce the indirect block size of
spacemaps from 128K to 16K.
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Albert Lee <trisk@forkgnu.org>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Conrad Meyer [Thu, 2 Aug 2018 19:25:43 +0000 (19:25 +0000)]
FUSE: Bump maximum IO size to enable more performant operation
Various components restrict size of IO passed up to the userspace filesystem
based on the mount's f_iosize value. The previous default of PAGE_SIZE
is anemic, even for normal filesystems, but especially considering every
FUSE operation involves a kernel <-> userspace IPC upcall.
Bump to DFLTPHYS (currently 64kB) to match other FUSE implementations.
Anecdotally, Jakub reports IO read performance increased from 600 MB/s ->
2700 MB/s with a basic RAM-backed FUSE filesystem.
PR: 230260
Reported by: Peter (MooseFS) <freebsd AT moosefs.com>
Tested by: Jakub Kruszona-Zawadzki <acid AT moosefs.com>
MFC after: 3 days
Only filesystems and volumes are valid "zfs remap" parameters: when passed
a snapshot name zfs_remap_indirects() does not handle the EINVAL returned
from libzfs_core, which results in failing an assertion and consequently
crashing.
Alexander Motin [Thu, 2 Aug 2018 19:09:13 +0000 (19:09 +0000)]
9512 zfs remap poolname@snapname coredumps
Only filesystems and volumes are valid "zfs remap" parameters: when passed
a snapshot name zfs_remap_indirects() does not handle the EINVAL returned
from libzfs_core, which results in failing an assertion and consequently
crashing.
Alexander Motin [Thu, 2 Aug 2018 18:55:55 +0000 (18:55 +0000)]
Do not blindly include illumos kernel headers instead of user-space.
It is not needed now, and I doubt it much helped at all, creating more
confusions then good.
Glen Barber [Thu, 2 Aug 2018 18:51:44 +0000 (18:51 +0000)]
Fix the ftp-stage target for arm embedded builds.
The images were renamed from KERNCONF to BOARDNAME when
specified, which would result in an image name of:
12.0-CURRENT-arm-armv7-GENERIC.img
which would then be renamed to use the BOARDNAME for the
SoC the image is targeted to use. BOARDNAME was specified
for all images as of r336994, which now causes the ftp-stage
target to fail, as the rename is no longer necessary.
Make sure the rtld(1) error messages go to stderr, not stdout.
While here fix capitalization of a few nearby strings, add the
rtld's file name prefix so it's obvious where the message come
from, and return zero when "-h" is used.
https://www.illumos.org/issues/7955
Libshare currently initializes all available filesystems when doing any
libshare operation. This requires iterating through all the filesystem
multiple times, which is a huge performance problem for sharing and
unsharing operations.
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
Approved by: Gordon Ross <gordon.w.ross@gmail.com>
Author: Daniel Hoffman <dj.hoffman@delphix.com>
For FreeBSD this is practically a NOP, just a diff reduction.
Kyle Evans [Wed, 1 Aug 2018 20:08:20 +0000 (20:08 +0000)]
ubldr: Bump heap size, 1MB -> 2MB
1MB was leaving very little margin in some of the worse-case scenarios with
lualoader. 2MB is still low enough that we shouldn't have any problems with
UBoot-supported boards.
Mark Johnston [Wed, 1 Aug 2018 19:45:04 +0000 (19:45 +0000)]
Fix some nits in the unix_passfd tests.
- Remove return statements in functions with a void return type.
- Allocate enough space for the SCM_CREDS and SCM_RIGHTS messages
received in the rights_creds_payload test.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Add ioctl to conveniently mmap a PCI device BAR into userspace.
Add the ioctl PCIOCBARMMAP on /dev/pci to conveniently create
userspace mapping of a PCI device BAR. This is enormously superior to
read the BAR value with PCIOCREAD and then try to mmap /dev/mem, and
should allow to automatically activate the mapped BARs when needed in
future.
Current implementation creates new sg pager for each user mmap
request. If the pointer (and reference) to a managed device pager is
stored in pci_map, we would be able to revoke all mappings on the BAR
deactivation or relocation. This is related to the unimplemented BAR
activation on mmap, and is postponed for the future.
Mark Johnston [Wed, 1 Aug 2018 18:45:33 +0000 (18:45 +0000)]
Don't hard-code the number of compression utility arguments.
The zstd invocation constructed by newsyslog contains one more parameter
than invocations for the other supported compression utilities. However,
the maximum number of arguments was hard-coded, leading to an
out-of-bounds array access when using zstd compression.
Alexander Motin [Wed, 1 Aug 2018 18:28:17 +0000 (18:28 +0000)]
8115 parallel zfs mount
Mounting of filesystems in "filesystem/local" is done using `zfs mount -a`,
which mounts each filesystems serially. The bottleneck for each mount is
the I/O done to load metadata for each filesystem. As such, mounting
filesystems using a parallel algorithm should be a big win, and bring down
the runtime of "filesystem/local"'s start method.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author: Sebastien Roy <seb@delphix.com>
Justin Hibbits [Wed, 1 Aug 2018 14:50:41 +0000 (14:50 +0000)]
snd_hda: Synchronize DMA buffers for the control path
Make sure both sides of the DMA buffer memory accesses for the CORB and RIRB
(control buffers) in snd_hda (device and CPU) can see coherent memory. This
is needed on weakly ordered architectures including PowerPC and ARM. Patch
originally by mmel, with small changes.
This does not cover the data path of snd_hda. We don't have sync operations
for in-progress DMA buffers, to sync ranges of a map.
Ravi Pokala [Wed, 1 Aug 2018 08:24:34 +0000 (08:24 +0000)]
Remove jedec_ts(4)
The jedec_ts(4) driver has been marked as deprecated in stable/11, and is
now being removed from -HEAD. Add a notice in UPDATING, and update the few
remaining references (regarding jedec_dimm(4)'s compatibility and history)
to reflect the fact that jedec_ts(4) is now deleted.
Mark Johnston [Wed, 1 Aug 2018 03:46:07 +0000 (03:46 +0000)]
Require that MAC label buffers be able to store a non-empty string.
The buffer size may be used to initialize an sbuf in
MAC_POLICY_EXTERNALIZE, and without this constraint it's possible to
trigger an assertion failure in the sbuf code. With INVARIANTS
disabled, the first attempt to write to the sbuf will fail.
Alexander Motin [Wed, 1 Aug 2018 03:21:17 +0000 (03:21 +0000)]
MFV r337029:
9426 metaslab size can exceed offset addressable by spacemap
metaslab size can exceed offset addressable by spacemap. The vdev can
address up to 2^63 * SPA_MAXBLOCKSIZE (512). A metaslab can address up to
2^47 * 2^vdev_ashift. Therefore we may need to increase the number of
metaslabs so that the maximum metaslab size is capped at the amount that
can be addressed by the spacemap. This should happen in
vdev_metaslab_set_size().
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Don Brady <don.brady@delphix.com>
Alexander Motin [Wed, 1 Aug 2018 03:19:30 +0000 (03:19 +0000)]
9426 metaslab size can exceed offset addressable by spacemap
metaslab size can exceed offset addressable by spacemap. The vdev can
address up to 2^63 * SPA_MAXBLOCKSIZE (512). A metaslab can address up to
2^47 * 2^vdev_ashift. Therefore we may need to increase the number of
metaslabs so that the maximum metaslab size is capped at the amount that
can be addressed by the spacemap. This should happen in
vdev_metaslab_set_size().
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Don Brady <don.brady@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Alexander Motin [Wed, 1 Aug 2018 02:39:44 +0000 (02:39 +0000)]
MFV r337022:
9403 assertion failed in arc_buf_destroy() when concurrently reading block with checksum error
This assertion (VERIFY) failure was reported when reading a block. Turns out
the problem is that if we get an i/o error (ECKSUM in this case), and there
are multiple concurrent ARC reads of the same block (from different clones),
then the ARC will put multiple buf's on the same ANON hdr, which isn't
supposed to happen, and then causes a panic when we try to arc_buf_destroy()
the buf.
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Alexander Motin [Tue, 31 Jul 2018 23:44:13 +0000 (23:44 +0000)]
9403 assertion failed in arc_buf_destroy() when concurrently reading block with checksum error
This assertion (VERIFY) failure was reported when reading a block. Turns out
the problem is that if we get an i/o error (ECKSUM in this case), and there
are multiple concurrent ARC reads of the same block (from different clones),
then the ARC will put multiple buf's on the same ANON hdr, which isn't
supposed to happen, and then causes a panic when we try to arc_buf_destroy()
the buf.
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author: Matthew Ahrens <mahrens@delphix.com>
Michael Tuexen [Tue, 31 Jul 2018 22:56:03 +0000 (22:56 +0000)]
Add a dtrace provider for UDP-Lite.
The dtrace provider for UDP-Lite is modeled after the UDP provider.
This fixes the bug that UDP-Lite packets were triggering the UDP
provider.
Thanks to dteske@ for providing the dwatch module.
Alexander Motin [Tue, 31 Jul 2018 22:50:50 +0000 (22:50 +0000)]
MFV r337014:
9421 zdb should detect and print out the number of "leaked" objects
9422 zfs diff and zdb should explicitly mark objects that are on the deleted queue
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author: Paul Dagnelie <pcd@delphix.com>
Alexander Motin [Tue, 31 Jul 2018 21:06:04 +0000 (21:06 +0000)]
MFV r336991, r337001:
9102 zfs should be able to initialize storage devices
The first access to a disk block can incur a performance penalty on some
platforms (e.g. AWS's EBS, VMware VMDKs). Therefore it is recommended that
volumes be "thick provisioned", where supported by the platform (VMware).
Thick provisioning is time consuming and often is ignored. If the thick
provision step is omitted, customers will see suboptimal performance until
we have written to all parts of the LUN. ZFS should be able to initialize
any unused storage to remove any first-write penalty that exists.
Reviewed by: John Wren Kennedy <john.kennedy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: George Wilson <george.wilson@delphix.com>
ofw_cpu: Add support for getting cpu clock via clock property
Nominal Mhz is either expressed via the clock-frequency property
or can be get via the clock property that holds the cpu clock.
Add support for the later.
Since we have now EFI framebuffer enabled for ARM64 if we boot on a board
with an screen, u-boot will set up a EFI GOP framebuffer and we won't boot
using the serial console.
Also on RPI3 the firmware always setup the framebuffer area resulting in u-boot
always setup the EFI GOP and FreeBSD never using the serial console.
This is not a problem for 12-CURRENT as EFI boot works but it doesn't
for 11.
While here some board arm_install_uboot also copy ubldr.bin et create
firstboot files but it's already done in arm_install_boot
Reviewed by: gjb
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D16481
The nvmem interface helps provider of nvmem data to expose themselves to consumer.
NVMEM is generally present on some embedded board in a form of eeprom or fuses.
The nvmem api are helpers for consumer to read/write the cell data from a provider.
FORCE_PKG_REGISTER is broken so multiple invocation of release.sh for the
same board will fails if /scratch isn't cleaned.
Leave it but deinstall the package first.
projects / FreeBSD / FreeBSD.git / log