]> CyberLeo.Net >> Repos - FreeBSD/FreeBSD.git/log
FreeBSD/FreeBSD.git
2 years agomodule: zfs: multilist: shim out multilist_d2l()
наб [Tue, 21 Dec 2021 17:44:04 +0000 (18:44 +0100)]
module: zfs: multilist: shim out multilist_d2l()

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12844

2 years agomodule: zfs: dsl: pool: shim out dsl_early_sync_task_verify()
наб [Tue, 21 Dec 2021 17:43:49 +0000 (18:43 +0100)]
module: zfs: dsl: pool: shim out dsl_early_sync_task_verify()

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12844

2 years agomodule: zfs: dnode: use debug-only in debug mode only
наб [Tue, 21 Dec 2021 17:38:30 +0000 (18:38 +0100)]
module: zfs: dnode: use debug-only in debug mode only

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12844

2 years agoinclude: sys/arc.h: shim out arc_referenced()
наб [Tue, 21 Dec 2021 17:37:52 +0000 (18:37 +0100)]
include: sys/arc.h: shim out arc_referenced()

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12844

2 years agoinclude: qat.h: mark unused macro arguments as used
наб [Sun, 12 Dec 2021 16:01:21 +0000 (17:01 +0100)]
include: qat.h: mark unused macro arguments as used

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12844

2 years agolibspl: kmem.h: mark unused kmem_*() macro arguments used
наб [Sun, 12 Dec 2021 14:20:26 +0000 (15:20 +0100)]
libspl: kmem.h: mark unused kmem_*() macro arguments used

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12844

2 years agolibspl: ASSERT*: mark arguments as used
наб [Sun, 12 Dec 2021 14:14:00 +0000 (15:14 +0100)]
libspl: ASSERT*: mark arguments as used

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12844

2 years agoZTS: Fix enospc_002_pos.ksh again
Brian Behlendorf [Thu, 23 Dec 2021 17:21:40 +0000 (09:21 -0800)]
ZTS: Fix enospc_002_pos.ksh again

This is a follow up commit for e03a41a60 which aimed to resolve
this same test failure.  The core "problem" here is that it takes
very little space to perform a clone/snapshot/bookmark, which
means if we want these commands to reliably fail the pool must
truely have exhausted all free space.

This commit increases the number of fill iterations to try and
consume every block which we can.  This still can't guarantee
the clone/snapshot/bookmark will fail, but it significantly
improves the odds.  The exception was kept since it's still
not a sure thing.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Igor Kozhukhov <igor@dilos.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12903

2 years agoReduce number of arc_prune threads
Alexander Motin [Thu, 23 Dec 2021 01:07:13 +0000 (20:07 -0500)]
Reduce number of arc_prune threads

On FreeBSD vnode reclamation is single-threaded, protected by single
global lock.  Linux seems to be able to use a thread per mount point,
but at this time it creates more harm than good.

Reduce number of threads to 1, adding tunable in case somebody wants
to try more.

Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Chris Dunlop <chris@onthe.net.au>
Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes #12896
Issue #9966

2 years agoZTS: Fix rollback_003_pos.ksh
Brian Behlendorf [Wed, 22 Dec 2021 19:05:07 +0000 (11:05 -0800)]
ZTS: Fix rollback_003_pos.ksh

Under Linux when rolling back a mounted filesystem negative dentries
may not be dropped from the cache.  This can result in an ENOENT
being incorrectly returned on first access.  Issuing a `df` before
the unmount results in the negative dentries being invalidated and
side steps the issue.

This is solely a workaround for the test case on Linux and not
correct behavior.  The core issue of invalidating negative dentries
needs to be handled with a kernel side change.  This is being
tracked as issue #6143.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12898
Issue #6143

2 years agoZTS: Fix refreserv_raidz.ksh
Brian Behlendorf [Wed, 22 Dec 2021 17:37:27 +0000 (09:37 -0800)]
ZTS: Fix refreserv_raidz.ksh

The rerefreserv_raidz test was failing on Linux because the sync being
issued doesn't guarantee a pool sync.  Switch to using the sync_pool
function and remove the ZTS exception for Linux.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12897

2 years agozfs-test/mmap_seek: fix build on musl
Georgy Yakovlev [Wed, 22 Dec 2021 00:44:18 +0000 (16:44 -0800)]
zfs-test/mmap_seek: fix build on musl

The build on musl needs linux/fs.h for SEEK_DATA and friends,
and sys/sysmacros.h for P2ROUNDUP.  Add the needed headers.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>
Closes #12891

2 years agozed: send notification email by default
shodanshok [Wed, 22 Dec 2021 00:24:05 +0000 (01:24 +0100)]
zed: send notification email by default

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
Closes #12806

2 years agocontrib/initrd hooks: properly quote @LIBFETCH_SONAME@
наб [Sat, 11 Dec 2021 01:41:23 +0000 (02:41 +0100)]
contrib/initrd hooks: properly quote @LIBFETCH_SONAME@

Bullseye shellcheck picks these up as SC2140, and it's right!
@LIBFETCH_SONAME@ is already quoted, so dracut had
  "$d/"libcurl.so.4""
and i-t had
  ""libcurl.so.4""

Partially reverts 34eef3e9a7a74d24a59d016051d547afc55dbaa0 (#12760),
which broke this

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agocontrib/pam_zfs_key: fix unused, remove argsused
наб [Sat, 11 Dec 2021 01:32:43 +0000 (02:32 +0100)]
contrib/pam_zfs_key: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agotests: fix unused, remove argsused
наб [Sat, 11 Dec 2021 01:24:35 +0000 (02:24 +0100)]
tests: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozfs: iter: fix unused, remove argsused
наб [Sat, 11 Dec 2021 01:00:02 +0000 (02:00 +0100)]
zfs: iter: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agoztest: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:57:41 +0000 (01:57 +0100)]
ztest: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozdb: il: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:40:42 +0000 (01:40 +0100)]
zdb: il: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozdb: main: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:37:25 +0000 (01:37 +0100)]
zdb: main: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozed: agents: fmd_api: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:26:20 +0000 (01:26 +0100)]
zed: agents: fmd_api: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozed: agents: zfs_retire: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:21:46 +0000 (01:21 +0100)]
zed: agents: zfs_retire: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozed: agents: zfs_mod: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:20:55 +0000 (01:20 +0100)]
zed: agents: zfs_mod: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozed: agents: zfs_diagnosis: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:18:01 +0000 (01:18 +0100)]
zed: agents: zfs_diagnosis: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozed: agents: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:08:22 +0000 (01:08 +0100)]
zed: agents: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozed: exec: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:07:57 +0000 (01:07 +0100)]
zed: exec: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozed: main: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:07:21 +0000 (01:07 +0100)]
zed: main: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozpool: vdev_os: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:06:29 +0000 (01:06 +0100)]
zpool: vdev_os: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozpool: main: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:05:17 +0000 (01:05 +0100)]
zpool: main: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozpool: iter: zpool_compare: fix unused, remove argsused
наб [Sat, 11 Dec 2021 00:05:01 +0000 (01:05 +0100)]
zpool: iter: zpool_compare: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozinject: cancel_one_handler: fix unused, remove argsused
наб [Fri, 10 Dec 2021 23:56:59 +0000 (00:56 +0100)]
zinject: cancel_one_handler: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agozhack: space_delta_cb: fix unused, remove argsused
наб [Fri, 10 Dec 2021 23:56:00 +0000 (00:56 +0100)]
zhack: space_delta_cb: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agoraidz_test: init_rand: fix unused, remove argsused
наб [Fri, 10 Dec 2021 23:53:04 +0000 (00:53 +0100)]
raidz_test: init_rand: fix unused, remove argsused

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12835

2 years agoZTS: speed up rsend tests
Brian Behlendorf [Fri, 17 Dec 2021 23:29:23 +0000 (23:29 +0000)]
ZTS: speed up rsend tests

With some minor tweaks several of rsend tests can be sped up
considerably without significantly reducing test coverage.

* send-c_verify_ratio:  ~120s -> ~60s
* send_realloc_*_files: ~330s -> ~65s

For the send_realloc* tests this also has the advantage of removing
(most of) the linux/freebsd conditional logic.  Note that for this
test more passes, and thus more incremental send/recvs, are preferable
to a larger number of files.

Total run time of the rsend test group was reduced from roughly 20 to
11 minutes in an environment similar to what's used by the CI.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12876

2 years agoZTS: rsend_007_pos failures
Brian Behlendorf [Fri, 17 Dec 2021 23:20:10 +0000 (23:20 +0000)]
ZTS: rsend_007_pos failures

The rsend_007_pos test reliably fails on Linux in the cleanup
function.  This is caused by an unmount error when attempting to
recursively destroy the newly received datasets.  Invoking `df`
prior to the `zfs destroy` interestingly avoids the unmont error.

Why this should matter is unclear and should be investigated.
However, this minor tweak may allow us to remove the ZTS rsend
exceptions.  The subsequent rsend_010_pos and rsend_011_pos
failures were a result of this initial failure.  The other
"maybe" failures I was unable to reproduce and have not been
recently observed in the master branch.

Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5665
Closes #6086
Closes #6087
Closes #6446
Closes #12876

2 years agoFreeBSD: fix world build after 143476ce8
Martin Matuška [Mon, 20 Dec 2021 22:28:43 +0000 (23:28 +0100)]
FreeBSD: fix world build after 143476ce8

Do not redefine the fallthrough macro when building with libcpp.

Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Martin Matuska <mm@FreeBSD.org>
Closes #12880

2 years agoFix error propagation from lzc_send_redacted
Philipp Riederer [Mon, 20 Dec 2021 18:50:46 +0000 (19:50 +0100)]
Fix error propagation from lzc_send_redacted

Any error from lzc_send_redacted is overwritten by the error of
send_conclusion_record; skip writing the conclusion record if there
was an earlier error.

Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Philipp Riederer <philipp@riederer.email>
Closes #12766

2 years agoLinux: Implement FS_IOC_GETVERSION
Ryan Moeller [Sat, 18 Dec 2021 00:18:37 +0000 (19:18 -0500)]
Linux: Implement FS_IOC_GETVERSION

Provide access to file generation number on Linux.

Add test coverage.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12856

2 years agolibshare: nfs: always try to mkdir()
наб [Tue, 18 May 2021 16:00:26 +0000 (18:00 +0200)]
libshare: nfs: always try to mkdir()

This also works out to one syscall if the directory exists,
but is one syscall shorter if it doesn't.

Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067

2 years agolibshare: nfs: set export file 644
наб [Mon, 17 May 2021 19:08:16 +0000 (21:08 +0200)]
libshare: nfs: set export file 644

The shares are publicly known anyway and can be interrogated by any
user, so this is a debugging aid more than anything.

Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067

2 years agolinux/libshare: nfs: don't needlessly strdup() hostspec
наб [Mon, 17 May 2021 18:34:04 +0000 (20:34 +0200)]
linux/libshare: nfs: don't needlessly strdup() hostspec

Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067

2 years agolibshare: nfs: share nfs_is_shared()
наб [Mon, 17 May 2021 18:25:29 +0000 (20:25 +0200)]
libshare: nfs: share nfs_is_shared()

Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067

2 years agolibshare: nfs: share nfs_copy_entries()
наб [Mon, 17 May 2021 17:56:48 +0000 (19:56 +0200)]
libshare: nfs: share nfs_copy_entries()

Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067

2 years agolibshare: nfs: open temporary file once
наб [Mon, 17 May 2021 16:13:18 +0000 (18:13 +0200)]
libshare: nfs: open temporary file once

Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067

2 years agolibshare: nfs: retry flock() when interrupted
наб [Mon, 17 May 2021 16:08:35 +0000 (18:08 +0200)]
libshare: nfs: retry flock() when interrupted

Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067

2 years agofreebsd/libshare: nfs: don't send SIGHUP to all processes
наб [Mon, 17 May 2021 16:03:26 +0000 (18:03 +0200)]
freebsd/libshare: nfs: don't send SIGHUP to all processes

pidfile_open() sets *pidptr to -1 if the process currently holding
the lock is between pidfile_open() and pidfile_write(),
the subsequent kill(mountdpid) would potentially SIGHUP all
non-system processes except init: just sleep for half a millisecond
and try again in that case

Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067

2 years agozfs-share.8: document -l flag
наб [Mon, 17 May 2021 16:09:46 +0000 (18:09 +0200)]
zfs-share.8: document -l flag

Description stolen from zfs-mount.8

Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067

2 years agocontrib/initrd: systemd-ask-password --no-tty before argument
наб [Fri, 17 Dec 2021 20:44:23 +0000 (21:44 +0100)]
contrib/initrd: systemd-ask-password --no-tty before argument

In systemd 249 (sid), sd-a-p processes its arguments in getopt + mode,
so "systemd-ask-password zupa --no-tty" prompts for "zupa --no-tty",
not "zupa" not on the tty, as expected (bullseye, 247).

Ref: https://github.com/systemd/systemd/commit/4b1c842d95bfd6ab352ade1a4655f9e512f35185
Ref: https://github.com/systemd/systemd/pull/19806
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12870

2 years agoWorkaround Debian's fake System.map behavior
Rich Ercolani [Fri, 17 Dec 2021 20:43:13 +0000 (15:43 -0500)]
Workaround Debian's fake System.map behavior

Debian ships fake System.map files by default, leading to the
invocation of depmod with them to flood you with errors about
missing symbols.

Let's notice and not do that.

Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12862

2 years agoZTS: alloc_class.ksh must wait for the process to exit
Brian Behlendorf [Fri, 17 Dec 2021 20:40:34 +0000 (12:40 -0800)]
ZTS: alloc_class.ksh must wait for the process to exit

The alloc_class_* tests may fail on Linux with an EBUSY error if
`zfs destroy` is run before the `dd` process has had a chance to
terminate.  Wait on the pid after the `kill -9` to make sure.

When testing I didn't observe any failures for the alloc_class
tests.  Remove them from the exceptions list, the CI was used to
verify the tests pass on all platforms.

Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12873

2 years agoZTS: Avoid piping send directly to /dev/null
Rich Ercolani [Fri, 17 Dec 2021 20:39:10 +0000 (15:39 -0500)]
ZTS: Avoid piping send directly to /dev/null

Unfortunately, #11445 means while we fail gracefully now, we still
fail, unless people want to implement a complex workaround just to
support /dev/null.

So let's just use the cheap workaround in a test for now.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12872

2 years agoZTS: Fix zpool_reopen_[1-5] on Fedora 35
Tony Hutter [Fri, 17 Dec 2021 20:37:21 +0000 (12:37 -0800)]
ZTS: Fix zpool_reopen_[1-5] on Fedora 35

The zpool_reopen_[1-5] tests are failing Fedora 35 with:

zpool_reopen_001_pos.ksh[64]: log_must[67]: log_pos[270]:
wait_for_resilver_end[98]: wait_for_action: line 71: func: is read only

Renaming 'func' -> 'funct' fixes the issue.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tony Hutter <hutter2@llnl.gov>
Closes #12871

2 years agoFix zvol_open() lock inversion
Brian Behlendorf [Fri, 17 Dec 2021 17:52:13 +0000 (09:52 -0800)]
Fix zvol_open() lock inversion

When restructuring the zvol_open() logic for the Linux 5.13 kernel
a lock inversion was accidentally introduced.  In the updated code
the spa_namespace_lock is now taken before the zv_suspend_lock
allowing the following scenario to occur:

    down_read <=== waiting for zv_suspend_lock
    zvol_open <=== holds spa_namespace_lock
    __blkdev_get
    blkdev_get_by_dev
    blkdev_open
    ...

     mutex_lock <== waiting for spa_namespace_lock
     spa_open_common
     spa_open
     dsl_pool_hold
     dmu_objset_hold_flags
     dmu_objset_hold
     dsl_prop_get
     dsl_prop_get_integer
     zvol_create_minor
     dmu_recv_end
     zfs_ioc_recv_impl <=== holds zv_suspend_lock via zvol_suspend()
     zfs_ioc_recv
     ...

This commit resolves the issue by moving the acquisition of the
spa_namespace_lock back to after the zv_suspend_lock which restores
the original ordering.

Additionally, as part of this change the error exit paths were
simplified where possible.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12863

2 years agoFreeBSD: Update argument types for VOP_READDIR
Alan Somers [Fri, 17 Dec 2021 17:50:12 +0000 (10:50 -0700)]
FreeBSD: Update argument types for VOP_READDIR

A recent commit to FreeBSD changed the type of
vop_readdir_args.a_cookies to a uint64_t**.  There is no functional
impact to ZFS because ZFS only uses 32-bit cookies, which will be
zero-extended to 64-bits by the existing code.

https://github.com/freebsd/freebsd-src/commit/b214fcceacad6b842545150664bd2695c1c2b34f

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Alan Somers <asomers@gmail.com>
Closes #12874

2 years agozcommon: pre-iterate over sysfs instead of statting every feature
наб [Fri, 17 Dec 2021 00:43:10 +0000 (01:43 +0100)]
zcommon: pre-iterate over sysfs instead of statting every feature

If sufficient memory (<2K, realistically) is available, libzfs_init()
can be significantly shorted by iterating over the correct sysfs
directory before registrations, we can turn 168 stats into 15/18
syscalls (3 opens (6 if built in), 3 fstats, 6 getdentses, and 3
closes), a tenfoldish reduction; this is probably a bit faster, too.

The list is always optional, and registration functions (and one-off
users) can simply pass NULL, which will fall back to the previous
mechanism

Also, don't allocate in zfs_mod_supported_impl, and use use access()
instead of stat(), since existence is really what we care about

Also, fix pre-prop-checking compat in fallback for built-in ZFS

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12089

2 years agozcommon: *_prop: make all zprop_index_t tables const
наб [Thu, 16 Dec 2021 21:26:04 +0000 (22:26 +0100)]
zcommon: *_prop: make all zprop_index_t tables const

They're already static, and there's no point in them being R/W
and living outside .rodata

Reviewed-by: RageLtMan <rageltman@sempervictus>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12836

2 years agoFreeBSD: Provide correct file generation number
Ryan Moeller [Thu, 16 Dec 2021 21:22:15 +0000 (16:22 -0500)]
FreeBSD: Provide correct file generation number

va_seq was actually a thin veil over va_gen, so z_gen is a more
appropriate value than z_seq to populate the field with.

Drop the unnecessary compat obfuscation and provide the correct
file generation number.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Ryan Moeller <freqlabs@freebsd.org>
Closes #12851

2 years agozfs list: Allow more fields in ZFS_ITER_SIMPLE mode
Allan Jude [Thu, 16 Dec 2021 19:56:22 +0000 (14:56 -0500)]
zfs list: Allow more fields in ZFS_ITER_SIMPLE mode

If the fields to be listed and sorted by are constrained
to those populated by dsl_dataset_fast_stat(), then
zfs list is much faster, as it does not need to open each
objset and reads its properties.

A previous optimization by Pawel Dawidek
(0cee24064a79f9c01fc4521543c37acea538405f) took advantage
of this to make listing snapshot names sorted only by name
much faster.

However, it was limited to `-o name -s name`, this work
extends this optimization to work with:
  - name
  - guid
  - createtxg
  - numclones
  - inconsistent
  - redacted
  - origin
and could be further extended to any other properties
supported by dsl_dataset_fast_stat() or similar, that do
not require extra locking or reading from disk.

Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Pawel Jakub Dawidek <pawel@dawidek.net>
Signed-off-by: Allan Jude <allan@klarasystems.com>
Closes #11080

2 years agosystemd: add weekly and monthly scrub timers
Georgy Yakovlev [Thu, 16 Dec 2021 19:47:22 +0000 (11:47 -0800)]
systemd: add weekly and monthly scrub timers

Timers can be enabled as follows:

systemctl enable zfs-scrub-weekly@rpool.timer --now
systemctl enable zfs-scrub-monthly@datapool.timer --now

Each timer will pull in zfs-scrub@${poolname}.service, which is not
schedule-specific.

Added PERIODIC SCRUB section to zpool-scrub.8.

Reviewed-by: Richard Laager <rlaager@wiktel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>
Closes #12193

2 years agot/z_diff/socket, zfs: main: fix unused argument warnings, ARGSUSED tags
наб [Thu, 9 Dec 2021 23:08:19 +0000 (00:08 +0100)]
t/z_diff/socket, zfs: main: fix unused argument warnings, ARGSUSED tags

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12829

2 years agolibzfs: diff: simplify superfluous stdio
наб [Thu, 9 Dec 2021 22:50:41 +0000 (23:50 +0100)]
libzfs: diff: simplify superfluous stdio

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12829

2 years agolibzfs: diff: print_what() can return the symbol => get_what()
наб [Thu, 9 Dec 2021 22:44:51 +0000 (23:44 +0100)]
libzfs: diff: print_what() can return the symbol => get_what()

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12829

2 years agolibzfs: diff: stream_bytes: use fputc, %hho formats chars
наб [Thu, 9 Dec 2021 22:42:02 +0000 (23:42 +0100)]
libzfs: diff: stream_bytes: use fputc, %hho formats chars

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12829

2 years agolibzfs: zpool_set_vdev_prop: remove unused vprop
наб [Thu, 9 Dec 2021 22:40:17 +0000 (23:40 +0100)]
libzfs: zpool_set_vdev_prop: remove unused vprop

Found by clang 14 with -Wunused-but-set-variable

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12829

2 years agolinux: libspl: getmntany: remove unused argument
наб [Thu, 9 Dec 2021 22:39:36 +0000 (23:39 +0100)]
linux: libspl: getmntany: remove unused argument

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12829

2 years agozfs, libzfs: diff: accept -h/ZFS_DIFF_NO_MANGLE, disabling path escaping
наб [Thu, 9 Dec 2021 23:02:52 +0000 (00:02 +0100)]
zfs, libzfs: diff: accept -h/ZFS_DIFF_NO_MANGLE, disabling path escaping

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12829

2 years agoAdd init script to load keys
ogelpre [Sun, 12 Dec 2021 19:17:14 +0000 (20:17 +0100)]
Add init script to load keys

Add new init scripts which allow automatic loading of keys if
keylocation property is set to a URI.

Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Benedikt Neuffer <ogelpre@itfriend.de>
Closes #11659
Closes #11662

2 years agozfs-dkms rpm: Fix scriptlets dependencies
Till Maas [Sun, 12 Dec 2021 19:15:25 +0000 (20:15 +0100)]
zfs-dkms rpm: Fix scriptlets dependencies

To ensure that the necessary packages are available during the %post and
%preun scriptlets, require them properly.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Till Maas <opensource@till.name>
Closes #12822
Closes #12832

2 years agoFreeBSD: Add vop_standard_writecount_nomsync
Ryan Moeller [Fri, 10 Dec 2021 14:15:27 +0000 (14:15 +0000)]
FreeBSD: Add vop_standard_writecount_nomsync

https://cgit.freebsd.org/src/commit?id=3ffcfa599e29686cf2b3c1a6087408c37acaed78

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12828

2 years agozfs: Fix a deadlock between page busy and the teardown lock
Mark Johnston [Sat, 20 Nov 2021 16:21:25 +0000 (11:21 -0500)]
zfs: Fix a deadlock between page busy and the teardown lock

When rolling back a dataset, ZFS has to purge file data resident in the
system page cache.  To do this, it loops over all vnodes for the
mountpoint and calls vn_pages_remove() to purge pages associated with
the vnode's VM object.  Each page is thus exclusively busied while the
dataset's teardown write lock is held.

When handling a page fault on a mapped ZFS file, FreeBSD's page fault
handler busies newly allocated pages and then uses VOP_GETPAGES to fill
them.  The ZFS getpages VOP acquires the teardown read lock with vnode
pages already busied.  This represents a lock order reversal which can
lead to deadlock.

To break the deadlock, observe that zfs_rezget() need only purge those
pages marked valid, and that pages busied by the page fault handler are,
by definition, invalid.  Furthermore, ZFS pages always transition from
invalid to valid with the teardown lock held, and ZFS never creates
partially valid pages.  Thus, zfs_rezget() can use the new
vn_pages_remove_valid() to skip over pages busied by the fault handler.

PR: 258208
Tested by: pho
Reviewed by: avg, sef, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32931

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12828

2 years agoFreeBSD: Catch up with more VFS changes
Ryan Moeller [Thu, 9 Dec 2021 18:04:56 +0000 (18:04 +0000)]
FreeBSD: Catch up with more VFS changes

Unused thread argument was removed from NDINIT*

https://cgit.freebsd.org/src/commit?id=7e1d3eefd410ca0fbae5a217422821244c3eeee4

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12828

2 years agoFreeBSD supports edonr follow up
наб [Thu, 9 Dec 2021 01:01:36 +0000 (02:01 +0100)]
FreeBSD supports edonr follow up

This chases 269b5dadcfd1d5732cf763dddcd46009a332eae4 (#12735),
which touched the actual code but didn't fix the comment

Additionally, ignore the name.

Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12823

2 years agoLinux 5.15 compat: META (#12824)
Brian Behlendorf [Tue, 7 Dec 2021 23:35:42 +0000 (15:35 -0800)]
Linux 5.15 compat: META (#12824)

The final 5.15 kernel is available and has been tested.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
2 years agocontrib/bash_completion.d: fix error spew from __zfs_match_snapshot()
наб [Tue, 7 Dec 2021 20:30:10 +0000 (21:30 +0100)]
contrib/bash_completion.d: fix error spew from __zfs_match_snapshot()

Given:
  /sbin/zfs list filling/a-zvol<TAB> -o space,refratio
The rest of the cmdline gets vored by:
  /sbin/zfs list filling/a-zvolcannot open 'filling/a-zvol':
  operation not applicable to datasets of this type

With -x (fragment):
  + COMPREPLY=($(compgen -W "$(__zfs_match_snapshot)" -- "$cur"))
  +++ __zfs_match_snapshot
  +++ local base_dataset=filling/dziadtop-nowe-duchy
  +++ [[ filling/dziadtop-nowe-duchy != filling/dziadtop-nowe-duchy ]]
  +++ [[ filling/dziadtop-nowe-duchy != '' ]]
  +++ __zfs_list_datasets filling/dziadtop-nowe-duchy
  +++ /sbin/zfs list -H -o name -s name -t filesystem
                     -r filling/dziadtop-nowe-duchy
  +++ tail -n +2
  cannot open 'filling/dziadtop-nowe-duchy':
  operation not applicable to datasets of this type
  +++ echo filling/dziadtop-nowe-duchy
  +++ echo filling/dziadtop-nowe-duchy@
  ++ compgen -W 'filling/dziadtop-nowe-duchy

This properly completes with:
  $ /sbin/zfs list filling/a-zvol<TAB> -o space,refratio
  filling/a-zvol   filling/a-zvol@
  $ /sbin/zfs list filling/a-zvol<cursor> -o space,refratio

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12820

2 years agoLinux 5.16: Resolve ZSTD_isError symbol collision in Linux kernel
Coleman Kane [Sun, 5 Dec 2021 20:18:46 +0000 (15:18 -0500)]
Linux 5.16: Resolve ZSTD_isError symbol collision in Linux kernel

Newer zstd code introduced in the main kernel tree now creates a symbol
collision with ZSTD_isError in our ZSTD code. This change relabels our
implementation with a ZFS-specific symbol name, and undoes some
macro-based micro-optimizations that conflict with the attempt to rename
our internal-use version.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Coleman Kane <ckane@colemankane.org>
Closes #12819

2 years agoLinux 5.16: The blk-cgroup.h header is where struct blkcg_gq is defined
Coleman Kane [Sat, 4 Dec 2021 03:00:10 +0000 (22:00 -0500)]
Linux 5.16: The blk-cgroup.h header is where struct blkcg_gq is defined

The definition of struct blkcg_gq was moved into blk-cgroup.h, which is
a header that's been in Linux since 2015. This is used by
vdev_blkg_tryget() in module/os/linux/zfs/vdev_disk.c. Since the kernel
for CentOS 7 and similar-generation releases doesn't have this header,
its inclusion is guarded by a configure test.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Coleman Kane <ckane@colemankane.org>
Closes #12819

2 years agoLinux 5.16: bio_set_dev is no longer a helper macro
Coleman Kane [Sat, 4 Dec 2021 02:45:28 +0000 (21:45 -0500)]
Linux 5.16: bio_set_dev is no longer a helper macro

This change adds a confiugre check to determine if bio_set_dev is a
helper macro or not. If not, then the attempt to override its internal
call to bio_associate_blkg(), with a macro definition to our own
version, is no longer possible, as the compiler won't use it when
compiling the new inline function replacement implemented in the header.
This change also creates a new vdev_bio_set_dev() function that performs
the same work, and also performs the work implemented in
vdev_bio_associate_blkg(), as it is the only thing calling that function
in our code. Our custom vdev_bio_associate_blkg() is now only compiled
if the bio_set_dev() is a macro in the Linux headers.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Coleman Kane <ckane@colemankane.org>
Closes #12819

2 years agoLinux 5.16: type member of iov_iter renamed iter_type
Coleman Kane [Fri, 3 Dec 2021 04:25:08 +0000 (23:25 -0500)]
Linux 5.16: type member of iov_iter renamed iter_type

The iov_iter->type member was renamed iov_iter->iter_type. However,
while looking into this, realized that in 2018 a iov_iter_type(*iov)
accessor function was introduced. So if that is present, use it,
otherwise fall back to trying the existing behavior of directly
accessing type from iov_iter.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Coleman Kane <ckane@colemankane.org>
Closes #12819

2 years agoLinux 5.16: block_device_operations->submit_bio now returns void
Coleman Kane [Fri, 3 Dec 2021 03:54:05 +0000 (22:54 -0500)]
Linux 5.16: block_device_operations->submit_bio now returns void

The return type for the submit_bio member of struct
block_device_operations was changed to no longer return a value.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Coleman Kane <ckane@colemankane.org>
Closes #12819

2 years agoZFS send/recv with ashift 9->12 leads to data corruption
Paul Dagnelie [Tue, 7 Dec 2021 18:27:59 +0000 (10:27 -0800)]
ZFS send/recv with ashift 9->12 leads to data corruption

Improve the ability of zfs send to determine if a block is compressed
or not by using information contained in the blkptr.

Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Reviewed-by: Matthew Ahrens <matthew.ahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #12770

2 years agoUpdate "tests/README.md"
Arshad Hussain [Tue, 7 Dec 2021 16:49:25 +0000 (22:19 +0530)]
Update "tests/README.md"

This patch adds detail section on adding and running
test-case. It also changes markdown number list to
more readeable headers

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Closes #12737

2 years agoAdd `const` to nvlist functions to properly expose their real behavior
Paul Dagnelie [Tue, 7 Dec 2021 01:19:13 +0000 (17:19 -0800)]
Add `const` to nvlist functions to properly expose their real behavior

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #12728

2 years agoZTS: import_rewind_device_replaced reliably fails
Brian Behlendorf [Mon, 6 Dec 2021 17:45:17 +0000 (09:45 -0800)]
ZTS: import_rewind_device_replaced reliably fails

The import_rewind_device_replaced.ksh test was never entirely reliable
because it depends on MOS data not being overwritten.  The MOS data is
not protected by the snapshot so occasional failures were always
expected.  However, this test is now failing reliably on all platforms
indicating something has changed in the code since the test was marked
"maybe".  Convert the test to a "known" failure until the root cause
is identified and resolved.

Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12821

2 years agoCorrected a case where we could read uninited ABD memory
Rich Ercolani [Fri, 3 Dec 2021 21:13:21 +0000 (16:13 -0500)]
Corrected a case where we could read uninited ABD memory

For my sins, I started running valgrind over ztest to try and fix
that pesky intermittent "zloop dies with malloc errors" problem.

This one seemed exciting enough to merit cutting a PR for before
the rest get polished.

Suggested-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12214

2 years agoStrip colons from all test result filenames
John Wren Kennedy [Thu, 2 Dec 2021 00:18:45 +0000 (17:18 -0700)]
Strip colons from all test result filenames

The upload artifact functionality in github can't handle colons in
filenames. The current code handles this for files under the most
recent set of results. With the ability to rerun failed tests, now
there can be multiple sets of results, and they all need to be
processed in the same way.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: John Kennedy <john.kennedy@delphix.com>
Closes #12815

2 years agoLinux 5.13 compat: retry zvol_open() when contended
Brian Behlendorf [Thu, 2 Dec 2021 00:07:12 +0000 (16:07 -0800)]
Linux 5.13 compat: retry zvol_open() when contended

Due to a possible lock inversion the zvol open call path on Linux
needs to be able to retry in the case where the spa_namespace_lock
cannot be acquired.

For Linux 5.12 an older kernel this was accomplished by returning
-ERESTARTSYS from zvol_open() to request that blkdev_get() drop
the bdev->bd_mutex lock, reaquire it, then call the open callback
again.  However, as of the 5.13 kernel this behavior was removed.

Therefore, for 5.12 and older kernels we preserved the existing
retry logic, but for 5.13 and newer kernels we retry internally in
zvol_open().  This should always succeed except in the case where
a pool's vdev are layed on zvols, in which case it may fail.  To
handle this case vdev_disk_open() has been updated to retry when
opening a device when -ERESTARTSYS is returned.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #12301
Closes #12759

2 years agoTemporarily remove tests from sanity runfile
John Wren Kennedy [Wed, 1 Dec 2021 21:22:52 +0000 (14:22 -0700)]
Temporarily remove tests from sanity runfile

With the addition of functionality to rerun failing tests, some
tests that fail only sometimes still fail often enough to degrade
the reliability of the sanity runs. Remove them from the runfile
until they reliably pass.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: John Kennedy <john.kennedy@delphix.com>
Closes #12814

2 years agoAdd zfs-test facility to automatically rerun failing tests
Paul Dagnelie [Wed, 1 Dec 2021 17:38:53 +0000 (09:38 -0800)]
Add zfs-test facility to automatically rerun failing tests

This was a project proposed as part of the Quality theme for the
hackthon for the 2021 OpenZFS Developer Summit. The idea is to improve
the usability of the automated tests that get run when a PR is created
by having failing tests automatically rerun in order to make flaky
tests less impactful.

Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #12740

2 years agoget_key_material: fix style
Attila Fülöp [Sun, 14 Nov 2021 17:50:49 +0000 (18:50 +0100)]
get_key_material: fix style

Reviewed-by: Felix Dörre <felix@dogcraft.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes #12765

2 years agoget_key_material: skip passphrase validation when loading keys
Harald van Dijk [Tue, 19 Oct 2021 23:32:28 +0000 (00:32 +0100)]
get_key_material: skip passphrase validation when loading keys

The restriction that an encryption key must be at least
MIN_PASSPHRASE_LEN characters long make sense when changing the
encryption key, but not when loading: as this restriction is not
enforced in the libraries, it is possible to bypass zfs change-key's
restrictions and end up with a key that becomes impossible to load with
zfs load-key, for example through pam_zfs_key.

Reviewed-by: Felix Dörre <felix@dogcraft.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Harald van Dijk <harald@gigawatt.nl>
Closes #12765

2 years agopam_zfs_key: tests: check if zfs load-key works on short passphrases
Attila Fülöp [Sun, 14 Nov 2021 17:08:45 +0000 (18:08 +0100)]
pam_zfs_key: tests: check if zfs load-key works on short passphrases

The pam_zfs_key pam module does not enforce a minimum password
length while changing the user password and thus the users home
dataset passphrase. To not end up with a dateset `zfs load-key`
can't load the key for, `zfs load-key` should not enforce a minimum
passphrase length. This adds a test for that.

Reviewed-by: Felix Dörre <felix@dogcraft.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes #12765
Closes #12651
Closes #12656

2 years agopam_zfs_key: tests: clean up the generated pam service config file
Attila Fülöp [Sun, 14 Nov 2021 16:36:12 +0000 (17:36 +0100)]
pam_zfs_key: tests: clean up the generated pam service config file

Remove the generated pam service config file
`/etc/pam.d/pam_zfs_key_test` on test cleanup, since the tests
shouldn't alter system state.

While here, move the pam service config file name into a variable.

Reviewed-by: Felix Dörre <felix@dogcraft.de>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes #12765

2 years agoRemove REMAKE_INITRD
jokersus [Tue, 30 Nov 2021 19:09:15 +0000 (22:09 +0300)]
Remove REMAKE_INITRD

The option has been deprecated in dkms and will break packaging in
future versions. See https://github.com/dell/dkms/commit/7114c62

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Amanakis <gamanakis@gmail.com>
Signed-off-by: jokersus <jokersus.cava@gmail.com>
Closes #12781

2 years agoDefault to zfs_dmu_offset_next_sync=1
Brian Behlendorf [Tue, 30 Nov 2021 18:38:09 +0000 (10:38 -0800)]
Default to zfs_dmu_offset_next_sync=1

Strict hole reporting was previously disabled by default as a
performance optimization.  However, this has lead to confusion
over the expected behavior and a variety of workarounds being
adopted by consumers of ZFS.  Change the default behavior to
always report holes and force the TXG sync.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12746

2 years agoStop segfaulting on unmount error case
Rich Ercolani [Tue, 30 Nov 2021 18:36:36 +0000 (13:36 -0500)]
Stop segfaulting on unmount error case

After interrupting ZTS runs that errored out, I found that
"zpool export testpool2" was segfaulting.

This seems unnecessary.

Reviewed-by: szubersk <szuberskidamian@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12804

2 years agoCode cleanups
Pawel Jakub Dawidek [Tue, 30 Nov 2021 18:32:38 +0000 (10:32 -0800)]
Code cleanups

- Allocate ve_search on the stack, so we avoid allocating memory for
  every I/O even if the VDEV cache is disabled.
- Reduce lock scope.
- Avoid locking in vdev_cache_read() when the VDEV cache is disabled.
- Sort file names properly.
- Correct comment.

Reviewed-by: Allan Jude <allan@klarasystems.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Pawel Jakub Dawidek <pawel@dawidek.net>
Closes #12749

2 years agoReplace wrong occurrences of `affect` by `effect` in the man pages
maxz [Tue, 30 Nov 2021 18:28:57 +0000 (19:28 +0100)]
Replace wrong occurrences of `affect` by `effect` in the man pages

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Amanakis <gamanakis@gmail.com>
Signed-off-by: Max Zettlmeißl <max@zettlmeissl.de>
Closes #12784

2 years agoAllow printing special vdev metaslab groups
Rich Ercolani [Tue, 30 Nov 2021 18:26:45 +0000 (13:26 -0500)]
Allow printing special vdev metaslab groups

Sometimes, we'd like to know info about the metaslab groups
on special vdevs too. So let's make -MM do something useful.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12750

2 years agoPass `--enable=all` to shellcheck within contrib/
Damian Szuberski [Tue, 30 Nov 2021 18:23:10 +0000 (19:23 +0100)]
Pass `--enable=all` to shellcheck within contrib/

- Remove `SHELLCHECK_IGNORE` in favor of inline suppressions
  and more general `SHELLCHECK_OPTS`.

- Exclude `SC2250` (turned on by `--enable=all`) globally

- Pass `--enable=all` to shellcheck for scripts in contrib/: it's
  very important to catch errors early in areas that are not easily
  testable.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes #12760

2 years agoetc/systemd/zfs-mount-generator: serialise, handle keylocation=http[s]://
наб [Tue, 30 Nov 2021 16:29:50 +0000 (17:29 +0100)]
etc/systemd/zfs-mount-generator: serialise, handle keylocation=http[s]://

* etc/systemd/zfs-mount-generator: serialise

The wins for a relatively normal workload are rather slim:
real 0.02119s/0.00985s=2.15029x
user 0.02130s/0.00346s=6.15560x
sys 0.03858s/0.00643s=6.00062x

wall-total 0.014518s/0.005925s=2.45009x
wall-init 0.014518s/0.002457s=5.90684x
wall-real 0.014518s/0.003467s=4.18668x

But this is a big win on machines with a lot of datasets and expensive
forks.

For example, the gain on a VM on my work laptop with 900+ legacy-mount
Docker datasets, the original gains from the C rewrite were
only five-fold:
real    0.516s/0.102s=5.05882x
user    0.237s/0.143s=1.65734x
sys     0.287s/0.100s=2.87x

And this serial variant gains this back there as well:
real    0.102s/0.008s=12.75x
user    0.143s/0.007s=20.42857
sys     0.100s/0.001s=100x

wall-total 0.09717s/0.00319s=30.40255x
wall-init 0.00203s/0.00200s=1.015941x
wall-real 0.09513s/0.00118s=80.02043x

For a total of
real    0.516s/0.008s=64.5x
user    0.237s/0.007s=33.85714x
sys     0.287s/0.001s=287x

Suggested-by: Richard Laager <rlaager@wiktel.com>
* etc/systemd/zfs-mount-generator: pull in network for keylocation=https

Also simplify RequiresMountsFor= handling
Ref: #11956

Reviewed-by: Richard Laager <rlaager@wiktel.com>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12138