Reviewed-by: Matthew Ahrens <mahrens@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Mark Maybee <mark.maybee@delphix.com> Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #12938
Allan Jude [Thu, 6 Jan 2022 18:57:09 +0000 (13:57 -0500)]
ZTS: normalize on use of `sync_pool` and `sync_all_pools`
- Replaces use of manual `zpool sync`
- Don't use `log_must sync_pool` as `sync_pool` uses it internally
- Replace many (but not all) uses of `sync` with `sync_pool`
This makes the tests more consistent, and makes searching easier.
Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Allan Jude <allan@klarasystems.com>
Closes #12894
Manoj Joseph [Thu, 6 Jan 2022 18:54:32 +0000 (10:54 -0800)]
Long opts for zdb
This change introduces long options for zdb. It updates the usage
message as well to include the long options.
Reviewed-by: Matthew Ahrens <mahrens@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Manoj Joseph <manoj.joseph@delphix.com>
Closes #12818
chrisrd [Wed, 5 Jan 2022 01:07:33 +0000 (12:07 +1100)]
zfs_prune: reset sc.nr_to_scan
sc.nr_to_scan is an input to super_cache_clean (via
shrinker->scan_objects), used to set the number of objects to scan
in the various caches. However super_cache_scan also modifies
sc.nr_to_scan, so when used in a loop we need to reset
sc.nr_to_scan back to our desired nr_to_scan for the next
iteration.
Issue discovered and solution suggested by
Tenzin Lhakhang @tlhakhan.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Chris Dunlop <chris@onthe.net.au>
Issue #12433
Closes #12908
Verify that all empty sectors are zero filled before using them to
calculate parity. Failure to do so can result in incorrect parity
columns being generated and written to disk if the contents of an
empty sector are non-zero. This was possible because the checksum
only protects the data portions of the buffer, not the empty sector
padding.
This issue has been addressed by updating raidz_parity_verify() to
check that all dRAID empty sectors are zero filled. Any sectors
which are non-zero will be fixed, repair IO issued, and a checksum
error logged. They can then be safely used to verify the parity.
This specific type of damage is unlikely to occur since it requires
a disk to have silently returned bad data, for an empty sector, while
performing a scrub. However, if a pool were to have been damaged
in this way, scrubbing the pool with this change applied will repair
both the empty sector and parity columns as long as the data checksum
is valid. Checksum errors will be reported in the `zpool status`
output for any repairs which are made.
Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Mark Maybee <mark.maybee@delphix.com> Reviewed-by: Brian Atkinson <batkinson@lanl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12857
Brian Behlendorf [Thu, 23 Dec 2021 17:21:40 +0000 (09:21 -0800)]
ZTS: Fix enospc_002_pos.ksh again
This is a follow up commit for e03a41a60 which aimed to resolve
this same test failure. The core "problem" here is that it takes
very little space to perform a clone/snapshot/bookmark, which
means if we want these commands to reliably fail the pool must
truely have exhausted all free space.
This commit increases the number of fill iterations to try and
consume every block which we can. This still can't guarantee
the clone/snapshot/bookmark will fail, but it significantly
improves the odds. The exception was kept since it's still
not a sure thing.
Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Igor Kozhukhov <igor@dilos.org> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12903
Alexander Motin [Thu, 23 Dec 2021 01:07:13 +0000 (20:07 -0500)]
Reduce number of arc_prune threads
On FreeBSD vnode reclamation is single-threaded, protected by single
global lock. Linux seems to be able to use a thread per mount point,
but at this time it creates more harm than good.
Reduce number of threads to 1, adding tunable in case somebody wants
to try more.
Reviewed-by: Ryan Moeller <ryan@ixsystems.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Chris Dunlop <chris@onthe.net.au> Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes #12896
Issue #9966
Brian Behlendorf [Wed, 22 Dec 2021 19:05:07 +0000 (11:05 -0800)]
ZTS: Fix rollback_003_pos.ksh
Under Linux when rolling back a mounted filesystem negative dentries
may not be dropped from the cache. This can result in an ENOENT
being incorrectly returned on first access. Issuing a `df` before
the unmount results in the negative dentries being invalidated and
side steps the issue.
This is solely a workaround for the test case on Linux and not
correct behavior. The core issue of invalidating negative dentries
needs to be handled with a kernel side change. This is being
tracked as issue #6143.
Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12898
Issue #6143
Brian Behlendorf [Wed, 22 Dec 2021 17:37:27 +0000 (09:37 -0800)]
ZTS: Fix refreserv_raidz.ksh
The rerefreserv_raidz test was failing on Linux because the sync being
issued doesn't guarantee a pool sync. Switch to using the sync_pool
function and remove the ZTS exception for Linux.
Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12897
Bullseye shellcheck picks these up as SC2140, and it's right!
@LIBFETCH_SONAME@ is already quoted, so dracut had
"$d/"libcurl.so.4""
and i-t had
""libcurl.so.4""
For the send_realloc* tests this also has the advantage of removing
(most of) the linux/freebsd conditional logic. Note that for this
test more passes, and thus more incremental send/recvs, are preferable
to a larger number of files.
Total run time of the rsend test group was reduced from roughly 20 to
11 minutes in an environment similar to what's used by the CI.
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12876
Brian Behlendorf [Fri, 17 Dec 2021 23:20:10 +0000 (23:20 +0000)]
ZTS: rsend_007_pos failures
The rsend_007_pos test reliably fails on Linux in the cleanup
function. This is caused by an unmount error when attempting to
recursively destroy the newly received datasets. Invoking `df`
prior to the `zfs destroy` interestingly avoids the unmont error.
Why this should matter is unclear and should be investigated.
However, this minor tweak may allow us to remove the ZTS rsend
exceptions. The subsequent rsend_010_pos and rsend_011_pos
failures were a result of this initial failure. The other
"maybe" failures I was unable to reproduce and have not been
recently observed in the master branch.
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5665
Closes #6086
Closes #6087
Closes #6446
Closes #12876
Do not redefine the fallthrough macro when building with libcpp.
Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Martin Matuska <mm@FreeBSD.org>
Closes #12880
Philipp Riederer [Mon, 20 Dec 2021 18:50:46 +0000 (19:50 +0100)]
Fix error propagation from lzc_send_redacted
Any error from lzc_send_redacted is overwritten by the error of
send_conclusion_record; skip writing the conclusion record if there
was an earlier error.
Reviewed-by: Paul Dagnelie <pcd@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Philipp Riederer <philipp@riederer.email>
Closes #12766
наб [Mon, 17 May 2021 16:03:26 +0000 (18:03 +0200)]
freebsd/libshare: nfs: don't send SIGHUP to all processes
pidfile_open() sets *pidptr to -1 if the process currently holding
the lock is between pidfile_open() and pidfile_write(),
the subsequent kill(mountdpid) would potentially SIGHUP all
non-system processes except init: just sleep for half a millisecond
and try again in that case
Reviewed-by: Don Brady <don.brady@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12067
наб [Fri, 17 Dec 2021 20:44:23 +0000 (21:44 +0100)]
contrib/initrd: systemd-ask-password --no-tty before argument
In systemd 249 (sid), sd-a-p processes its arguments in getopt + mode,
so "systemd-ask-password zupa --no-tty" prompts for "zupa --no-tty",
not "zupa" not on the tty, as expected (bullseye, 247).
Brian Behlendorf [Fri, 17 Dec 2021 20:40:34 +0000 (12:40 -0800)]
ZTS: alloc_class.ksh must wait for the process to exit
The alloc_class_* tests may fail on Linux with an EBUSY error if
`zfs destroy` is run before the `dd` process has had a chance to
terminate. Wait on the pid after the `kill -9` to make sure.
When testing I didn't observe any failures for the alloc_class
tests. Remove them from the exceptions list, the CI was used to
verify the tests pass on all platforms.
Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Rich Ercolani <rincebrain@gmail.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12873
Rich Ercolani [Fri, 17 Dec 2021 20:39:10 +0000 (15:39 -0500)]
ZTS: Avoid piping send directly to /dev/null
Unfortunately, #11445 means while we fail gracefully now, we still
fail, unless people want to implement a complex workaround just to
support /dev/null.
So let's just use the cheap workaround in a test for now.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #12872
Tony Hutter [Fri, 17 Dec 2021 20:37:21 +0000 (12:37 -0800)]
ZTS: Fix zpool_reopen_[1-5] on Fedora 35
The zpool_reopen_[1-5] tests are failing Fedora 35 with:
zpool_reopen_001_pos.ksh[64]: log_must[67]: log_pos[270]:
wait_for_resilver_end[98]: wait_for_action: line 71: func: is read only
Renaming 'func' -> 'funct' fixes the issue.
Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tony Hutter <hutter2@llnl.gov>
Closes #12871
Brian Behlendorf [Fri, 17 Dec 2021 17:52:13 +0000 (09:52 -0800)]
Fix zvol_open() lock inversion
When restructuring the zvol_open() logic for the Linux 5.13 kernel
a lock inversion was accidentally introduced. In the updated code
the spa_namespace_lock is now taken before the zv_suspend_lock
allowing the following scenario to occur:
This commit resolves the issue by moving the acquisition of the
spa_namespace_lock back to after the zv_suspend_lock which restores
the original ordering.
Additionally, as part of this change the error exit paths were
simplified where possible.
Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Rich Ercolani <rincebrain@gmail.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #12863
Alan Somers [Fri, 17 Dec 2021 17:50:12 +0000 (10:50 -0700)]
FreeBSD: Update argument types for VOP_READDIR
A recent commit to FreeBSD changed the type of
vop_readdir_args.a_cookies to a uint64_t**. There is no functional
impact to ZFS because ZFS only uses 32-bit cookies, which will be
zero-extended to 64-bits by the existing code.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Alan Somers <asomers@gmail.com>
Closes #12874
наб [Fri, 17 Dec 2021 00:43:10 +0000 (01:43 +0100)]
zcommon: pre-iterate over sysfs instead of statting every feature
If sufficient memory (<2K, realistically) is available, libzfs_init()
can be significantly shorted by iterating over the correct sysfs
directory before registrations, we can turn 168 stats into 15/18
syscalls (3 opens (6 if built in), 3 fstats, 6 getdentses, and 3
closes), a tenfoldish reduction; this is probably a bit faster, too.
The list is always optional, and registration functions (and one-off
users) can simply pass NULL, which will fall back to the previous
mechanism
Also, don't allocate in zfs_mod_supported_impl, and use use access()
instead of stat(), since existence is really what we care about
Also, fix pre-prop-checking compat in fallback for built-in ZFS
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes #12089
Ryan Moeller [Thu, 16 Dec 2021 21:22:15 +0000 (16:22 -0500)]
FreeBSD: Provide correct file generation number
va_seq was actually a thin veil over va_gen, so z_gen is a more
appropriate value than z_seq to populate the field with.
Drop the unnecessary compat obfuscation and provide the correct
file generation number.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Ryan Moeller <freqlabs@freebsd.org>
Closes #12851
Allan Jude [Thu, 16 Dec 2021 19:56:22 +0000 (14:56 -0500)]
zfs list: Allow more fields in ZFS_ITER_SIMPLE mode
If the fields to be listed and sorted by are constrained
to those populated by dsl_dataset_fast_stat(), then
zfs list is much faster, as it does not need to open each
objset and reads its properties.
A previous optimization by Pawel Dawidek
(0cee24064a79f9c01fc4521543c37acea538405f) took advantage
of this to make listing snapshot names sorted only by name
much faster.
However, it was limited to `-o name -s name`, this work
extends this optimization to work with:
- name
- guid
- createtxg
- numclones
- inconsistent
- redacted
- origin
and could be further extended to any other properties
supported by dsl_dataset_fast_stat() or similar, that do
not require extra locking or reading from disk.
Reviewed-by: Mark Maybee <mark.maybee@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Pawel Jakub Dawidek <pawel@dawidek.net> Signed-off-by: Allan Jude <allan@klarasystems.com>
Closes #11080
Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12828
Mark Johnston [Sat, 20 Nov 2021 16:21:25 +0000 (11:21 -0500)]
zfs: Fix a deadlock between page busy and the teardown lock
When rolling back a dataset, ZFS has to purge file data resident in the
system page cache. To do this, it loops over all vnodes for the
mountpoint and calls vn_pages_remove() to purge pages associated with
the vnode's VM object. Each page is thus exclusively busied while the
dataset's teardown write lock is held.
When handling a page fault on a mapped ZFS file, FreeBSD's page fault
handler busies newly allocated pages and then uses VOP_GETPAGES to fill
them. The ZFS getpages VOP acquires the teardown read lock with vnode
pages already busied. This represents a lock order reversal which can
lead to deadlock.
To break the deadlock, observe that zfs_rezget() need only purge those
pages marked valid, and that pages busied by the page fault handler are,
by definition, invalid. Furthermore, ZFS pages always transition from
invalid to valid with the teardown lock held, and ZFS never creates
partially valid pages. Thus, zfs_rezget() can use the new
vn_pages_remove_valid() to skip over pages busied by the fault handler.
Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12828
Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org>
Closes #12828
projects / FreeBSD / FreeBSD.git / log