2 .\" Copyright (c) 2013 by Turbo Fredriksson <turbo@bayour.com>. All rights reserved.
3 .\" Copyright (c) 2017 Datto Inc.
4 .\" The contents of this file are subject to the terms of the Common Development
5 .\" and Distribution License (the "License"). You may not use this file except
6 .\" in compliance with the License. You can obtain a copy of the license at
7 .\" usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing.
9 .\" See the License for the specific language governing permissions and
10 .\" limitations under the License. When distributing Covered Code, include this
11 .\" CDDL HEADER in each file and include the License file at
12 .\" usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this
13 .\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your
14 .\" own identifying information:
15 .\" Portions Copyright [yyyy] [name of copyright owner]
16 .TH ZFS-MODULE-PARAMETERS 5 "Oct 28, 2017"
18 zfs\-module\-parameters \- ZFS module parameters
22 Description of the different parameters to the ZFS module.
24 .SS "Module parameters"
31 \fBdbuf_cache_max_bytes\fR (ulong)
34 Maximum size in bytes of the dbuf cache. When \fB0\fR this value will default
35 to \fB1/2^dbuf_cache_shift\fR (1/32) of the target ARC size, otherwise the
36 provided value in bytes will be used. The behavior of the dbuf cache and its
37 associated settings can be observed via the \fB/proc/spl/kstat/zfs/dbufstats\fR
40 Default value: \fB0\fR.
46 \fBdbuf_cache_hiwater_pct\fR (uint)
49 The percentage over \fBdbuf_cache_max_bytes\fR when dbufs must be evicted
52 Default value: \fB10\fR%.
58 \fBdbuf_cache_lowater_pct\fR (uint)
61 The percentage below \fBdbuf_cache_max_bytes\fR when the evict thread stops
64 Default value: \fB10\fR%.
70 \fBdbuf_cache_shift\fR (int)
73 Set the size of the dbuf cache, \fBdbuf_cache_max_bytes\fR, to a log2 fraction
74 of the target arc size.
76 Default value: \fB5\fR.
82 \fBignore_hole_birth\fR (int)
85 When set, the hole_birth optimization will not be used, and all holes will
86 always be sent on zfs send. Useful if you suspect your datasets are affected
87 by a bug in hole_birth.
89 Use \fB1\fR for on (default) and \fB0\fR for off.
95 \fBl2arc_feed_again\fR (int)
98 Turbo L2ARC warm-up. When the L2ARC is cold the fill interval will be set as
101 Use \fB1\fR for yes (default) and \fB0\fR to disable.
107 \fBl2arc_feed_min_ms\fR (ulong)
110 Min feed interval in milliseconds. Requires \fBl2arc_feed_again=1\fR and only
111 applicable in related situations.
113 Default value: \fB200\fR.
119 \fBl2arc_feed_secs\fR (ulong)
122 Seconds between L2ARC writing
124 Default value: \fB1\fR.
130 \fBl2arc_headroom\fR (ulong)
133 How far through the ARC lists to search for L2ARC cacheable content, expressed
134 as a multiplier of \fBl2arc_write_max\fR
136 Default value: \fB2\fR.
142 \fBl2arc_headroom_boost\fR (ulong)
145 Scales \fBl2arc_headroom\fR by this percentage when L2ARC contents are being
146 successfully compressed before writing. A value of 100 disables this feature.
148 Default value: \fB200\fR%.
154 \fBl2arc_noprefetch\fR (int)
157 Do not write buffers to L2ARC if they were prefetched but not used by
160 Use \fB1\fR for yes (default) and \fB0\fR to disable.
166 \fBl2arc_norw\fR (int)
169 No reads during writes
171 Use \fB1\fR for yes and \fB0\fR for no (default).
177 \fBl2arc_write_boost\fR (ulong)
180 Cold L2ARC devices will have \fBl2arc_write_max\fR increased by this amount
181 while they remain cold.
183 Default value: \fB8,388,608\fR.
189 \fBl2arc_write_max\fR (ulong)
192 Max write bytes per interval
194 Default value: \fB8,388,608\fR.
200 \fBmetaslab_aliquot\fR (ulong)
203 Metaslab granularity, in bytes. This is roughly similar to what would be
204 referred to as the "stripe size" in traditional RAID arrays. In normal
205 operation, ZFS will try to write this amount of data to a top-level vdev
206 before moving on to the next one.
208 Default value: \fB524,288\fR.
214 \fBmetaslab_bias_enabled\fR (int)
217 Enable metaslab group biasing based on its vdev's over- or under-utilization
218 relative to the pool.
220 Use \fB1\fR for yes (default) and \fB0\fR for no.
226 \fBzfs_metaslab_segment_weight_enabled\fR (int)
229 Enable/disable segment-based metaslab selection.
231 Use \fB1\fR for yes (default) and \fB0\fR for no.
237 \fBzfs_metaslab_switch_threshold\fR (int)
240 When using segment-based metaslab selection, continue allocating
241 from the active metaslab until \fBzfs_metaslab_switch_threshold\fR
242 worth of buckets have been exhausted.
244 Default value: \fB2\fR.
250 \fBmetaslab_debug_load\fR (int)
253 Load all metaslabs during pool import.
255 Use \fB1\fR for yes and \fB0\fR for no (default).
261 \fBmetaslab_debug_unload\fR (int)
264 Prevent metaslabs from being unloaded.
266 Use \fB1\fR for yes and \fB0\fR for no (default).
272 \fBmetaslab_fragmentation_factor_enabled\fR (int)
275 Enable use of the fragmentation metric in computing metaslab weights.
277 Use \fB1\fR for yes (default) and \fB0\fR for no.
283 \fBmetaslabs_per_vdev\fR (int)
286 When a vdev is added, it will be divided into approximately (but no more than) this number of metaslabs.
288 Default value: \fB200\fR.
294 \fBmetaslab_preload_enabled\fR (int)
297 Enable metaslab group preloading.
299 Use \fB1\fR for yes (default) and \fB0\fR for no.
305 \fBmetaslab_lba_weighting_enabled\fR (int)
308 Give more weight to metaslabs with lower LBAs, assuming they have
309 greater bandwidth as is typically the case on a modern constant
310 angular velocity disk drive.
312 Use \fB1\fR for yes (default) and \fB0\fR for no.
318 \fBspa_config_path\fR (charp)
323 Default value: \fB/etc/zfs/zpool.cache\fR.
329 \fBspa_asize_inflation\fR (int)
332 Multiplication factor used to estimate actual disk consumption from the
333 size of data being written. The default value is a worst case estimate,
334 but lower values may be valid for a given pool depending on its
335 configuration. Pool administrators who understand the factors involved
336 may wish to specify a more realistic inflation factor, particularly if
337 they operate close to quota or capacity limits.
339 Default value: \fB24\fR.
345 \fBspa_load_verify_data\fR (int)
348 Whether to traverse data blocks during an "extreme rewind" (\fB-X\fR)
349 import. Use 0 to disable and 1 to enable.
351 An extreme rewind import normally performs a full traversal of all
352 blocks in the pool for verification. If this parameter is set to 0,
353 the traversal skips non-metadata blocks. It can be toggled once the
354 import has started to stop or start the traversal of non-metadata blocks.
356 Default value: \fB1\fR.
362 \fBspa_load_verify_metadata\fR (int)
365 Whether to traverse blocks during an "extreme rewind" (\fB-X\fR)
366 pool import. Use 0 to disable and 1 to enable.
368 An extreme rewind import normally performs a full traversal of all
369 blocks in the pool for verification. If this parameter is set to 0,
370 the traversal is not performed. It can be toggled once the import has
371 started to stop or start the traversal.
373 Default value: \fB1\fR.
379 \fBspa_load_verify_maxinflight\fR (int)
382 Maximum concurrent I/Os during the traversal performed during an "extreme
383 rewind" (\fB-X\fR) pool import.
385 Default value: \fB10000\fR.
391 \fBspa_slop_shift\fR (int)
394 Normally, we don't allow the last 3.2% (1/(2^spa_slop_shift)) of space
395 in the pool to be consumed. This ensures that we don't run the pool
396 completely out of space, due to unaccounted changes (e.g. to the MOS).
397 It also limits the worst-case time to allocate space. If we have
398 less than this amount of free space, most ZPL operations (e.g. write,
399 create) will return ENOSPC.
401 Default value: \fB5\fR.
407 \fBzfetch_array_rd_sz\fR (ulong)
410 If prefetching is enabled, disable prefetching for reads larger than this size.
412 Default value: \fB1,048,576\fR.
418 \fBzfetch_max_distance\fR (uint)
421 Max bytes to prefetch per stream (default 8MB).
423 Default value: \fB8,388,608\fR.
429 \fBzfetch_max_streams\fR (uint)
432 Max number of streams per zfetch (prefetch streams per file).
434 Default value: \fB8\fR.
440 \fBzfetch_min_sec_reap\fR (uint)
443 Min time before an active prefetch stream can be reclaimed
445 Default value: \fB2\fR.
451 \fBzfs_arc_dnode_limit\fR (ulong)
454 When the number of bytes consumed by dnodes in the ARC exceeds this number of
455 bytes, try to unpin some of it in response to demand for non-metadata. This
456 value acts as a ceiling to the amount of dnode metadata, and defaults to 0 which
457 indicates that a percent which is based on \fBzfs_arc_dnode_limit_percent\fR of
458 the ARC meta buffers that may be used for dnodes.
460 See also \fBzfs_arc_meta_prune\fR which serves a similar purpose but is used
461 when the amount of metadata in the ARC exceeds \fBzfs_arc_meta_limit\fR rather
462 than in response to overall demand for non-metadata.
465 Default value: \fB0\fR.
471 \fBzfs_arc_dnode_limit_percent\fR (ulong)
474 Percentage that can be consumed by dnodes of ARC meta buffers.
476 See also \fBzfs_arc_dnode_limit\fR which serves a similar purpose but has a
477 higher priority if set to nonzero value.
479 Default value: \fB10\fR%.
485 \fBzfs_arc_dnode_reduce_percent\fR (ulong)
488 Percentage of ARC dnodes to try to scan in response to demand for non-metadata
489 when the number of bytes consumed by dnodes exceeds \fBzfs_arc_dnode_limit\fR.
492 Default value: \fB10\fR% of the number of dnodes in the ARC.
498 \fBzfs_arc_average_blocksize\fR (int)
501 The ARC's buffer hash table is sized based on the assumption of an average
502 block size of \fBzfs_arc_average_blocksize\fR (default 8K). This works out
503 to roughly 1MB of hash table per 1GB of physical memory with 8-byte pointers.
504 For configurations with a known larger average block size this value can be
505 increased to reduce the memory footprint.
508 Default value: \fB8192\fR.
514 \fBzfs_arc_evict_batch_limit\fR (int)
517 Number ARC headers to evict per sub-list before proceeding to another sub-list.
518 This batch-style operation prevents entire sub-lists from being evicted at once
519 but comes at a cost of additional unlocking and locking.
521 Default value: \fB10\fR.
527 \fBzfs_arc_grow_retry\fR (int)
530 If set to a non zero value, it will replace the arc_grow_retry value with this value.
531 The arc_grow_retry value (default 5) is the number of seconds the ARC will wait before
532 trying to resume growth after a memory pressure event.
534 Default value: \fB0\fR.
540 \fBzfs_arc_lotsfree_percent\fR (int)
543 Throttle I/O when free system memory drops below this percentage of total
544 system memory. Setting this value to 0 will disable the throttle.
546 Default value: \fB10\fR%.
552 \fBzfs_arc_max\fR (ulong)
555 Max arc size of ARC in bytes. If set to 0 then it will consume 1/2 of system
556 RAM. This value must be at least 67108864 (64 megabytes).
558 This value can be changed dynamically with some caveats. It cannot be set back
559 to 0 while running and reducing it below the current ARC size will not cause
560 the ARC to shrink without memory pressure to induce shrinking.
562 Default value: \fB0\fR.
568 \fBzfs_arc_meta_adjust_restarts\fR (ulong)
571 The number of restart passes to make while scanning the ARC attempting
572 the free buffers in order to stay below the \fBzfs_arc_meta_limit\fR.
573 This value should not need to be tuned but is available to facilitate
574 performance analysis.
576 Default value: \fB4096\fR.
582 \fBzfs_arc_meta_limit\fR (ulong)
585 The maximum allowed size in bytes that meta data buffers are allowed to
586 consume in the ARC. When this limit is reached meta data buffers will
587 be reclaimed even if the overall arc_c_max has not been reached. This
588 value defaults to 0 which indicates that a percent which is based on
589 \fBzfs_arc_meta_limit_percent\fR of the ARC may be used for meta data.
591 This value my be changed dynamically except that it cannot be set back to 0
592 for a specific percent of the ARC; it must be set to an explicit value.
594 Default value: \fB0\fR.
600 \fBzfs_arc_meta_limit_percent\fR (ulong)
603 Percentage of ARC buffers that can be used for meta data.
605 See also \fBzfs_arc_meta_limit\fR which serves a similar purpose but has a
606 higher priority if set to nonzero value.
609 Default value: \fB75\fR%.
615 \fBzfs_arc_meta_min\fR (ulong)
618 The minimum allowed size in bytes that meta data buffers may consume in
619 the ARC. This value defaults to 0 which disables a floor on the amount
620 of the ARC devoted meta data.
622 Default value: \fB0\fR.
628 \fBzfs_arc_meta_prune\fR (int)
631 The number of dentries and inodes to be scanned looking for entries
632 which can be dropped. This may be required when the ARC reaches the
633 \fBzfs_arc_meta_limit\fR because dentries and inodes can pin buffers
634 in the ARC. Increasing this value will cause to dentry and inode caches
635 to be pruned more aggressively. Setting this value to 0 will disable
636 pruning the inode and dentry caches.
638 Default value: \fB10,000\fR.
644 \fBzfs_arc_meta_strategy\fR (int)
647 Define the strategy for ARC meta data buffer eviction (meta reclaim strategy).
648 A value of 0 (META_ONLY) will evict only the ARC meta data buffers.
649 A value of 1 (BALANCED) indicates that additional data buffers may be evicted if
650 that is required to in order to evict the required number of meta data buffers.
652 Default value: \fB1\fR.
658 \fBzfs_arc_min\fR (ulong)
661 Min arc size of ARC in bytes. If set to 0 then arc_c_min will default to
662 consuming the larger of 32M or 1/32 of total system memory.
664 Default value: \fB0\fR.
670 \fBzfs_arc_min_prefetch_ms\fR (int)
673 Minimum time prefetched blocks are locked in the ARC, specified in ms.
674 A value of \fB0\fR will default to 1000 ms.
676 Default value: \fB0\fR.
682 \fBzfs_arc_min_prescient_prefetch_ms\fR (int)
685 Minimum time "prescient prefetched" blocks are locked in the ARC, specified
686 in ms. These blocks are meant to be prefetched fairly aggresively ahead of
687 the code that may use them. A value of \fB0\fR will default to 6000 ms.
689 Default value: \fB0\fR.
695 \fBzfs_multilist_num_sublists\fR (int)
698 To allow more fine-grained locking, each ARC state contains a series
699 of lists for both data and meta data objects. Locking is performed at
700 the level of these "sub-lists". This parameters controls the number of
701 sub-lists per ARC state, and also applies to other uses of the
702 multilist data structure.
704 Default value: \fB4\fR or the number of online CPUs, whichever is greater
710 \fBzfs_arc_overflow_shift\fR (int)
713 The ARC size is considered to be overflowing if it exceeds the current
714 ARC target size (arc_c) by a threshold determined by this parameter.
715 The threshold is calculated as a fraction of arc_c using the formula
716 "arc_c >> \fBzfs_arc_overflow_shift\fR".
718 The default value of 8 causes the ARC to be considered to be overflowing
719 if it exceeds the target size by 1/256th (0.3%) of the target size.
721 When the ARC is overflowing, new buffer allocations are stalled until
722 the reclaim thread catches up and the overflow condition no longer exists.
724 Default value: \fB8\fR.
731 \fBzfs_arc_p_min_shift\fR (int)
734 If set to a non zero value, this will update arc_p_min_shift (default 4)
736 arc_p_min_shift is used to shift of arc_c for calculating both min and max
739 Default value: \fB0\fR.
745 \fBzfs_arc_p_dampener_disable\fR (int)
748 Disable arc_p adapt dampener
750 Use \fB1\fR for yes (default) and \fB0\fR to disable.
756 \fBzfs_arc_shrink_shift\fR (int)
759 If set to a non zero value, this will update arc_shrink_shift (default 7)
762 Default value: \fB0\fR.
768 \fBzfs_arc_pc_percent\fR (uint)
771 Percent of pagecache to reclaim arc to
773 This tunable allows ZFS arc to play more nicely with the kernel's LRU
774 pagecache. It can guarantee that the arc size won't collapse under scanning
775 pressure on the pagecache, yet still allows arc to be reclaimed down to
776 zfs_arc_min if necessary. This value is specified as percent of pagecache
777 size (as measured by NR_FILE_PAGES) where that percent may exceed 100. This
778 only operates during memory pressure/reclaim.
780 Default value: \fB0\fR% (disabled).
786 \fBzfs_arc_sys_free\fR (ulong)
789 The target number of bytes the ARC should leave as free memory on the system.
790 Defaults to the larger of 1/64 of physical memory or 512K. Setting this
791 option to a non-zero value will override the default.
793 Default value: \fB0\fR.
799 \fBzfs_autoimport_disable\fR (int)
802 Disable pool import at module load by ignoring the cache file (typically \fB/etc/zfs/zpool.cache\fR).
804 Use \fB1\fR for yes (default) and \fB0\fR for no.
810 \fBzfs_checksums_per_second\fR (int)
813 Rate limit checksum events to this many per second. Note that this should
814 not be set below the zed thresholds (currently 10 checksums over 10 sec)
815 or else zed may not trigger any action.
823 \fBzfs_commit_timeout_pct\fR (int)
826 This controls the amount of time that a ZIL block (lwb) will remain "open"
827 when it isn't "full", and it has a thread waiting for it to be committed to
828 stable storage. The timeout is scaled based on a percentage of the last lwb
829 latency to avoid significantly impacting the latency of each individual
830 transaction record (itx).
832 Default value: \fB5\fR%.
838 \fBzfs_dbgmsg_enable\fR (int)
841 Internally ZFS keeps a small log to facilitate debugging. By default the log
842 is disabled, to enable it set this option to 1. The contents of the log can
843 be accessed by reading the /proc/spl/kstat/zfs/dbgmsg file. Writing 0 to
844 this proc file clears the log.
846 Default value: \fB0\fR.
852 \fBzfs_dbgmsg_maxsize\fR (int)
855 The maximum size in bytes of the internal ZFS debug log.
857 Default value: \fB4M\fR.
863 \fBzfs_dbuf_state_index\fR (int)
866 This feature is currently unused. It is normally used for controlling what
867 reporting is available under /proc/spl/kstat/zfs.
869 Default value: \fB0\fR.
875 \fBzfs_deadman_enabled\fR (int)
878 When a pool sync operation takes longer than \fBzfs_deadman_synctime_ms\fR
879 milliseconds, or when an individual I/O takes longer than
880 \fBzfs_deadman_ziotime_ms\fR milliseconds, then the operation is considered to
881 be "hung". If \fBzfs_deadman_enabled\fR is set then the deadman behavior is
882 invoked as described by the \fBzfs_deadman_failmode\fR module option.
883 By default the deadman is enabled and configured to \fBwait\fR which results
884 in "hung" I/Os only being logged. The deadman is automatically disabled
885 when a pool gets suspended.
887 Default value: \fB1\fR.
893 \fBzfs_deadman_failmode\fR (charp)
896 Controls the failure behavior when the deadman detects a "hung" I/O. Valid
897 values are \fBwait\fR, \fBcontinue\fR, and \fBpanic\fR.
899 \fBwait\fR - Wait for a "hung" I/O to complete. For each "hung" I/O a
900 "deadman" event will be posted describing that I/O.
902 \fBcontinue\fR - Attempt to recover from a "hung" I/O by re-dispatching it
903 to the I/O pipeline if possible.
905 \fBpanic\fR - Panic the system. This can be used to facilitate an automatic
906 fail-over to a properly configured fail-over partner.
908 Default value: \fBwait\fR.
914 \fBzfs_deadman_checktime_ms\fR (int)
917 Check time in milliseconds. This defines the frequency at which we check
918 for hung I/O and potentially invoke the \fBzfs_deadman_failmode\fR behavior.
920 Default value: \fB60,000\fR.
926 \fBzfs_deadman_synctime_ms\fR (ulong)
929 Interval in milliseconds after which the deadman is triggered and also
930 the interval after which a pool sync operation is considered to be "hung".
931 Once this limit is exceeded the deadman will be invoked every
932 \fBzfs_deadman_checktime_ms\fR milliseconds until the pool sync completes.
934 Default value: \fB600,000\fR.
940 \fBzfs_deadman_ziotime_ms\fR (ulong)
943 Interval in milliseconds after which the deadman is triggered and an
944 individual IO operation is considered to be "hung". As long as the I/O
945 remains "hung" the deadman will be invoked every \fBzfs_deadman_checktime_ms\fR
946 milliseconds until the I/O completes.
948 Default value: \fB300,000\fR.
954 \fBzfs_dedup_prefetch\fR (int)
957 Enable prefetching dedup-ed blks
959 Use \fB1\fR for yes and \fB0\fR to disable (default).
965 \fBzfs_delay_min_dirty_percent\fR (int)
968 Start to delay each transaction once there is this amount of dirty data,
969 expressed as a percentage of \fBzfs_dirty_data_max\fR.
970 This value should be >= zfs_vdev_async_write_active_max_dirty_percent.
971 See the section "ZFS TRANSACTION DELAY".
973 Default value: \fB60\fR%.
979 \fBzfs_delay_scale\fR (int)
982 This controls how quickly the transaction delay approaches infinity.
983 Larger values cause longer delays for a given amount of dirty data.
985 For the smoothest delay, this value should be about 1 billion divided
986 by the maximum number of operations per second. This will smoothly
987 handle between 10x and 1/10th this number.
989 See the section "ZFS TRANSACTION DELAY".
991 Note: \fBzfs_delay_scale\fR * \fBzfs_dirty_data_max\fR must be < 2^64.
993 Default value: \fB500,000\fR.
999 \fBzfs_delays_per_second\fR (int)
1002 Rate limit IO delay events to this many per second.
1010 \fBzfs_delete_blocks\fR (ulong)
1013 This is the used to define a large file for the purposes of delete. Files
1014 containing more than \fBzfs_delete_blocks\fR will be deleted asynchronously
1015 while smaller files are deleted synchronously. Decreasing this value will
1016 reduce the time spent in an unlink(2) system call at the expense of a longer
1017 delay before the freed space is available.
1019 Default value: \fB20,480\fR.
1025 \fBzfs_dirty_data_max\fR (int)
1028 Determines the dirty space limit in bytes. Once this limit is exceeded, new
1029 writes are halted until space frees up. This parameter takes precedence
1030 over \fBzfs_dirty_data_max_percent\fR.
1031 See the section "ZFS TRANSACTION DELAY".
1033 Default value: \fB10\fR% of physical RAM, capped at \fBzfs_dirty_data_max_max\fR.
1039 \fBzfs_dirty_data_max_max\fR (int)
1042 Maximum allowable value of \fBzfs_dirty_data_max\fR, expressed in bytes.
1043 This limit is only enforced at module load time, and will be ignored if
1044 \fBzfs_dirty_data_max\fR is later changed. This parameter takes
1045 precedence over \fBzfs_dirty_data_max_max_percent\fR. See the section
1046 "ZFS TRANSACTION DELAY".
1048 Default value: \fB25\fR% of physical RAM.
1054 \fBzfs_dirty_data_max_max_percent\fR (int)
1057 Maximum allowable value of \fBzfs_dirty_data_max\fR, expressed as a
1058 percentage of physical RAM. This limit is only enforced at module load
1059 time, and will be ignored if \fBzfs_dirty_data_max\fR is later changed.
1060 The parameter \fBzfs_dirty_data_max_max\fR takes precedence over this
1061 one. See the section "ZFS TRANSACTION DELAY".
1063 Default value: \fB25\fR%.
1069 \fBzfs_dirty_data_max_percent\fR (int)
1072 Determines the dirty space limit, expressed as a percentage of all
1073 memory. Once this limit is exceeded, new writes are halted until space frees
1074 up. The parameter \fBzfs_dirty_data_max\fR takes precedence over this
1075 one. See the section "ZFS TRANSACTION DELAY".
1077 Default value: \fB10\fR%, subject to \fBzfs_dirty_data_max_max\fR.
1083 \fBzfs_dirty_data_sync\fR (int)
1086 Start syncing out a transaction group if there is at least this much dirty data.
1088 Default value: \fB67,108,864\fR.
1094 \fBzfs_fletcher_4_impl\fR (string)
1097 Select a fletcher 4 implementation.
1099 Supported selectors are: \fBfastest\fR, \fBscalar\fR, \fBsse2\fR, \fBssse3\fR,
1100 \fBavx2\fR, \fBavx512f\fR, and \fBaarch64_neon\fR.
1101 All of the selectors except \fBfastest\fR and \fBscalar\fR require instruction
1102 set extensions to be available and will only appear if ZFS detects that they are
1103 present at runtime. If multiple implementations of fletcher 4 are available,
1104 the \fBfastest\fR will be chosen using a micro benchmark. Selecting \fBscalar\fR
1105 results in the original, CPU based calculation, being used. Selecting any option
1106 other than \fBfastest\fR and \fBscalar\fR results in vector instructions from
1107 the respective CPU instruction set being used.
1109 Default value: \fBfastest\fR.
1115 \fBzfs_free_bpobj_enabled\fR (int)
1118 Enable/disable the processing of the free_bpobj object.
1120 Default value: \fB1\fR.
1126 \fBzfs_free_max_blocks\fR (ulong)
1129 Maximum number of blocks freed in a single txg.
1131 Default value: \fB100,000\fR.
1137 \fBzfs_vdev_async_read_max_active\fR (int)
1140 Maximum asynchronous read I/Os active to each device.
1141 See the section "ZFS I/O SCHEDULER".
1143 Default value: \fB3\fR.
1149 \fBzfs_vdev_async_read_min_active\fR (int)
1152 Minimum asynchronous read I/Os active to each device.
1153 See the section "ZFS I/O SCHEDULER".
1155 Default value: \fB1\fR.
1161 \fBzfs_vdev_async_write_active_max_dirty_percent\fR (int)
1164 When the pool has more than
1165 \fBzfs_vdev_async_write_active_max_dirty_percent\fR dirty data, use
1166 \fBzfs_vdev_async_write_max_active\fR to limit active async writes. If
1167 the dirty data is between min and max, the active I/O limit is linearly
1168 interpolated. See the section "ZFS I/O SCHEDULER".
1170 Default value: \fB60\fR%.
1176 \fBzfs_vdev_async_write_active_min_dirty_percent\fR (int)
1179 When the pool has less than
1180 \fBzfs_vdev_async_write_active_min_dirty_percent\fR dirty data, use
1181 \fBzfs_vdev_async_write_min_active\fR to limit active async writes. If
1182 the dirty data is between min and max, the active I/O limit is linearly
1183 interpolated. See the section "ZFS I/O SCHEDULER".
1185 Default value: \fB30\fR%.
1191 \fBzfs_vdev_async_write_max_active\fR (int)
1194 Maximum asynchronous write I/Os active to each device.
1195 See the section "ZFS I/O SCHEDULER".
1197 Default value: \fB10\fR.
1203 \fBzfs_vdev_async_write_min_active\fR (int)
1206 Minimum asynchronous write I/Os active to each device.
1207 See the section "ZFS I/O SCHEDULER".
1209 Lower values are associated with better latency on rotational media but poorer
1210 resilver performance. The default value of 2 was chosen as a compromise. A
1211 value of 3 has been shown to improve resilver performance further at a cost of
1212 further increasing latency.
1214 Default value: \fB2\fR.
1220 \fBzfs_vdev_max_active\fR (int)
1223 The maximum number of I/Os active to each device. Ideally, this will be >=
1224 the sum of each queue's max_active. It must be at least the sum of each
1225 queue's min_active. See the section "ZFS I/O SCHEDULER".
1227 Default value: \fB1,000\fR.
1233 \fBzfs_vdev_scrub_max_active\fR (int)
1236 Maximum scrub I/Os active to each device.
1237 See the section "ZFS I/O SCHEDULER".
1239 Default value: \fB2\fR.
1245 \fBzfs_vdev_scrub_min_active\fR (int)
1248 Minimum scrub I/Os active to each device.
1249 See the section "ZFS I/O SCHEDULER".
1251 Default value: \fB1\fR.
1257 \fBzfs_vdev_sync_read_max_active\fR (int)
1260 Maximum synchronous read I/Os active to each device.
1261 See the section "ZFS I/O SCHEDULER".
1263 Default value: \fB10\fR.
1269 \fBzfs_vdev_sync_read_min_active\fR (int)
1272 Minimum synchronous read I/Os active to each device.
1273 See the section "ZFS I/O SCHEDULER".
1275 Default value: \fB10\fR.
1281 \fBzfs_vdev_sync_write_max_active\fR (int)
1284 Maximum synchronous write I/Os active to each device.
1285 See the section "ZFS I/O SCHEDULER".
1287 Default value: \fB10\fR.
1293 \fBzfs_vdev_sync_write_min_active\fR (int)
1296 Minimum synchronous write I/Os active to each device.
1297 See the section "ZFS I/O SCHEDULER".
1299 Default value: \fB10\fR.
1305 \fBzfs_vdev_queue_depth_pct\fR (int)
1308 Maximum number of queued allocations per top-level vdev expressed as
1309 a percentage of \fBzfs_vdev_async_write_max_active\fR which allows the
1310 system to detect devices that are more capable of handling allocations
1311 and to allocate more blocks to those devices. It allows for dynamic
1312 allocation distribution when devices are imbalanced as fuller devices
1313 will tend to be slower than empty devices.
1315 See also \fBzio_dva_throttle_enabled\fR.
1317 Default value: \fB1000\fR%.
1323 \fBzfs_expire_snapshot\fR (int)
1326 Seconds to expire .zfs/snapshot
1328 Default value: \fB300\fR.
1334 \fBzfs_admin_snapshot\fR (int)
1337 Allow the creation, removal, or renaming of entries in the .zfs/snapshot
1338 directory to cause the creation, destruction, or renaming of snapshots.
1339 When enabled this functionality works both locally and over NFS exports
1340 which have the 'no_root_squash' option set. This functionality is disabled
1343 Use \fB1\fR for yes and \fB0\fR for no (default).
1349 \fBzfs_flags\fR (int)
1352 Set additional debugging flags. The following flags may be bitwise-or'd
1364 Enable dprintf entries in the debug log.
1366 2 ZFS_DEBUG_DBUF_VERIFY *
1367 Enable extra dbuf verifications.
1369 4 ZFS_DEBUG_DNODE_VERIFY *
1370 Enable extra dnode verifications.
1372 8 ZFS_DEBUG_SNAPNAMES
1373 Enable snapshot name verification.
1376 Check for illegally modified ARC buffers.
1379 Enable spa_dbgmsg entries in the debug log.
1381 64 ZFS_DEBUG_ZIO_FREE
1382 Enable verification of block frees.
1384 128 ZFS_DEBUG_HISTOGRAM_VERIFY
1385 Enable extra spacemap histogram verifications.
1387 256 ZFS_DEBUG_METASLAB_VERIFY
1388 Verify space accounting on disk matches in-core range_trees.
1390 512 ZFS_DEBUG_SET_ERROR
1391 Enable SET_ERROR and dprintf entries in the debug log.
1394 * Requires debug build.
1396 Default value: \fB0\fR.
1402 \fBzfs_free_leak_on_eio\fR (int)
1405 If destroy encounters an EIO while reading metadata (e.g. indirect
1406 blocks), space referenced by the missing metadata can not be freed.
1407 Normally this causes the background destroy to become "stalled", as
1408 it is unable to make forward progress. While in this stalled state,
1409 all remaining space to free from the error-encountering filesystem is
1410 "temporarily leaked". Set this flag to cause it to ignore the EIO,
1411 permanently leak the space from indirect blocks that can not be read,
1412 and continue to free everything else that it can.
1414 The default, "stalling" behavior is useful if the storage partially
1415 fails (i.e. some but not all i/os fail), and then later recovers. In
1416 this case, we will be able to continue pool operations while it is
1417 partially failed, and when it recovers, we can continue to free the
1418 space, with no leaks. However, note that this case is actually
1421 Typically pools either (a) fail completely (but perhaps temporarily,
1422 e.g. a top-level vdev going offline), or (b) have localized,
1423 permanent errors (e.g. disk returns the wrong data due to bit flip or
1424 firmware bug). In case (a), this setting does not matter because the
1425 pool will be suspended and the sync thread will not be able to make
1426 forward progress regardless. In case (b), because the error is
1427 permanent, the best we can do is leak the minimum amount of space,
1428 which is what setting this flag will do. Therefore, it is reasonable
1429 for this flag to normally be set, but we chose the more conservative
1430 approach of not setting it, so that there is no possibility of
1431 leaking space in the "partial temporary" failure case.
1433 Default value: \fB0\fR.
1439 \fBzfs_free_min_time_ms\fR (int)
1442 During a \fBzfs destroy\fR operation using \fBfeature@async_destroy\fR a minimum
1443 of this much time will be spent working on freeing blocks per txg.
1445 Default value: \fB1,000\fR.
1451 \fBzfs_immediate_write_sz\fR (long)
1454 Largest data block to write to zil. Larger blocks will be treated as if the
1455 dataset being written to had the property setting \fBlogbias=throughput\fR.
1457 Default value: \fB32,768\fR.
1463 \fBzfs_max_recordsize\fR (int)
1466 We currently support block sizes from 512 bytes to 16MB. The benefits of
1467 larger blocks, and thus larger IO, need to be weighed against the cost of
1468 COWing a giant block to modify one byte. Additionally, very large blocks
1469 can have an impact on i/o latency, and also potentially on the memory
1470 allocator. Therefore, we do not allow the recordsize to be set larger than
1471 zfs_max_recordsize (default 1MB). Larger blocks can be created by changing
1472 this tunable, and pools with larger blocks can always be imported and used,
1473 regardless of this setting.
1475 Default value: \fB1,048,576\fR.
1481 \fBzfs_metaslab_fragmentation_threshold\fR (int)
1484 Allow metaslabs to keep their active state as long as their fragmentation
1485 percentage is less than or equal to this value. An active metaslab that
1486 exceeds this threshold will no longer keep its active status allowing
1487 better metaslabs to be selected.
1489 Default value: \fB70\fR.
1495 \fBzfs_mg_fragmentation_threshold\fR (int)
1498 Metaslab groups are considered eligible for allocations if their
1499 fragmentation metric (measured as a percentage) is less than or equal to
1500 this value. If a metaslab group exceeds this threshold then it will be
1501 skipped unless all metaslab groups within the metaslab class have also
1502 crossed this threshold.
1504 Default value: \fB85\fR.
1510 \fBzfs_mg_noalloc_threshold\fR (int)
1513 Defines a threshold at which metaslab groups should be eligible for
1514 allocations. The value is expressed as a percentage of free space
1515 beyond which a metaslab group is always eligible for allocations.
1516 If a metaslab group's free space is less than or equal to the
1517 threshold, the allocator will avoid allocating to that group
1518 unless all groups in the pool have reached the threshold. Once all
1519 groups have reached the threshold, all groups are allowed to accept
1520 allocations. The default value of 0 disables the feature and causes
1521 all metaslab groups to be eligible for allocations.
1523 This parameter allows one to deal with pools having heavily imbalanced
1524 vdevs such as would be the case when a new vdev has been added.
1525 Setting the threshold to a non-zero percentage will stop allocations
1526 from being made to vdevs that aren't filled to the specified percentage
1527 and allow lesser filled vdevs to acquire more allocations than they
1528 otherwise would under the old \fBzfs_mg_alloc_failures\fR facility.
1530 Default value: \fB0\fR.
1536 \fBzfs_multihost_history\fR (int)
1539 Historical statistics for the last N multihost updates will be available in
1540 \fB/proc/spl/kstat/zfs/<pool>/multihost\fR
1542 Default value: \fB0\fR.
1548 \fBzfs_multihost_interval\fR (ulong)
1551 Used to control the frequency of multihost writes which are performed when the
1552 \fBmultihost\fR pool property is on. This is one factor used to determine
1553 the length of the activity check during import.
1555 The multihost write period is \fBzfs_multihost_interval / leaf-vdevs\fR milliseconds.
1556 This means that on average a multihost write will be issued for each leaf vdev every
1557 \fBzfs_multihost_interval\fR milliseconds. In practice, the observed period can
1558 vary with the I/O load and this observed value is the delay which is stored in
1561 On import the activity check waits a minimum amount of time determined by
1562 \fBzfs_multihost_interval * zfs_multihost_import_intervals\fR. The activity
1563 check time may be further extended if the value of mmp delay found in the best
1564 uberblock indicates actual multihost updates happened at longer intervals than
1565 \fBzfs_multihost_interval\fR. A minimum value of \fB100ms\fR is enforced.
1567 Default value: \fB1000\fR.
1573 \fBzfs_multihost_import_intervals\fR (uint)
1576 Used to control the duration of the activity test on import. Smaller values of
1577 \fBzfs_multihost_import_intervals\fR will reduce the import time but increase
1578 the risk of failing to detect an active pool. The total activity check time is
1579 never allowed to drop below one second. A value of 0 is ignored and treated as
1582 Default value: \fB10\fR.
1588 \fBzfs_multihost_fail_intervals\fR (uint)
1591 Controls the behavior of the pool when multihost write failures are detected.
1593 When \fBzfs_multihost_fail_intervals = 0\fR then multihost write failures are ignored.
1594 The failures will still be reported to the ZED which depending on its
1595 configuration may take action such as suspending the pool or offlining a device.
1597 When \fBzfs_multihost_fail_intervals > 0\fR then sequential multihost write failures
1598 will cause the pool to be suspended. This occurs when
1599 \fBzfs_multihost_fail_intervals * zfs_multihost_interval\fR milliseconds have
1600 passed since the last successful multihost write. This guarantees the activity test
1601 will see multihost writes if the pool is imported.
1603 Default value: \fB5\fR.
1609 \fBzfs_no_scrub_io\fR (int)
1612 Set for no scrub I/O. This results in scrubs not actually scrubbing data and
1613 simply doing a metadata crawl of the pool instead.
1615 Use \fB1\fR for yes and \fB0\fR for no (default).
1621 \fBzfs_no_scrub_prefetch\fR (int)
1624 Set to disable block prefetching for scrubs.
1626 Use \fB1\fR for yes and \fB0\fR for no (default).
1632 \fBzfs_nocacheflush\fR (int)
1635 Disable cache flush operations on disks when writing. Beware, this may cause
1636 corruption if disks re-order writes.
1638 Use \fB1\fR for yes and \fB0\fR for no (default).
1644 \fBzfs_nopwrite_enabled\fR (int)
1649 Use \fB1\fR for yes (default) and \fB0\fR to disable.
1655 \fBzfs_dmu_offset_next_sync\fR (int)
1658 Enable forcing txg sync to find holes. When enabled forces ZFS to act
1659 like prior versions when SEEK_HOLE or SEEK_DATA flags are used, which
1660 when a dnode is dirty causes txg's to be synced so that this data can be
1663 Use \fB1\fR for yes and \fB0\fR to disable (default).
1669 \fBzfs_pd_bytes_max\fR (int)
1672 The number of bytes which should be prefetched during a pool traversal
1673 (eg: \fBzfs send\fR or other data crawling operations)
1675 Default value: \fB52,428,800\fR.
1681 \fBzfs_per_txg_dirty_frees_percent \fR (ulong)
1684 Tunable to control percentage of dirtied blocks from frees in one TXG.
1685 After this threshold is crossed, additional dirty blocks from frees
1686 wait until the next TXG.
1687 A value of zero will disable this throttle.
1689 Default value: \fB30\fR and \fB0\fR to disable.
1697 \fBzfs_prefetch_disable\fR (int)
1700 This tunable disables predictive prefetch. Note that it leaves "prescient"
1701 prefetch (e.g. prefetch for zfs send) intact. Unlike predictive prefetch,
1702 prescient prefetch never issues i/os that end up not being needed, so it
1703 can't hurt performance.
1705 Use \fB1\fR for yes and \fB0\fR for no (default).
1711 \fBzfs_read_chunk_size\fR (long)
1714 Bytes to read per chunk
1716 Default value: \fB1,048,576\fR.
1722 \fBzfs_read_history\fR (int)
1725 Historical statistics for the last N reads will be available in
1726 \fB/proc/spl/kstat/zfs/<pool>/reads\fR
1728 Default value: \fB0\fR (no data is kept).
1734 \fBzfs_read_history_hits\fR (int)
1737 Include cache hits in read history
1739 Use \fB1\fR for yes and \fB0\fR for no (default).
1745 \fBzfs_recover\fR (int)
1748 Set to attempt to recover from fatal errors. This should only be used as a
1749 last resort, as it typically results in leaked space, or worse.
1751 Use \fB1\fR for yes and \fB0\fR for no (default).
1757 \fBzfs_resilver_min_time_ms\fR (int)
1760 Resilvers are processed by the sync thread. While resilvering it will spend
1761 at least this much time working on a resilver between txg flushes.
1763 Default value: \fB3,000\fR.
1769 \fBzfs_scan_ignore_errors\fR (int)
1772 If set to a nonzero value, remove the DTL (dirty time list) upon
1773 completion of a pool scan (scrub) even if there were unrepairable
1774 errors. It is intended to be used during pool repair or recovery to
1775 stop resilvering when the pool is next imported.
1777 Default value: \fB0\fR.
1783 \fBzfs_scrub_min_time_ms\fR (int)
1786 Scrubs are processed by the sync thread. While scrubbing it will spend
1787 at least this much time working on a scrub between txg flushes.
1789 Default value: \fB1,000\fR.
1795 \fBzfs_scan_checkpoint_intval\fR (int)
1798 To preserve progress across reboots the sequential scan algorithm periodically
1799 needs to stop metadata scanning and issue all the verifications I/Os to disk.
1800 The frequency of this flushing is determined by the
1801 \fBfBzfs_scan_checkpoint_intval\fR tunable.
1803 Default value: \fB7200\fR seconds (every 2 hours).
1809 \fBzfs_scan_fill_weight\fR (int)
1812 This tunable affects how scrub and resilver I/O segments are ordered. A higher
1813 number indicates that we care more about how filled in a segment is, while a
1814 lower number indicates we care more about the size of the extent without
1815 considering the gaps within a segment. This value is only tunable upon module
1816 insertion. Changing the value afterwards will have no affect on scrub or
1817 resilver performance.
1819 Default value: \fB3\fR.
1825 \fBzfs_scan_issue_strategy\fR (int)
1828 Determines the order that data will be verified while scrubbing or resilvering.
1829 If set to \fB1\fR, data will be verified as sequentially as possible, given the
1830 amount of memory reserved for scrubbing (see \fBzfs_scan_mem_lim_fact\fR). This
1831 may improve scrub performance if the pool's data is very fragmented. If set to
1832 \fB2\fR, the largest mostly-contiguous chunk of found data will be verified
1833 first. By deferring scrubbing of small segments, we may later find adjacent data
1834 to coalesce and increase the segment size. If set to \fB0\fR, zfs will use
1835 strategy \fB1\fR during normal verification and strategy \fB2\fR while taking a
1838 Default value: \fB0\fR.
1844 \fBzfs_scan_legacy\fR (int)
1847 A value of 0 indicates that scrubs and resilvers will gather metadata in
1848 memory before issuing sequential I/O. A value of 1 indicates that the legacy
1849 algorithm will be used where I/O is initiated as soon as it is discovered.
1850 Changing this value to 0 will not affect scrubs or resilvers that are already
1853 Default value: \fB0\fR.
1859 \fBzfs_scan_max_ext_gap\fR (int)
1862 Indicates the largest gap in bytes between scrub / resilver I/Os that will still
1863 be considered sequential for sorting purposes. Changing this value will not
1864 affect scrubs or resilvers that are already in progress.
1866 Default value: \fB2097152 (2 MB)\fR.
1872 \fBzfs_scan_mem_lim_fact\fR (int)
1875 Maximum fraction of RAM used for I/O sorting by sequential scan algorithm.
1876 This tunable determines the hard limit for I/O sorting memory usage.
1877 When the hard limit is reached we stop scanning metadata and start issuing
1878 data verification I/O. This is done until we get below the soft limit.
1880 Default value: \fB20\fR which is 5% of RAM (1/20).
1886 \fBzfs_scan_mem_lim_soft_fact\fR (int)
1889 The fraction of the hard limit used to determined the soft limit for I/O sorting
1890 by the sequential scan algorithm. When we cross this limit from bellow no action
1891 is taken. When we cross this limit from above it is because we are issuing
1892 verification I/O. In this case (unless the metadata scan is done) we stop
1893 issuing verification I/O and start scanning metadata again until we get to the
1896 Default value: \fB20\fR which is 5% of the hard limit (1/20).
1902 \fBzfs_scan_vdev_limit\fR (int)
1905 Maximum amount of data that can be concurrently issued at once for scrubs and
1906 resilvers per leaf device, given in bytes.
1908 Default value: \fB41943040\fR.
1914 \fBzfs_send_corrupt_data\fR (int)
1917 Allow sending of corrupt data (ignore read/checksum errors when sending data)
1919 Use \fB1\fR for yes and \fB0\fR for no (default).
1925 \fBzfs_sync_pass_deferred_free\fR (int)
1928 Flushing of data to disk is done in passes. Defer frees starting in this pass
1930 Default value: \fB2\fR.
1936 \fBzfs_sync_pass_dont_compress\fR (int)
1939 Don't compress starting in this pass
1941 Default value: \fB5\fR.
1947 \fBzfs_sync_pass_rewrite\fR (int)
1950 Rewrite new block pointers starting in this pass
1952 Default value: \fB2\fR.
1958 \fBzfs_sync_taskq_batch_pct\fR (int)
1961 This controls the number of threads used by the dp_sync_taskq. The default
1962 value of 75% will create a maximum of one thread per cpu.
1964 Default value: \fB75\fR%.
1970 \fBzfs_txg_history\fR (int)
1973 Historical statistics for the last N txgs will be available in
1974 \fB/proc/spl/kstat/zfs/<pool>/txgs\fR
1976 Default value: \fB0\fR.
1982 \fBzfs_txg_timeout\fR (int)
1985 Flush dirty data to disk at least every N seconds (maximum txg duration)
1987 Default value: \fB5\fR.
1993 \fBzfs_vdev_aggregation_limit\fR (int)
1996 Max vdev I/O aggregation size
1998 Default value: \fB131,072\fR.
2004 \fBzfs_vdev_cache_bshift\fR (int)
2007 Shift size to inflate reads too
2009 Default value: \fB16\fR (effectively 65536).
2015 \fBzfs_vdev_cache_max\fR (int)
2018 Inflate reads smaller than this value to meet the \fBzfs_vdev_cache_bshift\fR
2021 Default value: \fB16384\fR.
2027 \fBzfs_vdev_cache_size\fR (int)
2030 Total size of the per-disk cache in bytes.
2032 Currently this feature is disabled as it has been found to not be helpful
2033 for performance and in some cases harmful.
2035 Default value: \fB0\fR.
2041 \fBzfs_vdev_mirror_rotating_inc\fR (int)
2044 A number by which the balancing algorithm increments the load calculation for
2045 the purpose of selecting the least busy mirror member when an I/O immediately
2046 follows its predecessor on rotational vdevs for the purpose of making decisions
2049 Default value: \fB0\fR.
2055 \fBzfs_vdev_mirror_rotating_seek_inc\fR (int)
2058 A number by which the balancing algorithm increments the load calculation for
2059 the purpose of selecting the least busy mirror member when an I/O lacks
2060 locality as defined by the zfs_vdev_mirror_rotating_seek_offset. I/Os within
2061 this that are not immediately following the previous I/O are incremented by
2064 Default value: \fB5\fR.
2070 \fBzfs_vdev_mirror_rotating_seek_offset\fR (int)
2073 The maximum distance for the last queued I/O in which the balancing algorithm
2074 considers an I/O to have locality.
2075 See the section "ZFS I/O SCHEDULER".
2077 Default value: \fB1048576\fR.
2083 \fBzfs_vdev_mirror_non_rotating_inc\fR (int)
2086 A number by which the balancing algorithm increments the load calculation for
2087 the purpose of selecting the least busy mirror member on non-rotational vdevs
2088 when I/Os do not immediately follow one another.
2090 Default value: \fB0\fR.
2096 \fBzfs_vdev_mirror_non_rotating_seek_inc\fR (int)
2099 A number by which the balancing algorithm increments the load calculation for
2100 the purpose of selecting the least busy mirror member when an I/O lacks
2101 locality as defined by the zfs_vdev_mirror_rotating_seek_offset. I/Os within
2102 this that are not immediately following the previous I/O are incremented by
2105 Default value: \fB1\fR.
2111 \fBzfs_vdev_read_gap_limit\fR (int)
2114 Aggregate read I/O operations if the gap on-disk between them is within this
2117 Default value: \fB32,768\fR.
2123 \fBzfs_vdev_scheduler\fR (charp)
2126 Set the Linux I/O scheduler on whole disk vdevs to this scheduler. Valid options
2127 are noop, cfq, bfq & deadline
2129 Default value: \fBnoop\fR.
2135 \fBzfs_vdev_write_gap_limit\fR (int)
2138 Aggregate write I/O over gap
2140 Default value: \fB4,096\fR.
2146 \fBzfs_vdev_raidz_impl\fR (string)
2149 Parameter for selecting raidz parity implementation to use.
2151 Options marked (always) below may be selected on module load as they are
2152 supported on all systems.
2153 The remaining options may only be set after the module is loaded, as they
2154 are available only if the implementations are compiled in and supported
2155 on the running system.
2157 Once the module is loaded, the content of
2158 /sys/module/zfs/parameters/zfs_vdev_raidz_impl will show available options
2159 with the currently selected one enclosed in [].
2160 Possible options are:
2161 fastest - (always) implementation selected using built-in benchmark
2162 original - (always) original raidz implementation
2163 scalar - (always) scalar raidz implementation
2164 sse2 - implementation using SSE2 instruction set (64bit x86 only)
2165 ssse3 - implementation using SSSE3 instruction set (64bit x86 only)
2166 avx2 - implementation using AVX2 instruction set (64bit x86 only)
2167 avx512f - implementation using AVX512F instruction set (64bit x86 only)
2168 avx512bw - implementation using AVX512F & AVX512BW instruction sets (64bit x86 only)
2169 aarch64_neon - implementation using NEON (Aarch64/64 bit ARMv8 only)
2170 aarch64_neonx2 - implementation using NEON with more unrolling (Aarch64/64 bit ARMv8 only)
2172 Default value: \fBfastest\fR.
2178 \fBzfs_zevent_cols\fR (int)
2181 When zevents are logged to the console use this as the word wrap width.
2183 Default value: \fB80\fR.
2189 \fBzfs_zevent_console\fR (int)
2192 Log events to the console
2194 Use \fB1\fR for yes and \fB0\fR for no (default).
2200 \fBzfs_zevent_len_max\fR (int)
2203 Max event queue length. A value of 0 will result in a calculated value which
2204 increases with the number of CPUs in the system (minimum 64 events). Events
2205 in the queue can be viewed with the \fBzpool events\fR command.
2207 Default value: \fB0\fR.
2213 \fBzfs_zil_clean_taskq_maxalloc\fR (int)
2216 The maximum number of taskq entries that are allowed to be cached. When this
2217 limit is exceeded transaction records (itxs) will be cleaned synchronously.
2219 Default value: \fB1048576\fR.
2225 \fBzfs_zil_clean_taskq_minalloc\fR (int)
2228 The number of taskq entries that are pre-populated when the taskq is first
2229 created and are immediately available for use.
2231 Default value: \fB1024\fR.
2237 \fBzfs_zil_clean_taskq_nthr_pct\fR (int)
2240 This controls the number of threads used by the dp_zil_clean_taskq. The default
2241 value of 100% will create a maximum of one thread per cpu.
2243 Default value: \fB100\fR%.
2249 \fBzil_replay_disable\fR (int)
2252 Disable intent logging replay. Can be disabled for recovery from corrupted
2255 Use \fB1\fR for yes and \fB0\fR for no (default).
2261 \fBzil_slog_bulk\fR (ulong)
2264 Limit SLOG write size per commit executed with synchronous priority.
2265 Any writes above that will be executed with lower (asynchronous) priority
2266 to limit potential SLOG device abuse by single active ZIL writer.
2268 Default value: \fB786,432\fR.
2274 \fBzio_delay_max\fR (int)
2277 A zevent will be logged if a ZIO operation takes more than N milliseconds to
2278 complete. Note that this is only a logging facility, not a timeout on
2281 Default value: \fB30,000\fR.
2287 \fBzio_dva_throttle_enabled\fR (int)
2290 Throttle block allocations in the ZIO pipeline. This allows for
2291 dynamic allocation distribution when devices are imbalanced.
2292 When enabled, the maximum number of pending allocations per top-level vdev
2293 is limited by \fBzfs_vdev_queue_depth_pct\fR.
2295 Default value: \fB1\fR.
2301 \fBzio_requeue_io_start_cut_in_line\fR (int)
2304 Prioritize requeued I/O
2306 Default value: \fB0\fR.
2312 \fBzio_taskq_batch_pct\fR (uint)
2315 Percentage of online CPUs (or CPU cores, etc) which will run a worker thread
2316 for IO. These workers are responsible for IO work such as compression and
2317 checksum calculations. Fractional number of CPUs will be rounded down.
2319 The default value of 75 was chosen to avoid using all CPUs which can result in
2320 latency issues and inconsistent application performance, especially when high
2321 compression is enabled.
2323 Default value: \fB75\fR.
2329 \fBzvol_inhibit_dev\fR (uint)
2332 Do not create zvol device nodes. This may slightly improve startup time on
2333 systems with a very large number of zvols.
2335 Use \fB1\fR for yes and \fB0\fR for no (default).
2341 \fBzvol_major\fR (uint)
2344 Major number for zvol block devices
2346 Default value: \fB230\fR.
2352 \fBzvol_max_discard_blocks\fR (ulong)
2355 Discard (aka TRIM) operations done on zvols will be done in batches of this
2356 many blocks, where block size is determined by the \fBvolblocksize\fR property
2359 Default value: \fB16,384\fR.
2365 \fBzvol_prefetch_bytes\fR (uint)
2368 When adding a zvol to the system prefetch \fBzvol_prefetch_bytes\fR
2369 from the start and end of the volume. Prefetching these regions
2370 of the volume is desirable because they are likely to be accessed
2371 immediately by \fBblkid(8)\fR or by the kernel scanning for a partition
2374 Default value: \fB131,072\fR.
2380 \fBzvol_request_sync\fR (uint)
2383 When processing I/O requests for a zvol submit them synchronously. This
2384 effectively limits the queue depth to 1 for each I/O submitter. When set
2385 to 0 requests are handled asynchronously by a thread pool. The number of
2386 requests which can be handled concurrently is controller by \fBzvol_threads\fR.
2388 Default value: \fB0\fR.
2394 \fBzvol_threads\fR (uint)
2397 Max number of threads which can handle zvol I/O requests concurrently.
2399 Default value: \fB32\fR.
2405 \fBzvol_volmode\fR (uint)
2408 Defines zvol block devices behaviour when \fBvolmode\fR is set to \fBdefault\fR.
2409 Valid values are \fB1\fR (full), \fB2\fR (dev) and \fB3\fR (none).
2411 Default value: \fB1\fR.
2417 \fBzfs_qat_disable\fR (int)
2420 This tunable disables qat hardware acceleration for gzip compression and.
2421 AES-GCM encryption. It is available only if qat acceleration is compiled in
2422 and the qat driver is present.
2424 Use \fB1\fR for yes and \fB0\fR for no (default).
2427 .SH ZFS I/O SCHEDULER
2428 ZFS issues I/O operations to leaf vdevs to satisfy and complete I/Os.
2429 The I/O scheduler determines when and in what order those operations are
2430 issued. The I/O scheduler divides operations into five I/O classes
2431 prioritized in the following order: sync read, sync write, async read,
2432 async write, and scrub/resilver. Each queue defines the minimum and
2433 maximum number of concurrent operations that may be issued to the
2434 device. In addition, the device has an aggregate maximum,
2435 \fBzfs_vdev_max_active\fR. Note that the sum of the per-queue minimums
2436 must not exceed the aggregate maximum. If the sum of the per-queue
2437 maximums exceeds the aggregate maximum, then the number of active I/Os
2438 may reach \fBzfs_vdev_max_active\fR, in which case no further I/Os will
2439 be issued regardless of whether all per-queue minimums have been met.
2441 For many physical devices, throughput increases with the number of
2442 concurrent operations, but latency typically suffers. Further, physical
2443 devices typically have a limit at which more concurrent operations have no
2444 effect on throughput or can actually cause it to decrease.
2446 The scheduler selects the next operation to issue by first looking for an
2447 I/O class whose minimum has not been satisfied. Once all are satisfied and
2448 the aggregate maximum has not been hit, the scheduler looks for classes
2449 whose maximum has not been satisfied. Iteration through the I/O classes is
2450 done in the order specified above. No further operations are issued if the
2451 aggregate maximum number of concurrent operations has been hit or if there
2452 are no operations queued for an I/O class that has not hit its maximum.
2453 Every time an I/O is queued or an operation completes, the I/O scheduler
2454 looks for new operations to issue.
2456 In general, smaller max_active's will lead to lower latency of synchronous
2457 operations. Larger max_active's may lead to higher overall throughput,
2458 depending on underlying storage.
2460 The ratio of the queues' max_actives determines the balance of performance
2461 between reads, writes, and scrubs. E.g., increasing
2462 \fBzfs_vdev_scrub_max_active\fR will cause the scrub or resilver to complete
2463 more quickly, but reads and writes to have higher latency and lower throughput.
2465 All I/O classes have a fixed maximum number of outstanding operations
2466 except for the async write class. Asynchronous writes represent the data
2467 that is committed to stable storage during the syncing stage for
2468 transaction groups. Transaction groups enter the syncing state
2469 periodically so the number of queued async writes will quickly burst up
2470 and then bleed down to zero. Rather than servicing them as quickly as
2471 possible, the I/O scheduler changes the maximum number of active async
2472 write I/Os according to the amount of dirty data in the pool. Since
2473 both throughput and latency typically increase with the number of
2474 concurrent operations issued to physical devices, reducing the
2475 burstiness in the number of concurrent operations also stabilizes the
2476 response time of operations from other -- and in particular synchronous
2477 -- queues. In broad strokes, the I/O scheduler will issue more
2478 concurrent operations from the async write queue as there's more dirty
2483 The number of concurrent operations issued for the async write I/O class
2484 follows a piece-wise linear function defined by a few adjustable points.
2487 | o---------| <-- zfs_vdev_async_write_max_active
2494 |-------o | | <-- zfs_vdev_async_write_min_active
2495 0|_______^______|_________|
2496 0% | | 100% of zfs_dirty_data_max
2498 | `-- zfs_vdev_async_write_active_max_dirty_percent
2499 `--------- zfs_vdev_async_write_active_min_dirty_percent
2502 Until the amount of dirty data exceeds a minimum percentage of the dirty
2503 data allowed in the pool, the I/O scheduler will limit the number of
2504 concurrent operations to the minimum. As that threshold is crossed, the
2505 number of concurrent operations issued increases linearly to the maximum at
2506 the specified maximum percentage of the dirty data allowed in the pool.
2508 Ideally, the amount of dirty data on a busy pool will stay in the sloped
2509 part of the function between \fBzfs_vdev_async_write_active_min_dirty_percent\fR
2510 and \fBzfs_vdev_async_write_active_max_dirty_percent\fR. If it exceeds the
2511 maximum percentage, this indicates that the rate of incoming data is
2512 greater than the rate that the backend storage can handle. In this case, we
2513 must further throttle incoming writes, as described in the next section.
2515 .SH ZFS TRANSACTION DELAY
2516 We delay transactions when we've determined that the backend storage
2517 isn't able to accommodate the rate of incoming writes.
2519 If there is already a transaction waiting, we delay relative to when
2520 that transaction will finish waiting. This way the calculated delay time
2521 is independent of the number of threads concurrently executing
2524 If we are the only waiter, wait relative to when the transaction
2525 started, rather than the current time. This credits the transaction for
2526 "time already served", e.g. reading indirect blocks.
2528 The minimum time for a transaction to take is calculated as:
2530 min_time = zfs_delay_scale * (dirty - min) / (max - dirty)
2531 min_time is then capped at 100 milliseconds.
2534 The delay has two degrees of freedom that can be adjusted via tunables. The
2535 percentage of dirty data at which we start to delay is defined by
2536 \fBzfs_delay_min_dirty_percent\fR. This should typically be at or above
2537 \fBzfs_vdev_async_write_active_max_dirty_percent\fR so that we only start to
2538 delay after writing at full speed has failed to keep up with the incoming write
2539 rate. The scale of the curve is defined by \fBzfs_delay_scale\fR. Roughly speaking,
2540 this variable determines the amount of delay at the midpoint of the curve.
2544 10ms +-------------------------------------------------------------*+
2560 2ms + (midpoint) * +
2563 | zfs_delay_scale ----------> ******** |
2564 0 +-------------------------------------*********----------------+
2565 0% <- zfs_dirty_data_max -> 100%
2568 Note that since the delay is added to the outstanding time remaining on the
2569 most recent transaction, the delay is effectively the inverse of IOPS.
2570 Here the midpoint of 500us translates to 2000 IOPS. The shape of the curve
2571 was chosen such that small changes in the amount of accumulated dirty data
2572 in the first 3/4 of the curve yield relatively small differences in the
2575 The effects can be easier to understand when the amount of delay is
2576 represented on a log scale:
2580 100ms +-------------------------------------------------------------++
2589 + zfs_delay_scale ----------> ***** +
2600 +--------------------------------------------------------------+
2601 0% <- zfs_dirty_data_max -> 100%
2604 Note here that only as the amount of dirty data approaches its limit does
2605 the delay start to increase rapidly. The goal of a properly tuned system
2606 should be to keep the amount of dirty data out of that range by first
2607 ensuring that the appropriate limits are set for the I/O scheduler to reach
2608 optimal throughput on the backend storage, and then by changing the value
2609 of \fBzfs_delay_scale\fR to increase the steepness of the curve.
projects / FreeBSD / FreeBSD.git / blob