4 .\" The contents of this file are subject to the terms of the
5 .\" Common Development and Distribution License (the "License").
6 .\" You may not use this file except in compliance with the License.
8 .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9 .\" or http://www.opensolaris.org/os/licensing.
10 .\" See the License for the specific language governing permissions
11 .\" and limitations under the License.
13 .\" When distributing Covered Code, include this CDDL HEADER in each
14 .\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15 .\" If applicable, add the following below this CDDL HEADER, with the
16 .\" fields enclosed by brackets "[]" replaced with your own identifying
17 .\" information: Portions Copyright [yyyy] [name of copyright owner]
21 .\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved.
22 .\" Copyright 2011 Joshua M. Clulow <josh@sysmgr.org>
23 .\" Copyright (c) 2011, 2019 by Delphix. All rights reserved.
24 .\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
25 .\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
26 .\" Copyright (c) 2014 by Adam Stevko. All rights reserved.
27 .\" Copyright (c) 2014 Integros [integros.com]
28 .\" Copyright 2019 Richard Laager. All rights reserved.
29 .\" Copyright 2018 Nexenta Systems, Inc.
30 .\" Copyright 2019 Joyent, Inc.
38 .Nd generate backup stream of ZFS dataset
43 .Op Fl R Op Fl X Ar dataset Ns Oo , Ns Ar dataset Oc Ns …
44 .Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot
49 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
50 .Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot
53 .Fl -redact Ar redaction_bookmark
55 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
61 .Ar receive_resume_token
68 .Ar snapshot redaction_bookmark
69 .Ar redaction_snapshot Ns …
77 .Op Fl R Op Fl X Ar dataset Ns Oo , Ns Ar dataset Oc Ns …
78 .Op Oo Fl I Ns | Ns Fl i Oc Ar snapshot
81 Creates a stream representation of the second
83 which is written to standard output.
84 The output can be redirected to a file or to a different system
85 .Po for example, using
88 By default, a full stream is generated.
91 Deduplicated send is no longer supported.
92 This flag is accepted for backwards compatibility, but a regular,
93 non-deduplicated stream will be generated.
95 Generate a stream package that sends all intermediary snapshots from the first
96 snapshot to the second snapshot.
100 .Fl i Em @a Em fs@b Ns \&; Fl i Em @b Em fs@c Ns \&; Fl i Em @c Em fs@d .
101 The incremental source may be specified as with the
104 .It Fl L , -large-block
105 Generate a stream which may contain blocks larger than 128 KiB.
106 This flag has no effect if the
108 pool feature is disabled, or if the
110 property of this filesystem has never been set above 128 KiB.
111 The receiving system must have the
113 pool feature enabled as well.
116 for details on ZFS feature flags and the
120 Print machine-parsable verbose information about the stream package generated.
121 .It Fl R , -replicate
122 Generate a replication stream package, which will replicate the specified
123 file system, and all descendent file systems, up to the named snapshot.
124 When received, all properties, snapshots, descendent file systems, and clones
131 flags are used in conjunction with the
133 flag, an incremental replication stream is generated.
134 The current values of properties, and current snapshot and file system names are
135 set when the stream is received.
138 flag is specified when this stream is received, snapshots and file systems that
139 do not exist on the sending side are destroyed.
142 flag is used to send encrypted datasets, then
144 must also be specified.
145 .It Fl X , -exclude Ar dataset Ns Oo , Ns Ar dataset Oc Ns …
149 specifies a set of datasets (and, hence, their descendants),
150 to be excluded from the send stream.
151 The root dataset may not be excluded.
154 .Fl X Ar a , Ns Ar b .
156 Generate a more compact stream by using
158 records for blocks which are stored more compactly on disk by the
161 This flag has no effect if the
164 The receiving system must have the
169 feature is active on the sending system, then the receiving system must have
170 that feature enabled as well.
171 Datasets that are sent with this flag may not be
172 received as an encrypted dataset, since encrypted datasets cannot use the
177 for details on ZFS feature flags and the
181 Sends only received property values whether or not they are overridden by local
182 settings, but only if the dataset has ever been received.
183 Use this option when you want
185 to restore received properties backed up on the sent dataset and to avoid
186 sending local settings that may have nothing to do with the source dataset,
187 but only with how the data is backed up.
188 .It Fl c , -compressed
189 Generate a more compact stream by using compressed WRITE records for blocks
190 which are compressed on disk and in memory
197 feature is active on the sending system, then the receiving system must have
198 that feature enabled as well.
201 feature is enabled on the sending system but the
203 option is not supplied in conjunction with
205 then the data will be decompressed before sending so it can be split into
209 will not have their data recompressed on the receiver side using
210 .Fl o Sy compress Ns = Ar value .
211 The data will stay compressed as it was from the sender.
212 The new compression property will be set for future data.
213 Note that uncompressed data from the sender will still attempt to
214 compress on the receiver, unless you specify
215 .Fl o Sy compress Ns = Em off .
217 For encrypted datasets, send data exactly as it exists on disk.
218 This allows backups to be taken even if encryption keys are not currently loaded.
219 The backup may then be received on an untrusted machine since that machine will
220 not have the encryption keys to read the protected data or alter it without
222 Upon being received, the dataset will have the same encryption
223 keys as it did on the send side, although the
225 property will be defaulted to
227 if not otherwise provided.
228 For unencrypted datasets, this flag will be equivalent to
230 Note that if you do not use this flag for sending encrypted datasets, data will
231 be sent unencrypted and may be re-encrypted with a different encryption key on
232 the receiving system, which will disable the ability to do a raw send to that
233 system for incrementals.
235 Generate a stream package that includes any snapshot holds (created with the
237 command), and indicating to
239 that the holds be applied to the dataset on the receiving system.
241 Generate an incremental stream from the first
243 .Pq the incremental source
246 .Pq the incremental target .
247 The incremental source can be specified as the last component of the snapshot
251 character and following
253 and it is assumed to be from the same file system as the incremental target.
255 If the destination is a clone, the source may be the origin snapshot, which must
266 Do not generate any actual send data.
267 This is useful in conjunction with the
271 flags to determine what data will be sent.
272 In this case, the verbose output will be written to standard output
273 .Po contrast with a non-dry-run, where the stream is written to standard output
274 and the verbose output goes to standard error
277 Include the dataset's properties in the stream.
278 This flag is implicit when
281 The receiving system must also support this feature.
282 Sends of encrypted datasets must use
284 when using this flag.
285 .It Fl s , -skip-missing
286 Allows sending a replication stream even when there are snapshots missing in the
288 When a snapshot is missing, instead of throwing an error and aborting the send,
289 a warning is printed to the standard error stream and the dataset to which it belongs
290 and its descendents are skipped.
291 This flag can only be used in conjunction with
294 Print verbose information about the stream package generated.
295 This information includes a per-second report of how much data has been sent.
297 The format of the stream is committed.
298 You will be able to receive your streams on future versions of ZFS.
304 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
305 .Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot
307 Generate a send stream, which may be of a filesystem, and may be incremental
309 If the destination is a filesystem or volume, the pool must be read-only, or the
310 filesystem must not be mounted.
311 When the stream generated from a filesystem or volume is received, the default
312 snapshot name will be
316 Deduplicated send is no longer supported.
317 This flag is accepted for backwards compatibility, but a regular,
318 non-deduplicated stream will be generated.
319 .It Fl L , -large-block
320 Generate a stream which may contain blocks larger than 128 KiB.
321 This flag has no effect if the
323 pool feature is disabled, or if the
325 property of this filesystem has never been set above 128 KiB.
326 The receiving system must have the
328 pool feature enabled as well.
331 for details on ZFS feature flags and the
335 Print machine-parsable verbose information about the stream package generated.
336 .It Fl c , -compressed
337 Generate a more compact stream by using compressed WRITE records for blocks
338 which are compressed on disk and in memory
345 feature is active on the sending system, then the receiving system must have
346 that feature enabled as well.
349 feature is enabled on the sending system but the
351 option is not supplied in conjunction with
353 then the data will be decompressed before sending so it can be split into
356 For encrypted datasets, send data exactly as it exists on disk.
357 This allows backups to be taken even if encryption keys are not currently loaded.
358 The backup may then be received on an untrusted machine since that machine will
359 not have the encryption keys to read the protected data or alter it without
361 Upon being received, the dataset will have the same encryption
362 keys as it did on the send side, although the
364 property will be defaulted to
366 if not otherwise provided.
367 For unencrypted datasets, this flag will be equivalent to
369 Note that if you do not use this flag for sending encrypted datasets, data will
370 be sent unencrypted and may be re-encrypted with a different encryption key on
371 the receiving system, which will disable the ability to do a raw send to that
372 system for incrementals.
374 Generate a more compact stream by using
376 records for blocks which are stored more compactly on disk by the
379 This flag has no effect if the
382 The receiving system must have the
387 feature is active on the sending system, then the receiving system must have
388 that feature enabled as well.
389 Datasets that are sent with this flag may not be received as an encrypted dataset,
390 since encrypted datasets cannot use the
395 for details on ZFS feature flags and the
398 .It Fl i Ar snapshot Ns | Ns Ar bookmark
399 Generate an incremental send stream.
400 The incremental source must be an earlier snapshot in the destination's history.
401 It will commonly be an earlier snapshot in the destination's file system, in
402 which case it can be specified as the last component of the name
407 character and following
410 If the incremental target is a clone, the incremental source can be the origin
411 snapshot, or an earlier snapshot in the origin's filesystem, or the origin's
417 Do not generate any actual send data.
418 This is useful in conjunction with the
422 flags to determine what data will be sent.
423 In this case, the verbose output will be written to standard output
424 .Po contrast with a non-dry-run, where the stream is written to standard output
425 and the verbose output goes to standard error
428 Print verbose information about the stream package generated.
429 This information includes a per-second report of how much data has been sent.
434 .Fl -redact Ar redaction_bookmark
436 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
439 Generate a redacted send stream.
440 This send stream contains all blocks from the snapshot being sent that aren't
441 included in the redaction list contained in the bookmark specified by the
446 The resulting send stream is said to be redacted with respect to the snapshots
447 the bookmark specified by the
448 .Fl -redact No flag was created with.
449 The bookmark must have been created by running
451 on the snapshot being sent.
453 This feature can be used to allow clones of a filesystem to be made available on
454 a remote system, in the case where their parent need not (or needs to not) be
456 For example, if a filesystem contains sensitive data, and it has clones where
457 that sensitive data has been secured or replaced with dummy data, redacted sends
458 can be used to replicate the secured data without replicating the original
459 sensitive data, while still sharing all possible blocks.
460 A snapshot that has been redacted with respect to a set of snapshots will
461 contain all blocks referenced by at least one snapshot in the set, but will
462 contain none of the blocks referenced by none of the snapshots in the set.
463 In other words, if all snapshots in the set have modified a given block in the
464 parent, that block will not be sent; but if one or more snapshots have not
465 modified a block in the parent, they will still reference the parent's block, so
466 that block will be sent.
467 Note that only user data will be redacted.
469 When the redacted send stream is received, we will generate a redacted
471 Due to the nature of redaction, a redacted dataset can only be used in the
473 .Bl -enum -width "a."
475 To receive, as a clone, an incremental send from the original snapshot to one
476 of the snapshots it was redacted with respect to.
477 In this case, the stream will produce a valid dataset when received because all
478 blocks that were redacted in the parent are guaranteed to be present in the
480 This use case will produce a normal snapshot, which can be used just like other
484 To receive an incremental send from the original snapshot to something
485 redacted with respect to a subset of the set of snapshots the initial snapshot
486 was redacted with respect to.
487 In this case, each block that was redacted in the original is still redacted
488 (redacting with respect to additional snapshots causes less data to be redacted
489 (because the snapshots define what is permitted, and everything else is
491 This use case will produce a new redacted snapshot.
493 To receive an incremental send from a redaction bookmark of the original
494 snapshot that was created when redacting with respect to a subset of the set of
495 snapshots the initial snapshot was created with respect to
497 A send stream from such a redaction bookmark will contain all of the blocks
498 necessary to fill in any redacted data, should it be needed, because the sending
499 system is aware of what blocks were originally redacted.
500 This will either produce a normal snapshot or a redacted one, depending on
501 whether the new send stream is redacted.
503 To receive an incremental send from a redacted version of the initial
504 snapshot that is redacted with respect to a subject of the set of snapshots the
505 initial snapshot was created with respect to.
506 A send stream from a compatible redacted dataset will contain all of the blocks
507 necessary to fill in any redacted data.
508 This will either produce a normal snapshot or a redacted one, depending on
509 whether the new send stream is redacted.
511 To receive a full send as a clone of the redacted snapshot.
512 Since the stream is a full send, it definitionally contains all the data needed
513 to create a new dataset.
514 This use case will either produce a normal snapshot or a redacted one, depending
515 on whether the full send stream was redacted.
518 These restrictions are detected and enforced by
520 a redacted send stream will contain the list of snapshots that the stream is
521 redacted with respect to.
522 These are stored with the redacted snapshot, and are used to detect and
523 correctly handle the cases above.
524 Note that for technical reasons,
525 raw sends and redacted sends cannot be combined at this time.
531 .Ar receive_resume_token
533 Creates a send stream which resumes an interrupted receive.
535 .Ar receive_resume_token
536 is the value of this property on the filesystem or volume that was being
538 See the documentation for
539 .Nm zfs Cm receive Fl s
545 .Op Fl i Ar snapshot Ns | Ns Ar bookmark
549 Generate a send stream from a dataset that has been partially received.
552 This flag requires that the specified filesystem previously received a resumable
553 send that did not finish and was interrupted.
554 In such scenarios this flag
555 enables the user to send this partially received state.
556 Using this flag will always use the last fully received snapshot
557 as the incremental source if it exists.
562 .Ar snapshot redaction_bookmark
563 .Ar redaction_snapshot Ns …
565 Generate a new redaction bookmark.
566 In addition to the typical bookmark information, a redaction bookmark contains
567 the list of redacted blocks and the list of redaction snapshots specified.
568 The redacted blocks are blocks in the snapshot which are not referenced by any
569 of the redaction snapshots.
570 These blocks are found by iterating over the metadata in each redaction snapshot
571 to determine what has been changed since the target snapshot.
572 Redaction is designed to support redacted zfs sends; see the entry for
574 for more information on the purpose of this operation.
575 If a redact operation fails partway through (due to an error or a system
576 failure), the redaction can be resumed by rerunning the same command.
579 ZFS has support for a limited version of data subsetting, in the form of
584 .Sy redaction bookmark
585 can be created that stores a list of blocks containing sensitive information.
591 Redacted sends omit the blocks containing sensitive information,
592 replacing them with REDACT records.
593 When these send streams are received, a
596 A redacted dataset cannot be mounted by default, since it is incomplete.
597 It can be used to receive other send streams.
598 In this way datasets can be used for data backup and replication,
599 with all the benefits that zfs send and receive have to offer,
600 while protecting sensitive information from being
601 stored on less-trusted machines or services.
603 For the purposes of redaction, there are two steps to the process.
604 A redact step, and a send/receive step.
605 First, a redaction bookmark is created.
606 This is done by providing the
608 command with a parent snapshot, a bookmark to be created, and a number of
610 These redaction snapshots must be descendants of the parent snapshot,
611 and they should modify data that is considered sensitive in some way.
612 Any blocks of data modified by all of the redaction snapshots will
613 be listed in the redaction bookmark, because it represents the truly sensitive
615 When it comes to the send step, the send process will not send
616 the blocks listed in the redaction bookmark, instead replacing them with
618 When received on the target system, this will create a
619 redacted dataset, missing the data that corresponds to the blocks in the
620 redaction bookmark on the sending system.
621 The incremental send streams from
622 the original parent to the redaction snapshots can then also be received on
623 the target system, and this will produce a complete snapshot that can be used
625 Incrementals from one snapshot on the parent filesystem and another
626 can also be done by sending from the redaction bookmark, rather than the
627 snapshots themselves.
629 In order to make the purpose of the feature more clear, an example is provided.
630 Consider a zfs filesystem containing four files.
631 These files represent information for an online shopping service.
632 One file contains a list of usernames and passwords, another contains purchase histories,
633 a third contains click tracking data, and a fourth contains user preferences.
634 The owner of this data wants to make it available for their development teams to
635 test against, and their market research teams to do analysis on.
636 The development teams need information about user preferences and the click
637 tracking data, while the market research teams need information about purchase
638 histories and user preferences.
639 Neither needs access to the usernames and passwords.
640 However, because all of this data is stored in one ZFS filesystem,
641 it must all be sent and received together.
642 In addition, the owner of the data
643 wants to take advantage of features like compression, checksumming, and
644 snapshots, so they do want to continue to use ZFS to store and transmit their data.
645 Redaction can help them do so.
646 First, they would make two clones of a snapshot of the data on the source.
647 In one clone, they create the setup they want their market research team to see;
648 they delete the usernames and passwords file,
649 and overwrite the click tracking data with dummy information.
650 In another, they create the setup they want the development teams
651 to see, by replacing the passwords with fake information and replacing the
652 purchase histories with randomly generated ones.
653 They would then create a redaction bookmark on the parent snapshot,
654 using snapshots on the two clones as redaction snapshots.
655 The parent can then be sent, redacted, to the target
656 server where the research and development teams have access.
657 Finally, incremental sends from the parent snapshot to each of the clones can be sent
658 to and received on the target server; these snapshots are identical to the
659 ones on the source, and are ready to be used, while the parent snapshot on the
660 target contains none of the username and password data present on the source,
661 because it was removed by the redacted send operation.
664 .\" These are, respectively, examples 12, 13 from zfs.8
665 .\" Make sure to update them bidirectionally
666 .Ss Example 1 : No Remotely Replicating ZFS Data
667 The following commands send a full stream and then an incremental stream to a
668 remote machine, restoring them into
669 .Em poolB/received/fs@a
671 .Em poolB/received/fs@b ,
674 must contain the file system
676 and must not initially contain
677 .Em poolB/received/fs .
678 .Bd -literal -compact -offset Ds
679 .No # Nm zfs Cm send Ar pool/fs@a |
680 .No " " Nm ssh Ar host Nm zfs Cm receive Ar poolB/received/fs Ns @ Ns Ar a
681 .No # Nm zfs Cm send Fl i Ar a pool/fs@b |
682 .No " " Nm ssh Ar host Nm zfs Cm receive Ar poolB/received/fs
685 .Ss Example 2 : No Using the Nm zfs Cm receive Fl d No Option
686 The following command sends a full stream of
687 .Ar poolA/fsA/fsB@snap
688 to a remote machine, receiving it into
689 .Ar poolB/received/fsA/fsB@snap .
692 portion of the received snapshot's name is determined from the name of the sent
695 must contain the file system
698 .Ar poolB/received/fsA
699 does not exist, it is created as an empty file system.
700 .Bd -literal -compact -offset Ds
701 .No # Nm zfs Cm send Ar poolA/fsA/fsB@snap |
702 .No " " Nm ssh Ar host Nm zfs Cm receive Fl d Ar poolB/received
projects / FreeBSD / FreeBSD.git / blob