1 This file describes the design, layouts, and file formats of a
2 libsvn_fs_fs repository.
7 In FSFS, each committed revision is represented as an immutable file
8 containing the new node-revisions, contents, and changed-path
9 information for the revision, plus a second, changeable file
10 containing the revision properties.
12 In contrast to the BDB back end, the contents of recent revision of
13 files are stored as deltas against earlier revisions, instead of the
14 other way around. This is less efficient for common-case checkouts,
15 but brings greater simplicity and robustness, as well as the
16 flexibility to make commits work without write access to existing
17 revisions. Skip-deltas and delta combination mitigate the checkout
20 In-progress transactions are represented with a prototype rev file
21 containing only the new text representations of files (appended to as
22 changed file contents come in), along with a separate file for each
23 node-revision, directory representation, or property representation
24 which has been changed or added in the transaction. During the final
25 stage of the commit, these separate files are marshalled onto the end
26 of the prototype rev file to form the immutable revision file.
28 Layout of the FS directory
29 --------------------------
31 The layout of the FS directory (the "db" subdirectory of the
34 revs/ Subdirectory containing revs
35 <shard>/ Shard directory, if sharding is in use (see below)
36 <revnum> File containing rev <revnum>
37 <shard>.pack/ Pack directory, if the repo has been packed (see below)
38 pack Pack file, if the repository has been packed (see below)
39 manifest Pack manifest file, if a pack file exists (see below)
40 revprops/ Subdirectory containing rev-props
41 <shard>/ Shard directory, if sharding is in use (see below)
42 <revnum> File containing rev-props for <revnum>
43 <shard>.pack/ Pack directory, if the repo has been packed (see below)
44 <rev>.<count> Pack file, if the repository has been packed (see below)
45 manifest Pack manifest file, if a pack file exists (see below)
46 revprops.db SQLite database of the packed revprops (format 5 only)
47 transactions/ Subdirectory containing transactions
48 <txnid>.txn/ Directory containing transaction <txnid>
49 txn-protorevs/ Subdirectory containing transaction proto-revision files
50 <txnid>.rev Proto-revision file for transaction <txnid>
51 <txnid>.rev-lock Write lock for proto-rev file
52 txn-current File containing the next transaction key
53 locks/ Subdirectory containing locks
54 <partial-digest>/ Subdirectory named for first 3 letters of an MD5 digest
55 <digest> File containing locks/children for path with <digest>
56 node-origins/ Lazy cache of origin noderevs for nodes
57 <partial-nodeid> File containing noderev ID of origins of nodes
58 current File specifying current revision and next node/copy id
59 fs-type File identifying this filesystem as an FSFS filesystem
60 write-lock Empty file, locked to serialise writers
61 pack-lock Empty file, locked to serialise 'svnadmin pack' (f. 7+)
62 txn-current-lock Empty file, locked to serialise 'txn-current'
63 uuid File containing the repository IDs
64 format File containing the format number of this filesystem
65 fsfs.conf Configuration file
66 min-unpacked-rev File containing the oldest revision not in a pack file
67 min-unpacked-revprop Same for revision properties (format 5 only)
68 rep-cache.db SQLite database mapping rep checksums to locations
70 Files in the revprops directory are in the hash dump format used by
73 The format of the "current" file is:
75 * Format 3 and above: a single line of the form
76 "<youngest-revision>\n" giving the youngest revision for the
79 * Format 2 and below: a single line of the form "<youngest-revision>
80 <next-node-id> <next-copy-id>\n" giving the youngest revision, the
81 next unique node-ID, and the next unique copy-ID for the
84 The "write-lock" file is an empty file which is locked before the
85 final stage of a commit and unlocked after the new "current" file has
86 been moved into place to indicate that a new revision is present. It
87 is also locked during a revprop propchange while the revprop file is
88 read in, mutated, and written out again. Furthermore, it will be used
89 to serialize the repository structure changes during 'svnadmin pack'
90 (see also next section). Note that readers are never blocked by any
91 operation - writers must ensure that the filesystem is always in a
94 The "pack-lock" file is an empty file which is locked before an 'svnadmin
95 pack' operation commences. Thus, only one process may attempt to modify
96 the repository structure at a time while other processes may still read
97 and write (commit) to the repository during most of the pack procedure.
98 It is only available with format 7 and newer repositories. Older formats
99 use the global write-lock instead which disables commits completely
100 for the duration of the pack process.
102 The "txn-current" file is a file with a single line of text that
103 contains only a base-36 number. The current value will be used in the
104 next transaction name, along with the revision number the transaction
105 is based on. This sequence number ensures that transaction names are
106 not reused, even if the transaction is aborted and a new transaction
107 based on the same revision is begun. The only operation that FSFS
108 performs on this file is "get and increment"; the "txn-current-lock"
109 file is locked during this operation.
111 "fsfs.conf" is a configuration file in the standard Subversion/Python
112 config format. It is automatically generated when you create a new
113 repository; read the generated file for details on what it controls.
115 When representation sharing is enabled, the filesystem tracks
116 representation checksum and location mappings using a SQLite database in
117 "rep-cache.db". The database has a single table, which stores the sha1
118 hash text as the primary key, mapped to the representation revision, offset,
119 size and expanded size. This file is only consulted during writes and never
120 during reads. Consequently, it is not required, and may be removed at an
121 abritrary time, with the subsequent loss of rep-sharing capabilities for
122 revisions written thereafter.
127 The "format" file defines what features are permitted within the
128 filesystem, and indicates changes that are not backward-compatible.
129 It serves the same purpose as the repository file of the same name.
131 The filesystem format file was introduced in Subversion 1.2, and so
132 will not be present if the repository was created with an older
133 version of Subversion. An absent format file should be interpreted as
134 indicating a format 1 filesystem.
136 The format file is a single line of the form "<format number>\n",
137 followed by any number of lines specifying 'format options' -
138 additional information about the filesystem's format. Each format
139 option line is of the form "<option>\n" or "<option> <parameters>\n".
141 Clients should raise an error if they encounter an option not
142 permitted by the format number in use.
146 Format 1, understood by Subversion 1.1+
147 Format 2, understood by Subversion 1.4+
148 Format 3, understood by Subversion 1.5+
149 Format 4, understood by Subversion 1.6+
150 Format 5, understood by Subversion 1.7-dev, never released
151 Format 6, understood by Subversion 1.8
152 Format 7, understood by Subversion 1.9
153 Format 8, understood by Subversion 1.10
155 The differences between the formats are:
157 Delta representation in revision files
158 Format 1: svndiff0 only
159 Formats 2-7: svndiff0 or svndiff1
160 Formats 8: svndiff0, svndiff1 or svndiff2
163 Formats 1-2: none permitted
164 Format 3+: "layout" option
165 Format 7+: "addressing" option
167 Transaction name reuse
168 Formats 1-2: transaction names may be reused
169 Format 3+: transaction names generated using txn-current file
171 Location of proto-rev file and its lock
172 Formats 1-2: transactions/<txnid>/rev and
173 transactions/<txnid>/rev-lock.
174 Format 3+: txn-protorevs/<txnid>.rev and
175 txn-protorevs/<txnid>.rev-lock.
177 Node-ID and copy-ID generation
178 Formats 1-2: Node-IDs and copy-IDs are guaranteed to form a
179 monotonically increasing base36 sequence using the "current"
181 Format 3+: Node-IDs and copy-IDs use the new revision number to
182 ensure uniqueness and the "current" file just contains the
186 Format 1-2: minfo-here and minfo-count node-revision fields are not
187 stored. svn_fs_get_mergeinfo returns an error.
188 Format 3+: minfo-here and minfo-count node-revision fields are
189 maintained. svn_fs_get_mergeinfo works.
191 Revision changed paths list:
192 Format 1-3: Does not contain the node's kind.
193 Format 4+: Contains the node's kind.
194 Format 7+: Contains the mergeinfo-mod flag.
197 Format 4: Applied to revision data only.
198 Format 5: Revprops would be packed independently of revision data.
199 Format 6+: Applied equally to revision data and revprop data
200 (i.e. same min packed revision)
203 Format 1+: Physical addressing; uses fixed positions within a rev file
204 Format 7+: Logical addressing; uses item index that will be translated
205 on-the-fly to the actual rev / pack file location (default for 7+ created)
208 Format 1+: The first line of db/uuid contains the repository UUID
209 Format 7+: The second line contains the instance ID (in UUID formatting)
211 # Incomplete list. See SVN_FS_FS__MIN_*_FORMAT
214 Filesystem format options
215 -------------------------
217 Currently, the only recognised format options are "layout" and "addressing".
218 The first specifies the paths that will be used to store the revision
219 files and revision property files. The second specifies that logical to
220 physical address translation is required.
222 The "layout" option is followed by the name of the filesystem layout
223 and any required parameters. The default layout, if no "layout"
224 keyword is specified, is the 'linear' layout.
226 The known layouts, and the parameters they require, are as follows:
229 Revision files and rev-prop files are named after the revision they
230 represent, and are placed directly in the revs/ and revprops/
231 directories. r1234 will be represented by the revision file
232 revs/1234 and the rev-prop file revprops/1234.
234 "sharded <max-files-per-directory>"
235 Revision files and rev-prop files are named after the revision they
236 represent, and are placed in a subdirectory of the revs/ and
237 revprops/ directories named according to the 'shard' they belong to.
239 Shards are numbered from zero and contain between one and the
240 maximum number of files per directory specified in the layout's
243 For the "sharded 1000" layout, r1234 will be represented by the
244 revision file revs/1/1234 and rev-prop file revprops/1/1234. The
245 revs/0/ directory will contain revisions 0-999, revs/1/ will contain
246 1000-1999, and so on.
248 The "addressing" option is followed by the name of the addressing mode
249 and any required parameters. The default addressing, if no "addressing"
250 keyword is specified, is the 'physical' addressing.
252 The supported modes, and the parameters they require, are as follows:
255 All existing and future revision files will use the traditional
256 physical addressing scheme. All references are given as rev/offset
257 pairs with "offset" being the byte offset relative to the beginning of
258 the revision in the respective rev or pack file.
261 All existing and future revision files will use logical
262 addressing. It is illegal to use logical addressing on non-sharded
269 Two addressing modes are supported in format 7: physical and logical
270 addressing. Both use the same address format but apply a different
271 interpretation to it. Older formats only support physical addressing.
273 All items are addressed using <rev> <item_index> pairs. In physical
274 addressing mode, item_index is the (ASCII decimal) number of bytes from
275 the start of the revision file to the start of the respective item. For
276 non-packed files that is also the absolute file offset. Revision pack
277 files simply concatenate multiple rev files, i.e. the absolute file offset
280 absolute offset = rev offset taken from manifest + item_index
282 This simple addressing scheme makes it hard to change the location of
283 any item since that may break references from later revisions.
285 Logical addressing uses an index file to translate the rev / item_index
286 pairs into absolute file offsets. There is one such index for every rev /
287 pack file using logical addressing and both are created in sync. That
288 makes it possible to reorder items during pack file creation, particularly
289 to mix items from different revisions.
291 Some item_index values are pre-defined and apply to every revision:
293 0 ... not used / invalid
294 1 ... changed path list
295 2 ... root node revision
297 A reverse index (phys-to-log) is being created as well that allows for
298 translating arbitrary file locations into item descriptions (type, rev,
299 item_index, on-disk length). Known item types
301 0 ... unused / empty section
302 1 ... file representation
303 2 ... directory representation
304 3 ... file property representation
305 4 ... directory property representation
307 6 ... changed paths list
309 The various representation types all share the same morphology. The
310 distinction is only made to allow for more effective reordering heuristics.
311 Zero-length items are allowed.
317 A filesystem can optionally be "packed" to conserve space on disk. The
318 packing process concatenates all the revision files in each full shard to
319 create a pack file. The original shard is removed, and reads are
320 redirected to the pack file.
322 With physical addressing, a manifest file is created for each shard which
323 records the indexes of the corresponding revision files in the pack file.
324 The manifest file consists of a list of offsets, one for each revision in
325 the pack file. The offsets are stored as ASCII decimal, and separated by
328 Revision pack files using logical addressing don't use manifest files but
329 appends index data to the revision contents. The revisions inside a pack
330 file will also get interleaved to reduce I/O for typical access patterns.
331 There is no structural difference between packed and non-packed revision
335 Packing revision properties (format 5: SQLite)
336 ---------------------------
338 This was supported by 1.7-dev builds but never included in a blessed release.
340 See r1143829 of this file:
341 http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_fs_fs/structure?view=markup&pathrev=1143829
344 Packing revision properties (format 6+)
345 ---------------------------
347 Similarly to the revision data, packing will concatenate multiple
348 revprops into a single file. Since they are mutable data, we put an
349 upper limit to the size of these files: We will concatenate the data
350 up to the limit and then use a new file for the following revisions.
352 The limit can be set and changed at will in the configuration file.
353 It is 64kB by default. Because a pack file must contain at least one
354 complete property list, files containing just one revision may exceed
357 Furthermore, pack files can be compressed which saves about 75% of
358 disk space. A configuration file flag enables the compression; it is
359 off by default and may be switched on and off at will. The pack size
360 limit is always applied to the uncompressed data. For this reason,
361 the default is 256kB while compression has been enabled.
363 Files are named after their start revision as "<rev>.<counter>" where
364 counter will be increased whenever we rewrite a pack file due to a
365 revprop change. The manifest file contains the list of pack file
366 names, one line for each revision.
368 Many tools track repository global data in revision properties at
369 revision 0. To minimize I/O overhead for those applications, we
370 will never pack that revision, i.e. its data is always being kept
375 Top level: <packed container>
377 We always apply data compression to the pack file - using the
378 SVN_DELTA_COMPRESSION_LEVEL_NONE level if compression is disabled.
379 (Note that compression at SVN_DELTA_COMPRESSION_LEVEL_NONE is not
380 a no-op stream transformation although most of the data will remain
383 container := header '\n' (revprops)+
384 header := start_rev '\n' rev_count '\n' (size '\n')+
386 All numbers in the header are given as ASCII decimals. rev_count
387 is the number of revisions packed into this container. There must
388 be exactly as many "size" and serialized "revprops". The "size"
389 values in the list are the length in bytes of the serialized
390 revprops of the respective revision.
392 Writing to packed revprops
394 The old pack file is being read and the new revprops serialized.
395 If they fit into the same pack file, a temp file with the new
396 content gets written and moved into place just like an non-packed
397 revprop file would. No name change or manifest update required.
399 If they don't fit into the same pack file, i.e. exceed the pack
400 size limit, the pack will be split into 2 or 3 new packs just
401 before and / or after the modified revision.
403 In the current implementation, they will never be merged again.
404 To minimize fragmentation, the initial packing process will only
405 use about 90% of the limit, i.e. leave some room for growth.
407 When a pack file gets split, its counter is being increased
408 creating a new file and leaving the old content in place and
409 available for concurrent readers. Only after the new manifest
410 file got moved into place, will the old pack files be deleted.
412 Write access to revprops is being serialized by the global
413 filesystem write lock. We only need to build a few retries into
414 the reader code to gracefully handle manifest changes and pack
421 A node-rev ID consists of the following three fields:
423 node_revision_id ::= node_id '.' copy_id '.' txn_id
425 At this level, the form of the ID is the same as for BDB - see the
426 section called "ID's" in <../libsvn_fs_base/notes/structure>.
428 In order to support efficient lookup of node-revisions by their IDs
429 and to simplify the allocation of fresh node-IDs during a transaction,
430 we treat the fields of a node-rev ID in new and interesting ways.
432 Within a new transaction:
434 New node-revision IDs assigned within a transaction have a txn-id
435 field of the form "t<txnid>".
437 When a new node-id or copy-id is assigned in a transaction, the ID
438 used is a "_" followed by a base36 number unique to the transaction.
442 Within a revision file, node-revs have a txn-id field of the form
443 "r<rev>/<item_index>", to support easy lookup. See addressing modes
446 During the final phase of a commit, node-revision IDs are rewritten
447 to have repository-wide unique node-ID and copy-ID fields, and to have
448 "r<rev>/<item_index>" txn-id fields.
450 In Format 3 and above, this uniqueness is done by changing a temporary
451 id of "_<base36>" to "<base36>-<rev>". Note that this means that the
452 originating revision of a line of history or a copy can be determined
453 by looking at the node ID.
455 In Format 2 and below, the "current" file contains global base36
456 node-ID and copy-ID counters; during the commit, the counter value is
457 added to the transaction-specific base36 ID, and the value in
458 "current" is adjusted.
460 (It is legal for Format 3 repositories to contain Format 2-style IDs;
461 this just prevents I/O-less node-origin-rev lookup for those nodes.)
463 The temporary assignment of node-ID and copy-ID fields has
464 implications for svn_fs_compare_ids and svn_fs_check_related. The ID
465 _1.0.t1 is not related to the ID _1.0.t2 even though they have the
466 same node-ID, because temporary node-IDs are restricted in scope to
467 the transactions they belong to.
469 There is a lazily created cache mapping from node-IDs to the full
470 node-revision ID where they are created. This is in the node-origins
471 directory; the file name is the node-ID without its last character (or
472 "0" for single-character node IDs) and the contents is a serialized
473 hash mapping from node-ID to node-revision ID. This cache is only
474 used for node-IDs of the pre-Format 3 style.
476 Copy-IDs and copy roots
477 -----------------------
479 Copy-IDs are assigned in the same manner as they are in the BDB
482 * A node-rev resulting from a creation operation (with no copy
483 history) receives the copy-ID of its parent directory.
485 * A node-rev resulting from a copy operation receives a fresh
486 copy-ID, as one would expect.
488 * A node-rev resulting from a modification operation receives a
489 copy-ID depending on whether its predecessor derives from a
490 copy operation or whether it derives from a creation operation
491 with no intervening copies:
493 - If the predecessor does not derive from a copy, the new
494 node-rev receives the copy-ID of its parent directory. If the
495 node-rev is being modified through its created-path, this will
496 be the same copy-ID as the predecessor node-rev has; however,
497 if the node-rev is being modified through a copied ancestor
498 directory (i.e. we are performing a "lazy copy"), this will be
501 - If the predecessor derives from a copy and the node-rev is
502 being modified through its created-path, the new node-rev
503 receives the copy-ID of the predecessor.
505 - If the predecessor derives from a copy and the node-rev is not
506 being modified through its created path, the new node-rev
507 receives a fresh copy-ID. This is called a "soft copy"
508 operation, as distinct from a "true copy" operation which was
509 actually requested through the svn_fs interface. Soft copies
510 exist to ensure that the same <node-ID,copy-ID> pair is not
511 used twice within a transaction.
513 Unlike the BDB implementation, we do not have a "copies" table.
514 Instead, each node-revision record contains a "copyroot" field
515 identifying the node-rev resulting from the true copy operation most
516 proximal to the node-rev. If the node-rev does not itself derive from
517 a copy operation, then the copyroot field identifies the copy of an
518 ancestor directory; if no ancestor directories derive from a copy
519 operation, then the copyroot field identifies the root directory of
525 A revision file contains a concatenation of various kinds of data:
527 * Text and property representations
529 * The changed-path data
530 * Two offsets at the very end (physical addressing only)
531 * Index data (logical addressing only)
532 * Revision / pack file footer (logical addressing only)
534 A representation begins with a line containing either "PLAIN\n" or
535 "DELTA\n" or "DELTA <rev> <item_index> <length>\n", where <rev>,
536 <item_index>, and <length> give the location of the delta base of the
537 representation and the amount of data it contains (not counting the header
538 or trailer). If no base location is given for a delta, the base is the
539 empty stream. After the initial line comes raw svndiff data, followed
540 by a cosmetic trailer "ENDREP\n".
542 If the representation is for the text contents of a directory node,
543 the expanded contents are in hash dump format mapping entry names to
544 "<type> <id>" pairs, where <type> is "file" or "dir" and <id> gives
545 the ID of the child node-rev.
547 If a representation is for a property list, the expanded contents are
548 in the form of a dumped hash map mapping property names to property
551 The marshalling syntax for node-revs is a series of fields terminated
552 by a blank line. Fields have the syntax "<name>: <value>\n", where
553 <name> is a symbolic field name (each symbolic name is used only once
554 in a given node-rev) and <value> is the value data. Unrecognized
555 fields are ignored, for extensibility. The following fields are
558 id The ID of the node-rev
560 pred The ID of the predecessor node-rev
561 count Count of node-revs since the base of the node
562 text "<rev> <item_index> <length> <size> <digest>" for text rep
563 props "<rev> <item_index> <length> <size> <digest>" for props rep
564 <rev> and <item_index> give location of rep
565 <length> gives length of rep, sans header and trailer
566 <size> gives size of expanded rep (*)
567 <digest> gives hex MD5 digest of expanded rep
568 ### in formats >=4, also present:
569 <sha1-digest> gives hex SHA1 digest of expanded rep
570 <uniquifier> see representation_t->uniquifier in fs.h
571 ### Starting from format 8, a special notation "-"
572 can be used for optional values that are not present
573 (<sha1-digest> and <uniquifier>).
574 cpath FS pathname node was created at
575 copyfrom "<rev> <path>" of copyfrom data
576 copyroot "<rev> <created-path>" of the root of this copy
577 minfo-cnt The number of nodes under (and including) this node
578 which have svn:mergeinfo.
579 minfo-here Exists if this node itself has svn:mergeinfo.
581 (*) Earlier versions of this document would state that <size> may be 0
582 if the actual value matches <length>. This is only true for property
583 and directory representations and should be avoided in general. File
584 representations may not be handled correctly by SVN before 1.7.20,
585 1.8.12 and 1.9.0, if they have 0 <size> fields for non-empty contents.
586 Releases 1.8.0 through 1.8.11 may have falsely created instances of
587 that (see issue #4554). Finally, 0 <size> fields are only ever legal
588 for DELTA representations if the reconstructed full-text is actually
591 The predecessor of a node-rev crosses both soft and true copies;
592 together with the count field, it allows efficient determination of
593 the base for skip-deltas. The first node-rev of a node contains no
594 "pred" field. A node-revision with no properties may omit the "props"
595 field. A node-revision with no contents (a zero-length file or an
596 empty directory) may omit the "text" field. In a node-revision
597 resulting from a true copy operation, the "copyfrom" field gives the
598 copyfrom data. The "copyroot" field identifies the root node-revision
599 of the copy; it may be omitted if the node-rev is its own copy root
600 (as is the case for node-revs with copy history, and for the root node
601 of revision 0). Copy roots are identified by revision and
602 created-path, not by node-rev ID, because a copy root may be a
603 node-rev which exists later on within the same revision file, meaning
604 its location is not yet known.
606 The changed-path data is represented as a series of changed-path
607 items, each consisting of two lines. The first line has the format
608 "<id> <action> <text-mod> <prop-mod> <mergeinfo-mod> <path>\n",
609 where <id> is the node-rev ID of the new node-rev, <action> is "add",
610 "delete", "replace", or "modify", <text-mod>, <prop-mod>, and
611 <mergeinfo-mod> are "true" or "false" indicating whether the text,
612 properties and/or mergeinfo changed, and <path> is the changed pathname.
613 For deletes, <id> is the node-rev ID of the deleted node-rev, and
614 <text-mod> and <prop-mod> are always "false". The second line has the
615 format "<rev> <path>\n" containing the node-rev's copyfrom information
616 if it has any; if it does not, the second line is blank.
618 Starting with FS format 4, <action> may contain the kind ("file" or
619 "dir") of the node, after a hyphen; for example, an added directory
620 may be represented as "add-dir".
622 Prior to FS format 7, <mergeinfo-mod> flag is not available. It may
623 also be missing in revisions upgraded from pre-f7 formats.
625 In physical addressing mode, at the very end of a rev file is a pair of
626 lines containing "\n<root-offset> <cp-offset>\n", where <root-offset> is
627 the offset of the root directory node revision and <cp-offset> is the
628 offset of the changed-path data.
630 In logical addressing mode, the revision footer has the form
632 <l2p offset> <l2p checksum> <p2l offset> <p2l checksum><terminal byte>
634 The terminal byte contains the length (as plain 8 bit value) of the footer
635 excluding that length byte. The first offset is the start of the log-to-
636 phys index, followed by the digest of the MD5 checksum over its content.
637 The other pair gives the same of for the phys-to-log index.
639 All numbers in the rev file format are unsigned and are represented as
645 A transaction directory has the following layout:
647 props Transaction props
648 props-final Final transaction props (optional)
649 next-ids Next temporary node-ID and copy-ID
650 changes Changed-path information so far
651 node.<nid>.<cid> New node-rev data for node
652 node.<nid>.<cid>.props Props for new node-rev, if changed
653 node.<nid>.<cid>.children Directory contents for node-rev
654 <sha1> Text representation of that sha1
656 In FS formats 1 and 2, it also contains:
658 rev Prototype rev file with new text reps
659 rev-lock Lockfile for writing to the above
661 (In newer formats, these files are in the txn-protorevs/ directory.)
663 In format 7+ logical addressing mode, it contains two additional index
664 files (see structure-indexes for a detailed description) and one more
667 itemidx Next item_index value as decimal integer
668 index.l2p Log-to-phys proto-index
669 index.p2l Phys-to-log proto-index
671 The prototype rev file is used to store the text representations as
672 they are received from the client. To ensure that only one client is
673 writing to the file at a given time, the "rev-lock" file is locked for
674 the duration of each write.
676 The three kinds of props files are all in hash dump format. The "props"
677 file will always be present. The "node.<nid>.<cid>.props" file will
678 only be present if the node-rev properties have been changed. The
679 "props-final" only exists while converting the transaction into a revision.
682 The <sha1> files have been introduced in FS format 6. Their content
683 is that of text rep references: "<rev> <item_offset> <length> <size> <digest>"
684 They will be written for text reps in the current transaction and be
685 used to eliminate duplicate reps within that transaction.
687 The "next-ids" file contains a single line "<next-temp-node-id>
688 <next-temp-copy-id>\n" giving the next temporary node-ID and copy-ID
689 assignments (without the leading underscores). The next node-ID is
690 also used as a uniquifier for representations which may share the same
693 The "children" file for a node-revision begins with a copy of the hash
694 dump representation of the directory entries from the old node-rev (or
695 a dump of the empty hash for new directories), and then an incremental
696 hash dump entry for each change made to the directory.
698 The "changes" file contains changed-path entries in the same form as
699 the changed-path entries in a rev file, except that <id> and <action>
700 may both be "reset" (in which case <text-mod> and <prop-mod> are both
701 always "false") to indicate that all changes to a path should be
702 considered undone. Reset entries are only used during the final merge
703 phase of a transaction. Actions in the "changes" file always contain
704 a node kind, even if the FS format is older than format 4.
706 The node-rev files have the same format as node-revs in a revision
707 file, except that the "text" and "props" fields are augmented as
710 * The "props" field may have the value "-1" if properties have
711 been changed and are contained in a "props" file within the
712 node-rev subdirectory.
714 * For directory node-revs, the "text" field may have the value
715 "-1" if entries have been changed and are contained in a
716 "contents" file in the node-rev subdirectory.
718 * For the directory node-rev representing the root of the
719 transaction, the "is-fresh-txn-root" field indicates that it has
720 not been made mutable yet (see Issue #2608).
722 * For file node-revs, the "text" field may have the value "-1
723 <offset> <length> <size> <digest>" if the text representation is
724 within the prototype rev file.
726 * The "copyroot" field may have the value "-1 <created-path>" if the
727 copy root of the node-rev is part of the transaction in process.
732 Locks in FSFS are stored in serialized hash format in files whose
733 names are MD5 digests of the FS path which the lock is associated
734 with. For the purposes of keeping directory inode usage down, these
735 digest files live in subdirectories of the main lock directory whose
736 names are the first 3 characters of the digest filename.
738 Also stored in the digest file for a given FS path are pointers to
739 other digest files which contain information associated with other FS
740 paths that are beneath our path (an immediate child thereof, or a
741 grandchild, or a great-grandchild, ...).
743 To answer the question, "Does path FOO have a lock associated with
744 it?", one need only generate the MD5 digest of FOO's
745 absolute-in-the-FS path (say, 3b1b011fed614a263986b5c4869604e8), look
746 for a file located like so:
748 /path/to/repos/locks/3b1/3b1b011fed614a263986b5c4869604e8
750 And then see if that file contains lock information.
752 To inquire about locks on children of the path FOO, you would
753 reference the same path as above, but look for a list of children in
754 that file (instead of lock information). Children are listed as MD5
755 digests, too, so you would simply iterate over those digests and
756 consult the files they reference for lock information.
762 Format 7 introduces logical addressing that requires item indexes
763 to be translated / mapped to physical rev / pack file offsets.
764 These indexes are appended to the respective rev / pack file.
766 The indexes map (revision number, item-index) pairs to absolute file offsets
767 and absolute file offsets to (revision number, item-index, item metadata).
769 Details of the binary format used by these index files can be
770 found in structure-indexes.