2 .\" Copyright (c) 2002 Poul-Henning Kamp
3 .\" Copyright (c) 2002 Networks Associates Technology, Inc.
4 .\" All rights reserved.
6 .\" This software was developed for the FreeBSD Project by Poul-Henning Kamp
7 .\" and NAI Labs, the Security Research Division of Network Associates, Inc.
8 .\" under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the
9 .\" DARPA CHATS research program.
11 .\" Redistribution and use in source and binary forms, with or without
12 .\" modification, are permitted provided that the following conditions
14 .\" 1. Redistributions of source code must retain the above copyright
15 .\" notice, this list of conditions and the following disclaimer.
16 .\" 2. Redistributions in binary form must reproduce the above copyright
17 .\" notice, this list of conditions and the following disclaimer in the
18 .\" documentation and/or other materials provided with the distribution.
19 .\" 3. The names of the authors may not be used to endorse or promote
20 .\" products derived from this software without specific prior written
23 .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
24 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
25 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
26 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
27 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
28 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
29 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
30 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
31 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
32 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
42 .Nd modular disk I/O request transformation framework.
44 The GEOM framework provides an infrastructure in which "classes"
45 can perform transformations on disk I/O requests on their path from
46 the upper kernel to the device drivers and back.
48 Transformations in a GEOM context range from the simple geometric
49 displacement performed in typical disk partitioning modules over RAID
50 algorithms and device multipath resolution to full blown cryptographic
51 protection of the stored data.
53 Compared to traditional "volume management", GEOM differs from most
54 and in some cases all previous implementations in the following ways:
58 It is trivially simple to write a new class
59 of transformation and it will not be given stepchild treatment.
61 someone for some reason wanted to mount IBM MVS diskpacks, a class
62 recognizing and configuring their VTOC information would be a trivial
65 GEOM is topologically agnostic.
66 Most volume management implementations
67 have very strict notions of how classes can fit together, very often
68 one fixed hierarchy is provided for instance subdisk - plex -
72 Being extensible means that new transformations are treated no differently
73 than existing transformations.
75 Fixed hierarchies are bad because they make it impossible to express
76 the intent efficiently.
77 In the fixed hierarchy above it is not possible to mirror two
78 physical disks and then partition the mirror into subdisks, instead
79 one is forced to make subdisks on the physical volumes and to mirror
80 these two and two resulting in a much more complex configuration.
81 GEOM on the other hand does not care in which order things are done,
82 the only restriction is that cycles in the graph will not be allowed.
84 .Sh "TERMINOLOGY and TOPOLOGY"
85 GEOM is quite object oriented and consequently the terminology
86 borrows a lot of context and semantics from the OO vocabulary:
88 A "class", represented by the data structure g_class implements one
89 particular kind of transformation.
90 Typical examples are MBR disk
91 partition, BSD disklabel, and RAID5 classes.
93 An instance of a class is called a "geom" and represented by the
94 data structure "g_geom".
95 In a typical i386 FreeBSD system, there
96 will be one geom of class MBR for each disk.
98 A "provider", represented by the data structure "g_provider", is
99 the front gate at which a geom offers service.
100 A provider is "a disk-like thing which appears in /dev" - a logical
102 All providers have three main properties: name, sectorsize and size.
104 A "consumer" is the backdoor through which a geom connects to another
105 geom provider and through which I/O requests are sent.
107 The topological relationship between these entities are as follows:
110 A class has zero or more geom instances.
112 A geom has exactly one class it is derived from.
114 A geom has zero or more consumers.
116 A geom has zero or more providers.
118 A consumer can be attached to zero or one providers.
120 A provider can have zero or more consumers attached.
123 All geoms have a rank-number assigned, which is used to detect and
124 prevent loops in the acyclic directed graph.
129 A geom with no attached consumers has rank=1
131 A geom with attached consumers has a rank one higher than the
132 highest rank of the geoms of the providers its consumers are
135 .Sh "SPECIAL TOPOLOGICAL MANEUVERS"
136 In addition to the straightforward attach, which attaches a consumer
137 to a provider, and detach, which breaks the bond, a number of special
138 topological maneuvers exists to facilitate configuration and to
139 improve the overall flexibility.
142 is a process that happens whenever a new class or new provider
143 is created and it provides the class a chance to automatically configure an
144 instance on providers, which it recognize as its own.
145 A typical example is the MBR disk-partition class which will look for
146 the MBR table in the first sector and if found and validated it will
147 instantiate a geom to multiplex according to the contents of the MBR.
149 A new class will be offered to all existing providers in turn and a new
150 provider will be offered to all classes in turn.
152 Exactly what a class does to recognize if it should accept the offered
153 provider is not defined by GEOM, but the sensible set of options are:
156 Examine specific data structures on the disk.
158 Examine properties like sectorsize or mediasize for the provider.
160 Examine the rank number of the provider's geom.
162 Examine the method name of the provider's geom.
166 is the process by which a provider is removed while
167 it potentially is still being used.
169 When a geom orphans a provider, all future I/O requests will
170 "bounce" on the provider with an error code set by the geom.
172 consumers attached to the provider will receive notification about
173 the orphanization when the eventloop gets around to it, and they
174 can take appropriate action at that time.
176 A geom which came into being as a result of a normal taste operation
177 should selfdestruct unless it has a way to keep functioning lacking
178 the orphaned provider.
179 Geoms like diskslicers should therefore selfdestruct whereas
180 RAID5 or mirror geoms will be able to continue, as long as they do
183 When a provider is orphaned, this does not necessarily result in any
184 immediate change in the topology: any attached consumers are still
185 attached, any opened paths are still open, any outstanding I/O
186 requests are still outstanding.
188 The typical scenario is
189 .Bl -bullet -offset indent -compact
191 A device driver detects a disk has departed and orphans the provider for it.
193 The geoms on top of the disk receive the orphanization event and
194 orphans all their providers in turn.
195 Providers, which are not attached to, will typically self-destruct
197 This process continues in a quasi-recursive fashion until all
198 relevant pieces of the tree has heard the bad news.
200 Eventually the buck stops when it reaches geom_dev at the top
203 Geom_dev will call destroy_dev(9) to stop any more request from
205 It will sleep until all (if any) outstanding I/O requests have
207 It will explicitly close (ie: zero the access counts), a change
208 which will propagate all the way down through the mesh.
209 It will then detach and destroy its geom.
211 The geom whose provider is now attached will destroy the provider,
212 detach and destroy its consumer and destroy its geom.
214 This process percolates all the way down through the mesh, until
215 the cleanup is complete.
218 While this approach seems byzantine, it does provide the maximum
219 flexibility and robustness in handling disappearing devices.
221 The one absolutely crucial detail to be aware is that if the
222 device driver does not return all I/O requests, the tree will
226 is a special case of orphanization used to protect
227 against stale metadata.
228 It is probably easiest to understand spoiling by going through
231 Imagine a disk, "da0" on top of which a MBR geom provides
232 "da0s1" and "da0s2" and on top of "da0s1" a BSD geom provides
233 "da0s1a" through "da0s1e", both the MBR and BSD geoms have
234 autoconfigured based on data structures on the disk media.
235 Now imagine the case where "da0" is opened for writing and those
236 data structures are modified or overwritten: Now the geoms would
237 be operating on stale metadata unless some notification system
238 can inform them otherwise.
240 To avoid this situation, when the open of "da0" for write happens,
241 all attached consumers are told about this, and geoms like
242 MBR and BSD will selfdestruct as a result.
243 When "da0" is closed again, it will be offered for tasting again
244 and if the data structures for MBR and BSD are still there, new
245 geoms will instantiate themselves anew.
247 Now for the fine print:
249 If any of the paths through the MBR or BSD module were open, they
250 would have opened downwards with an exclusive bit rendering it
251 impossible to open "da0" for writing in that case and conversely
252 the requested exclusive bit would render it impossible to open a
253 path through the MBR geom while "da0" is open for writing.
255 From this it also follows that changing the size of open geoms can
256 only be done with their cooperation.
258 Finally: the spoiling only happens when the write count goes from
259 zero to non-zero and the retasting only when the write count goes
260 from non-zero to zero.
263 are a very special operation which allows a new geom
264 to be instantiated between a consumer and a provider attached to
265 each other and to remove it again.
267 To understand the utility of this, imagine a provider with
268 being mounted as a file system.
269 Between the DEVFS geoms consumer and its provider we insert
270 a mirror module which configures itself with one mirror
271 copy and consequently is transparent to the I/O requests
273 We can now configure yet a mirror copy on the mirror geom,
274 request a synchronization, and finally drop the first mirror
276 We have now in essence moved a mounted file system from one
277 disk to another while it was being used.
278 At this point the mirror geom can be deleted from the path
279 again, it has served its purpose.
282 is the process where the administrator issues instructions
283 for a particular class to instantiate itself.
285 ways to express intent in this case, a particular provider can be
286 specified with a level of override forcing for instance a BSD
287 disklabel module to attach to a provider which was not found palatable
288 during the TASTE operation.
290 Finally IO is the reason we even do this: it concerns itself with
291 sending I/O requests through the graph.
294 represented by struct bio, originate at a consumer,
295 are scheduled on its attached provider, and when processed, returned
297 It is important to realize that the struct bio which
298 enters through the provider of a particular geom does not "come
299 out on the other side".
300 Even simple transformations like MBR and BSD will clone the
301 struct bio, modify the clone, and schedule the clone on their
303 Note that cloning the struct bio does not involve cloning the
304 actual data area specified in the IO request.
306 In total four different IO requests exist in GEOM: read, write,
307 delete, and get attribute.
309 Read and write are self explanatory.
311 Delete indicates that a certain range of data is no longer used
312 and that it can be erased or freed as the underlying technology
314 Technologies like flash adaptation layers can arrange to erase
315 the relevant blocks before they will become reassigned and
316 cryptographic devices may want to fill random bits into the
317 range to reduce the amount of data available for attack.
319 It is important to recognize that a delete indication is not a
320 request and consequently there is no guarantee that the data actually
321 will be erased or made unavailable unless guaranteed by specific
323 If "secure delete" semantics are required, a
324 geom should be pushed which converts delete indications into (a
325 sequence of) write requests.
327 Get attribute supports inspection and manipulation
328 of out-of-band attributes on a particular provider or path.
329 Attributes are named by ascii strings and they will be discussed in
330 a separate section below.
332 (stay tuned while the author rests his brain and fingers: more to come.)
334 Several flags are provided for tracing GEOM operations and unlocking
335 protection mechanisms via the
336 .Va kern.geom.debugflags
338 All of these flags are off by default, and great care should be taken in
341 .It 0x01 Pq Dv G_T_TOPOLOGY
342 Provide tracing of topology change events.
343 .It 0x02 Pq Dv G_T_BIO
344 Provide tracing of buffer I/O requests.
345 .It 0x04 Pq Dv G_T_ACCESS
346 Provide tracing of access check controls.
348 .It 0x10 (allow foot shooting)
349 Allow writing to Rank 1 providers.
350 This would, for example, allow the super-user to overwrite the MBR on the root
351 disk or write random sectors elsewhere to a mounted disk.
352 The implications are obvious.
353 .It 0x20 Pq Dv G_T_DETAILS
354 This appears to be unused at this time.
355 .It 0x40 Pq Dv G_F_DISKIOCTL
356 This appears to be unused at this time.
357 .It 0x80 Pq Dv G_F_CTLDUMP
358 Dump contents of gctl requests.
361 This software was developed for the FreeBSD Project by Poul-Henning Kamp
362 and NAI Labs, the Security Research Division of Network Associates, Inc.\&
363 under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the
364 DARPA CHATS research program.
366 The first precursor for GEOM was a gruesome hack to Minix 1.2 and was
368 An earlier attempt to implement a less general scheme
369 in FreeBSD never succeeded.
371 .An "Poul-Henning Kamp" Aq phk@FreeBSD.org
projects / FreeBSD / FreeBSD.git / blob