2 .\" Copyright (c) 2010 The FreeBSD Foundation
3 .\" All rights reserved.
5 .\" This documentation was written by CK Software GmbH under sponsorship from
6 .\" the FreeBSD Foundation.
8 .\" Redistribution and use in source and binary forms, with or without
9 .\" modification, are permitted provided that the following conditions
11 .\" 1. Redistributions of source code must retain the above copyright
12 .\" notice, this list of conditions and the following disclaimer.
13 .\" 2. Redistributions in binary form must reproduce the above copyright
14 .\" notice, this list of conditions and the following disclaimer in the
15 .\" documentation and/or other materials provided with the distribution.
17 .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
18 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
19 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
20 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
21 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
22 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
23 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
24 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
25 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
26 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
36 .Nd "network subsystem virtualization infrastructure"
39 .Cd "options VNET_DEBUG"
43 .\"------------------------------------------------------------
44 .Ss "Constants and Global Variables"
50 .Vt extern struct vnet *vnet0;
51 .\"------------------------------------------------------------
52 .Ss "Variable Declaration"
70 #define V_name VNET(name)
72 .\" ------------------------------------------------------------
73 .Ss "Virtual Instance Selection"
105 .Fo VNET_ITERATOR_DECL
112 .\" ------------------------------------------------------------
116 .Fn VNET_LIST_RUNLOCK
117 .Fn VNET_LIST_RLOCK_NOSLEEP
118 .Fn VNET_LIST_RUNLOCK_NOSLEEP
119 .\" ------------------------------------------------------------
120 .Ss "Startup and Teardown Functions"
134 .Fa "enum sysinit_sub_id subsystem"
135 .Fa "enum sysinit_elem_order order"
136 .Fa "sysinit_cfunc_t func"
137 .Fa "const void *arg"
142 .Fa "enum sysinit_sub_id subsystem"
143 .Fa "enum sysinit_elem_order order"
144 .Fa "sysinit_cfunc_t func"
145 .Fa "const void *arg"
147 .\" ------------------------------------------------------------
150 .Fo VNET_GLOBAL_EVENTHANDLER_REGISTER
151 .Fa "const char *name"
157 .Fo VNET_GLOBAL_EVENTHANDLER_REGISTER_TAG
158 .Fa "eventhandler_tag tag"
159 .Fa "const char *name"
164 .\" ------------------------------------------------------------
165 .Ss "Sysctl Handling"
167 .Fa parent nbr name access ptr val descr
170 .Fa parent nbr name access ptr arg handler fmt descr
172 .Fo SYSCTL_VNET_STRING
173 .Fa parent nbr name access arg len descr
175 .Fo SYSCTL_VNET_STRUCT
176 .Fa parent nbr name access ptr type descr
179 .Fa parent nbr name access ptr val descr
184 .\" ------------------------------------------------------------
187 is the name of a technique to virtualize the network stack.
188 The basic idea is to change global resources most notably variables into
189 per network stack resources and have functions, sysctls, eventhandlers,
190 etc. access and handle them in the context of the correct instance.
191 Each (virtual) network stack is attached to a
195 being the unrestricted default network stack of the base system.
197 The global defines for
203 to access internals for debugging reasons.
204 .\" ------------------------------------------------------------
205 .Ss "Variable Declaration"
207 Variables are virtualized by using the
209 macro rather than writing them out as
211 One can still use static initialization or storage class specifiers, e.g.,
213 .Dl Li static VNET_DEFINE(int, foo) = 1;
215 .Dl Li static VNET_DEFINE(SLIST_HEAD(, bar), bars);
217 Static initialization is not possible when the virtualized variable
218 would need to be referenced, e.g., with
219 .Dq TAILQ_HEAD_INITIALIZER() .
222 based initialization function must be used.
224 External variables have to be declared using the
227 In either case the convention is to define another macro,
228 that is then used throughout the implementation to access that variable.
229 The variable name is usually prefixed by
231 to express that it is virtualized.
234 macro will then translate accesses to that variable to the copy of the
235 currently selected instance (see the
236 .Sx "Virtual instance selection"
239 .Dl Li #define V_name VNET(name)
242 Do not confuse this with the convention used by
247 macro returns the offset within the memory region of the virtual network
249 It is usually only used with
252 .\" ------------------------------------------------------------
253 .Ss "Virtual Instance Selection"
255 There are three different places where the current virtual
256 network stack pointer is stored and can be taken from:
257 .Bl -enum -offset indent
261 .Dl "(struct prison *)->pr_vnet"
263 For convenience the following macros are provided:
264 .Bd -literal -compact -offset indent
265 .Fn CRED_TO_VNET "struct ucred *"
266 .Fn TD_TO_VNET "struct thread *"
267 .Fn P_TO_VNET "struct proc *"
272 .Dl "(struct socket *)->so_vnet"
276 .Dl "(struct ifnet *)->if_vnet"
280 In addition the currently active instance is cached in
281 .Dq "curthread->td_vnet"
282 which is usually only accessed through the
287 To set the correct context of the current virtual network instance, use the
290 .Fn CURVNET_SET_QUIET
293 .Fn CURVNET_SET_QUIET
294 version will not record vnet recursions in case the kernel was compiled
296 .Cd "options VNET_DEBUG"
297 and should thus only be used in well known cases, where recursion is
299 Both macros will save the previous state on the stack and it must be restored
305 As the previous state is saved on the stack, you cannot have multiple
307 calls in the same block.
310 As the previous state is saved on the stack, a
312 call has to be in the same block as the
314 call or in a subblock with the same idea of the saved instances as the
318 As each macro is a set of operations and, as previously explained, cannot
319 be put into its own block when defined, one cannot conditionally set
320 the current vnet context.
324 .Bd -literal -offset indent
330 .Bd -literal -offset indent
338 Sometimes one needs to loop over all virtual instances, for example to update
339 virtual from global state, to run a function from a
341 for each instance, etc.
343 .Fn VNET_ITERATOR_DECL
347 The former macro defines the variable that iterates over the loop,
348 and the latter loops over all of the virtual network stack instances.
351 for how to savely traverse the list of all virtual instances.
356 macro provides a safe way to check whether the currently active instance is the
357 unrestricted default network stack of the base system
363 macro provides a way to conditionally add assertions that are only active with
365 compiled in and either
366 .Cd "options VNET_DEBUG"
368 .Cd "options INVARIANTS"
370 It uses the same semantics as
372 .\" ------------------------------------------------------------
375 For public access to the list of virtual network stack instances
378 macro, read locks are provided.
379 Macros are used to abstract from the actual type of the locks.
380 If a caller may sleep while traversing the list, it must use the
383 .Fn VNET_LIST_RUNLOCK
385 Otherwise, the caller can use
386 .Fn VNET_LIST_RLOCK_NOSLEEP
388 .Fn VNET_LIST_RUNLOCK_NOSLEEP .
389 .\" ------------------------------------------------------------
390 .Ss "Startup and Teardown Functions"
392 To start or tear down a virtual network stack instance the internal
397 are provided and called from the jail framework.
398 They run the publicly provided methods to handle network stack
399 startup and teardown.
401 For public control, the system startup interface has been enhanced
402 to not only handle a system boot but to also handle a virtual
403 network stack startup and teardown.
404 To the base system the
408 macros look exactly as if there were no virtual network stack.
411 is not compiled in they are compiled to the standard
414 In addition to that they are run for each virtual network stack
415 when starting or, in reverse order, when shutting down.
416 .\" ------------------------------------------------------------
419 Eventhandlers can be handled in two ways:
421 .Bl -enum -offset indent -compact
425 returned in each virtual instance and properly free the eventhandlers
426 on teardown using those, or
428 use one eventhandler that will iterate over all virtual network
432 For the first case one can just use the normal
434 functions, while for the second case the
435 .Fn VNET_GLOBAL_EVENTHANDLER_REGISTER
437 .Fn VNET_GLOBAL_EVENTHANDLER_REGISTER_TAG
440 .Fn VNET_GLOBAL_EVENTHANDLER_REGISTER_TAG
441 takes an extra first argument that will carry the
444 Eventhandlers registered with either of these will not run
448 will be called from an internal iterator function for each vnet.
449 Both macros can only be used for eventhandlers that do not take
450 additional arguments, as the variadic arguments from an
451 .Xr EVENTHANDLER_INVOKE 9
452 call will be ignored.
453 .\" ------------------------------------------------------------
454 .Ss "Sysctl Handling"
458 can be virtualized by using one of the
462 They take the same arguments as the standard
464 functions, with the only difference, that the
466 argument has to be passed as
470 so that the variable can be selected from the correct memory
471 region of the virtual network stack instance of the caller.
473 For the very rare case a sysctl handler function would want to
477 .Fn VNET_SYSCTL_ARG req arg1
478 is provided that will translate the
480 argument to the correct memory address in the virtual network stack
481 context of the caller.
482 .\" ------------------------------------------------------------
492 The virtual network stack implementation first appeared in
495 This manual page was written by
496 .An Bjoern A. Zeeb, CK Software GmbH,
497 under sponsorship from the FreeBSD Foundation.