1 .\" Copyright (c) 1996-1999 Whistle Communications, Inc.
2 .\" All rights reserved.
4 .\" Subject to the following obligations and disclaimer of warranty, use and
5 .\" redistribution of this software, in source or object code forms, with or
6 .\" without modifications are expressly permitted by Whistle Communications;
7 .\" provided, however, that:
8 .\" 1. Any and all reproductions of the source or object code must include the
9 .\" copyright notice above and the following disclaimer of warranties; and
10 .\" 2. No rights are granted, in any manner or form, to use Whistle
11 .\" Communications, Inc. trademarks, including the mark "WHISTLE
12 .\" COMMUNICATIONS" on advertising, endorsements, or otherwise except as
13 .\" such appears in the above copyright notice or in the software.
15 .\" THIS SOFTWARE IS BEING PROVIDED BY WHISTLE COMMUNICATIONS "AS IS", AND
16 .\" TO THE MAXIMUM EXTENT PERMITTED BY LAW, WHISTLE COMMUNICATIONS MAKES NO
17 .\" REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, REGARDING THIS SOFTWARE,
18 .\" INCLUDING WITHOUT LIMITATION, ANY AND ALL IMPLIED WARRANTIES OF
19 .\" MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.
20 .\" WHISTLE COMMUNICATIONS DOES NOT WARRANT, GUARANTEE, OR MAKE ANY
21 .\" REPRESENTATIONS REGARDING THE USE OF, OR THE RESULTS OF THE USE OF THIS
22 .\" SOFTWARE IN TERMS OF ITS CORRECTNESS, ACCURACY, RELIABILITY OR OTHERWISE.
23 .\" IN NO EVENT SHALL WHISTLE COMMUNICATIONS BE LIABLE FOR ANY DAMAGES
24 .\" RESULTING FROM OR ARISING OUT OF ANY USE OF THIS SOFTWARE, INCLUDING
25 .\" WITHOUT LIMITATION, ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
26 .\" PUNITIVE, OR CONSEQUENTIAL DAMAGES, PROCUREMENT OF SUBSTITUTE GOODS OR
27 .\" SERVICES, LOSS OF USE, DATA OR PROFITS, HOWEVER CAUSED AND UNDER ANY
28 .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
29 .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
30 .\" THIS SOFTWARE, EVEN IF WHISTLE COMMUNICATIONS IS ADVISED OF THE POSSIBILITY
33 .\" Authors: Julian Elischer <julian@whistle.com>
34 .\" Archie Cobbs <archie@whistle.com>
37 .\" $Whistle: netgraph.4,v 1.7 1999/01/28 23:54:52 julian Exp $
44 .Nd graph based kernel networking subsystem
48 system provides a uniform and modular system for the implementation
49 of kernel objects which perform various networking functions. The objects,
52 can be arranged into arbitrarily complicated graphs. Nodes have
54 which are used to connect two nodes together, forming the edges in the graph.
55 Nodes communicate along the edges to process data, implement protocols, etc.
59 is to supplement rather than replace the existing kernel networking
60 infrastructure. It provides:
62 .Bl -bullet -compact -offset 2n
64 A flexible way of combining protocol and link level drivers
66 A modular way to implement new protocols
68 A common framework for kernel entities to inter-communicate
70 A reasonably fast, kernel-based implementation
73 The most fundamental concept in
77 All nodes implement a number of predefined methods which allow them
78 to interact with other nodes in a well defined manner.
82 which is a static property of the node determined at node creation time.
83 A node's type is described by a unique ASCII type name.
84 The type implies what the node does and how it may be connected
87 In object-oriented language, types are classes and nodes are instances
88 of their respective class. All node types are subclasses of the generic node
89 type, and hence inherit certain common functionality and capabilities
90 (e.g., the ability to have an ASCII name).
92 Nodes may be assigned a globally unique ASCII name which can be
93 used to refer to the node.
94 The name must not contain the characters ``.'' or ``:'' and is limited to
96 characters (including NUL byte).
98 Each node instance has a unique
100 which is expressed as a 32-bit hex value. This value may be used to
101 refer to a node when there is no ASCII name assigned to it.
103 Nodes are connected to other nodes by connecting a pair of
105 one from each node. Data flows bidirectionally between nodes along
106 connected pairs of hooks. A node may have as many hooks as it
107 needs, and may assign whatever meaning it wants to a hook.
109 Hooks have these properties:
111 .Bl -bullet -compact -offset 2n
113 A hook has an ASCII name which is unique among all hooks
114 on that node (other hooks on other nodes may have the same name).
115 The name must not contain a ``.'' or a ``:'' and is
118 characters (including NUL byte).
120 A hook is always connected to another hook. That is, hooks are
121 created at the time they are connected, and breaking an edge by
122 removing either hook destroys both hooks.
125 A node may decide to assign special meaning to some hooks.
126 For example, connecting to the hook named ``debug'' might trigger
127 the node to start sending debugging information to that hook.
129 Two types of information flow between nodes: data messages and
130 control messages. Data messages are passed in mbuf chains along the edges
131 in the graph, one edge at a time. The first mbuf in a chain must have the
133 flag set. Each node decides how to handle data coming in on its hooks.
135 Control messages are type-specific structures sent from one node directly
136 to an arbitrary other node. There are two ways to address such a message. If
137 there is a sequence of edges connecting the two nodes, the message
138 may be ``source routed'' by specifying the corresponding sequence
139 of hooks as the destination address for the message (relative
140 addressing). Otherwise, the recipient node global ASCII name
141 (or equivalent ID based name) is used as the destination address
142 for the message (absolute addressing). The two types of addressing
143 may be combined, by specifying an absolute start node and a sequence
146 Messages often represent commands that are followed by a reply message
147 in the reverse direction. To facilitate this, the recipient of a
148 control message is supplied with a ``return address'' that is suitable
149 for addressing a reply.
151 Each control message contains a 32 bit value called a
153 indicating the type of the message, i.e., how to interpret it.
154 Typically each type defines a unique typecookie for the messages
155 that it understands. However, a node may choose to recognize and
156 implement more than one type of message.
157 .Sh Netgraph is Functional
158 In order to minimize latency, most
160 operations are functional.
161 That is, data and control messages are delivered by making function
162 calls rather than by using queues and mailboxes. For example, if node
163 A wishes to send a data mbuf to neighboring node B, it calls the
166 data delivery function. This function in turn locates
167 node B and calls B's ``receive data'' method. While this mode of operation
168 results in good performance, it has a few implications for node
171 .Bl -bullet -compact -offset 2n
173 Whenever a node delivers a data or control message, the node
174 may need to allow for the possibility of receiving a returning message
175 before the original delivery function call returns.
177 Netgraph nodes and support routines generally run at
179 However, some nodes may want to send data and control messages
180 from a different priority level. Netgraph supplies queueing routines which
181 utilize the NETISR system to move message delivery to
183 Note that messages are always received at
186 It's possible for an infinite loop to occur if the graph contains cycles.
189 So far, these issues have not proven problematical in practice.
190 .Sh Interaction With Other Parts of the Kernel
191 A node may have a hidden interaction with other components of the
192 kernel outside of the
194 subsystem, such as device hardware,
195 kernel protocol stacks, etc. In fact, one of the benefits of
197 is the ability to join disparate kernel networking entities together in a
198 consistent communication framework.
200 An example is the node type
202 which is both a netgraph node and a
204 BSD socket in the protocol family
206 Socket nodes allow user processes to participate in
208 Other nodes communicate with socket nodes using the usual methods, and the
209 node hides the fact that it is also passing information to and from a
210 cooperating user process.
212 Another example is a device driver that presents
213 a node interface to the hardware.
215 Nodes are notified of the following actions via function calls
216 to the following node methods (all at
218 and may accept or reject that action (by returning the appropriate
221 .It Creation of a new node
222 The constructor for the type is called. If creation of a new node is
223 allowed, the constructor must call the generic node creation
224 function (in object-oriented terms, the superclass constructor)
225 and then allocate any special resources it needs. For nodes that
226 correspond to hardware, this is typically done during the device
227 attach routine. Often a global ASCII name corresponding to the
228 device name is assigned here as well.
229 .It Creation of a new hook
230 The hook is created and tentatively
231 linked to the node, and the node is told about the name that will be
232 used to describe this hook. The node sets up any special data structures
233 it needs, or may reject the connection, based on the name of the hook.
234 .It Successful connection of two hooks
235 After both ends have accepted their
236 hooks, and the links have been made, the nodes get a chance to
237 find out who their peer is across the link and can then decide to reject
238 the connection. Tear-down is automatic.
239 .It Destruction of a hook
240 The node is notified of a broken connection. The node may consider some hooks
241 to be critical to operation and others to be expendable: the disconnection
242 of one hook may be an acceptable event while for another it
243 may effect a total shutdown for the node.
244 .It Shutdown of a node
245 This method allows a node to clean up
246 and to ensure that any actions that need to be performed
247 at this time are taken. The method must call the generic (i.e., superclass)
248 node destructor to get rid of the generic components of the node.
249 Some nodes (usually associated with a piece of hardware) may be
251 in that a shutdown breaks all edges and resets the node,
252 but doesn't remove it, in which case the generic destructor is not called.
254 .Sh Sending and Receiving Data
255 Three other methods are also supported by all nodes:
257 .It Receive data message
258 An mbuf chain is passed to the node.
259 The node is notified on which hook the data arrived,
260 and can use this information in its processing decision.
261 The node must must always
263 the mbuf chain on completion or error, or pass it on to another node
264 (or kernel module) which will then be responsible for freeing it.
266 In addition to the mbuf chain itself there is also a pointer to a
267 structure describing meta-data about the message
268 (e.g. priority information). This pointer may be
270 if there is no additional information. The format for this information is
273 The memory for meta-data must allocated via
277 As with the data itself, it is the receiver's responsibility to
279 the meta-data. If the mbuf chain is freed the meta-data must
280 be freed at the same time. If the meta-data is freed but the
281 real data on is passed on, then a
283 pointer must be substituted.
285 The receiving node may decide to defer the data by queueing it in the
287 NETISR system (see below).
289 The structure and use of meta-data is still experimental, but is presently used in
290 frame-relay to indicate that management packets should be queued for transmission
291 at a higher priority than data packets. This is required for
292 conformance with Frame Relay standards.
294 .It Receive queued data message
295 Usually this will be the same function as
296 .Em Receive data message.
297 This is the entry point called when a data message is being handed to
298 the node after having been queued in the NETISR system.
299 This allows a node to decide in the
300 .Em Receive data message
301 method that a message should be deferred and queued,
302 and be sure that when it is processed from the queue,
303 it will not be queued again.
304 .It Receive control message
305 This method is called when a control message is addressed to the node.
306 A return address is always supplied, giving the address of the node
307 that originated the message so a reply message can be sent anytime later.
309 It is possible for a synchronous reply to be made, and in fact this
310 is more common in practice.
311 This is done by setting a pointer (supplied as an extra function parameter)
312 to point to the reply.
313 Then when the control message delivery function returns,
314 the caller can check if this pointer has been made non-NULL,
315 and if so then it points to the reply message allocated via
317 and containing the synchronous response. In both directions,
318 (request and response) it is up to the
319 receiver of that message to
321 the control message buffer. All control messages and replies are
328 Much use has been made of reference counts, so that nodes being
329 free'd of all references are automatically freed, and this behaviour
330 has been tested and debugged to present a consistent and trustworthy
331 framework for the ``type module'' writer to use.
335 framework provides an unambiguous and simple to use method of specifically
336 addressing any single node in the graph. The naming of a node is
337 independent of its type, in that another node, or external component
338 need not know anything about the node's type in order to address it so as
339 to send it a generic message type. Node and hook names should be
340 chosen so as to make addresses meaningful.
342 Addresses are either absolute or relative. An absolute address begins
343 with a node name, (or ID), followed by a colon, followed by a sequence of hook
344 names separated by periods. This addresses the node reached by starting
345 at the named node and following the specified sequence of hooks.
346 A relative address includes only the sequence of hook names, implicitly
347 starting hook traversal at the local node.
349 There are a couple of special possibilities for the node name.
350 The name ``.'' (referred to as ``.:'') always refers to the local node.
351 Also, nodes that have no global name may be addressed by their ID numbers,
352 by enclosing the hex representation of the ID number within square brackets.
353 Here are some examples of valid netgraph addresses:
354 .Bd -literal -offset 4n -compact
363 Consider the following set of nodes might be created for a site with
364 a single physical frame relay line having two active logical DLCI channels,
365 with RFC-1490 frames on DLCI 16 and PPP frames over DLCI 20:
368 [type SYNC ] [type FRAME] [type RFC1490]
369 [ "Frame1" ](uplink)<-->(data)[<un-named>](dlci16)<-->(mux)[<un-named> ]
370 [ A ] [ B ](dlci20)<---+ [ C ]
377 One could always send a control message to node C from anywhere
379 .Em "Frame1:uplink.dlci16" .
381 .Em "Frame1:uplink.dlci20"
382 could reliably be used to reach node D, and node A could refer
387 Conversely, B can refer to A as
391 could be used by both nodes C and D to address a message to node A.
393 Note that this is only for
394 .Em control messages .
395 Data messages are routed one hop at a time, by specifying the departing
396 hook, with each node making the next routing decision. So when B
397 receives a frame on hook
399 it decodes the frame relay header to determine the DLCI,
400 and then forwards the unwrapped frame to either C or D.
402 A similar graph might be used to represent multi-link PPP running
406 [ type BRI ](B1)<--->(link1)[ type MPP ]
407 [ "ISDN1" ](B2)<--->(link2)[ (no name) ]
412 +->(switch)[ type Q.921 ](term1)<---->(datalink)[ type Q.931 ]
413 [ (no name) ] [ (no name) ]
415 .Sh Netgraph Structures
416 Interesting members of the node and hook structures are shown below:
419 char *name; /* Optional globally unique name */
420 void *private; /* Node implementation private info */
421 struct ng_type *type; /* The type of this node */
422 int refs; /* Number of references to this struct */
423 int numhooks; /* Number of connected hooks */
424 hook_p hooks; /* Linked list of (connected) hooks */
426 typedef struct ng_node *node_p;
429 char *name; /* This node's name for this hook */
430 void *private; /* Node implementation private info */
431 int refs; /* Number of references to this struct */
432 struct ng_node *node; /* The node this hook is attached to */
433 struct ng_hook *peer; /* The other hook in this connected pair */
434 struct ng_hook *next; /* Next in list of hooks for this node */
436 typedef struct ng_hook *hook_p;
439 The maintenance of the name pointers, reference counts, and linked list
440 of hooks for each node is handled automatically by the
443 Typically a node's private info contains a back-pointer to the node or hook
444 structure, which counts as a new reference that must be registered by
448 From a hook you can obtain the corresponding node, and from
449 a node the list of all active hooks.
451 Node types are described by this structure:
454 u_int32_t version; /* Must equal NG_VERSION */
455 const char *name; /* Unique type name */
457 /* Module event handler */
458 modeventhand_t mod_event; /* Handle load/unload (optional) */
461 int (*constructor)(node_p *node); /* Create a new node */
463 /** Methods using the node **/
464 int (*rcvmsg)(node_p node, /* Receive control message */
465 struct ng_mesg *msg, /* The message */
466 const char *retaddr, /* Return address */
467 struct ng_mesg **resp); /* Synchronous response */
468 int (*shutdown)(node_p node); /* Shutdown this node */
469 int (*newhook)(node_p node, /* create a new hook */
470 hook_p hook, /* Pre-allocated struct */
471 const char *name); /* Name for new hook */
473 /** Methods using the hook **/
474 int (*connect)(hook_p hook); /* Confirm new hook attachment */
475 int (*rcvdata)(hook_p hook, /* Receive data on a hook */
476 struct mbuf *m, /* The data in an mbuf */
477 meta_p meta); /* Meta-data, if any */
478 int (*disconnect)(hook_p hook); /* Notify disconnection of hook */
482 Control messages have the following structure:
484 #define NG_CMDSTRLEN 15 /* Max command string (16 with null) */
488 u_char version; /* Must equal NG_VERSION */
489 u_char spare; /* Pad to 2 bytes */
490 u_short arglen; /* Length of cmd/resp data */
491 u_long flags; /* Message status flags */
492 u_long token; /* Reply should have the same token */
493 u_long typecookie; /* Node type understanding this message */
494 u_long cmd; /* Command identifier */
495 u_char cmdstr[NG_CMDSTRLEN+1]; /* Cmd string (for debug) */
497 char data[0]; /* Start of cmd/resp data */
500 #define NG_VERSION 1 /* Netgraph version */
501 #define NGF_ORIG 0x0000 /* Command */
502 #define NGF_RESP 0x0001 /* Response */
505 Control messages have the fixed header shown above, followed by a
506 variable length data section which depends on the type cookie
507 and the command. Each field is explained below:
510 Indicates the version of netgraph itself. The current version is
513 This is the length of any extra arguments, which begin at
516 Indicates whether this is a command or a response control message.
520 is a means by which a sender can match a reply message to the
521 corresponding command message; the reply always has the same token.
524 The corresponding node type's unique 32-bit value.
525 If a node doesn't recognize the type cookie it must reject the message
529 Each type should have an include file that defines the commands,
530 argument format, and cookie for its own messages.
532 insures that the same header file was included by both sender and
533 receiver; when an incompatible change in the header file is made,
537 The de facto method for generating unique type cookies is to take the
538 seconds from the epoch at the time the header file is written
540 .Dv "date -u +'%s'" ")."
542 There is a predefined typecookie
543 .Dv NGM_GENERIC_COOKIE
544 for the ``generic'' node type, and
545 a corresponding set of generic messages which all nodes understand.
546 The handling of these messages is automatic.
548 The identifier for the message command. This is type specific,
549 and is defined in the same header file as the typecookie.
551 Room for a short human readable version of ``command'' (for debugging
555 Some modules may choose to implement messages from more than one
556 of the header files and thus recognize more than one type cookie.
557 .Sh Generic Control Messages
558 There are a number of standard predefined messages that will work
559 for any node, as they are supported directly by the framework itself.
562 along with the basic layout of messages and other similar information.
565 Connect to another node, using the supplied hook names on either end.
567 Construct a node of the given type and then connect to it using the
570 The target node should disconnect from all its neighbours and shut down.
571 Persistent nodes such as those representing physical hardware
572 might not disappear from the node namespace, but only reset themselves.
573 The node must disconnect all of its hooks.
574 This may result in neighbors shutting themselves down, and possibly a
575 cascading shutdown of the entire connected graph.
577 Assign a name to a node. Nodes can exist without having a name, and this
578 is the default for nodes created using the
580 method. Such nodes can only be addressed relatively or by their ID number.
582 Ask the node to break a hook connection to one of its neighbours.
583 Both nodes will have their ``disconnect'' method invoked.
584 Either node may elect to totally shut down as a result.
586 Asks the target node to describe itself. The four returned fields
587 are the node name (if named), the node type, the node ID and the
588 number of hooks attached. The ID is an internal number unique to that node.
590 This returns the information given by
593 includes an array of fields describing each link, and the description for
594 the node at the far end of that link.
596 This returns an array of node descriptions (as for
598 where each entry of the array describes a named node.
599 All named nodes will be described.
603 except that all nodes are listed regardless of whether they have a name or not.
605 This returns a list of all currently installed netgraph types.
606 .It Dv NGM_TEXT_STATUS
607 The node may return a text formatted status message.
608 The status information is determined entirely by the node type.
609 It is the only "generic" message
610 that requires any support within the node itself and as such the node may
611 elect to not support this message. The text response must be less than
613 bytes in length (presently 1024). This can be used to return general
614 status information in human readable form.
617 Data moving through the
619 system can be accompanied by meta-data that describes some
620 aspect of that data. The form of the meta-data is a fixed header,
621 which contains enough information for most uses, and can optionally
622 be supplemented by trailing
624 structures, which contain a
626 (see the section on control messages), an identifier, a length and optional
627 data. If a node does not recognize the cookie associated with an option,
628 it should ignore that option.
630 Meta data might include such things as priority, discard eligibility,
631 or special processing requirements. It might also mark a packet for
632 debug status, etc. The use of meta-data is still experimental.
636 code may either be statically compiled
637 into the kernel or else loaded dynamically as a KLD via
639 In the former case, include
640 .Bd -literal -offset 4n -compact
645 in your kernel configuration file. You may also include selected
646 node types in the kernel compilation, for example:
647 .Bd -literal -offset 4n -compact
650 options NETGRAPH_SOCKET
651 options NETGRAPH_ECHO
657 subsystem is loaded, individual node types may be loaded at any time
662 knows how to automatically do this; when a request to create a new
667 will attempt to load the KLD module
670 Types can also be installed at boot time, as certain device drivers
671 may want to export each instance of the device as a netgraph node.
673 In general, new types can be installed at any time from within the
676 supplying a pointer to the type's
681 .Dv "NETGRAPH_INIT()"
682 macro automates this process by using a linker set.
683 .Sh EXISTING NODE TYPES
684 Several node types currently exist. Each is fully documented
688 The socket type implements two new sockets in the new protocol domain
690 The new sockets protocols are
696 Typically one of each is associated with a socket node.
697 When both sockets have closed, the node will shut down. The
699 socket is used for sending and receiving data, while the
701 socket is used for sending and receiving control messages.
702 Data and control messages are passed using the
707 .Dv struct sockaddr_ng
711 Responds only to generic messages and is a ``black hole'' for data,
712 Useful for testing. Always accepts new hooks.
715 Responds only to generic messages and always echoes data back through the
716 hook from which it arrived. Returns any non generic messages as their
717 own response. Useful for testing. Always accepts new hooks.
720 This node is useful for ``snooping.'' It has 4 hooks:
726 Data entering from the right is passed to the left and duplicated on
728 and data entering from the left is passed to the right and
733 is sent to the right and data from
738 Encapsulates/de-encapsulates frames encoded according to RFC 1490.
739 Has a hook for the encapsulated packets (``downstream'') and one hook
740 for each protocol (i.e., IP, PPP, etc.).
743 Encapsulates/de-encapsulates Frame Relay frames.
744 Has a hook for the encapsulated packets (``downstream'') and one hook
748 Automatically handles frame relay
749 ``LMI'' (link management interface) operations and packets.
750 Automatically probes and detects which of several LMI standards
751 is in use at the exchange.
754 This node is also a line discipline. It simply converts between mbuf
755 frames and sequential serial data, allowing a tty to appear as a netgraph
756 node. It has a programmable ``hotkey'' character.
759 This node encapsulates and de-encapsulates asynchronous frames
760 according to RFC 1662. This is used in conjunction with the TTY node
761 type for supporting PPP links over asynchronous serial lines.
764 This node is also a system networking interface. It has hooks representing
765 each protocol family (IP, AppleTalk, IPX, etc.) and appears in the output of
767 The interfaces are named
773 Whether a named node exists can be checked by trying to send a control message
777 If it does not exist,
781 All data messages are mbuf chains with the M_PKTHDR flag set.
783 Nodes are responsible for freeing what they allocate.
784 There are three exceptions:
787 Mbufs sent across a data link are never to be freed by the sender.
789 Any meta-data information traveling with the data has the same restriction.
790 It might be freed by any node the data passes through, and a
792 passed onwards, but the caller will never free it.
794 .Dv "NG_FREE_META(meta)"
796 .Dv "NG_FREE_DATA(m, meta)"
797 should be used if possible to free data and meta data (see
801 .Dv ng_send_message()
802 are freed by the callee. As in the case above, the addresses
803 associated with the message are freed by whatever allocated them so the
804 recipient should copy them if it wants to keep that information.
807 .Bl -tag -width xxxxx -compact
808 .It Pa /sys/netgraph/netgraph.h
809 Definitions for use solely within the kernel by
812 .It Pa /sys/netgraph/ng_message.h
813 Definitions needed by any file that needs to deal with
816 .It Pa /sys/netgraph/ng_socket.h
817 Definitions needed to use
820 .It Pa /sys/netgraph/ng_{type}.h
821 Definitions needed to use
824 nodes, including the type cookie definition.
825 .It Pa /modules/netgraph.ko
826 Netgraph subsystem loadable KLD module.
827 .It Pa /modules/ng_{type}.ko
828 Loadable KLD module for node type {type}.
830 .Sh USER MODE SUPPORT
831 There is a library for supporting user-mode programs that wish
832 to interact with the netgraph system. See
836 Two user-mode support programs,
840 are available to assist manual configuration and debugging.
842 There are a few useful techniques for debugging new node types.
843 First, implementing new node types in user-mode first
844 makes debugging easier.
847 node type is also useful for debugging, especially in conjunction with
859 .Xr ng_frame_relay 8 ,
876 system was designed and first implemented at Whistle Communications, Inc.
877 in a version FreeBSD 2.2 customized for the Whistle InterJet.
879 Julian Elischer <julian@whistle.com>, with contributions by
880 Archie Cobbs <archie@whistle.com>.