2 shaharf@voltaire.com, halr@voltaire.com
6 Model of operation: All utilities use direct MAD access to perform their
7 operations. Operations that require QP 0 mads only, may use direct routed
8 mads, and therefore may work even in unconfigured subnets. Almost all
9 utilities can operate without accessing the SM, unless GUID to lid translation
12 Dependencies: Most utilities depend on libibmad and libibumad.
13 All utilities depend on the ib_umad kernel module.
15 Multiple port/Multiple CA support: when no IB device or port is specified
16 (see the "local umad parameters" below), the libibumad library
17 selects the port to use by the following criteria:
18 1. the first port that is ACTIVE.
19 2. if not found, the first port that is UP (physical link up).
21 If a port and/or CA name is specified, the libibumad library
22 attempts to fulfill the user request, and will fail if it is not
25 ibaddr # use the 'best port'
26 ibaddr -C mthca1 # pick the best port from mthca1 only.
27 ibaddr -P 2 # use the second (active/up) port from the
28 first available IB device.
29 ibaddr -C mthca0 -P 2 # use the specified port only.
31 Common options & flags:
32 Most diagnostics take the following flags. The exact list of supported
33 flags per utility can be found in the usage message and can be shown
34 using util_name -h syntax.
37 -d raise the IB debugging level. May be used
38 several times (-ddd or -d -d -d).
39 -e show umad send receive errors (timeouts and others)
40 -h show the usage message
41 -v increase the application verbosity level.
42 May be used several times (-vv or -v -v -v)
43 -V show the internal version info.
46 -D use directed path address arguments. The path
47 is a comma separated list of out ports.
50 "0,1,2,1,4" # out via port 1, then 2, ...
51 -G use GUID address arguments. In most cases, it is the Port GUID.
54 -s <smlid> use 'smlid' as the target lid for SA queries.
56 # Local umad parameters:
57 -C <ca_name> use the specified ca_name.
58 -P <ca_port> use the specified ca_port.
59 -t <timeout_ms> override the default timeout for the solicited mads.
61 CLI notation: all utilities use the POSIX style notation,
62 meaning that all options (flags) must precede all arguments
66 Utilities descriptions:
71 ibstatus is a script which displays basic information obtained from the local
72 IB driver. Output includes LID, SMLID, port state, link width active, and port
76 ibstatus [-h] [devname[:port]]...
79 ibstatus # display status of all IB ports
80 ibstatus mthca1 # status of mthca1 ports
81 ibstatus mthca1:1 mthca0:2 # show status of specified ports
89 Similar to the ibstatus utility but implemented as a binary and not a script.
90 It has options to list CAs and/or ports.
93 ibstat [-d(ebug) -l(ist_of_cas) -p(ort_list) -s(hort)] <ca_name> [portnum]
96 ibstat # display status of all IB ports
97 ibstat mthca1 # status of mthca1 ports
98 ibstat mthca1 2 # show status of specified ports
99 ibstat -p mthca0 # list the port guids of mthca0
100 ibstat -l # list all CA names
108 ibroute uses SMPs to display the forwarding tables (unicast
109 (LinearForwardingTable or LFT) or multicast (MulticastForwardingTable or MFT))
110 for the specified switch LID and the optional lid (mlid) range.
111 The default range is all valid entries in the range 1...FDBTop.
114 ibroute [options] <switch_addr> [<startlid> [<endlid>]]]
117 -a show all lids in range, even invalid entries.
118 -n do not try to resolve destinations.
119 -M show multicast forwarding tables. In this case the range
120 parameters are specifying mlid range.
123 ibroute 2 # dump all valid entries of switch lid 2
124 ibroute 2 15 # dump entries in the range 15...FDBTop.
125 ibroute -a 2 10 20 # dump all entries in the range 10..20
126 ibroute -n 2 # simple format
127 ibroute -M 2 # show multicast tables
135 ibtracert uses SMPs to trace the path from a source GID/LID to a
136 destination GID/LID. Each hop along the path is displayed until the destination
137 is reached or a hop does not respond. By using the -m option, multicast path
138 tracing can be performed between source and destination nodes.
141 ibtracert [options] <src-addr> <dest-addr>
144 -n simple format; don't show additional information.
145 -m <mlid> show the multicast trace of the specified mlid.
148 ibtracert 2 23 # show trace between lid 2 and 23
149 ibtracert -m 0xc000 3 5 # show multicast trace between lid 3 and 5
150 for mcast lid 0xc000.
155 smpquery allows a basic subset of standard SMP queries including the following:
156 node info, node description, switch info, port info. Fields are displayed in
157 human readable format.
160 smpquery [options] <op> <dest_addr> [op_params]
162 Current supported operations and their parameters:
165 portinfo <addr> [<portnum>] # default port is zero
167 pkeys <addr> [<portnum>]
168 sl2vl <addr> [<portnum>]
169 vlarb <addr> [<portnum>]
172 smpquery nodeinfo 2 # show nodeinfo for lid 2
173 smpquery portinfo 2 5 # show portinfo for lid 2 port 5
178 smpdump is a general purpose SMP utility which gets SM attributes from a
179 specified SMA. The result is dumped in hex by default.
182 smpdump [options] <dest_addr> <attr> [mod]
185 -s show output as string
188 smpdump -D 0,1,2 0x15 2 # port info, port 2
189 smpdump 3 0x15 2 # port info, lid 3 port 2
194 ibaddr can be used to show the lid and GID addresses of the specified port,
195 or the local port by default.
196 Note: this utility can be used as simple address resolver.
199 ibaddr [options] [<dest_addr>]
202 ibaddr # show local address
203 ibaddr 2 # show address of the specified port lid
204 ibaddr -G 0x8f1040023 # show address of the specified port guid
209 sminfo issue and dumps the output of a sminfo query in human readable format.
210 The target SM is the one listed in the local port info, or the SM specified
211 by the optional SM lid or by the SM direct routed path.
212 Note: using sminfo for any purposes other then simple query may be very
213 dangerous, and may result in a malfunction of the target SM.
216 sminfo [options] <sm_lid|sm_dr_path> [sminfo_modifier]
219 -s <state> # use the specified state in sminfo mad
220 -p <priority> # use the specified priority in sminfo mad
221 -a <activity> # use the specified activity in sminfo mad
224 sminfo # show sminfo of SM listed in local portinfo
225 sminfo 2 # query SM on port lid 2
230 perfquery uses PerfMgt GMPs to obtain the PortCounters (basic performance
231 and error counters) from the PMA at the node specified. Optionally
232 show aggregated counters for all ports of node. Also, optionally, reset
233 after read, or only reset counters.
236 perfquery [options] [<lid|guid> [[port] [reset_mask]]]
239 -a show aggregated counters for all ports of the destination lid.
240 -r reset counters after read.
241 -R only reset counters.
244 perfquery # read local port's performance counters
245 perfquery 32 1 # read performance counters from lid 32, port 1
246 perfquery -a 32 # read node aggregated performance counters
247 perfquery -r 32 1 # read performance counters and reset
248 perfquery -R 32 1 # reset performance counters of port 1 only
249 perfquery -R -a 32 # reset performance counters of all ports
250 perfquery -R 32 2 0xf000 # reset only non-error counters of port 2
255 ibping uses vendor mads to validate connectivity between IB nodes.
256 On exit, (IP) ping like output is show. ibping is run as client/server.
257 Default is to run as client. Note also that a default ping server is
258 implemented within the kernel.
261 ibping [options] <dest lid|guid>
264 -c <count> stop after count packets
265 -f flood destination: send packets back to back w/o delay
266 -o <oui> use specified OUI number to multiplex vendor mads
267 -S start in server mode (do not return)
272 ibnetdiscover performs IB subnet discovery and outputs a human readable
273 topology file. GUIDs, node types, and port numbers are displayed
274 as well as port LIDs and NodeDescriptions. All nodes (and links) are displayed
275 (full topology). Optionally this utility can be used to list the current
276 connected nodes. The output is printed to the standard output unless a
277 topology file is specified.
280 ibnetdiscover [options] [<topology-filename>]
283 -l List of connected nodes
284 -H List of connected HCAs
285 -S List of connected switches
291 ibhosts either walks the IB subnet topology or uses an already saved topology
292 file and extracts the CA nodes.
295 ibhosts [-h] [<topology-file>]
298 ibnetdiscover, ibnetdiscover format
303 ibswitches either walks the IB subnet topology or uses an already saved
304 topology file and extracts the IB switches.
307 ibswitches [-h] [<topology-file>]
310 ibnetdiscover, ibnetdiscover format
315 ibchecknet uses a full topology file that was created by ibnetdiscover,
316 scans the network to validate the connectivity and reports errors
317 (from port counters).
320 ibchecknet [-h] [<topology-file>]
323 ibnetdiscover, ibnetdiscover format, ibchecknode, ibcheckport, ibcheckerrs
328 Check connectivity and do some simple sanity checks for the specified port.
329 Port address is lid unless -G option is used to specify a GUID address.
332 ibcheckport [-h] [-G] <lid|guid> <port_number>
335 ibcheckport 2 3 # check lid 2 port 3
338 smpquery, smpquery output format, ibaddr
343 Check connectivity and do some simple sanity checks for the specified node.
344 Port address is lid unless -G option is used to specify a GUID address.
347 ibchecknode [-h] [-G] <lid|guid>
350 ibchecknode 2 # check node via lid 2
353 smpquery, smpquery output format, ibaddr
360 Check specified port (or node) and report errors that surpassed their predefined
361 threshold. Port address is lid unless -G option is used to specify a GUID
362 address. The predefined thresholds can be dumped using the -s option, and a
363 user defined threshold_file (using the same format as the dump) can be
364 specified using the -t <file> option.
367 ibcheckerrs [-h] [-G] [-t <threshold_file>] [-s(how_thresholds)] <lid|guid> [<port>]
370 ibcheckerrs 2 # check aggregated node counter for lid 2
371 ibcheckerrs 2 4 # check port counters for lid 2 port 4
372 ibcheckerrs -t xxx 2 # check node using xxx threshold file
375 perfquery, perfquery output format, ibaddr
380 ibportstate allows the port state and port physical state of an IB port
381 to be queried or a switch port to be disabled or enabled.
384 ibportstate [-d(ebug) -e(rr_show) -v(erbose) -D(irect) -G(uid) -s smlid
385 -V(ersion) -C ca_name -P ca_port -t timeout_ms] <dest dr_path|lid|guid>
387 supported ops: enable, disable, query
390 ibportstate 3 1 disable # by lid
391 ibportstate -G 0x2C9000100D051 1 enable # by guid
392 ibportstate -D 0 1 # by direct route
397 ibcheckwidth uses a full topology file that was created by ibnetdiscover,
398 scans the network to validate the active link widths and reports any 1x
402 ibcheckwidth [-h] [<topology-file>]
405 ibnetdiscover, ibnetdiscover format, ibchecknode, ibcheckportwidth
410 Check connectivity and check the specified port for 1x link width.
411 Port address is lid unless -G option is used to specify a GUID address.
414 ibcheckportwidth [-h] [-G] <lid|guid> <port>
417 ibcheckportwidth 2 3 # check lid 2 port 3
420 smpquery, smpquery output format, ibaddr
425 ibcheckstate uses a full topology file that was created by ibnetdiscover,
426 scans the network to validate the port state and port physical state,
427 and reports any ports which have a port state other than Active or
428 a port physical state other than LinkUp.
431 ibcheckstate [-h] [<topology-file>]
434 ibnetdiscover, ibnetdiscover format, ibchecknode, ibcheckportstate
439 Check connectivity and check the specified port for proper port state
440 (Active) and port physical state (LinkUp).
441 Port address is lid unless -G option is used to specify a GUID address.
444 ibcheckportstate [-h] [-G] <lid|guid> <port_number>
447 ibcheckportstate 2 3 # check lid 2 port 3
450 smpquery, smpquery output format, ibaddr
454 ibcheckerrors uses a full topology file that was created by ibnetdiscover,
455 scans the network to validate the connectivity and reports errors
456 (from port counters).
459 ibnetcheckerrors [-h] [<topology-file>]
462 ibnetdiscover, ibnetdiscover format, ibchecknode, ibcheckport, ibcheckerrs
466 ibdiscover.pl uses a topology file create by ibnetdiscover and a discover.map
467 file which the network administrator creates which indicates the nodes
468 to be expected and a ibdiscover.topo file which is the expected connectivity
469 and produces a new connectivity file (discover.topo.new) and outputs
470 the changes to stdout. The network administrator can choose to replace
471 the "old" topo file with the new one or certain changes in.
473 The syntax of the ibdiscover.map file is:
474 <nodeGUID>|port|"Text for node"|<NodeDescription from ibnetdiscover format>
476 8f10400410015|8|"ISR 6000"|# SW-6IB4 Voltaire port 0 lid 5
477 8f10403960558|2|"HCA 1"|# MT23108 InfiniHost Mellanox Technologies
479 The syntax of the old and new topo files (ibdiscover.topo and
480 ibdiscover.topo.new) are:
481 <LocalPort>|<LocalNodeGUID>|<RemotePort>|<RemoteNodeGUID>
483 10|5442ba00003080|1|8f10400410015
485 These topo files are produced by the ibdiscover.pl tool.
488 ibnetdiscover | ibdiscover.pl
491 ibnetdiscover, ibnetdiscover format
496 ibnodes either walks the IB subnet topology or uses an already saved topology
497 file and extracts the IB nodes (CAs and switches).
500 ibnodes [<topology-file>]
503 ibnetdiscover, ibnetdiscover format
508 ibclearerrors clears the PMA error counters in PortCounters by either walking
509 the IB subnet topology or using an already saved topology file.
512 ibclearerrors [-h] [<topology-file>]
515 ibnetdiscover, ibnetdiscover format, perfquery
520 ibclearcounters clears the PMA port counters by either walking
521 the IB subnet topology or using an already saved topology file.
524 ibclearcounters [-h] [<topology-file>]
527 ibnetdiscover, ibnetdiscover format, perfquery
532 Issue some SA queries.
535 Usage: saquery [-h -d -P -N -L -G -s -g][<name>]
536 Queries node records by default
538 -P get PathRecord info
539 -N get NodeRecord info
540 -L Return just the Lid of the name specified
541 -G Return just the Guid of the name specified
542 -s Return the PortInfoRecords with isSM capability mask bit on
543 -g get multicast group info
546 OpenSM libvendor, OpenSM libopensm, libibumad
551 ibsysstat uses vendor mads to validate connectivity between IB nodes
552 and obtain other information about the IB node. ibsysstat is run as
553 client/server. Default is to run as client.
556 ibsysstat [options] <dest lid|guid> [<op>]
559 Current supported operations:
560 ping - verify connectivity to server (default)
561 host - obtain host information from server
562 cpu - obtain cpu information from server
563 -o <oui> use specified OUI number to multiplex vendor mads
564 -S start in server mode (do not return)