2 shaharf@voltaire.com, halr@voltaire.com
6 Model of operation: All utilities use direct MAD access to perform their
7 operations. Operations that require QP 0 mads only, may use direct routed
8 mads, and therefore may work even in unconfigured subnets. Almost all
9 utilities can operate without accessing the SM, unless GUID to lid translation
12 Dependencies: Most utilities depend on libibmad and libibumad.
13 All utilities depend on the ib_umad kernel module.
15 Multiple port/Multiple CA support: when no IB device or port is specified
16 (see the "local umad parameters" below), the libibumad library
17 selects the port to use by the following criteria:
18 1. the first port that is ACTIVE.
19 2. if not found, the first port that is UP (physical link up).
21 If a port and/or CA name is specified, the libibumad library
22 attempts to fulfill the user request, and will fail if it is not
25 ibaddr # use the 'best port'
26 ibaddr -C mthca1 # pick the best port from mthca1 only.
27 ibaddr -P 2 # use the second (active/up) port from the
28 first available IB device.
29 ibaddr -C mthca0 -P 2 # use the specified port only.
31 Common options & flags:
32 Most diagnostics take the following flags. The exact list of supported
33 flags per utility can be found in the usage message and can be shown
34 using util_name -h syntax.
37 -d raise the IB debugging level. May be used
38 several times (-ddd or -d -d -d).
39 -e show umad send receive errors (timeouts and others)
40 -h show the usage message
41 -v increase the application verbosity level.
42 May be used several times (-vv or -v -v -v)
43 -V show the internal version info.
46 -D use directed path address arguments. The path
47 is a comma separated list of out ports.
50 "0,1,2,1,4" # out via port 1, then 2, ...
51 -G use GUID address arguments. In most cases, it is the Port GUID.
54 -s <smlid> use 'smlid' as the target lid for SA queries.
56 # Local umad parameters:
57 -C <ca_name> use the specified ca_name.
58 -P <ca_port> use the specified ca_port.
59 -t <timeout_ms> override the default timeout for the solicited mads.
61 CLI notation: all utilities use the POSIX style notation,
62 meaning that all options (flags) must precede all arguments
66 Utilities descriptions:
71 ibstatus is a script which displays basic information obtained from the local
72 IB driver. Output includes LID, SMLID, port state, link width active, and port
76 ibstatus [-h] [devname[:port]]...
79 ibstatus # display status of all IB ports
80 ibstatus mthca1 # status of mthca1 ports
81 ibstatus mthca1:1 mthca0:2 # show status of specified ports
89 Similar to the ibstatus utility but implemented as a binary and not a script.
90 It has options to list CAs and/or ports.
93 ibstat [-d(ebug) -l(ist_of_cas) -p(ort_list) -s(hort)] <ca_name> [portnum]
96 ibstat # display status of all IB ports
97 ibstat mthca1 # status of mthca1 ports
98 ibstat mthca1 2 # show status of specified ports
99 ibstat -p mthca0 # list the port guids of mthca0
100 ibstat -l # list all CA names
108 ibroute uses SMPs to display the forwarding tables (unicast
109 (LinearForwardingTable or LFT) or multicast (MulticastForwardingTable or MFT))
110 for the specified switch LID and the optional lid (mlid) range.
111 The default range is all valid entries in the range 1...FDBTop.
114 ibroute [options] <switch_addr> [<startlid> [<endlid>]]]
117 -a show all lids in range, even invalid entries.
118 -n do not try to resolve destinations.
119 -M show multicast forwarding tables. In this case the range
120 parameters are specifying mlid range.
123 ibroute 2 # dump all valid entries of switch lid 2
124 ibroute 2 15 # dump entries in the range 15...FDBTop.
125 ibroute -a 2 10 20 # dump all entries in the range 10..20
126 ibroute -n 2 # simple format
127 ibroute -M 2 # show multicast tables
135 ibtracert uses SMPs to trace the path from a source GID/LID to a
136 destination GID/LID. Each hop along the path is displayed until the destination
137 is reached or a hop does not respond. By using the -m option, multicast path
138 tracing can be performed between source and destination nodes.
141 ibtracert [options] <src-addr> <dest-addr>
144 -n simple format; don't show additional information.
145 -m <mlid> show the multicast trace of the specified mlid.
148 ibtracert 2 23 # show trace between lid 2 and 23
149 ibtracert -m 0xc000 3 5 # show multicast trace between lid 3 and 5
150 for mcast lid 0xc000.
155 smpquery allows a basic subset of standard SMP queries including the following:
156 node info, node description, switch info, port info. Fields are displayed in
157 human readable format.
160 smpquery [options] <op> <dest_addr> [op_params]
162 Current supported operations and their parameters:
165 portinfo <addr> [<portnum>] # default port is zero
167 pkeys <addr> [<portnum>]
168 sl2vl <addr> [<portnum>]
169 vlarb <addr> [<portnum>]
172 smpquery nodeinfo 2 # show nodeinfo for lid 2
173 smpquery portinfo 2 5 # show portinfo for lid 2 port 5
178 smpdump is a general purpose SMP utility which gets SM attributes from a
179 specified SMA. The result is dumped in hex by default.
182 smpdump [options] <dest_addr> <attr> [mod]
185 -s show output as string
188 smpdump -D 0,1,2 0x15 2 # port info, port 2
189 smpdump 3 0x15 2 # port info, lid 3 port 2
194 ibaddr can be used to show the lid and GID addresses of the specified port,
195 or the local port by default.
196 Note: this utility can be used as simple address resolver.
199 ibaddr [options] [<dest_addr>]
202 ibaddr # show local address
203 ibaddr 2 # show address of the specified port lid
204 ibaddr -G 0x8f1040023 # show address of the specified port guid
209 sminfo issue and dumps the output of a sminfo query in human readable format.
210 The target SM is the one listed in the local port info, or the SM specified
211 by the optional SM lid or by the SM direct routed path.
212 Note: using sminfo for any purposes other then simple query may be very
213 dangerous, and may result in a malfunction of the target SM.
216 sminfo [options] <sm_lid|sm_dr_path> [sminfo_modifier]
219 -s <state> # use the specified state in sminfo mad
220 -p <priority> # use the specified priority in sminfo mad
221 -a <activity> # use the specified activity in sminfo mad
224 sminfo # show sminfo of SM listed in local portinfo
225 sminfo 2 # query SM on port lid 2
230 perfquery uses PerfMgt GMPs to obtain the PortCounters (basic performance
231 and error counters) from the PMA at the node specified. Optionally reset all
235 perfquery [options] [<lid|guid> [[port] [reset_mask]]]
238 -a show aggregated counters for all ports of the destination lid.
239 -r reset counters after read.
240 -R only reset counters.
243 perfquery # read local port's performance counters
244 perfquery 32 1 # read performance counters from lid 32, port 1
245 perfquery -a 32 # read node aggregated performance counters
246 perfquery -r 32 1 # read performance counters and reset
247 perfquery -R 32 1 # reset performance counters of port 1 only
248 perfquery -R -a 32 # reset performance counters of all ports
249 perfquery -R 32 2 0xf000 # reset only non-error counters of port 2
254 ibping uses vendor mads to validate connectivity between IB nodes.
255 On exit, (IP) ping like output is show. ibping is run as client/server.
256 Default is to run as client. Note also that a default ping server is
257 implemented within the kernel.
260 ibping [options] <dest lid|guid>
263 -c <count> stop after count packets
264 -f flood destination: send packets back to back w/o delay
265 -o <oui> use specified OUI number to multiplex vendor mads
266 -S start in server mode (do not return)
271 ibnetdiscover performs IB subnet discovery and outputs a human readable
272 topology file. GUIDs, node types, and port numbers are displayed
273 as well as port LIDs and NodeDescriptions. All nodes (and links) are displayed
274 (full topology). Optionally this utility can be used to list the current
275 connected nodes. The output is printed to the standard output unless a
276 topology file is specified.
279 ibnetdiscover [options] [<topology-filename>]
282 -l List of connected nodes
283 -H List of connected HCAs
284 -S List of connected switches
290 ibhosts either walks the IB subnet topology or uses an already saved topology
291 file and extracts the CA nodes.
294 ibhosts [-h] [<topology-file>]
297 ibnetdiscover, ibnetdiscover format
302 ibswitches either walks the IB subnet topology or uses an already saved
303 topology file and extracts the IB switches.
306 ibswitches [-h] [<topology-file>]
309 ibnetdiscover, ibnetdiscover format
314 ibchecknet uses a full topology file that was created by ibnetdiscover,
315 scans the network to validate the connectivity and reports errors
316 (from port counters).
319 ibchecknet [-h] [<topology-file>]
322 ibnetdiscover, ibnetdiscover format, ibchecknode, ibcheckport, ibcheckerrs
327 Check connectivity and do some simple sanity checks for the specified port.
328 Port address is lid unless -G option is used to specify a GUID address.
331 ibcheckport [-h] [-G] <lid|guid> <port_number>
334 ibcheckport 2 3 # check lid 2 port 3
337 smpquery, smpquery output format, ibaddr
342 Check connectivity and do some simple sanity checks for the specified node.
343 Port address is lid unless -G option is used to specify a GUID address.
346 ibchecknode [-h] [-G] <lid|guid>
349 ibchecknode 2 # check node via lid 2
352 smpquery, smpquery output format, ibaddr
359 Check specified port (or node) and report errors that surpassed their predefined
360 threshold. Port address is lid unless -G option is used to specify a GUID
361 address. The predefined thresholds can be dumped using the -s option, and a
362 user defined threshold_file (using the same format as the dump) can be
363 specified using the -t <file> option.
366 ibcheckerrs [-h] [-G] [-t <threshold_file>] [-s(how_thresholds)] <lid|guid> [<port>]
369 ibcheckerrs 2 # check aggregated node counter for lid 2
370 ibcheckerrs 2 4 # check port counters for lid 2 port 4
371 ibcheckerrs -t xxx 2 # check node using xxx threshold file
374 perfquery, perfquery output format, ibaddr
379 ibportstate allows the port state and port physical state of an IB port
380 to be queried or a switch port to be disabled or enabled.
383 ibportstate [-d(ebug) -e(rr_show) -v(erbose) -D(irect) -G(uid) -s smlid
384 -V(ersion) -C ca_name -P ca_port -t timeout_ms] <dest dr_path|lid|guid>
386 supported ops: enable, disable, query
389 ibportstate 3 1 disable # by lid
390 ibportstate -G 0x2C9000100D051 1 enable # by guid
391 ibportstate -D 0 1 # by direct route
396 ibcheckwidth uses a full topology file that was created by ibnetdiscover,
397 scans the network to validate the active link widths and reports any 1x
401 ibcheckwidth [-h] [<topology-file>]
404 ibnetdiscover, ibnetdiscover format, ibchecknode, ibcheckportwidth
409 Check connectivity and check the specified port for 1x link width.
410 Port address is lid unless -G option is used to specify a GUID address.
413 ibcheckportwidth [-h] [-G] <lid|guid> <port>
416 ibcheckportwidth 2 3 # check lid 2 port 3
419 smpquery, smpquery output format, ibaddr
424 ibcheckstate uses a full topology file that was created by ibnetdiscover,
425 scans the network to validate the port state and port physical state,
426 and reports any ports which have a port state other than Active or
427 a port physical state other than LinkUp.
430 ibcheckstate [-h] [<topology-file>]
433 ibnetdiscover, ibnetdiscover format, ibchecknode, ibcheckportstate
438 Check connectivity and check the specified port for proper port state
439 (Active) and port physical state (LinkUp).
440 Port address is lid unless -G option is used to specify a GUID address.
443 ibcheckportstate [-h] [-G] <lid|guid> <port_number>
446 ibcheckportstate 2 3 # check lid 2 port 3
449 smpquery, smpquery output format, ibaddr
453 ibcheckerrors uses a full topology file that was created by ibnetdiscover,
454 scans the network to validate the connectivity and reports errors
455 (from port counters).
458 ibnetcheckerrors [-h] [<topology-file>]
461 ibnetdiscover, ibnetdiscover format, ibchecknode, ibcheckport, ibcheckerrs
465 ibdiscover.pl uses a topology file create by ibnetdiscover and a discover.map
466 file which the network administrator creates which indicates the nodes
467 to be expected and a ibdiscover.topo file which is the expected connectivity
468 and produces a new connectivity file (discover.topo.new) and outputs
469 the changes to stdout. The network administrator can choose to replace
470 the "old" topo file with the new one or certain changes in.
472 The syntax of the ibdiscover.map file is:
473 <nodeGUID>|port|"Text for node"|<NodeDescription from ibnetdiscover format>
475 8f10400410015|8|"ISR 6000"|# SW-6IB4 Voltaire port 0 lid 5
476 8f10403960558|2|"HCA 1"|# MT23108 InfiniHost Mellanox Technologies
478 The syntax of the old and new topo files (ibdiscover.topo and
479 ibdiscover.topo.new) are:
480 <LocalPort>|<LocalNodeGUID>|<RemotePort>|<RemoteNodeGUID>
482 10|5442ba00003080|1|8f10400410015
484 These topo files are produced by the ibdiscover.pl tool.
487 ibnetdiscover | ibdiscover.pl
490 ibnetdiscover, ibnetdiscover format
495 ibnodes either walks the IB subnet topology or uses an already saved topology
496 file and extracts the IB nodes (CAs and switches).
499 ibnodes [<topology-file>]
502 ibnetdiscover, ibnetdiscover format
507 ibclearerrors clears the PMA error counters in PortCounters by either walking
508 the IB subnet topology or using an already saved topology file.
511 ibclearerrors [-h] [<topology-file>]
514 ibnetdiscover, ibnetdiscover format, perfquery
519 ibclearcounters clears the PMA port counters by either walking
520 the IB subnet topology or using an already saved topology file.
523 ibclearcounters [-h] [<topology-file>]
526 ibnetdiscover, ibnetdiscover format, perfquery
531 Issue some SA queries.
534 Usage: saquery [-h -d -P -N -L -G -s -g][<name>]
535 Queries node records by default
537 -P get PathRecord info
538 -N get NodeRecord info
539 -L Return just the Lid of the name specified
540 -G Return just the Guid of the name specified
541 -s Return the PortInfoRecords with isSM capability mask bit on
542 -g get multicast group info
545 OpenSM libvendor, OpenSM libopensm, libibumad
550 ibsysstat uses vendor mads to validate connectivity between IB nodes
551 and obtain other information about the IB node. ibsysstat is run as
552 client/server. Default is to run as client.
555 ibsysstat [options] <dest lid|guid> [<op>]
558 Current supported operations:
559 ping - verify connectivity to server (default)
560 host - obtain host information from server
561 cpu - obtain cpu information from server
562 -o <oui> use specified OUI number to multiplex vendor mads
563 -S start in server mode (do not return)