.\" Copyright (c) 1985, 1990, 1993 .\" The Regents of the University of California. All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" 3. Neither the name of the University nor the names of its contributors .\" may be used to endorse or promote products derived from this software .\" without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" @(#)systat.1 8.2 (Berkeley) 12/30/93 .\" $FreeBSD$ .\" .Dd April 1, 2022 .Dt SYSTAT 1 .Os .Sh NAME .Nm systat .Nd display system statistics .Sh SYNOPSIS .Nm .Op Fl Ar display .Op Ar display-commands .Op Ar refresh-interval .Sh DESCRIPTION The .Nm utility displays various system statistics in a screen oriented fashion using the curses screen display library, .Xr ncurses 3 . .Pp While .Nm is running the screen is usually divided into two windows (an exception is the vmstat display which uses the entire screen). The upper window depicts the current system load average. The information displayed in the lower window may vary, depending on user commands. The last line on the screen is reserved for user input and error messages. .Pp By default .Nm displays the processes getting the largest percentage of the processor in the lower window. Other displays show swap space usage, disk I/O statistics (a la .Xr iostat 8 ) , virtual memory statistics (a la .Xr vmstat 8 ) , TCP/IP statistics, and network connections (a la .Xr netstat 1 ) . .Pp Input is interpreted at two different levels. A ``global'' command interpreter processes all keyboard input. If this command interpreter fails to recognize a command, the input line is passed to a per-display command interpreter. This allows each display to have certain display-specific commands. .Pp Command line options: .Bl -tag -width "refresh_interval" .It Fl Ns Ar display The .Fl flag expects .Ar display to be one of: .Ic icmp , .Ic icmp6 , .Ic ifstat , .Ic iolat , .Ic iostat , .Ic ip , .Ic ip6 , .Ic netstat , .Ic pigs , .Ic sctp , .Ic swap , .Ic tcp , .Ic vmstat , or .Ic zarc , These displays can also be requested interactively (without the .Dq Fl ) and are described in full detail below. .It Ar refresh-interval The .Ar refresh-value specifies the screen refresh time interval in seconds. Time interval can be fractional. .It Ar display-commands A list of commands specific to this display. These commands can also be entered interactively and are described for each display separately below. If the command requires arguments, they can be specified as separate command line arguments. A command line argument .Fl - will finish display commands. For example: .Pp .Dl Nm Fl ifstat Fl match Ar bge0,em1 Fl pps .Pp This will display statistics of packets per second for network interfaces named as bge0 and em1. .Pp .Dl Nm Fl iostat Fl numbers Fl - Ar 2.1 .Pp This will display all IO statistics in a numeric format and the information will be refreshed each 2.1 seconds. .El .Pp Certain characters cause immediate action by .Nm . These are .Bl -tag -width Fl .It Ic \&^L Refresh the screen. .It Ic \&^G Print the name of the current ``display'' being shown in the lower window and the refresh interval. .It Ic \&: Move the cursor to the command line and interpret the input line typed as a command. While entering a command the current character erase, word erase, and line kill characters may be used. .El .Pp The following commands are interpreted by the ``global'' command interpreter. .Bl -tag -width Fl .It Ic help Print the names of the available displays on the command line. .It Ic load Print the load average over the past 1, 5, and 15 minutes on the command line. .It Ic stop Stop refreshing the screen. .It Xo .Op Ic start .Op Ar number .Xc Start (continue) refreshing the screen. If a second, numeric, argument is provided it is interpreted as a refresh interval (in seconds). Supplying only a number will set the refresh interval to this value. .It Ic quit Exit .Nm . (This may be abbreviated to .Ic q . ) .El .Pp The available displays are: .Bl -tag -width Ic .It Ic pigs Display, in the lower window, those processes resident in main memory and getting the largest portion of the processor (the default display). When less than 100% of the processor is scheduled to user processes, the remaining time is accounted to the ``idle'' process. .It Ic icmp Display, in the lower window, statistics about messages received and transmitted by the Internet Control Message Protocol .Pq Dq ICMP . The left half of the screen displays information about received packets, and the right half displays information regarding transmitted packets. .Pp The .Ic icmp display understands two commands: .Ic mode and .Ic reset . The .Ic mode command is used to select one of four display modes, given as its argument: .Bl -tag -width absoluteXX -compact .It Ic rate : show the rate of change of each value in packets (the default) per second .It Ic delta : show the rate of change of each value in packets per refresh interval .It Ic since : show the total change of each value since the display was last reset .It Ic absolute : show the absolute value of each statistic .El .Pp The .Ic reset command resets the baseline for .Ic since mode. The .Ic mode command with no argument will display the current mode in the command line. .It Ic icmp6 This display is like the .Ic icmp display, but displays statistics for IPv6 ICMP. .It Ic ip Otherwise identical to the .Ic icmp display, except that it displays IP and UDP statistics. .It Ic ip6 Like the .Ic ip display, except that it displays IPv6 statistics. It does not display UDP statistics. .It Ic sctp Like .Ic icmp , but with SCTP statistics. .It Ic tcp Like .Ic icmp , but with TCP statistics. .It Ic iolat Display statistics describing the hardware latencies of I/O operations as computed by the .Va CAM_IOSCHED_DYNAMIC option. This option must be in the kernel config file of the running kernel for this display to work. All devices are displayed as there is currently no way to filter them. The statistics displayed for the I/O latencies are the percentiles with sufficient data during the polling interval to compute. If a value cannot be estimated ``-'' is displayed. The P50 (also known as the median), P90, P99 and P99.9 values are computed if more than 2, 10, 100 or 1000 operations occurred during the polling interval. The latency is the hardware latency values, and does not include any software queuing time. The latencies are estimated based on histogram data computed by the CAM I/O scheduler and represent estimates of the actual value that are only good to two or three significant digits. The display of latency changes based on the scale of the latency to reflect the precision of the estimates and to fit on the available screen space. All latencies are reported in milliseconds. When color is enabled .Bl -bullet .It Values below the medium latency threshold are displayed in green. .It Values between the minimum latency and high latency thresholds are displayed in magenta. .It Values above the high latency thresholds are displayed in red. .Pp When color is disabled, the default foreground and background colors are always used. .Pp The following commands are specific to the .Ic iolat display; the minimum unambiguous prefix may be supplied. .Pp .Bl -tag -width Fl -compact .It Cm color Toggle the use of color in the display. The default is on. .It Cm hi=XXX Set the high latency threshold to XXX milliseconds. .It Cm med=XXX Set the medium latency threshold to XXX milliseconds. .It Cm read Toggle the display of statistics about read operations. The default is on. .It Cm write Toggle the display of statistics about write operations. The default is on. .It Cm trim Toggle the display of statistics about trim operations. The default is on. .El .El .It Ic iostat Display, in the lower window, statistics about processor use and disk throughput. Statistics on processor use appear as bar graphs of the amount of time executing in user mode (``user''), in user mode running low priority processes (``nice''), in system mode (``system''), in interrupt mode (``interrupt''), and idle (``idle''). Statistics on disk throughput show, for each drive, megabytes per second, average number of disk transactions per second, and average kilobytes of data per transaction. This information may be displayed as bar graphs or as rows of numbers which scroll downward. Bar graphs are shown by default. .Pp The following commands are specific to the .Ic iostat display; the minimum unambiguous prefix may be supplied. .Pp .Bl -tag -width Fl -compact .It Cm numbers Show the disk I/O statistics in numeric form. Values are displayed in numeric columns which scroll downward. .It Cm bars Show the disk I/O statistics in bar graph form (default). .It Cm kbpt Toggle the display of kilobytes per transaction. (the default is to not display kilobytes per transaction). .El .It Ic swap Show information about swap space usage on all the swap areas compiled into the kernel and processes that are swapped out as well as a summary of disk activity. .Pp The swap areas are displayed first with their name, sizes and usage percentage. The .Ar Used column indicates the total blocks used so far; the graph shows the percentage of space in use on each partition. If there are more than one swap partition in use, a total line is also shown. Areas known to the kernel, but not in use are shown as not available. .Pp Below the swap space statistics, processes are listed in order of higher swap area usage. Pid, username, a part of command line, the total use of swap space in bytes, the size of process, as well as per-process swap usage percentage and per-system swap space percentage are shown per process. .Pp At the bottom left is the disk usage display. It reports the number of kilobytes per transaction, transactions per second, megabytes per second and the percentage of the time the disk was busy averaged over the refresh period of the display (by default, five seconds). The system keeps statistics on most every storage device. In general, up to seven devices are displayed. The devices displayed by default are the first devices in the kernel's device list. See .Xr devstat 3 and .Xr devstat 9 for details on the devstat system. .It Ic vmstat Take over the entire display and show a (rather crowded) compendium of statistics related to virtual memory usage, process scheduling, device interrupts, system name translation caching, disk I/O etc. .Pp The upper left quadrant of the screen shows the number of users logged in and the load average over the last one, five, and fifteen minute intervals. Below this line are statistics on memory utilization. The first row of the table reports memory usage only among active processes, that is processes that have run in the previous twenty seconds. The second row reports on memory usage of all processes. The first column reports on the number of kilobytes in physical pages claimed by processes. The second column reports the number of kilobytes in physical pages that are devoted to read only text pages. The third and fourth columns report the same two figures for virtual pages, that is the number of kilobytes in pages that would be needed if all processes had all of their pages. Finally the last column shows the number of kilobytes in physical pages on the free list. .Pp Below the memory display is a list of the average number of threads (over the last refresh interval) that are runnable (`r'), in page wait (`p'), in disk wait other than paging (`d'), sleeping (`s'), and swapped out but desiring to run (`w'). The row also shows the average number of context switches (`Csw'), traps (`Trp'; includes page faults), system calls (`Sys'), interrupts (`Int'), network software interrupts (`Sof'), and page faults (`Flt'). .Pp Below the process queue length listing is a numerical listing and a bar graph showing the amount of system (shown as `='), interrupt (shown as `+'), user (shown as `>'), nice (shown as `-'), and idle time (shown as ` '). .Pp Below the process display are statistics on name translations. It lists the number of names translated in the previous interval, the number and percentage of the translations that were handled by the system wide name translation cache, and the number and percentage of the translations that were handled by the per process name translation cache. .Pp To the right of the name translations display are lines showing the number of dirty buffers in the buffer cache (`dtbuf'), desired maximum size of vnode cache (`desvn'), number of vnodes actually allocated (`numvn'), and number of allocated vnodes that are free (`frevn'). .Pp At the bottom left is the disk usage display. It reports the number of kilobytes per transaction, transactions per second, megabytes per second and the percentage of the time the disk was busy averaged over the refresh period of the display (by default, five seconds). The system keeps statistics on most every storage device. In general, up to seven devices are displayed. The devices displayed by default are the first devices in the kernel's device list. See .Xr devstat 3 and .Xr devstat 9 for details on the devstat system. .Pp Under the date in the upper right hand quadrant are statistics on paging and swapping activity. The first two columns report the average number of pages brought in and out per second over the last refresh interval due to page faults and the paging daemon. The third and fourth columns report the average number of pages brought in and out per second over the last refresh interval due to swap requests initiated by the scheduler. The first row of the display shows the average number of disk transfers per second over the last refresh interval; the second row of the display shows the average number of pages transferred per second over the last refresh interval. .Pp Below the paging statistics is a column of lines regarding the virtual memory system. The first few lines describe, in units (except as noted below) of pages per second averaged over the sampling interval, pages copied on write (`cow'), pages zero filled on demand (`zfod'), pages optimally zero filled on demand (`ozfod'), the ratio of the (average) ozfod / zfod as a percentage (`%ozfod'), pages freed by the page daemon (`daefr'), pages freed by exiting processes (`prcfr'), total pages freed (`totfr'), pages reactivated from the free list (`react'), the average number of times per second that the page daemon was awakened (`pdwak'), pages analyzed by the page daemon (`pdpgs'), and in-transit blocking page faults (`intrn'). Note that the units are special for `%ozfod' and `pdwak'. The next few lines describe, as amounts of memory in kilobytes, pages wired down (`wire'), active pages (`act'), inactive pages (`inact'), dirty pages queued for laundering (`laund'), and free pages (`free'). Note that the values displayed are the current transient ones; they are not averages. .Pp At the bottom of this column is a line showing the amount of virtual memory, in kilobytes, mapped into the buffer cache (`buf'). This statistic is not useful. It exists only as a placeholder for the corresponding useful statistic (the amount of real memory used to cache disks). The most important component of the latter (the amount of real memory used by the vm system to cache disks) is not available, but can be guessed from the `inact' amount under some system loads. .Pp Running down the right hand side of the display is a breakdown of the interrupts being handled by the system. At the top of the list is the total interrupts per second over the time interval. The rest of the column breaks down the total on a device by device basis. Only devices that have interrupted at least once since boot time are shown. .Pp The following commands are specific to the .Ic vmstat display; the minimum unambiguous prefix may be supplied. .Pp .Bl -tag -width Ar -compact .It Cm boot Display cumulative statistics since the system was booted. .It Cm run Display statistics as a running total from the point this command is given. .It Cm time Display statistics averaged over the refresh interval (the default). .It Cm zero Reset running statistics to zero. .El .It Ic zarc display arc cache usage and hit/miss statistics. .It Ic netstat Display, in the lower window, network connections. By default, network servers awaiting requests are not displayed. Each address is displayed in the format ``host.port'', with each shown symbolically, when possible. It is possible to have addresses displayed numerically, limit the display to a set of ports, hosts, and/or protocols (the minimum unambiguous prefix may be supplied): .Pp .Bl -tag -width Ar -compact .It Cm all Toggle the displaying of server processes awaiting requests (this is the equivalent of the .Fl a flag to .Xr netstat 1 ) . .It Cm numbers Display network addresses numerically. .It Cm names Display network addresses symbolically. .It Cm proto Ar protocol Display only network connections using the indicated .Ar protocol . Supported protocols are ``tcp'', ``udp'', and ``all''. .It Cm ignore Op Ar items Do not display information about connections associated with the specified hosts or ports. Hosts and ports may be specified by name (``vangogh'', ``ftp''), or numerically. Host addresses use the Internet dot notation (``128.32.0.9''). Multiple items may be specified with a single command by separating them with spaces. .It Cm display Op Ar items Display information about the connections associated with the specified hosts or ports. As for .Ar ignore , .Op Ar items may be names or numbers. .It Cm show Op Ar ports\&|hosts Show, on the command line, the currently selected protocols, hosts, and ports. Hosts and ports which are being ignored are prefixed with a `!'. If .Ar ports or .Ar hosts is supplied as an argument to .Cm show , then only the requested information will be displayed. .It Cm reset Reset the port, host, and protocol matching mechanisms to the default (any protocol, port, or host). .El .It Ic ifstat Display the network traffic going through active interfaces on the system. Idle interfaces will not be displayed until they receive some traffic. .Pp For each interface being displayed, the current, peak and total statistics are displayed for incoming and outgoing traffic. By default, the .Ic ifstat display will automatically scale the units being used so that they are in a human-readable format. The scaling units used for the current and peak traffic columns can be altered by the .Ic scale command. .Bl -tag -width ".Cm scale Op Ar units" .It Cm scale Op Ar units Modify the scale used to display the current and peak traffic over all interfaces. The following units are recognised: kbit, kbyte, mbit, mbyte, gbit, gbyte and auto. .It Cm pps Show statistics in packets per second instead of bytes/bits per second. A subsequent call of .Ic pps switches this mode off. .It Cm match Op Ar patterns Display only interfaces that match pattern provided as an argument. Patterns should be in shell syntax separated by whitespaces or commas. If this command is called without arguments then all interfaces are displayed. For example: .Pp .Dl match em0, bge1 .Pp This will display em0 and bge1 interfaces. .Pp .Dl match em*, bge*, lo0 .Pp This will display all .Ic em interfaces, all .Ic bge interfaces and the loopback interface. .El .El .Pp Commands to switch between displays may be abbreviated to the minimum unambiguous prefix; for example, ``io'' for ``iostat''. Certain information may be discarded when the screen size is insufficient for display. For example, on a machine with 10 drives the .Ic iostat bar graph displays only 3 drives on a 24 line terminal. When a bar graph would overflow the allotted screen space it is truncated and the actual value is printed ``over top'' of the bar. .Pp The following commands are common to each display which shows information about disk drives. These commands are used to select a set of drives to report on, should your system have more drives configured than can normally be displayed on the screen. .Pp .Bl -tag -width Ar -compact .It Cm ignore Op Ar drives Do not display information about the drives indicated. Multiple drives may be specified, separated by spaces. .It Cm display Op Ar drives Display information about the drives indicated. Multiple drives may be specified, separated by spaces. .It Cm only Op Ar drives Display only the specified drives. Multiple drives may be specified, separated by spaces. .It Cm drives Display a list of available devices. .It Cm match Xo .Ar type , Ns Ar if , Ns Ar pass .Op | Ar ... .Xc Display devices matching the given pattern. The basic matching expressions are the same as those used in .Xr iostat 8 with one difference. Instead of specifying multiple .Fl t arguments which are then ORed together, the user instead specifies multiple matching expressions joined by the pipe .Pq Ql \&| character. The comma separated arguments within each matching expression are ANDed together, and then the pipe separated matching expressions are ORed together. Any device matching the combined expression will be displayed, if there is room to display it. For example: .Pp .Dl match da,scsi | cd,ide .Pp This will display all SCSI Direct Access devices and all IDE CDROM devices. .Pp .Dl match da | sa | cd,pass .Pp This will display all Direct Access devices, all Sequential Access devices, and all passthrough devices that provide access to CDROM drives. .El .Sh FILES .Bl -tag -width /boot/kernel/kernel -compact .It Pa /boot/kernel/kernel For the namelist. .It Pa /dev/kmem For information in main memory. .It Pa /etc/hosts For host names. .It Pa /etc/networks For network names. .It Pa /etc/services For port names. .El .Sh SEE ALSO .Xr netstat 1 , .Xr kvm 3 , .Xr icmp 4 , .Xr icmp6 4 , .Xr ip 4 , .Xr ip6 4 , .Xr tcp 4 , .Xr udp 4 , .Xr gstat 8 , .Xr iostat 8 , .Xr vmstat 8 .Sh HISTORY The .Nm program appeared in .Bx 4.3 . The .Ic icmp , .Ic ip , and .Ic tcp displays appeared in .Fx 3.0 ; the notion of having different display modes for the ICMP, IP, TCP, and UDP statistics was stolen from the .Fl C option to .Xr netstat 1 in Silicon Graphics' IRIX system. .Sh BUGS Certain displays presume a minimum of 80 characters per line. Ifstat does not detect new interfaces. The .Ic vmstat display looks out of place because it is (it was added in as a separate display rather than created as a new program). The .Ic iolat command does not implement the common device commands including filtering, as it does not use the .Xr devstat 3 mechanism to obtain its statistics.