Unix tips: Making troubleshooting with lsof easier

In last week’s post, we looked at a series of commands that used the lsof (list open files) command to provide information that can help with troubleshooting on the Unix systems you manage. Since lsof has such a huge collection of options, remembering which option to use for what sometimes makes the command hard to use as often or as effectively as you might like. So what we’re doing today is looking at several ways to make the use of this very helpful tool a bit easier. We do that by creating useful aliases, by providing something of a “cheat sheet,” and by deploying a number of lsof options in a script that makes educated guesses about what you’re going after.

Using aliases

Both of the aliases below will list whatever files are open on your behalf when you are logged in. I suspect that few sysadmins will want to type “showmyopenfiles.” It might be less of a problem to remember the lsof option or print out a cheat sheet. On the other hand, “showmine” would be somewhat ambiguous – my open files or my processes?

Note that most of these aliases require root privilege and assume that you have sudo provileges.

alias showmyopenfiles='sudo lsof -u `whoami`'
alias showmine='sudo lsof -u `whoami`'

Maybe “showfiles” ,“showmyfiles” or just “ofiles” would work better.

alias showfiles='sudo lsof'
alias showmyfiles='sudo lsof -u `whoami`'
alias ofiles='sudo lsof -u `whoami`'

In the command shown below, we’re looking for the processes that have opened /usr/sbin/lsof — the lsof command itself.

$ showfiles /usr/sbin/lsof
COMMAND   PID USER  FD   TYPE DEVICE SIZE/OFF   NODE NAME
lsof    27473 root txt    REG  202,1   141048 407194 /usr/sbin/lsof
lsof    27474 root txt    REG  202,1   141048 407194 /usr/sbin/lsof

Of course, getting overly clever with your aliases might make them harder to use than just going with basic lsof commands. Another option is to create aliases for the handful of options that you’re likely to use most of the time.

alias byuser='sudo lsof -u'
alias bypid='sudo lsof -p'
alias byfile='sudo lsof'
alias byprogram='sudo lsof -c'

Anyone using these aliases just has to remember to add the argument (username, PID, etc.).

In a similar way, you can set up aliases that show information for your network connections.

alias shownet='sudo lsof -i'
alias showtcp='sudo lsof -i tcp'
alias showudp='sudo lsof -i udp'

Using a cheat sheet

Having a cheat sheet on hand with quick explanations of the lsof command’s options can also help you take advantage of its many features without having to memorize them. Simple explanations and sample commands seem to be the most helpful. Here’s an example:

What files are open?                       lsof
What process has a particular file open?   lsof /path/to/the/file
What files in some directory are open?     lsof +D /path/to/the/dir
What files does some user have open?       lsof -u username
What files do a group of users have open?  lsof -u user1,user2
What files are open by process name?       lsof -c procname
What files are open by PID?                lsof -p 123
What files are open by other PIDs?         lsof -p ^123
Show network activity                      lsof -i
What files are open by port?               lsof -i :25
                                           lsof -i :smtp
List PIDs                                  lsof -t
Show network activity for a user           lsof -a -u username -i
Show socket use                            lsof -U
Show NFS activity                          lsof -N

Using a script

You can also simplify use of the lsof command by creating a script. The one below tries to determine what you’re looking for by evaluating the argument that you provide. For example, if you enter an IP address, it assumes that you want to see network activity for that particular IP. Feel free to modify it to better represent your own troubleshooting focus.

#!/bin/bash

if [ $# == 0 ]; then
    echo "USAGE: $0 "
    echo "Example: $0 procid"
fi

case $1 in
    [0-9]*.[0-9]*.[0-9]*.[0-9]*) sudo lsof -i@$1;;
    [0-99999]*) lsof -p $1;;
    net)        sudo lsof -i;;
    [a-z]*)     who | grep $1;
                if [ $? == 0 ]; then
                    sudo lsof -u $1
                else
                    if [ -e $1 ]; then
                        sudo lsof $1
                    else
                        sudo lsof -i :$1
                    fi
                fi;;
    /*)         if [ -f $1 ]; then
                    sudo lsof $1
                fi;;
    *)          echo "Sorry -- target not recognized";;
esac

The items in the case statement need to be ordered in such a way that the more restrictive choices come first (e.g., IP addresses before numbers). This script looks for IP addresses, numbers (which it assumes are PIDs), strings (which it first tries to identify as usernames, then looks for a matching local file, and then tries it as a port. If the argument starts with a /, it assumes it’s a file.

Wrap Up

There are many ways to make routine use of the lsof command easier and more likely. I hope some of the options presented in this post will prove to be useful.