Work the Shell

It has been a while since I spent some time looking at basic command-line features and how they tie in to shell scripts, so I thought it'd be a good time to go back to basics. Let's start with some fundamentals of command-line history, actually.

If you're like me, you're stuck in a rut, either using !xx to repeat the most recent command starting with that letter or those letters, or using the arrow up/down keys to step through your command history and find a specific command. I definitely get into a !v !cc !. loop when I'm developing software, for example—useful.

But, there's a lot more that your Bash shell can do in terms of manipulating your command history. A common one is to search for a specific command, then repeat it with the command's numeric ID, like this:

$ history | grep find
  213   find $HOME -name "*.zip" -print
$ !213

Make a typo on the command? You can use arrow keys to monkey around and fix it, but that can be tedious. Instead, use the ^old^new replacement sequence:

$ pc oldfile newfile
-bash: pc: command not found
$ ^pc^cp
cp oldfile newfile
$

It's super helpful, and notice that Bash echoes the corrected command so you know what's going on too.

One of my favorites is to repeat all the arguments of the previous command but not the command itself. Use !*, like this:

$ vi file1 file2 file3
$ cp !* ~

Got that? The cp command will copy each of the three files to your home directory for safekeeping.

You also can select individual parameters by index number too, as I demonstrate here:

$ echo listening to the soundtrack from cloud atlas
listening to the soundtrack from cloud atlas
$ echo !!:3
echo the
the
$ echo !!:3-5
-bash: :3-5: bad word specifier

Hmm...what went wrong on that second example? Did you figure it out? The !! applies to the command immediately before the current command, so the long string of listening to the soundtrack from cloud atlas is not what's being modified, and there are only two words in the immediately previous command (echo the).

There are two ways to get the latter example to work, either have it come immediately after the longer echo statement or tap into the power of Bash history modifiers even further by referencing the command by its number:

$ history | grep cloud atlas
  508   echo listening to the soundtrack from cloud atlas
$ echo !508:6-7
echo cloud atlas
cloud atlas
$

Obviously, the shell's habit of echoing the expanded command before it actually executes it makes these examples a bit annoying, but they demonstrate the concept, and quite frankly, sometimes when you're working with the shell, a bit of redundancy isn't such a horrible thing anyway.

Enough command-line history though. Let's shift to the rather insanely complicated find command instead. There's actually quite a bit you can do with find, and that's why it ends up being a really complex utility. Worse, its command-line parameters are specified in a format quite different from most Linux commands because they're generally in -parameter value pairs, except the very first parameter, which specifies the starting directory for the search.

Unless it's command-line flags that aren't in the -parameter value format.

To make this more fun, there are different versions of find floating around, depending on what flavor of Linux you're running—UNIX, NetBSD, Darwin and so on. I'll try to talk about general approaches to working with find, but if you encounter hiccups, check your man page (man find) to see what your local variant should be.

The most basic use of find is to search for filenames that match a specific pattern—for example, all “C” source files in the home directory or any subdirectory thereunder:

$ find ~ -name "*.c" -print

On modern find commands, the -print parameter is redundant and can be omitted, but if you've an old-school version of find, omitting it might result in zero output—not so useful.

find can check a lot of different characteristics of files too. Here's the same search, but this time, let's say you want only C source files that are more than 1K and have been created 30 or fewer days old:

$ find ~ -name "*.c" -ctime -30d -size +1k -print

And now, here's an explanation, because I know you need one.

Times can be specified in seconds, minutes, hours, days or weeks by specifying “smhdw”, respectively. Size is specified in kilobytes, megabytes, gigabytes, terabytes and petabytes with “kmgtp”, respectively.

Then, a specified value defaults to an exact match, so -ctime 30d will match only files that were created exactly 30 days earlier—not quite so useful. So, the - prefix means “less than or equal to”, and + means “greater than or equal to”. Got it?

You also can search for files by permission string, which is darn useful for administrators. In fact, a very common search that you should do occasionally is this:

$ find / -perm -4000 -user root -print

This introduces yet another parameter: -owner. There's also a group value if you want to check group ID, although in modern Linux that seems to be far less utilized than it was back in the old days—progress, or something like that.

find has an -exec parameter that's worth exploring too, because it lets you invoke arbitrary Linux commands on each and every matching file. So you started with a basic command to list all C source files by looking for matches with the simple pattern *.c, but the output of that is just the filenames—boring.

Instead, -exec can let you output an ls -l result for each one just as easily:

$ find ~ -name "*.c" -exec ls -l '{}' \;

-rw-r--r--  1 taylor  staff  250 Jul  4 09:07
/Users/taylor/Documents/Books/Wicked-Cool-Scripts-10th-Ed/
↪Scripts.new/077-epoch.c

-rwxrwxrwx  1 taylor  staff  631 Sep  9  2009 /Users/taylor/
↪Documents/Linux Journal/distance-between.c

-rw-r--r--  1 taylor  staff  377 Apr  6  2012 /Users/taylor/
↪Documents/Linux Journal/fixdict.c

-rw-r--r--@ 1 taylor  staff  6848 May  1  2013
/Users/taylor/Documents/old.intuitive.com/intuitive.com/
↪scripts/adt/get-adt-news.c

-rw-r--r--  1 taylor  staff  6848 Jun  8  2013
/Users/taylor/Documents/server cli files and
archives/scripts/adt/get-adt-news.c

-rw-r--r--  1 taylor  staff  3085 Aug  1 22:47
/Users/taylor/Library/Application Support/Plex Media
Server/Plug-ins/Framework.bundle/Contents/Resources/
↪Platforms/Shared/Libraries/tornado/epoll.c

It seems like pretty complicated output, but really, it's just an ls -l with a lot of subdirectories making things look more complicated. You can handle that, right?

The parameters required for -exec are tricky. The {} sequence is replaced with the matching filename, but without the quotes, it'll be preprocessed and removed. And, the weird \; sequence is because -exec needs to know how far to read on the parameter list, so it reads until the ;. But again, that'd be interpreted by Bash before even being given to find without the backslash—hence \;.

So, a really common way for sysadmins to search for setuid root programs is a slight variant on the above:

$ find / -perm -4000 -user root -exec ls -l '{}' \; | 
   mail -s "setuid programs and scripts" admin

That'll send an e-mail report that lists all files with setuid and owner root to the e-mail address “admin”.

That's it for this article. Next time, I'll talk about the find|xargs command pipe pair and the substantial problem with files and directories that contain spaces.