The Open-Source Classroom

If you think hacking is breaking into Pentagon computers to play “Global Thermonuclear War” with Joshua, you have good taste in movies, but unfortunately, not a clear picture of what hackers do. Yes, there is a subset of folks who take advantage of system vulnerabilities to compromise computer systems. There's a much larger group of people, however, who just use quick bits of code to get their jobs done. These “hacks” aren't nefarious, but are generally not well planned and executed code. Hacks are like digital duct tape, and although you probably can hold an airplane's wing on with duct tape, you wouldn't want to fly it very far. The same is true with the hacks I talk about here. They're generally good for a quick fix, but not something you want to build your infrastructure on. (Unfortunately, simple hacks often get hacked on more and more, and become production systems, which is not ideal, but nonetheless can happen. Use your hacking powers wisely and know when your digital duct tape isn't appropriate.)

In my last article, I described a bunch of simple skills that I now want to demonstrate in action. Basically, I'm just going to think up a handful of things I've done through the years and show you an example script. Then I'll go through them. You probably won't have the same needs I do, but hopefully the concepts will get you thinking. For example, let's start with a script I used to use on my file server to create home directories for newly added users. On a standalone system, the home directories are created when you add a user, but on a large network, the processes are often separate. My users would get added to an LDAP database, and then I'd run the following to create their home directories:

#!/bin/bash
# 
# Create home directories on file server

cp -R /etc/skel /home/$1
chown -R $1.$1 /home/$1
chmod 751 /home/$1
echo "Unless you saw an error, everything is good."

If you remember from my last article, the $1 variable is filled with the first argument given to the script. In this case, it's a user name (like “spowers”). The script then copies the /etc/skel folder and all of its contents to /home with the name of the new user. Then ownership is changed to the user's user name and group, and finally the permissions are set on the user's folder. In my case, it allows non-owners to enter the directory so Apache can read the user's public Web folder. This is a real-world example of how to use the $1 variable. If you had a separate group, you could use $2 to specify. Building on this example, you can come up with elaborate variations to suit your needs.

The next script is far smaller, but it serves an interesting purpose. If you're running a program that is known to crash occasionally (Mono programs are notorious for this, at least in my experience), it's helpful to have them automatically restart. It's possible to create an init script or upstart configuration that will respawn dead processes, but it's often challenging to get the configuration just right. A quick hack is to put the program in an endless loop. Here's an example:

#!/bin/bash
#
# Restart program when it dies
while true
do
  /usr/bin/crashy_program
  sleep 10
done

If you start this script from rc.local or from crontab on boot (see my article in the UpFront section of this issue on using cron to start programs at boot), it will run crashy_program until it crashes, wait 10 seconds, and then the loop starts over, which launches the program again. You do have to be careful, because if “crashy_program” is something that launches itself into the background, in dæmon mode, the script will just keep starting new instances of the program until your RAM fills up. You could add a pkill crashy_program line inside the loop if you want to clean up any remaining processes before starting the loop over, but this method of keeping a script running will work well only if the program doesn't release control of the shell while it's running. Hopefully that makes sense.

From here out, the scripts I show you will be more and more complex. There might even be some stuff thrown in that I didn't cover in my last article, but that's okay, it should be pretty easy to figure out what's going on. Take this script for example, which I use to check my Internet connection at home:

#!/bin/bash
#
# Test Google by IP
wget -q --tries=3 --timeout=5 \
   http://173.194.46.49 -O /tmp/google.idx &> /dev/null

if [ ! -s /tmp/google.idx ]
then
   /usr/local/bin/powercontrol reboot
   sleep 180
   echo "Charter sucks." | mail -s \
        "DANGER WILL ROBINSON: Rebooted Home Router" me@example.com
fi

rm -rf /tmp/google.idx

This is literally the code I use to check my Internet connection and power cycle my modem if need be. First things first, the backslash in a script is just a way of making the commands more readable. All the \ character does is break a single command into multiple lines. The system doesn't actually see the \ character, it sees the entire line. So above, the wget command is a one-liner that ends with /dev/null.

The script itself uses wget to download the Google search page to /tmp/google.idx. I use an IP address because often when my modem is off-line, DNS lookups fail, so that IP address is one of Google's. Anyway, wget tries to download the Google page, allowing for three failed attempts with a five-second timeout. Then the “if” statement checks to see if it failed at downloading the file. (That's what the ! does, it negates the test command.) If it failed, it issues a reboot command to my serial-port-connected power-cycling machine, waits three minutes for the connection to come back up, and then e-mails me a notification of the failure. If wget successfully downloads the file, which it usually does, the if statement is skipped, the downloaded file is erased and the script ends. I run this via cron every 15 minutes or so, and it works well to keep my flaky connection stable.

The next script goes back to the home directory situation. This time, however, I use a “for” loop to affect change to all the folders in the /home directory. See if you can figure out what this does:

#!/bin/bash
#
for x in `ls /home`
do
mkdir /home/$x/public_html
chown $x.nobody /home/$x/public_html
chmod 755 /home/$x/public_html
done

This script basically creates a set of objects from the ls /home command (because it's in backticks), and then executes one loop iteration for each object in the set. The beauty of this is that it will work whether you have three users or 3,000 users. Each iteration of the loop (the part between do and done) creates a public_html folder inside the user's folder and gives it the correct ownership and permissions. You can imagine how much typing this saves for large numbers of users! I use a variation on this type of loop for lots of maintenance issues on user files. If I need to copy a single file to everyone's desktop, a for loop saves the day.

This is probably a good time to remind everyone that quick Bash hacks like these aren't foolproof. It's best if you first have your script do something innocent like echo instead of mkdir, so that it prints on the screen what it is doing. A simple typo could cause you to wipe out millions of user files, so it's best to test your script before using it on your live servers or personal system. This is especially true if you start running rm commands in a loop—that's some powerful mojo, which you don't want to use incorrectly.

Finally, I'm going to demonstrate another way I use quick Bash scripts on a regular basis, and that is to create configuration files. Basically, any time you see repetitious data in a configuration file, chances are you can write a script that will save you lots of time. This script is fairly complex, but it uses lots of the tools I've been talking about. This configuration file is actually part of a script I use to monitor Bitcoin miners, for those who are curious:

#!/bin/bash
# 
BASE_ADDRESS="172.20.1."
LOOP_NUMBER=$(($2 - $1))
# First part of config file
echo "<?php"
# This loop should run for all miners
for MINERLOOP in $(seq 0 $LOOP_NUMBER);
do
echo "\$r[$MINERLOOP]['name'] = 'MINER$(($1 + $MINERLOOP))';"
echo "\$r[$MINERLOOP]['ip'] = '$BASE_ADDRESS$(($1 + $MINERLOOP))';"
echo "\$r[$MINERLOOP]['port'] = '4028';"
echo "\$r[$MINERLOOP]['sick'] = 'FALSE';"
echo " "
done
# And finish off the file
echo "?>"

To make this program run, it needs two arguments. The last octet of the IP addresses of the miners I'm configuring must be entered, so I'd type something like:

./myscript 100 102

And the output is:

<?php
$r[0]['name'] = 'MINER100';
$r[0]['ip'] = '172.20.1.100';
$r[0]['port'] = '4028';
$r[0]['sick'] = 'FALSE';

$r[1]['name'] = 'MINER101';
$r[1]['ip'] = '172.20.1.101';
$r[1]['port'] = '4028';
$r[1]['sick'] = 'FALSE';

$r[2]['name'] = 'MINER102';
$r[2]['ip'] = '172.20.1.102';
$r[2]['port'] = '4028';
$r[2]['sick'] = 'FALSE';

?>

If you follow the logic of the script, you'll see it starts by figuring out the number of loops needed by subtracting the beginning IP octet from the ending one—in this case, 102-100=2. You'll notice there are actually three iterations of the loop, and that's because I'm a little sneaky. I start the loop iterations at zero, so there are three total loops done. Little quirks like this are figured out as you test your scripts and are the reason you must test your scripts before depending on them, even if they're for something simple like this.

Anyway, there are some confusing things in this script that I had to learn how to do while I was debugging it originally. The $(seq 0 $LOOP_NUMBER) statement, for example, is really confusing looking. The reason it's required, however, is because it's not possible to put a variable in a standard range statement for creating a for loop. My first instinct was to say for MINERLOOP in {0..$LOOP_NUMBER}, but that just doesn't work. My brain thinks it should work, but alas, it doesn't. So, using the seq command along with the $() structure, provides the same effect, only with seq, it works.

There might be some confusion with the echo statements too, because since I needed the $ character in my final output, I needed to use a backslash to “escape” the next character. The same thing with the + symbol inside the echo statement. I included the output so you can see what actually happens with the syntax. Please don't think I wrote this script without pulling my hair out in frustration several times. Trying to get an exact format can be incredibly frustrating. In this instance, however, all the script does is print to the screen. That means it's fairly safe to run just to see if the output is what you expected. Once it looks correct, you simply can redirect the output of the script into a file like this:

./myscript 100 102 > config.php

And, you end up with a configuration file completely created with minimal input required by the user. It's important to check that file to make sure it looks like you expect, but generally, you'll see exactly what printed on the screen when you ran the script in the first place.

Well, I'm glad. Being a system administrator, or just a skilled end user, doesn't have to be some mystical dark art. Being a system administrator is more about thinking differently and problem-solving than anything else. It's great to have an arsenal of knowledge and know-how under your belt, but just having the right attitude often is more valuable than having all the answers. If all we needed were answers, Google would manage all of our servers. Coming up with the right questions and knowing what tools to use—that's the real value.

I know some of these scripting examples seem absurdly simple. Some of the most useful scripts are! The idea with this article was to get you thinking about how to combine the various scripting basics into something powerful, something useful and ultimately something that saves you time.