Work the Shell

I'm in the middle of writing what I'll call a Mad Libs for dream interpretation script, as detailed in my article in the October 2014 issue, but before I get back to it, I have to say that more people have written to me about the leap year function presented many months ago than any other topic in the history of this column.

I never realized people were so passionate about their leap years—and to consider that it's to compensate for the fact that our 365-day calendar is shorter than a solar year by almost six hours per year, starting way back in 1592, an extra day was added every four years (approximately).

The variety of solutions sent in were quite impressive, including some that were presented in FORTRAN and other classic scientific programming languages. Yes, FORTRAN.

The simplest solution proved to be letting Linux itself do the heavy lifting and just check to see how many days were in a given calendar year by using GNU date for a given year:

date -d 12/31/YEAR +%j

If it's 366, it's a leap year. If it's 365, it isn't—easy.

But the winner is reader Norbert Zacharias who sent in this link: aa.usno.navy.mil/faq/docs/JD_Formula.php. You can go there and enjoy the delightful complexity of this US Navy solution!

Now, back to dreams—a perfect segue!

In my last article, I started working on a reader-suggested script that would let people type in a few sentences describing a dream, then extract all the nouns and prompt the user for a free association synonym (or, I suppose, antonym), then echo back the original description with all the substitutions.

With the addition of a noun list and a simple technique for deconstructing what has been given to identify the nouns, most of the code actually is written. Even better, the noun → free association phrase mapping is a one-way translation, so we don't even really need to save it. This means that a sed sequence like:

s/old/new/g

will work just fine, and because that can be appended to multiple substitutions, it just might prove super easy.

Here's the code stub that prompts users for a new word for each existing noun they've entered:

for word in $nouns
do
  echo "What comes to mind when I say $word?"
done

To expand it as needed is easy:

echo "What comes to mind when I say $word?"
read newword
sedstring="$sedstring;s/$word/$newword/g"

That's it. Let's put that in place and see what happens when we create a half-dozen noun substitutions. I'll skip some of the I/O and just tell you that the phrase I entered was “The rain in spain falls mainly on the plain” and that the script then dutifully identified “rain”, “spain” and “plain” as nouns.

The result:

What comes to mind when I say rain?
storm
What comes to mind when I say spain?
soccer
What comes to mind when I say plain?
jane
build sed string 
 ↪;s/rain/storm/g;s/spain/soccer/g;s/plain/jane/g

Great. We're close to being done with the script—really close. In fact, all that's left is:

cat $dream | sed $sedstring

Let's try it:

$ dreamer.sh
Welcome to Dreamer. To start, please describe in a few sentences 
the dream you'd like to explore. End with DONE in all caps on its 
own line. The rain in Spain falls mainly on the plain.
DONE
Hmm.... okay. I have identified the following words as nouns:
 rain spain plain
Are you ready to do some free association? Let's begin...
What comes to mind when I say rain?
storm
What comes to mind when I say spain?
soccer
What comes to mind when I say plain?
jane
The result:
  The storm in Spain falls mainly on the jane.

By George, I think we have it!

Here's the final code:

#!/bin/sh

# dreamer - script to help interpret dreams. does this by 
#    asking users to describe their most recent dream, 
#    then prompts them to free associate words 
#    for each of the nouns in their original description.

nounlist="nounlist.txt"
dream="/tmp/dreamer.$$"

input=""; nouns=""

trap "/bin/rm -f $dream" 0      # no tempfile left behind

echo "Welcome to Dreamer. To start, please describe in a 
 ↪few sentences"
echo "the dream you'd like to explore. End with "DONE" 
 ↪in all caps on "
echo "its own line."

until [ "$input" = "DONE" -o "$input" = "done" ]
do
  echo "$input" >> $dream
  read input    # let's read another line from the user...
done

for word in $( sed 's/[[:punct:]]//g' $dream | tr '[A-Z]' 
 ↪'[a-z]' | tr ' ' '\n')
do
  # is the word a noun? Let's look!
  if [ ! -z "$(grep -E "^${word}$" $nounlist)" ] ; then
    nouns="$nouns $word"
  fi
done

echo "Hmm.... okay. I have identified the following words as nouns:"
echo "$nouns"

echo "Are you ready to do some free association? Let's begin..."

for word in $nouns
do
  echo "What comes to mind when I say $word?"
  read newword
  sedstring="$sedstring;s/$word/$newword/g"
done

echo "The result:"
cat $dream | sed "$sedstring" | fmt | sed 's/^/  /'
echo ""
exit 0

To be fair, this is a bit of an odd script to write, but the basic concept of breaking input down into individual words, processing those words and reassembling the output is something that does have wider applicability. For example, you might use common acronyms but need to have them spelled out for a final report, or language substitution or redacting specific names.

There's also another trick worth noting on the last output line. Let's look at the statement again:

cat $dream | sed "$sedstring" | fmt | sed 's/^/  /'

The first two sections of this pipe do the word substitution. No rocket science there (well, unless your rocket happens to run Bourne Shell, but that's a somewhat anxiety-provoking concept). What's interesting are the last two elements.

The fmt command wraps overly long or short lines to make them all fill in to be around 80 characters long, and then the final sed statement prefaces every line with a double space. I actually use this frequently because I like my scripts to be able to output arbitrary length text that's nice and neat.

Let's grab that great journal from Ishmael and use it as an example:

$ cat moby.txt
Call me Ishmael.
Some years ago - never mind how long precisely - having little or no 
money in my purse, and nothing particular to interest me on shore, I 
thought I would sail about a little and see the watery part
of the world.
It is a way I have of driving off the spleen and regulating the 
circulation. Whenever I find myself growing grim about the mouth; 
whenever it is a damp, drizzly November in my soul; whenever I find 
myself involuntarily pausing
before coffin
warehouses, and bringing up the rear
of every funeral I meet; and especially whenever my hypos get such an 
upper hand of me, that it requires a strong moral principle to prevent 
me from deliberately stepping into the street, and methodically 
knocking people's hats off - then, I account it high time to get to 
sea as soon as I can.

Run that output through the fmt command, however, and it all cleans up perfectly:

$ cat moby.txt | fmt
Call me Ishmael. Some years ago - never mind how long 
precisely - having little or no money in my purse, and nothing 
particular to interest me on shore, I thought I would sail 
about a little and see the watery part of the world. It is 
a way I have of driving off the spleen and regulating the 
circulation. Whenever I find myself growing grim about the 
mouth; whenever it is a damp, drizzly November in my soul; 
whenever I find myself involuntarily pausing before coffin 
warehouses, and bringing up the rear of every funeral I meet; 
and especially whenever my hypos get such an upper hand of me, 
that it requires a strong moral principle to prevent me from 
deliberately stepping into the street, and methodically knocking 
people's hats off - then, I account it high time to get to sea 
as soon as I can.

Now let's indent each line by those two spaces:

$ cat moby.txt | fmt | sed 's/^/  /'
  Call me Ishmael.  Some years ago - never mind how long
  precisely - having little or no money in my purse, and 
  nothing particular to interest me on shore, I thought I 
  would sail about a little and see the watery part of the
  world.  It is a way I have of driving off the spleen and
  regulating the circulation.  Whenever I find myself growing
  grim about the mouth; whenever it is a damp, drizzly November
  in my soul; whenever I find myself involuntarily pausing
  before coffin warehouses, and bringing up the rear of every
  funeral I meet; and especially whenever my hypos get such an
  upper hand of me, that it requires a strong moral principle to
  prevent me from deliberately stepping into the street, and 
  methodically knocking people's hats off - then, I account it
  high time to get to sea as soon as I can.
$

See how that works? You also can preface each line with “>” or any other sequence you'd like. Easy enough!

Well, that's it for this month. Next month, we'll dig into, um, I don't know. What should we explore next month, dear reader?