Reading Dave Taylor's excellent article on extracting movie information from IMDb for a Twitter movie trivia game [LJ, July 2008], I could not help but think, “why doesn't he ...”, on several occasions. To put my money where my mouth is, I rewrote his code snippets as a—IMHO—more readable bash script using more concise code snippets. Here it is:
#!/bin/bash
# imdb-top-250-movies.sh
#
# Felix C. Stegerman <flx@obfusk.net>
# 2008-07-12 [14:15]
#
# --
#
# Output "title | year" for each of the top 250 movies at IMDb.
#
# --
#
# Usage: imdb-top-250-movies.sh > IMDB-TOP-250-MOVIES
#
# --
PREFIX='http://www.imdb.com'
CHART="/chart/top"
TITLE='/title/tt[0-9]+/'
function get_top_250_chart ()
{
wget -O - "$PREFIX/$CHART" \
| grep -E -o "$TITLE" \
| sed 's!^!'"$PREFIX"'!'
}
function get_movie_and_year ()
{
wget -O - "$1" \
| grep '<title>' \
| sed -r 's!^.*>(.*)<.*!\1!' \
| sed -r 's!\(([0-9]{4})(/(I{1,3}|I?V|VI{1,3}|I?X))?\)$!| \1!'
}
for URL in $(get_top_250_chart); do
get_movie_and_year "$URL"
done
# vim: set ft=sh tw=70 sw=2 sts=2 et:
Also, this version correctly parses—for example, “Crash
(2004/I)” is changed to
"Crash | 2004".
—
Felix C. Stegerman
I gave renewing a lot of thought this time around. You see, it used to be
automatic, but you folks seem to be catering primarily to the software
crowd. That's okay, but spread yourselves a bit more freely, please. I am a
hardware engineer and would appreciate some more from that side of the
house. After thinking long and hard, I decided to give you one more try for
one year. You see, I actually found that I am looking forward to my issue each
month; it's just that the programming stuff is hard to get exited about
after many years in hardware. I'm sure I am not alone.
—
Des Cavin
We greatly appreciate your feedback, and we will keep your request in mind.—Ed.
In the Letters section of the last few issues, there has been mention of
Linux being used in different consumer products. Today, while surfing the
Web looking for a new car stereo, I came across the SoundStream VIR-4100N,
which is listed on the Web site as a “4.3"-wide Touch Screen, Din Size,
In-Dash, Fully Motorized TFT Monitor, with Navigation/DVD/AM/FM”. The
third bullet, in a long list of features, says “OS 2.6.x Linux-based
software, 400MHz processor for fast recalculation times”.
—
David Baldock
Regarding Dan Sawyer's review of the Cradlepoint PHS300 [titled “Hot and Bothered at Starbucks”], in the August 2008 issue: the CTR350 does not come with a battery, which is what separates it from the PHS300.
In response to the article “One Box. Sixteen Trillion Bytes” by Eric Pearce in the August 2008 issue of Linux Journal: I also was excited about the prospects of using larger, cheaper, SATA RAID solutions to cut costs on our ever-growing storage needs. I'm not saying it's a bad idea, but there are problems that are not apparent until after you make the investment. These problems can be dealt with, and I dare say that a niche market is waiting on someone to do this, but I've seen other companies fail to do it in the past.
The key problem that does not show up until later is that of SATA drive firmware compatibility. We had eight 400G drives in a box that got turned into the file server for the company. At first, we had a RocketRaid card. Some research suggested the problems we were having were due to the drivers and the card being substandard. So, we invested in an eight-port, 9000 series 3ware card.
It turns out that 3ware has to test every firmware revision of every SATA drive with its cards, because many perform very poorly. After losing three drives, we tried to replace a bad one with a 500G drive, because we couldn't get a 400G drive at local stores, and performance fell through the floor. I updated the firmware, researched on 3ware.com and called 3ware. It came down to poor quality control on the part of SATA manufacturers, and there was nothing we could do about the drives we had. In summary, if you use SATA RAID, make sure you check 3ware's compatibility list.
The bad news is that you probably don't know the firmware revision
until after you open the drive and use it. Refunds get really tricky after
that, so you'll probably go through a few drives trying to find good ones.
It's probably a good idea to have multiple hot spares in your system and to
have some pretested drives lying around.
—
JT Moree
I'm writing in response to Jeremiah Bowling's article on Zenoss in the August 2008 issue of LJ. Although I'm sure it is a good product, there is another one that provides the same, if not more, capability with less setup time: Hyperic HQ. I believe Linux Journal mentioned it in the New Products section a while back.
Hyperic uses XML over HTTPS, using agents installed on hosts to provide metrics back to the HQ Server. It can monitor 10s of 100s of different metrics across different platforms, and it is open source. The process of adding devices and services is performed by the agent through its detection capabilities, which are reported back to the server automatically when the agent is registered. SNMP isn't even needed unless desired. It is open source, but Hyperic sells an enterprise version that comes with support. The only thing it seems that Hyperic doesn't have that Zenoss does have is the geographic mapping capability. Hyperic provides all the metric charts, alerting, roles and monitoring that Zenoss does. I urge those who try out Zenoss to take Hyperic for a spin also. I'm using it at work, and it makes monitoring systems a breeze.
One final note, Jeremiah discusses the setup of SNMP and the use of community strings. Although specifying a custom community string is better than the default, anyone still can sniff the network to determine what the string is, so it really doesn't provide much security. As I mentioned above, Hyperic agents communicate with the Hyperic server using HTTPS, which we all know really is secure.
I appreciate Jeremiah's article, despite my mention of a similar product. I
wasn't aware of Zenoss and always am glad to have my awareness expanded.
—
Brandon McCombs
Look for an article on Hyperic HQ in an upcoming issue.—Ed.
I was interested to see an article about building E-Ink devices in the
August 2008
issue of LJ. I acknowledge that the article involves a clever bit of engineering, but
you fail to mention that display you used, AM200, costs $3,000 US
(store.nexternal.com/shared/StoreFront/default.asp?CS=eink&StoreType=BtoC&Count1=555858790&Count2=472999215).
It's disappointing this fact was left out to make the project sound more
reasonable. Normally, I've found your articles are more honest about the
costs.
—
Jim Putman
Although I always enjoy reading about scientific applications for Linux,
“One Tale of Two Scientific Distros” by Doc Searls in your
August 2008 Upfront section,
hit particularly close to home. My cousin, David Auty, is an
astrophysicist, currently researching neutrino oscillations at Fermilab.
Here's a link to a brief description of his work: www.sussex.ac.uk/physics/profile104724.html.
—
James Knott
The article “One Box. Sixteen Trillion Bytes” [August 2008] showed the use of a 3ware utility to interrogate performance of the RAID-connected SATA disks. Having some systems with problematic hard disks, that got me wondering what general utilities are available to support SATA disks—for example, what goes beyond hdparm?
I discovered that the newer sdparm utility is designed to support SATA disks. However, despite extensive Google searches, other than one good overview, there is a dearth of how-to guides that highlight what one can safely change, or the likely or possible implications of changing the various parameters.
I'm sure a future article in Linux Journal by an sdparm expert would be of
great interest to many since most systems are now using SATA or SAS drives
rather than PATA.
—
David Cartwright
Have a photo you'd like to share with LJ readers? Send your submission to info@linuxjournal.com. If we run yours in the magazine, we'll send you a free T-shirt.
Hannah C. Bigrigg, five years old, Canton, Ohio. Submitted by Travis Bigrigg.