An interesting thing happened. I dropped reading LJ a while back due to
hating to stare into a monitor. But last week I finally purchased an Amazon
tablet and re-subscribed to LJ because of the
LJ app. It's now easy to read
on a nice screen. Even though I still enjoy printed magazines, I do respect
the environment and agree that chopping down green for this is not good. So
good choice on an environmentally-friendly mag.
—
Peter K.
Thanks Peter! Paper magazines have a dear place in my heart as well, but I can't deny the digital format has some advantages too. I'm glad you're back in the fold, welcome home!—Shawn Powers
Regarding Shawn Powers' “Vagrant Simplified” in the January
2015 issue: great article. I tried Vagrant a few months back, and I couldn't get the
light bulb to turn on. Thus, I put it aside. Shawn's article supplied the
understanding I was missing. Many thanks.
—
Tim Parks
That's exactly what I was hoping for! I'm glad it worked, and I'm glad Vagrant is demystified for a few more people. Thank you for the kind words.—Shawn Powers
A while ago I wrote a C++ program for downloading stock and option information from Yahoo in Windows. I remember it took a lot of code to parse the information, most particularly the option information.
Since then, I have graduated to Linux, and I am currently running on Xubuntu. A friend piqued my interest in option trading that caused me to revisit coding a Linux version of option tracking. I have not yet gotten around to coding any GUI stuff, and I wanted something quickly, so I just wrote a few C++ programs using Geany to run on the terminal.
I really didn't want to write hundreds of lines of code to parse the data and looked around for some XML parsers. None looked easy enough for me to use, but then I looked at the source page, which I downloaded, and tried grep, which led me to develop the following few, or one, line(s) of code that I thought you might be interested as a source for some future articles:
wget -O /tmp/_option.html ↪http://finance.yahoo.com/q/op?s=SPY&date=1429228800 grep 'option_entry\|:volume' /tmp/_option.html | sed -n ↪'s/\r//;s/[^>]*//;s/>//;s/<\/div>//;p' | sed ↪'s/<\/strong>//;s/[^>]*>//;s/<\/a>//;s/%//' > /tmp/_option.txt grep -A8 SPY141226P00230000 _option.txt
These three lines extracted the following information:
SPY141226P00230000 24.27 22.87 24.09 0.00 0.00 10 21 65.14
Of course, with a slight change to:
wget -q -O /tmp/_option.html ↪http://finance.yahoo.com/q/op?s=SPY&date=1429228800 ↪&& sleep 1 && grep ↪'option_entry\|:volume' /tmp/_option.html | sed -n ↪'s/\r//;s/[^>]*//;s/>//;s/<\/div>//;p' | sed ↪'s/<\/strong>//;s/[^>]*>//;s/<\/a>//;s/%//' ↪| grep -A8 SPY141226P00230000
I could say that we can use “one line” of code to parse the Yahoo finance page for the option information!
This is C++ in that I use system to execute. Maybe a popen function might be better as an alternative, but I didn't think of it at the time.
In summary, I thought this was pretty cool, and you may have already done
something similar, but as I said, I thought it might give you some ideas for
future articles. I enjoy and find your articles educational, which are usually
one of the first I read after the titles that catch my eye.
—
Roger
Dave Taylor replies: Thanks for your note and code snippet, Roger. It is rather amazing what you can do with sed, although when it gets that complex, you might consider having the script in a separate file and using the -f FILE option to sed to retain your sanity as you debug it. The problem with all of these crude HTML parsers, of course, is that if they make the slightest tweak on the page, your code's broken. I know; it happens to me all the time.
Regarding Chris Wills' letter in the January 2015 issue [this letter is from David Barton, author of the article “Ideal Backups with zbackup” in the November 2014 issue]: currently I use rsnapshot to back up the zbackup stores with hourly, daily, weekly rotations. Because the zbackup store changes very slightly each time, it is very space-effective. Due to the IO load caused by large numbers of files, very large numbers of servers may want to look at options that don't require linking all the files, such as rotating thin provisioned snapshots, Btrfs/ZFS snapshots or rotating onto removable storage media like tape. I don't think the snapshots need to be replicated, since it is a guard against malicious file corruption—for example, an administrator inserting random bits into the files.
Also, for readers who are interested in using zbackup to back up very large
directory structures, there is a pre-release of software on
https://github.com/davidbartonau/zbackup-tar that backs up directories about 10x
faster on a non-SSD.
—
David Barton
Regarding Kyle Rankin's article “Dr Hjkl on the Command Line” in the December 2014 issue of LJ: it seems that Mr Rankin wants to use vi keystrokes to manipulate the shell command line, so why is he explaining Emacs mode keystrokes? In the shell, all he needs to type is set -o vi and use vi mode from then on. Hit Esc to enter command mode, then hop to the previous word with b, next word with w, change the current word with cw and so on. Even “hjkl” are active, for moving the cursor to the previous/next letter/shell command.
The mechanism is called GNU readline; it supports both Emacs (default)
and vi mode (put set editing-mode vi in ~/.inputrc),
and most command-line tools like the shell or the MySQL client or the GDB debugger will
behave accordingly because they're using the library.
—
Mike
In the December 2014 issue,
seriously, Kyle: set -o vi.
—
Xaveer
In Kyle Rankin's December 2014 column, he describes being comfortable with the vi command set, as I am too. He then goes on to describe bash command-line editing capabilities, using lots of Ctrl and Alt keys. If you've ever been sucked into editor wars, I'm sure it occurred to you that those key sequences seem awfully Emacs-like.
And, in fact, that's exactly what they are. Bash starts out with its command-line editing in Emacs mode. However, bash also has a perfectly functional vi mode that may seem more familiar to you. Just do set -o vi to turn on vi mode.
Now, having said that, I have to admit that I leave my bash sessions in
Emacs mode almost exclusively. The vi mode, like the vi editor, is
modal, and that modality is somewhat non-intuitive in command-line
editing. However, for a vi fan, it's certainly worth exploring.
—
Tim Roberts
Kyle Rankin replies: I remember when I first got really interested in vi that I changed the command line to vi mode. I realized pretty quickly though that I didn't like having modes on the command line and switched it back. In general, I try to keep my environments set to their defaults, so you won't find me with custom bashrc files that set a lot of aliases or anything like that. It's just too much of a pain to ship custom settings like that throughout all my home and work systems, so instead, I try to make the most with the defaults I get.
Happy New Year to you and your team. I had stop subscribing to this
magazine
some time ago and came back because of the digital format. Why?
Because I am a seaman who is away for six months at a time.
Without the digital format, it's hard to keep reading such a fine magazine.
Seriously, I do not understand 60% of what is written, but if one keeps
reading, surely one's knowledge will gradually improve.
Keep up the good work.
—
KokYY
Awesome! Yes, please keep reading. Then after a couple months go back and see if any of the older stuff makes sense. (Don't worry if it doesn't all make sense, however; sometimes the articles make my head spin too!)—Shawn Powers
I just wanted to put in a good word for the They Said It column. The quotes are
not always memorable (although they often are), but they always put me in a
good frame of mind for enjoying the rest of the issue.
—
Steven Janke
Thank you Steven. I enjoy looking for good quotes every month. The hardest part is making sure I don't repeat any (unless they're really good ones!)—Shawn Powers
In the “Secure Server Deployments in Hostile Territory” article in the January 2015 issue, Mr Rankin says: “... it's important to know that Security Groups are assigned only when an instance is created—you can't add or remove Security Groups from an instance after you create it.”
In a VPC, which has been EC2's default for a good while now, this is not true.
You easily can change the Security Group(s) associated with an instance.
—
Gx
Kyle Rankin replies: Thanks for the e-mail. It could be that I'm just showing my age in how long I've been working with EC2, since none of my accounts default into VPCs, and I'm still in the wild west of “EC2 Classic”. That said, I do think having to think in terms of the limitations of the classic EC2 Security Group model helps build more robust security since you can take less for granted.