This edition of diff -u is dedicated to Andre Hedrick, who committed suicide in July 2012. He was best known for his work on the IDE driver and later the ATA subsystem, which provided essentially the same features.
I knew Andre only from his e-mails to the kernel mailing list. He seemed supremely confident in his understanding of the intricacies of IDE hard drive technology. He had a deep understanding of the specifications and standards, but he also had a deep understanding of the way the different pieces of hardware violated those specs and standards. Sometimes his understanding derived from conversations with the manufacturers and engineers who developed the hardware. IDE/ATA was one of the biggest nightmares of the entire kernel, for years. It was one of those areas that just had to work right, but that was steeped in mystery and confusion. Even detecting which hard drive was on the system was a nightmarish puzzle in many cases.
Andre really could lose his temper though when someone disagreed with him. Once upon a time in 2000, he discovered that the root user could physically damage the hard drive on a given system. To him, this was an important bug, because if malicious users gained root, he felt they shouldn't be able to do real physical harm. But when he submitted his patch to fix the problem, no one wanted to take it, on the grounds that if someone gains root privileges, it's already game-over. During the ensuing flame war, Andre famously said in multiple different e-mail messages, “Here is your SECURITY HOLE! JOE-SIX-PACK-HACKER can fry your butt.”
He may have been hard to deal with sometimes, but I believe his heart always was in the right place. The cantankerous rantings that seemed so abusive at times, all stemmed from a desire to help and protect users, in the face of a truly ugly yet absolutely essential hardware industry.
The kernel configuration system is about to become much simpler for the great mass of users in the world. Linus Torvalds recently put out a call to action, of sorts. He'd noticed that Linux distributions often had odd and unpredictable kernel configurations, and the system might break in subtle ways if users compiled a kernel that lacked some obscure option or other. This typically was not a problem for kernel developers themselves who tend to have a deep understanding of configuration options, but for regular users, he felt it resulted in fewer people being comfortable compiling their own kernels. Linus asked the kernel developers in general to work on providing a minimal default kernel configuration for all versions of all main distributions. This would ensure a working kernel and give users a jumping-off point for their own explorations.
Kernel development often is pretty crazy, and kernel developers like their little jokes. Recently when Alexandre Pereira da Silva suggested adding a “Tested-by:” signature to all of the kernel's git commits, it led to a discussion of all the different types of git signatures that have been accepted into the kernel. Some of the choicest were things like “Fatfingered-by:” and “Heckled-for-on-IRC-by:”. These are actually in the kernel, or at least in the git log. On one level, it's a shame, because people doing data mining to analyze kernel development may have a harder time with their analysis, but on another level, it's all pretty funny.
Last month (the October 2012 issue of LJ), I showed you an awesome audiobook player app for Android, but I didn't share my frustration in getting the audio files on to my phone. When I plugged my phone in to the computer, I couldn't get the SD card to mount, no matter what settings I changed. It was very frustrating and forced me to come up with a better way. Enter: FolderSync.
First off, it's important to note that FolderSync isn't free. There is a free version, but it's limited to a single account and has ads, and although it works, it really isn't the same as the full app. If you're like me, you don't think twice about spending $5 on a fancy cup of coffee, but if you have to pay for an app on your phone, you debate internally for hours. I've never been so happy I spent $2.29 on an app.
Basically, FolderSync works on your phone sort of like Dropbox works on your desktop computer. Instead of syncing only your Dropbox files, however, FolderSync supports a wide variety of data sources. At the time of this writing, data can be synchronized with Amazon S3, Google Docs, Google Drive, SkyDrive, Dropbox, SugarSync, Box.net, Ubuntu One, NetDocuments, FTP/FTPES/FTPS, SFTP, WebDAV/WebDAVs and Samba/SMB/CIFS.
Image from the FolderSync Home Page: www.tacit.dk
In my case, I keep an Audiobooks folder on a share at home, and every night the files are synced up. With the paid version, synchronizations can be forced as well. Although the Audiobook syncing is the only thing I use FolderSync for, it's the perfect tool to keep music, photos, documents or anything else synchronized on your phone without ever plugging it in to the computer. Because FolderSync supports two-way synchronization, it is possible to delete files from your home server, so be careful!
Image from the FolderSync Home Page: www.tacit.dk
After configuring FolderSync to sync my Audiobooks automatically, but only over Wi-Fi and only while plugged in, I realized it had to be Editors' Choice for this month. That means two Android apps in a row win the coveted title, but once you try it, I suspect you'll agree. Check out FolderSync at the Google Play Store: https://play.google.com/store/apps/details?id=dk.tacit.android.foldersync.full, or if you'd prefer to keep your pumpkin latte money, try out the Lite version: https://play.google.com/store/apps/details?id=dk.tacit.android.foldersync.lite.
Usually the Non-Linux FOSS article is dedicated to open-source software for Windows or OS X, but since this month's The Open-Source Classroom column is about surviving in a Windows world as a Linux user, I thought I'd take some liberties here and talk about open-source soda!
If you've been to Penguicon (www.penguicon.org), it's likely you've been able to taste OpenCola, the fully open-source soda-pop drink designed to take the mystery out of cola. The original formula is available all over the Internet (because it's open source!), but there are some great step-by-step procedures as well. Wiki-How has a great outline here: www.wikihow.com/Make-OpenCola.
For a more complex recipe and at least one other flavor, be sure to visit www.opensoda.org as well. Sharing recipes is nothing new for homebrewers of beer, but it's surprisingly difficult to find soda-pop recipes. Perhaps with the supermarket availability of SodaStream (www.sodastream.com), open-source recipes will become more common. In the meantime, if you'd like to try open-source soda, be sure to stop by Penguicon's Consuite next year.
I have a huge collection of NASA photos taken from the Astronomy Pic of the Day Web site (apod.nasa.gov/apod/astropix.html) stored in a folder in my Dropbox. No matter what computer system I'm using, I rotate those images on my background, getting a virtual tour of the universe on every screen. Oddly enough, it can be challenging to get that image rotation to work well in Linux. I've mentioned some wallpaper-rotating applications before, but Slidewall is really pretty cool.
Slidewall includes a small dæmon process to change GNOME-based desktop wallpaper images. It's in the Ubuntu 12.04+ repositories, but it will work with any GNOME-based system. Rotating a wallpaper collection works well, but Slidewall goes the extra mile and will fetch collections live from the Internet, or it can display a live picture of the Earth showing which parts have sunlight and which don't. Slidewall is a huge step forward in desktop wallpaper management, and if you have a difficult time deciding what picture to use as your background, now you don't have to choose just one!
Although many thought Duke Nukem Forever was nothing more than vaporware meant to be used as a metaphor for “never gonna happen”, we were all shocked when it was actually released. Linux users have a similar love/hate relationship with with Valve, the makers of the Steam platform for gaming. Rumors of Linux-native Steam have been circulating for years, but nothing apart from running Steam under Wine ever has brought the popular platform to Linux.
Several months ago, Valve announced it was working on a client, and I excitedly wrote about it, only to look foolish when months went by and nothing happened. Based on communication from Valve, it looks like in October 2012 there will be 1,000 real-life Linux users chosen for an external beta release! Will those chosen few be sworn to secrecy? Is Valve messing with journalists and not even have a Linux plan? For news right from the horse's mouth, check out Valve's Linux blog: blogs.valvesoftware.com/linux.
Many programming languages include libraries to do more complicated math. You can do statistics, numerical analysis or handle big numbers. One topic many programming languages have difficulty with is symbolic math. If you use Python though, you have access to sympy, the symbolic math library. Sympy is under constant development, and it's aiming to be a full-featured computer algebra system (CAS). It also is written completely in Python, so you won't need to install any extra requirements. You can download a source tarball or a git repository if you want the latest and greatest. Most distributions also provide a package for sympy for those of you less concerned about being bleeding-edge. Once it is installed, you will be able to access the sympy library in two ways. You can access it like any other library with the import statement. But, sympy also provides a binary called isympy that is modeled after ipython.
In its simplest mode, sympy can be used as a calculator. Sympy has built-in support for three numeric types: float, rational and integer. Float and integer are intuitive, but what is a rational? A rational number is made of a numerator and a denominator. So, Rational(5,2) is equivalent to 5/2. There is also support for complex numbers. The imaginary part of a complex number is tagged with the constant I. So, a basic complex number is:
a + b*I
You can get the imaginary part with “im”, and the real part with “re”. You need to tell functions explicitly when they need to deal with complex numbers. For example, when doing a basic expansion, you get:
exp(I*x).expand() exp(I*x)
To get the actual expansion, you need to tell expand that it is dealing with complex numbers. This would look like:
exp(I*x).expand(complex=True)
All of the standard arithmetic operators, like addition, multiplication and power are available. All of the usual functions also are available, like trigonometric functions, special functions and so on. Special constants, like e and pi, are treated symbolically in sympy. They won't actually evaluate to a number, so something like “1+pi” remains “1+pi”. You actually have to use evalf explicitly to get a numeric value. There is also a class, called oo, which represents the concept of infinity—a handy extra when doing more complicated mathematics.
Although this is useful, the real power of a CAS is the ability to do symbolic mathematics, like calculus or solving equations. Most other CASes automatically create symbolic variables when you use them. In sympy, these symbolic entities exist as classes, so you need to create them explicitly. You create them by using:
x = Symbol('x')
y = Symbol('y')
If you have more than one symbol at a time to define, you can use:
x,y = symbols('x', 'y')
Then, you can use them in other operations, like looking at equations. For example:
(x+y)**2
You then can apply operations to these equations, like expanding it:
((x+y)**2).expand() x**2 + 2*x*y + y**2
You also can substitute these variables for other variables, or even numbers, using the substitution operator. For example:
((x+y)**2).subs(x,1) (1+y)**2
You can decompose or combine more complicated equations too. For example, let's say you have the following:
(x+1)/(x-1)
Then, you can do a partial fraction decomposition with:
apart((x+1)/(x-1),x) 1 + 2/(x-1)
You can combine things back together again with:
together(1 + 2/(x-1)) (x+1)/(x-1)
When dealing with trigonometric functions, you need to tell operators like expand and together about it. For example, you could use:
sin(x+y).expand(trig=True) sin(x)*cos(y) + sin(y)*cos(x)
The really big use case for a CAS is calculus. Calculus is the backbone of scientific calculations and is used in many situations. One of the fundamental ideas in calculus is the limit. Sympy provides a function called limit to handle exactly that. You need to provide a function, a variable and the value toward which the limit is being calculated. So, if you wanted to calculate the limit of (sin(x)/x) as x goes to 0, you would use:
limit(sin(x)/x, x, 0) 1
Because sympy provides an infinity object, you can calculate limits as they go to infinity. So, you can calculate:
limit(1/x, x, oo) 0
Sympy also allows you to do differentiation. It can understand basic polynomials, as well as trigonometric functions. If you wanted to differentiate sin(x), then you could use:
x = Symbol('x')
diff(sin(x), x)
cos(x)
You can calculate higher derivatives by adding an extra parameter to the diff function call. So, calculating the first derivative of (x**2) can be done with:
diff(x**2, x, 1) 2*x
While the second derivative can be done with:
diff(x**2, x, 2) 2
Sympy provides for calculating solutions to differential equations. You can define a differential equation with the diff function. For example:
f(x).diff(x,x) + f(x)
where f(x) is the function of interest, and diff(x,x) takes the second derivative of f(x) with respect to x. To solve this equation, you would use the function dsolve:
dsolve(f(x).diff(x,x) + f(x), f(x)) f(x) = C1*cos(x) + C2*sin(x)
This is a very common task in scientific calculations.
The opposite of differentiation is integration. Sympy provides support for both indefinite and definite integrals. You can integrate elementary functions with:
integrate(sin(x), x) -cos(x)
You can integrate special functions too. For example:
integrate(exp(-x**2)*erf(x), x)
Definite integrals can be calculated by adding limits to the integration. If you integrate sin(x) from 0 to pi/2, you would use:
integrate(sin(x), (x, 0, pi/2)) 1
Sympy also can handle some improper integrals. For example:
integrate(exp(x), (x, 0, oo)) 1
Sometimes, equations are too complex to deal with analytically. In those cases, you need to generate a series expansion and calculate an approximation. Sympy provides the operator series to do this. For example, if you wanted a fourth-order series expansion of cos(x) about 0, you would use:
cos(x).series(x, 0, 4) 1 - (x**2)/2 + (x**4)/24
Sympy handles linear algebra through the use of the Matrixclass. If you are dealing with just numbers, you can use:
Matrix([[1,0], [0,1]])
If you want to, you can define the dimensions of your matrix explicitly. This would look like:
Matrix(2, 2, [1, 0, 0, 1])
You also can use symbolic variables in your matrices:
x = Symbol('x')
y = Symbol('y')
A = Matrix([[1,x], [y,1]])
Once a matrix is created, you can operate on it. There are functions to do dot products, cross products or calculate determinants. Vectors are simply matrices made of either one row or one column.
Doing all of these calculations is a bit of a waste if you can't print out what you are doing in a form you can use. The most basic output is generated with the print command. If you want to dress it up some, you can use the pprint command. This command does some ASCII pretty-printing, using ASCII characters to display things like integral signs. If you want to generate output that you can use in a published article, you can make sympy generate LaTeX output. This is done with the latex function. Simply using the plain function will generate generic LaTeX output. For example:
latex(x**2)
x^{2}
You can hand in modes, however, for special cases. If you wanted to generate inline LaTeX, you could use:
latex(x**2, mode='inline')
$x^{2}$
You can generate full LaTeX equation output with:
latex(x**2, mode='equation')
\begin{equation}x^{2}\end{equation}
To end, let's look at some gotchas that may crop up. The first thing to consider is the equal sign. A single equal sign is the assignment operator, while two equal signs are used for equality testing. Equality testing applies only to actual equality, not symbolic. So, testing the following will return false:
(x+1)**2 == x**2 + 2*x + 1
If you want to test whether two equations are equal, you need to subtract one from the other, and through careful use of expand, simplify and trigsimp, see whether you end up with 0. Sympy doesn't use the default Python int and float, because it provides more control. If you have an expression that contains only numbers, the default Python types are used. If you want to use the sympy data types, you can use the function sympify(), or S(). So, using Python data types, you get:
6.2 -> 6.2000000000000002
Whereas the sympy data types give:
S(6.2) -> 6.20000000000000
Expressions are immutable in sympy. Any functions applied to them do not change the expressions themselves, but instead return new expressions.
This article touched on only the most basic elements of sympy. But, I hope you have seen that it can be very useful in doing scientific calculations. And by using the isympy console, you have the flexibility to do interactive scientific analysis and work. If some functionality isn't there yet, remember that it is under active development, and also remember that you always can chip in and offer to help out.
Let him who would enjoy a good future waste none of his present.
—Roger Babson
Happiness is not something you postpone for the future; it is something you design for the present.
—Jim Rohn
Change is the law of life. And those who look only to the past or present are certain to miss the future.
—John F. Kennedy
If you want a vision of the future, imagine a boot stamping on a human face—forever.
—George Orwell
The best way to predict the future is to create it.
—Peter Drucker