Pages

Wednesday, October 27, 2010

Son of Hamas

A very moving book, and I think potentially very important book.

The gloss: the first son of one of the founders of Hamas grows up full of hatred for his Israeli oppressors, gets caught before he can do anything really criminal, suffers inhumane conditions in Israeli prisons, witnesses Palestinian prisoners torture other Palestinian prisoners, and comes to question the cause he was prepared to fight for. He witnesses the brutality of people in his own culture and comes to think that there may be something wrong with it; seeing that Palestinians were treating each other and the Israelis at least as badly as the Israelis were treating them. He becomes appalled at the universal violence and just wants to make it stop. The Israeli intelligence agency asks him to act as a spy; exposing suicide bombers and violent plots. He agrees on the condition that his information won't be used to kill anyone, only to imprison them. Ultimately this stipulation proves impossible, he's worn down by years of living a double life, and he escapes to the US around 2005 and writes this book about his experiences.

Its moving because of Mosab's empathy, passion, bravery, and his willingness to admit that he had been wrong. Its potentially important because it is an outspoken and clear instance of a person with plenty of reason to be filled with hatred (and the position to act on it) forsaking violence and devoting his life to making peace. Not peace in the sense of a negotiated truce, but a genuine, deep healing that obviates violence. More on this later.

Mosab is truly admirable.

Monday, October 25, 2010

I must post this in the midst of my elation before I run into another crippling error

I was composing an email to the R mailing list to ask for help on the issues I described below. Wanting to seem very serious and well-researched, I double checked everything I was doing before sending it. In the process, I found the error. Maybe I should take that as an approach to problem solving in general: write out my problems as if I'm addressing the ultimate experts on the issue. That way if I fail to figure it out, I have a nice message ready to send, but if I figure it out, then I figure it out!

Here's the email:

I'm trying to set up a large cluster of computers at my university
lab. All clients are running XP SP3, and I'm working from a OS X
10.6.4 machine set up as the master; using R 2.11.1 on all machines.

I've succeeded at getting password-less SSH access to the client
machines using Cygwin, but I'm having issues with initializing sleighs
that seem to stem from the "DOS vs UNIX" path-style issues.
I've tried what seem to be two different correct approches, and get
two different errors.

1. The unix-style way; accessing the contents of the C drive through
/cygdrive/c/...:

s=sleigh(
nodeList=c("
lab211-18.mydomain.edu"),
launch=sshcmd,
user="authorizedUser",
scriptExec=envcmd,
scriptDir="'/cygwin/c/Program Files/R/R-2.11.1/library/nws/bin/'",
scriptName="RNWSSleighWorker.py",
verbose=TRUE)

2. The DOS-style way, where the changes are to the "scriptExec" and
"scriptDir" options:

s=sleigh(
nodeList=c("
lab211-18.mydomain.edu"),
launch=sshcmd,
user="authorizedUser",
scriptExec=scriptcmd,
scriptDir="'c:\\Program Files\\R\\R-2.11.1\\library\\nws\\bin\\'",
scriptName="RNWSSleighWorker.py",
verbose=TRUE)

I've verified that the "RNWSSleighWorker.py" exists in the path
"C:\Program Files\R\R-2.11.1\library\nws\bin\" on the client
machine...

At this point I was going to paste in the exact errors I was seeing
and I realized I was missing a directory ("\R\") from all the path's
above, and that I was doing the unix style one with "cygwin" instead
of "cygdrive". Now, the "dos-style way" seems to be working completely
and totally.

So now I can supposedly do parallel computation. I'm almost afraid to take the next step, for fear of what horrors inevitably await!

unix R and windows not playing nice.


This is beginning to have a very quixotic feeling to it. One of the lab consultants who's been away for five weeks on an internship stopped by today, and as I was updating him on my progress on the project it struck me how little I've actually accomplished in five weeks. True, I've overcome some major bariers and discovered some interesting things, but ultimately I'm still not doing distributed computing.

So far, my accomplishments include figuring out the SSH paswordless login with RSA encription, and getting the client machines to respond to unix commands from the host. Before (when I was using COPSSH), I kept getting an error saying that the 'env' command couldn't be found on the remote machine. 'env' is a really basic unix command, but it seems that it wasn't included in the version of Cygwin that was packaged with COPSSH. I resolved the issue by just installing Cygwin directly. The first time through, I didn't realize how enormous the full Cygwin set of packages is and ended up installing 6gb of mostly unknown material to a computer. I was gratified to see that SSH still works with just the minimal subset of Cygwin packages.

The problem I'm having now is even more frustrating, and gives credit to the recommendation from ReVolution not to mess with Windows clients. The issue is with the quoting of file paths, of course. Unix uses forward slashes and Windows uses backslashes; and in most programming languages the backslash is an escape character. Why oh why would Microsoft have chosen to ignore this reality? Unix and C were around and well established before they came on the scene with their dos-style paths... could that have done it explicitly to make it difficult to operate between platforms (and thus increase switching-costs?).


The crux of the issue is in the last line, right after the "\nws\bin\" bit. See that extra forward slash that follows? It seems thats inserted automatically by the stored function, regardless of what I enter as a parameter. Which is, incidentally, the following:

s=sleigh(

nodeList=c("coblab211-18.business.uc.edu"),

launch=sshcmd,

user="murphytj1",

scriptExec=scriptcmd,

scriptDir="'c:\\Program Files\\R-2.11.1\\library\\nws\\bin\\'",

scriptName="RNWSSleighWorker.py",

verbose=TRUE)


The double backslashes are what's required to overcome the escape character issue. The "scriptDir" is supposed to be the path to the file on the remote machine where the NWS client script is located. But despite the fact that I'm supposedly executing it with a windows style command (what the scriptExec option scriptcmd does), the function appends the forward slash. It is, I think, an error on the part of the writers of the NWS package. Now I'm trying to dig through their code to see if I can fix it; chances are slim.

Equally grim news on the Snowfall front; I can't even get sfCluster running on the host machine. Many many dependencies are apparently missing; starting with the gcc compiler (which I fixed). Even more frustrating, when I call make install, I get another error that looks like it was an issue with the coding:

Ok. Now call 'make install'.
sh-3.2# make install
Making/probing required folders...
mkdir -pv /usr/local/bin
mkdir -pv /usr/local/etc/sfCluster
mkdir -pv /usr/local/lib/sfCluster
mkdir -pv /usr/local/share/sfCluster
mkdir -pv --mode=700 /usr/local/var/sfCluster/tmp
mkdir: illegal option -- -
usage: mkdir [-pv] [-m mode] directory ...
make: *** [install] Error 64
sh-3.2#

What I should probably do is just drop both the macbook and the windows machines and try it on my Ubuntu laptop.

Tuesday, October 19, 2010

Distributed Computing

I achieved a milestone in accomplishing my distributed computing project at the computer lab: I was able to get SSH commands sent from one machine to another without using a password. This is the second major achievement: it allows the master machine to distribute the workload to the workers. The first major achievement was the netWorkSpace server, which allows the workers to access the same variable-space.

HOWEVER, its a limited achievement. I was able to get passwordless SSH working from my Mac, off campus, connecting through the VPN to a windows machine. When I try to do the same thing from a windows machine in the same room, I fail. Go figure: reduce three major layers of complexity, get a bad result.

Ironically, I think the issue arises from connecting from a windows machine. SSH apparently works by checking a certain file for public keys, checking them against the private keys on the remote machine if it finds any, and asking for a password if it doesn't. I think the problem lies in where it's looking for the keys. On a *nix machine (like my mac), it looks in the /User/Home/.ssh directory by default. The closest corollary to that on a windows machine (where I've installed COPSSH (which implies Open SSH and Cygwin)) is buried in "program files/icw/home/user/.ssh." It seems that when I try to connect from the windows machine, even using the BASH terminal emulator, it's not looking in that directory for the key. I've also tried adding that directory to the %PATH% variable (which lets me launch SSH from a CMD prompt), but I still get asked for a password.

I'm quite sure there's a way to make it work, I just haven't found it yet. The consultants in the lab office have all become quite amused at my exasperation... I've tried to recruit a few of the more savy folks into helping me, but thus far I haven't had an enthusiastic response. I guess supercomputing isn't attractive to everyone, eh? How do I reach these kiiids?

On the other hand, a few of the people I've talked with about running problems on it have been enthusiastic. One guy I know has a friend who's working on a protein folding simulation in Matlab (which netWorkSpace also supports), which I think would be an ideal sort of problem to run on such a setup. His description of it sounded incredibly computationally intensive, and it sounds like the problem has the attributes that make it viable for distributed computing; ie: many independent sub-problems arising from a small amount of data.

I've also found some lectures on the subject from Google, which is exciting. http://code.google.com/edu/submissions/mapreduce-minilecture/listing.html
Apparently they teach this series of classes to their summer interns. I'm gratified that my post-graduate studies have enabled me to follow along and understand, at least through the parts I've seen so far. I learned, among other more technical things, that its distributed computing, that I'm trying to achieve, not parallel computing. The latter refers to tasks broken up between processing cores in a single machine, the former refers to tasks broken up between multiple machines. Same spirit; differing challenges.

Added: Dad mentioned that the "before" picture "actually doesn't look that bad..." Its just the light. It was bad. But of course, the most significant parts are the ones you can't see; all the internals that didn't work now (mostly) do.

Comparison Shots



Before:
After:


It's taken some time, and I'll admit that there were moments when I was ready to give it up as a lost cause. But at last, this 1969(ish) CB450 has been rebuilt, somewhat restored, and wholly resurrected as a cafe racer! It starts on the first kick (usually), goes really fast, and only stalls occasionally!

Short list of things I've done to the bike:
rebuilt the engine,
rebuilt the carburetors,
replaced air filters with pods,
replaced mufflers with short chrome exhaust pipes,
replaced clutch cable and lever,
replaced front break cable,
replaced throttle cable (with a hacked-together one),
replaced and sealed the gas tank,
rewired the electrical system for the lights and replaced the headlight,
replaced the chain,
hacked the old seat into something in the cafe-spirit and made a seat cover,
made the right hand tank badge,
bought a vintage left hand tank badge,
painted everything that wasn't chrome,
installed clubman handlebars,
installed bar-end mirrors.

And a short list of things I still need to do:
jet the carbs,
rebuild the stator,
rebuild the clutch.

But that stuff should be easy, right? I think I may also do something non-standard with the chrome panel on the gas tank, I was thinking of doing a checkered-diamond pattern on it in black.

Friday, October 15, 2010

Wedding

Rachael and I went to an awesome wedding last weekend between Steve and Cassie Kemple. It was at the Mockbee, which is an old factory building that's usually used as a music venue. The ceilings are all vaulted and 20 feet high, and there's giant arched windows in the main space with the stage. The did the ceremony downstairs in a long vault that was painted white, and they played Beatles songs. For dinner, there was Adriaticos. Any other pizza at a wedding is sort of 'meh,' I feel, but Adriaticos is like a gift.

The wedding itself was really sweet, you could tell it was a couple that really cared about each other.
We had a blast dancing and helping the bar make money. We pretty much danced straight through (with minor gasping, stich-soothing resting periods) till everyone was gone, then we helped clean up.

Rachael was layin down some serious next-level dance moves with Eric. I was too entranced to get a picture, sadly, but here she is with Patrica and Matt:




Thursday, October 14, 2010

Also, HDR is nice.

Wednesday, October 13, 2010

This what I was saving the piston rings for,

turns out.

This is pretty cool: http://mathworld.wolfram.com/ReuleauxTriangle.html . I knew that that figure had some background; its neat to see that its a formal mathematical concept with a name and lots of history.

Seat cover!

A few too many people said "is that... electrical tape on the seat?" when I showed them the otherwise super-slick photo below (it was actually black duct tape), so I felt motivated to finish the seat.

[Added] A part of this post got cut off when I sent it though the email interface:

I had made the seat cover a few months ago, but I couldn't figure out a good way to attach it to the seat. I was going to just glue it on, but I couldn't get the mounting brackets off of the bottom (they're rusted on). Plus, getting it on tight and smooth would be a nightmare (and likely a total disaster).

Rachael suggested that I sew it on, to which I responded: "Pfffff! The bottom is made of metal, how'm I gonna sew something to metal?" Gradually it dawned on me that she probably meant that I should lace it on, which is, of course, the perfect way to get it tight and deal with the brackets. So thank you, Rachael, for your insight.

Tuesday, October 12, 2010

Vrooooom

Wednesday, October 6, 2010

Parallel computing

After a bout of extreme dwarf fortress enthusiasm (which is about to end in a tantrum spiral), I've recovered some of my enthusiasm for more productive geekery. After work today, I figured out how to remotely uninstall R from windows machines, and to install the newest version. This is essential, you see, for my massive parallel computing experiment that I've been stewing on for a little while.

I got my boss's go-ahead to run the scripts about an hour ago (I can't sleep, and he's got a late-email habit); and now I've got all 141 lab machines ready for paralellization. Now I've just gotta get Snowfall figured out and a good problem coded into it. I talked with a professor today on that subject; he was curious about R's optimization capacities. I discovered (after writing a naive email to him on the subject), that R can interface with all the best solvers, including CPLEX and COIN-OR. I think that means that the project is a good idea.