It’s been years since I submitted a job to a computer cluster, and I’ve since purchased a Windows 7 machine, and have a Windows 7 machine in my office, Windows XP in my lab… Right now we’re testing out how well different reconstruction algorithms work, and how changing the parameters of reconstruction lead to differences in reconstruction speed and quality. We can change some parameters manually and run the recons locally. However for a full-scale comparison over all parameters we need much more power. Thus I’ve embarked on two parallel projects: 1. Get a nice bash shell on Windows 7. 2. Set up an account on the Rice grid, SUG@R: Shared University Grid @ Rice and remember how to submit jobs/write up job requests. A related goal is to get Python going on my x86-64 bit Windows machine. And even though I really love matlab, I want to make sure I can survive without it if there’s a project that would be better off without it, e.g., if you want some web crawling or other “web 2.0” functionality. I mean I don’t think matlab has many dom parsing tools… Matlab does have dom parsing tools OF COURSE. Anyway, matplotlib and numpy together are the open source equivalent. I think the plots look better straight out of the box and the plotting commands are very smooth. The Python shell gives (almost) the same interactive environment as the Matlab command line. Plus I think numpy simply wraps fast linear algebra algorithms in C, so the speed is at least comparable. Here’s how the first part is going:
At first I installed Cygwin, which allows for easy installation of a bash scripting shell and automatically grabs whatever packages, e.g. Python, numpy, vim, and many many many more, the user wants. This all convinced me, so I went with it. However the packages that Cygwin grabs seem to not necessarily be compatible with my 64 bit system. Primarily I was getting an error message that numpy couldn’t build because Python.h was not compiling. I was building from the tarball because although I had added the home python directory, C:\Python, to my %PATH environment variable, the numpy .exe for windows (32 bit) was claiming that Python 2.7 was not on the “windows registry”. After thoroughly confusing myself trying to figure out wtf the windows registry is, I decided to try installing an unofficial 64 bit release of numpy, which worked! The 64-bit numpy installer was able to recognize Python 2.7 on my %PATH, and afterwards numpy and matplotlib were working normally. So probably the problems compiling and the recognition problem were related and the problem has to do with 32 vs. 64 bit. But really understanding all that is beyond me right now…
So the next step was to run Python from the command line in Cygwin. At this point I realized that Cygwin was holding on to its own environment variables and executables, including Python. According to Cygwin, I was still on Python 2.6 and it knew nothing of my victories over numpy and matplotlib. Luckily I still had MinGW installed from a similar previous foray, so I brought up its bash shell and it was completely up to date with the changes I’d made to my Windows environment. Needless to say I will be getting rid of Cygwin… or maybe keep it around but forget about it.
Next: cluster computing!!!!