I had a whole lot of old computers in my basement. They were cool, but not real efficiet at doing anything. So my friend Mike Luther says, and I quote, “Beowulf!!!”. It’s all history from there.
Note: this article is outdated, though much of the information is still valid. I grew the cluster to 16 nodes at one point, until the circuit breaker ketps blowing. My cluster now consists of just two SGI 1400′s, each with 4 Xeon proccesors and 2 gigs of RAM, as well as dual NICs. It’s quite good at running MPI apps, as well as spreading image rendering across all 8 procs with OpenMOSIX.
The Devices
Basically, a “Beowulf Cluster” is a cluster of computers built from “commodity hardware” (read: crap) that no one in their right mind would want. Generally they run linux, though some run other types of unix. There are pro’s and con’s, namely that FreeBSD has a faster TCP/IP stack, but hey, whatever runs. (Note: NO ONE uses windows. )
I have a varying number of nodes (because they occasionally get stolen for things, some of them are also dual boot and do other things too). These are connected by 100Mb full-duplex links. (the master node has 2 NIC’s) The nodes are generally between 166 and 550 MHz, with 16 to 300 MBs ram. Sure, I’d like better stuff, but money is not something I have a lot of. (If you’d like to donate a box (or money…), I’d be happy to add you or your company to the donor’s list.) I’m yet to run a benchmark, though no doubt it’d be intersting.
In terms of software, I use RedHat 8. I then installed Lam MPI as my cluster software. This provides an API to send data between nodes. After that, I just shared the home directories on NFS, and made a hosts file so that the nodes could talk to each other. Voila! I know that’s not very specific, but really this is easy! You can email or post in the forums if you want to know more.
By the way, this generates some pretty wicked bandiwdth. If you’re in a production environment, you probably want to give the cluster it’s own network switch(es), because it takes it’s toll. (unlike me: I don’t have enough money for that, so all the boxes in the machine room AND the office are connected via 1 cheap unmanaged switch
The Test
Well, I needed to test this out. I spent quite a lot of time getting it running…. and it sat there.
I though, hey lets look for primes! However, I soon hit a wall: all the previously written code that I could find was for some deep math concept that I didn’t understand. So I wrote my own.
Bascially, all you have to do to test for primality is to test divisibility of a number N by all (prime) numbers up to sqrt(N). That sounds easy…. but think about it. Having a master node spit out number ranges to test is fine, but what happens when we find a prime? By the above test, we need to inform the other nodes so that they can update their list.
That’s hard.
In the end, and do to very high time constraints, I just tossed out the divison by only primes: I just tested it against ALL numbers below the root of N. This results in a very large efficiancy reduction, but at least it works.
So I coded this up, (see the end) and set it running while I went on vacation. I tested the code the week before, and in a day or two I got to about 1.6 billion. So I figured that for a week, I could get to 5 Billion or so.
That’s a problem. In c, the largest size for an integer is 2^32 -1: about 2.2 billion I guess. I talked to some forums, etc, and found a 64-bit (2^64 -1) integer. Yay. That’s several gazillion.
I wrote this in, and on the night I was leaving, at about 11 PM I screamed: printf() doesn’t support 64-bit integers. But I had no choice but to run it.
So I came back home to a 12.3 Gb file of prime numbers….. but all of them about 2.2 billion were in stange terms, cut off, negative, etc, because C was reading the memory wrong.
You can find the code here:
primes.c
Update:
I re-did the entire cluster. It’s now running both OpenMosix AND mpi, and I have several more nodes. I’ve been using it for rendering 3d animations in parrellel with POVray, as well as “crashing galaxees together” with my NBody app.