6 replies to this topic

[AdvMutant]

Hey guys!

I'm in good mood because today I'm almost ready to finally build my own computer cluster, something I've been dreaming on for a while... I hacked ebook68.com(:D) and downloaded "Sam's Linux Cluster Architecture" for free(:D:D), but the English there is kinda hard to understand for me[no idea why] so I still have some questions.

First off, the nodes - The first one has a 1.4GHz Intel processor, can't remember which. 384(256 + 128)MB of RAM, a 20GB HDD and a lame video card. I'll give more info if needed.
I don't yet have the other computers, but I visited my teacher from a school I used to study in, we're good friends so he managed it to get me 2 free Win98 computers, a keyboard, some power cables and a 20GB HDD. Both computers didn't work. lol, he had lifetime warranty so he's gonna call a technician soon. As soon as both are fixed, they're mine. The technical specs are unknown yet.
We also found a compeltly dissassembled PC with the same specs as the broken 2, but I'm not sure if I can have it so I'm just gonna ignore it for now.
So we have 3 computers that I wanna connect to make a cluster, and run a web server on.

First off - Windows Server 2003 or Linux Rocks? I'm doing this project with a friend, he wants Windows and I want Linux. Both have ups and downs. Just to make it clear I'm not gonna purcahse a legal copy of Widnows. What do you think, and why?

I'm gonna connect things somehow like so-

[COLOR="red"]          |

          |[/COLOR]

       [SERVER]

[COLOR="seagreen"]      /        \[/COLOR]

[NODE-0][COLOR="seagreen"]----[/COLOR][NODE-1]


[COLOR="seagreen"]eth0 - private LAN network between the computers in the cluster[/COLOR]

[COLOR="red"]eth1 - internet connection[/COLOR]

Should I get a desktop switch, or multiple network cards on the server?

And another general question about clusters. If I wanna run an application that's not designed to work on a cluster, it obviously won't take advantage of the cluster. Is there a program that will somehow force it to use the other computers? I mean, is there a way to make the three computers act like a singe tiple-core system?

Thanks everybody for any kind of help. Mike :D

[Jrb]

IMO the switch would be easier. Nice job on hacking the website, lol. I've heard of having two computers act as a single unit, but not three. Not saying it can't be done, though. :c-blink:

[Alexander]

If you go the BSD/Linux route a high performance option is Beowulf, some reference documents on what is required to set it up:
http://en.wikipedia....wulf_(computing)
The Beowulf HOWTO

An option for distributively owned systems to be deployed in high performance computing is condor, which was the common infrastructure for PS3 grid arrays
Condor Project Homepage

That hooked up with an LVM (logical volume manager) you can pool together each system's hard disks with an abstraction, which can promote redundancy.

For Windows based systems I believe there are clustering wizards (2003 can utilize this), but it is mainly used for SQL storage on a SAN array.

Edited by Alexander, 28 January 2011 - 10:41 PM.

[John]

1. Use a switch. It will decrease the communication diameter, and increase the bisection width thereby increasing the fault tolerance.
2. How about the best of both operating systems? Make one node Windows and the other node Linux. I believe Beowulf and (I know) Condor work in heterogeneous environments.


As a side note, the industry is seeing a paradigm shift away from compute clusters and toward distributed (data intensive) computing [clusters]. Therefore it would be more educationally beneficial to implement Hadoop and familiarize yourself with technologies like HBase, Hive, Cassandra, BigTable, MapReduce, etc...


(Me with a Condor flock)
 DSCF2039.JPG   213.9K   114 downloads

Edited by John, 28 January 2011 - 10:31 PM.

[AdvMutant]

So I'll get a switch. I can't see much point in having both Linux and Windows systems, what kind of "best" of both worlds will it give me?

And as I said before, will I be able to force running an application with cluster support, without having to re-program it? For example, rendering a heavy 3D model or running a modern FPS. It's mainly for my gamer friend, I'm curious if it's possible.

And yes there is a cluster wizard in both Win Server 2003 and 2008. My friend wants it for it's ease of use. I, however don't give a **** on ease of use stuff, only performance.

PS. Nice system bro!

[Alexander]

You need an algorithm to assign specific portions of rendering, or processing to each node. With gaming you've got two major problems, LAN speed compared to RAM and your game coding - which will doubtfully have parallel coding in mind.

[AdvMutant]

That's true. One of my goals in building a cluster is to get something decent with old Pentium III machines. I coundn't think of something better than running some modern games on it. Well, turns out that it's not possible. My friend just called and said that he spoke with another friend that works in a computer company, and they have some old PCs. Those are Pentium D machines, and we might get ~5 of those. We're gonna stick with a cluster web server then.

Also, take a look at this - Welcome - Synergy
A very neat program I just found. We might also be able to get some monitors. If we get at least 2, a Linux-Windows workstation is a must! I never knew you can have this much fun with PCs that nobody uses.:lol: