Building the Testbed

...or, The eTowers of Power

The heart of the project is the NetGear FS509 ethernet switch. It's got eight 100baseT ports, and one gigabit fiber ethernet port. Costs about $800. The gig ether port gets hooked up to the server under test, via a NetGear GA620 gigabit ethernet NIC for about $300, or a SysKonnect SK-9843, price unknown. Then the question becomes, what's the most cost-effective way to be able to completely saturate the eight 100baseT ports on the switch?

From previous experiments in web server benchmarking, I knew it takes about 200 MHz of CPU power to saturate a single 100baseT link. So, the options were: eight 200MHz machines, four 400MHz machines with two NICs each, or two 800MHz machines with four NICs each. The 800MHz CPUs are pretty expensive at the moment, so that end of the spectrum is no good. New consumer-grade machines are around 400-500 MHz right now, so that was looking like a good option. But then a friend ran across a clearance sale at buy.com. They were selling factory-refurbished eMachines from previous years at bargain basement prices. In particular, the eTower 266, original price $399, was available for only $229 including shipping. The 266 runs a Cyrix M-II CPU at 200MHz (the 266 was Cyrix's fakey "PR" rating), so it's just fast enough for this application. I bought eight of them, and also eight NetGear FX310 100baseT NICs because the eTowers don't come with ethernet. Add in assorted support paraphenalia - a Belkin KVM switch, cables, UPSs, shelves, and I had myself a testbed. The eTowers run FreeBSD 3.4 just fine, and tests show they are indeed able to saturate their ethernet - over 90Mbps.

Another more recent option might be the new Oracle network PC. Only $199, but that doesn't include any hard disk, so you'd have to either add one or run without it (probably reasonable). On the other hand it comes with 100baseT ethernet, so you don't have to add that. It also has twice the RAM, but it doesn't have a floppy. It uses the exact same 200MHz Cyrix M-II as the eTowers.

Anyway, after setting up the eTowers I got to thinking: what if someday I want to do some other parallel-processing application with these machines. 200MHz is kinda slow - could I upgrade them? The answer is yes! The Cyrix M-II CPU is a socket-7 part, running with a 66MHz front-side bus at a multiplier of 3.0x. The TriGem motherboard in the eTower has switches and jumper settings that let you set it to a 100MHz FSB and multipliers up to 4.5x. Well, a 450MHz AMD K6-2 fits in socket-7 and costs only $50-$60. I got one, stuck it into one of the eTowers, and it worked! For a day, anyway - then it wouldn't boot. Well, that's a typical "stability problem" that overclockers run into, so I tried lowering the FSB frequency back to 66MHz, and then it booted again. However, with the multiplier set to the motherboard's maximum of 4.5x, that gives only a 300MHz CPU clock - not great. But!! K6-2s have a little hack built into them, specifically for older motherboards - when they see a multiplier setting of 2.0x, they interpret it as 6.0x. I tried this and it worked, giving a 400MHz CPU clock.

Then I tried raising the FSB freq one step to 75MHz, bringing the CPU clock back up to 450MHz. Two of the eight systems wouldn't boot at this speed, so I had to back them off to 66MHz/400MHz. The rest are running fine at 75MHz/450MHz. They actually test out as slightly slower on non-cache memory access this way. I'm not sure why yet - some sort of timing thing. However most problems are small enough to fit into cache, so the 12.5% improvement in CPU speed is worth the tiny slowdown in non-cache memory speed.

Two other CPU options are the K6-III and the newer K6-2+, both of which have an internal L2 cache. These both do the same 2.0x to 6.0x multiplier hack, and are definitely faster than the K6-2, but they cost substantially more. I decided they weren't enough faster to justify the extra cost, especially since the CPU upgrade is basically a frill - for the main purpose of the client array I don't actually need any CPU upgrade at all. However, if you were upgrading a single machine, say for use as a cheap games platform, one of these would probably be the way to go.

In addition to changing the clocks, you also have to use the switches to change the core VCC - the K6 chips want voltages of 2.0V, 2.2V, or 2.4V, depending on which kind you have. If you have a choice, lower-voltage parts are preferable because they use less power and run cooler, but the 2.4V ones will definitely work. The eTower settings table doesn't have entries for 2.0V or 2.4V, but if you interpolate its missing entries you can figure out the proper settings.

Here's a table showing only the things you need to change:

One other note, the power supply fans in these refurbished eTowers are apparently somewhat marginal. One of my eight arrived making noise, a second started making noise a few weeks later, and a third a few months after that. I found a company that makes replacement power supplies specifically for the eTowers. It's the 145 SFX-L, marked "eMachines". Got one, installed it, works great. I keep an extra one around now. If you get an eTower you should save this link!