Adventures in creating an OctoCore Mac Pro – Part 1

Considerations in Application of a GeoThermal Heat Exchange for Environmental Comfort Following the Upgrade of a MacPro1,1 with Two Over-Clocked Intel X5350 Engineering Samples that Necessitated the Addition of a Water Cooling System

or

“How I Learned To Stop Worrying And Love Intel”

By Robert Bruce Campbell, DO

43,000 attended the 2010 Texas High School Football Championshipat Cowboy’s Stadium, Arlington, Texas

Background

It was the 2010 Texas Class 5A Division 1 High School Football Championship that set me on this road.

BOY! WHAT A GAME!

My son and I flew to Dallas to attend, and Iʼd set my computer to record it while we were gone (in HD, of course).

43,000 attended the 2010 Texas High School Football Championship at Cowboy’s Stadium, Arlington, Texas

BOY! WHAT A GAME!

Did I already say that?

Me and Bubba couldnʼt stop talking about it, discussing and picking apart every little detail over and over. Even the wife, not usually excited by this kinda stuff, got into it. So I decided to take my legally acquired copy of the game on my computer and “time/place-shift” my copy so the whole family could watch it on my 5 foot Panny Plasma together, legally, as provided for by the US Supreme court.

Off I went into my office and got to work on this three hour long HD file. Cutting and slashing things like commercial breaks and talking heads, I put chapter markers on each play. Two versions, one for the Panny, and one for the iPad/iPhone.

It was a big project and the new version of iMovie, unlike itʼs predecessor (or even Adobe Premier, which I learned on), is kinda counter-intuitive. El Gato, in their EyeTV package, had a nice piece of editing software as well as a hardware encoder (the turbo.264HD), but it also fell short (but nonetheless is quite impressive!).

So I defaulted to QuickTime Pro. Simple and powerful, you could place chapter markers, subtitles, transitions, ect., and it kept audio and voice-overs in perfect sync.

But I was already kind of tired from the time I spent experimenting with the new iMovie and other products and felt the time encoding was just taking too long. Not only that, but my MacPro (MacPro1,1), with 2 Intel 5160 processors, was kinda slow & was throwing off tons of heat.

So I had two problems; Speed, and Heat. Now, Iʼve always had heat issues. My home office is an add-on and with me, the MacPro, two 30 inch Apple Cinema Displays (themselves 150 watts per), a scanner, printer, and a multifunction—well, thatʼs a lot of heat in there. Worse, as you get older and fatter the heat bothers you more. I also donʼt think the neighbor kids like seeing an old fat guy in his underwear at a computer. Much less when heʼs sweating like a pig and wiping his face and underarms with a shop towel. Hell, I wouldnʼt want to see that (but on the upside, they donʼt come around as often).

But one thing atta time…

The the folks in Cupertino had already thought up a solution to the Speed issue. You see, built into the core of OS X is an app called X-grid that allows parallelization with other computers, and some other fine folks at a group called Dauger Research, Inc. made a neat little app called “Pooch” to work with QuickTime that let me share the encoding load amongst our household computers.

So things were really looking up for me for a while until the wife unhooked her MacBook Pro from the grid to go outside, and Bubba shut down his iMac to go to sleep.

Can’t we all just leave my grid alone?

Nah, really, it woulda worked well if everyone had just been away for a weekend.

So nothing left to do but upgrade. I was a bit unhappy with this. You see, Iʼve owned Macʼs since 1990 (first being a PowerBook 100) and have real issues with buying a new Mac before the six year minimum. But Iʼd read about the “Cloverfield” upgrades people were doing & it seemed a reasonable solution.

So off to eBay I went looking for a pair of the magical quad core 5365ʼs, only to find that though they could be purchased, the sellerʼs were requiring an exorbitant price and un-natural acts from the buyers. And this was bad because besides being cheap, I, uhh…, I donʼt go there.

But Lo and Behold, after a bit more searching “Intel pair”, I found for sale “a pair of Intel 5350 engineering samples”. From what Iʼd read, this was a completely different kind of Magic from the PC Worlds beyond this Universe. You see, with a clock of 2.66Ghz and a FSB of 1033, you could cast the BSEL charm on them, so not only were they powerful in their own right, but say the right spell while drawing with a circuit pen and itʼs like an un-ending supply of Adrenaline pills in UT2K4. Oh yeah baby, we had us a “Buy Now” moment.

So I swapped my new 5350ʼs for the 5160ʼs and finished the project.

The End

Boy, I wish I could say that…

BSEL “tape” mod (upper left hand corner tape covering a pin)

You know that story Jerry Clower tells about coon hunting, and his friend John is stuck up in the tree doing battle with a lynx? I think I know how John felt now.

So because Iʼm a masochistic retard, I went ahead with the BSEL mod. Over and over I tried, but it just wouldnʼt work. It would chime (POST) but immediately choke like Cass Elliot on a ham sandwich.

Google, Google, Google.

Read, Read, Read.

You see, to get a 5350 to run at 3.3 Ghz you gotta do the volt mod too.

Wait? Whu’d he say”

A whut mod an a whut mod?

(Hold ya hosses boys, I’ll be gettin’ there…)

And whuudya know… it worked!

Sun-E-Beaches, I got me a 3.3GHz Octo-core MacPro! WooHoo! Letʼs do some video!

Begin Encode…
Panic.

Hmmmmm…

Restart, Begin Encode…
Panic.

Restart, Begin Encode…
Panic.

Restart, Begin Encode…
Panic.

Arrrrrrgghhh!

It was the heat. Within minutes, Handbrake would drive the CPUs over 70ºC and the computer would auger-in like it was carrying Buddy Holly and Ritchie Valens to a show.

I know I should have undone everything then & there and thought myself the better for it. But remember how your Dad used to look at you incredulously and ask, “Dammit Boy, ainʼt you got no sense at all”? Yeah? Well me too, so “Onward thru the fog” I went.

Well, the short form is this: If I waited until 3 am, opened the windows to the outside until the ambient room temperature was 35ºF, took the silly cardboard and duct tape contraption I made that directed the air from my shop fan into the front of the case, AND set smcFanControl to run all my fans at 3,000 RPM…

“What in God’s name…!? it’s sounds like a 737 in here!” (What can I say? I married a Flight Attendant)

“Go back to bed, babe”

“It’s 3:30 in the morning! What on earth are you doing?”

“Proving Dad wrong, babe. Now go back to bed”

“Honey, your father has been gone now 5 years…”

“And he’s the one that always said ‘better late than never’. Go back to bed, babe”

and I almost finished the entire encode.

I gotta find a better way to cool these pups…

Google, Google, Google.

Read, Read, Read.

Yup, so this is where the water cooling comes in.

You see, it was simply amazing how fast these these CPUʼs would heat up. If they didnʼt literally start in a frozen state, you had no chance with the OEM Apple supplied copper-to- aluminum heatsinks. (Now, thru the retrospectoscope, I think a very large all copper conventional air-type heatsink may have had enough heat capacity and diffusion ability to handle it from a near frozen state, but how much fun would that have been, huh?)

Physics tells us if you canʼt get rid of it, youʼre gonna have to hold it and, my friends, nothing holds heat as good as water. (Just ask anyone who has ever seen a hurricane up close and personal, theyʼll tell yaʼ.)

Water cooling it is. Sooooooo….

Google, Google, Google.

Read, Read, Read.

I got a little educated. I learned things like laminar flow, turbulence, and “Delta-Tea” (I think itʼs kinda like a Mint Julep, except for folks from Mississippi), and went out to spend some money. And boy howdy, did I.

Materials and Methods Part 1- The Whyʼs

This was painful. All that I initially wanted was something that could rapidly absorb enough heat to keep the the thing running, and I thought Iʼd found the answer. You see, Iʼm not the first person to water cool an over-clocked MacPro—some guy named Zack Fanning at Asetek had already done it, so all I needed to do was copy him. But Zack had his own pet engineer from the company to do him a complete custom job (and like all administrators, had no freakinʼ clue how it was done). Worse yet, that engineer had gone and found himself a new owner. (I heard NASCAR. That may be sour grapes though.)

But youʼd think that with Intel selling 20 billion or so 5000X type servers that thereʼd been a few folks that put water cooling systems on them. Yeah well, Homey donʼt think so. I mean who in their right mind would water cool a rack mounted server?

“Well, Hell… Iʼd just add more air conditioning…” I was told by the Computer/IT Engineers (their facial expression I interpreted as them thinking that I had too many- or too few- base pairs to play in their sandbox).

By now it was obvious from every engineer that I talked to they werenʼt going to help, so I grabbed my helmet, crayons, and went back out to the corner to wait for that short yellow bus to take me home.

The problem was finding a way to mount a water block to the motherboard. See, not too many aftermarket water blocks have mounts for the LGA (Land Grid Array) 771 socket which is what the MacPro1,1 uses.

Koolance CPU-340 with mount

But I did find one. Finally.

The discontinued Koolance “CPU-340” had a mount that could be used on a 771. And although it was big and heavy, should be mountable on a 5000X type server with the accompanying mount. But notice your dear MacPro isnʼt a rack mounted server. And please take special notice of the memory cage used to support the DIMMS and provide the laminar airflow that you wouldnʼt need to if it was a 1U server. In other words, Dorothy, youʼre not just gonna slap a pocketwatch in the Tin Manʼs chest an have it work. This thing is doable, but I was going to have to find a way to get a square peg into a round hole. Good thing Iʼm an ER Doctor, huh? We eat problems like this for lunch everyday. You give me a big enough hammer and Iʼll drive the Empire State Building through the smallest round hole you can think of.

Next, remember how I told you that heat bothers us fat people? You do? Good, because my office is still going to be hotter than it was before this little brain surgery was done. You see, each original 5160 in the MacPro has a TDP (Thermal Dynamic Power) of 80 watts, but a 5365 has a TDP of 150 watts (but twice the number of cores!). And for arguments sake, letʼs say the top thermal power of a 5365 would get to is 120 watts at full load, OK? Well, a 5365 is runs at 3 Ghz- Iʼm at 3.3 GHZ- and assuming (rightly so) power is proportional to frequency then my chips should draw 132 watts (3.3/3=1.1, or 110%).

Like the math, huh? (Always makes the guy writing the article look like he knows what heʼs talking about.) Then hereʼs a little more for you.

P=v²/R. It means the power is proportional to the square of the voltage. And thatʼs what the VID mod does to the core voltage. So is an estimation of 132+ watts per chip reasonable? Yeah. Probably even a bit conservative.

What does all this mean? Imagine of adding two 100 watt incandescent bulbs inside your Mac and then shut the side panel. Now hold that thought in your head for a minute. OK? Good. Then as soon as youʼre done sitting there screwing off with that stupid look on your face you can go wring out my shop towel ʻcause things are gonna get hot!

basic water flow diagram

Next, all that a water block does is transport the CPUʼs heat somewhere else. Most “Over Clockers” will have the water pumped to a radiator with fans attached to dump the heat into their room. The cool water, having lost its heat to the room, loops back to the CPUʼs to start the process all over again. They even have to back off their overclocks in the summer due to the heat dumped into their rooms. You see the problem here, right?

(Sigh…)

OK. One more time: Fat guy, hot room, summer coming. Got it?

Hmmm. I should remind you again that Iʼm a cheap bastard. But you probably shoulda figured that one on your own since I didnʼt just buy an air conditioner.

So you see, it isnʼt just that I needed to cool the processors, I needed to get their heat out of my office. The rub is that I donʼt wanna pay for it. Heat as it turns out, flows down hill, from hot to cold, for free. Thatʼs what makes the ice in your tea melt. But in Texas, when its 90º outside, the heat is going to have to flow uphill to get out of my office and that, my friends, is the what that makes the guys at the STNP smile.

A radiator with fans blowing hot air over it just heats the water more. I needed a free cold “sink” for my heat to go to. I looked all around me, over and over, but no real sink could I find. The it occurred to me- donʼt look “around”, look “up and down.”

Up. Space. The universe. Now thatʼs cold! Like only several degrees Kelvin cold. Just look up some night. That blackness is essentially the absence of any heat at all. A “downhill” flow”? Itʼs a bloody downhill plummet. And believe it or not, itʼs a doable technology too, but just not with the “space” I had available to me.

So then I looked down, and guess what? There was a big ʻole heatsink right at my feet!

Google, Google, Google.

Read, Read, Read.

The idea of a geothermal heatsink again isnʼt original, but isnʼt common either—at least for computer people. Fortunately, the guys that did do it published their experiences on the Web to help the rest of us out.

And thereʼs more than a couple of ways to do things, depending on such things as where you live, the type of home, and your budget.

Not only that, I didnʼt need much space to make this thing work—after all, it was to be buried. Buried where you ask? Well, itʼs buried in the only logical place I had to bury it.

Right outside my office window.

In the wifeʼs flower bed.

Now, for you married gents out there, Iʼm sure you see the problem I faced here. The solution however, beautiful for its grand scale and elegance, really is beyond the scope of this article. Actually, statistically speaking, itʼs beyond the scope of 97% of you fellas reading this. But to that last 3% remember that story that goes, “the way to a manʼs heart is through his stomach?” Well, the way to a womanʼs heart takes a slightly different route.

There. It is decided. Iʼm gonna have me a 3.3 Ghz GeoThermally Water-Cooled OctoCore MacPro1,1! Whoop! (a “Whoop” is an Aggie thing. You wouldnʼt understand.)

The decision on the type of exchange really was decided by the space available to me rather than the ease of materials Iʼd be working with. Initially Iʼd considered is burying a length of PEX (high density polyethylene) tubing, but on paper its Thermal Conductivity looks horrible. Now, for you folks out there that say its actually much better than the “Engineering Toolbox” lists- Yeah. OK. Iʼm hip. But I still was going to have to dig up way more than what I had available in my allotted space, so now as then weʼll stick to copper.

Copper you see is the God of Thermal Conductivity. Better than gold, silver, or aluminum. Itʼs fairly cheap, and easy to work with also. But the biggie is the 67.8 watts/hour/ºF/ft2 selling point. And thatʼs big. Like “André the Giant” big.

So how much pipe will I need?

Although I was seeing my power at the wall closing in on 500 watts, I didnʼt think it would be realistic to believe that I could recover near that much heat. But it did seem like a nice round number that would also leave me room for growth (in case I wanted to expand my water cooling to include say.. two 30 inch ACDʼs?). Short form- 1/2 inch nominal pipe (5/8” outer diameter) is about 1 square foot surface area for every 6.1 feet of pipe (cylinder surface area= 2ᴨrh). It comes in 50 foot rolls. Figure losing 10 feet per roll in screw-ups. That leaves 6.5 ft2 to radiate heat, which works out to 440 watts/ºF/hour, which is another nice round number.

The Easter Bunny with a craniotomy. Remember this image if you’re considering following in my path.

But the deciding factor? It comes in 50 foot rolls. Duh.

Stay tuned for the next exciting episode
Materials and Methods Part 2—The Howʼs

See how Dr. Campbell maintains complete irreverence to authority as he and his trusty AK-47 “Bess” battle to save a small Vietnamese fishing village from the ravages of scurvy…


The opinions expressed here are those of the author and do not necessarily reflect the position of Other World Computing.

LEAVE A COMMENT


  • Hey Robert,

    Nice and entertaining blog! I couldn’t help but chuckle a bit when my name popped up on OWC’s blog. I’ve been a fan for many years, going back to the golden years of MacWorld SF.

    Sorry I couldn’t be of more help getting you set up with liquid cooling, but as you know mine was a one off job.

    Maybe if we get enough people to tell Apple to include Asetek liquid cooling in their Mac Pro’s, they’ll do it! ;-)

    -Zack (Asetek.com)




  • Best sentence ever:

    Considerations in Application of a GeoThermal Heat Exchange for Environmental Comfort Following the Upgrade of a MacPro1,1 with Two Over-Clocked Intel X5350 Engineering Samples that Necessitated the Addition of a Water Cooling System

    I’ll get back to after I actually read the article.

    O-Dub Scott




  • “Like “André the Giant” big.”

    You gotta be kidding me!

    What happened to the “Fat Elvis dead on the crapper” metaphor? Y’all cain’t be afraid offending the Cracker Hillbilly readership ’cause I’m the only one of them that can actually read!

    Wait… I did that?

    Hmmm. Guess I shouldn’t have used that lead solder in sweating that still together. And I think I’m going blind too. Nuts!




    • Well Robert…we try to be a bit PC here….and I’m not talkin’ computers…;-)

      We do love all your metaphors…but realize some readers’ children might be reading it too…and for sure I don’t want to have to explain to mine any part of that phrase! lol Because then it will likely become part of a rap song, recorded, and put up on You Tube to entertain…esp by my 9 yr old son who would find that phrase hilarious and beat it into the ground.