Thinking Outside The Case

Nice Rack!

Note: The below is a straight off-the-top-of-my head rant I dashed off to my editor at a technology journal I occasionally write for. I'm looking for feedback to tighten it up. Feel free to tear it apart!

When it comes to data center metrics the one most often talked about is square footage. Nobody ever announces that they’ve built a facility with Y-tons of cooling, or Z-Megawatts. The first metric quoted is X-square feet. Talk to any data center manager however and they’ll tell you that floor space is completely irrelevant these days. It only matters to the real estate people. All that matters to the rest of us is power and cooling – Watts per square foot. How much space you have available is nowhere near as important as what you can actually do with it.

If you look at your datacenter with a fresh eye, where is the waste really happening?

Since liquid-cooled servers are at the far right-hand side of the bell curve, achieving electrical density for the majority of us is usually a matter of effectively moving air. So what is REALLY preventing the air from moving in your data center? I won’t rehash the raised floor vs. solid floor debate (since we all know that solid floors are better) but even I know that the perforated tiles, or the overhead duct work is not the REAL constraint. A lot of folks have focused a lot of energy on containment; hot aisle containment systems, cold aisle containment systems, and even in-row supplemental cooling systems.

In reality however, all of these solutions are addressing the environment around the servers, not the servers themselves which are after all, the source of all the heat. Why attack symptoms? Let’s go after the problem directly: The server.

First of all, the whole concept of a “rack unit” needs to be discarded. I’ve ranted before on the absurdity of 1U servers, and how they actually decrease datacenter density when deployed as they are currently built. I’d like to take this a step further and just get rid of the whole idea of a server case. Wrapping a computer in a steel and plastic box, a constrained space, a bottleneck for efficient airflow is a patently absurd thing. It was a good idea in the day of 66 Mhz CPUs and hard drives that were bigger than your head, but in today’s reality of multi-core power hogs burning like magnesium flares it is just asking for trouble. Trouble is what we’ve got right now. Trouble in the form of hot little boxes, be they 1U or blade servers. They are just too much heat in too constrained spaces. Virtualization won’t solve this problem. If anything it will just make it worse by increasing the efficiency of the individual CPUs making them run hotter more of the time. Virtualization might lower the power bills of the users inside the server, but it won’t really change anything for the facility that surrounds the servers in question. The watts per square foot impact won’t be as big as we hoped and we’ll still be faced with cooling a hot box within a constrained space.

So here is my challenge to the server manufactures: Think outside of the case.

This isn’t a new idea really, nor is it mine. We’ve all seen how Google has abandoned cases for their servers. Conventional wisdom says that only a monolithic deployment such as a Google datacenter can really make use of this innovation. Baloney. How often does anyone deploy single servers anymore? Hardly ever. If server manufacturers would think outside of the case, they could design and sell servers in 10 or 20 rack unit scale enclosures. They could even sell entire racks. By shedding cases altogether, both server cases and blade chassis, they could create dense, electrically simple, easy to maintain, and most importantly easy to cool servers. The front could be made of I/O ports, fans, and drives. Big fans for quiet efficiency. The backs could be left open, with electrical down one side and network connections down the other. Minimize the case itself to as little as possible… think of Colin Chapman‘s famous directive about building a better race car: “Just add lightness.” The case of a server should serve one purpose only: To anchor it to the rack. Everything else is a superfluous obstruction of airflow. No need for steel, as plenty of lighter weight materials exist that can do the job with less mass.

Go look in your datacenter with this new eye and envision all those server cases and chassis removed. No more artificial restriction of airflow. Your racks also weigh less than half of what they do today. You could pack twice the computing horsepower into the same amount of space and cool it more effectively than what you have installed.

Ten years from now we’ll look back at servers of this era and ask ourselves “what were we thinking??” The case as we know it will vanish from the data center, much like the horse and buggy a century before. We’ll be so much better without them.

My April Fool’s Day prank.

I love practical jokes, but most years I forget about the license to do them on others that is the 1st of April. It just escapes my mind until it is too late.

This year however I was inspired by something that happened back in December. My friend (and client) John Welch posted something of a love letter to my employer on his website. Basically it said “You’ll never see a “bandwidth quota exceeded” page on my site because my hosting provider doesn’t do that sort of thing.”

That is true, but when I saw it I thought to myself, what a great practical joke this will make! 😉 … and made a note in my calendar to do this on 4/1/2008.

So when people went to John’s site today this is what they saw:

LOL

That “(reference)” link on the “bandwidth exceeded” page goes right to John’s post that inspired the prank.

I actually wanted to make the joke more John-specific… like “This user has expended the maximum number of input keystrokes allowed” or “this user has been suspended for excessive profanity” or something… Unfortunately I spent the day yesterday at our facility in Vancouver BC replacing a broken switch, and fixing a broken server, so I didn’t have the time to write up the HTML. Bill Dickson however, whipped up the PHP code required to pull off the stunt, and wrote that HTML page. Without his help, this one would not have been pulled off.

I hope John liked it.

Just a nice picture, with fond memories

Nick

This is a photo from my 2003 “Summer Roadtrip” with Nick. It is a view of him passing time with his gameboy as we drove from Colorado to Washington in the Jaguar. Note the rainbow out the window.

That trip was indeed magical and I have many fond memories from that week. Nick was nine years old, and a joy to travel with. I had hoped to go on another road trip with him (and his older brother) this week. I was planning to drive down to southern California with one kid, then have the other fly down and send the southbound kid home by plane as well. We have some Jag friends down there that would be fun to pay a return visit. Then I’d return home with the other boy. Chris will be leaving for College in the fall, and I’d dearly like to spend some quality time with them both in the meantime. This week would have worked well since it is spring break for both.

Unfortunately the car is still up on jacks in the barn (I need to order a new body heat shield … more on that later) and it is snowing. WTF is up with that?

Published, again.

I wrote a lengthy bit about communications as a key to surviving an IT disaster, which in many ways was a written version of the session I delivered at the MacIT conference at Macworld Expo last month. I tackle the stereotype of geeks as poor communicators, and lay out a strategy for getting IT departments into the communication habit. The stunning revelation that lead me down this road is a conclusion I came to when discussing an outage with a “layperson”… that is a user of technology rather than a maintainer of it. To him awareness was more important than downtime. Downtime didn’t bother him so much, so long as he was kept informed of what was going on, why, and when things would be back up. Forewarning would be even better. His downtime came about during a datacenter migration. A light bulb went off over my head, as I had successfully pulled off more than one datacenter migration within the past few years. Did everything go perfectly? Of course not, but the difference was that I put a huge emphasis on communication with our customers way before, before, during, and after the moves. I’m not some IT genius by any stretch of the imagination, and I’m not the first to use this tool effectively. It just seems that most IT professionals forget this critical part of their management strategy.

Anyway, for the terminally curious, the series is linked below. My editor wisely split it into two parts.

Part One

Part Two

Finally updating my reading list.

I’m a compulsive reader. My whole life I’ve rarely let much time go by without some book as my constant companion. I try to keep my website here in sync with what I’m reading, but I’ve failed miserably over the last year. I think two books were listed the whole year, while I likely plowed through 4x that number. Right now it still lists “Fiasco“… which was an excellent read. I’ve been through a couple of other books since – I just forgot to update my site here. So in penance, here are a few mini-reviews of stuff I’ve read of late. I will update the site soon with the book I am *actually* reading at the moment.

One of the most profound books I read this past year was “A Nation Of Enemies” about Chile under Pinochet. I have a partially written essay about it stashed away in my “drafts” database, so you’ll have to wait for that.

The Looming Tower” is a fascinating read concerning just about everything we know about Al Qaeda, right up to 9/11. It’s origins in Eqyptian Islamic thought, the principle players, their lives, their philosophies, their methodologies, etc. Right alongside that is a study of the intelligence community, specifically the CIA & FBI, and their continual stumbling over each other due to their basic, fundamental differences of philosophy. The CIA (subject of the book I’m reading now) had always been externally facing and saw the world as a whole, and understood how US Law was irrelevant in many respects in remote places of the world. I may or may not agree with that stance, but the book does detail how the CIA solution to Al Qaeda was to just kill it’s leadership. This was attempted a few times under the Clinton administration, but did not succeed. It was stopped on all but one occasion by the FBI, whose “law & order” mentality conflicted deeply with the CIA’s “kill ’em” plan of action. The FBI wanted to prosecute and punish, which of course requires evidence, due process, etc. They created elaborate schemes to capture bin Laden and extradite him to stand trial for the embassy bombings and the USS Cole. The delays, along with an administration change lead to neither happening. The events of September 11, 2001 therefore were not really a surprise to those in the intelligence community. They knew who it was, why, and in a lot of ways what and when it was going to happen. In hindsight perhaps the CIA’s option would have been a lot more effective and less expensive, but then again we would have been enjoying another decade of peace and prosperity had they did what the CIA suggested.

That leads us to “Fiasco”. To those that feel that the Bush Administration was a united front concerning Iraq, this book contrasts that sharply. On the one side you have the Department of Defense (Rumsfeld, Wolfowitz, et. al.) + Cheney, forging ahead with rosy projections – minimal troop commitments, self-funding effort through oil revenues, “They’ll greet us as liberators!” and PowerPoint based sales pitches. On the other hand you have the State Department asking tough questions … and getting no answers… from anyone. The Military of course does what the American military does best, Logistics… all the while ignoring Strategy, Intelligence, Counterinsurgency, post-war infrastructure, sealing the borders, not to mention exit strategies. They assumed the State Department had all that stuff worked out. What resulted was of course, a Fiasco. A fascinating read, I highly recommend it.

I’ve also read the classic text “This Kind of War” by T. R. Fehrenbach, which covers the Korean War. Fehrenbach is a blunt, plain spoken man with a keen awareness of Military History. Korea is a largely forgotten conflict but it set the precedent for post-atomic ground wars, both politically and militarily. An excellent read though do not expect a coolly detached historian’s view. An interesting aspect of reading this book for me is firing up Google Earth and having a look at the actual terrain being described. I wholeheartedly suggest anyone reading a book about any sort of military conflict do this simple thing. Terrain is a significant part of how battle is played out, and seeing the relief (Korea is an exceptionally hilly place) truly puts a lot in perspective. The simplistic map in the book of “The Gauntlet” run by the 2nd Division on the Sunch’on – Kunu-ri Road, November 30th 1950 is what lead me to look at the actual terrain. From that moment I was hooked.

At the moment I’m reading my Xmas gift from John Welch (Thanks John!), I’ll try to summarize it once I’m done.