Category: life

where everything else happens

Day Five of the big snow of 2008

Not an actual photo of today's labor

I had to get up on the roof today and shovel off some of the snow. We’ve accumulated over 2 feet of the stuff over the past 5 days and it is still coming down. This region is not really adapted to that level of snowfall. Our roof has taken 28 years of rain and a bit of snow, but I figured it would be best to lighten the load before more comes down. The forecast has this going on for a while, so better to minimize chances of problems, right? I can recall the big blizzard of 1995 when I was awakened in the night by the sound of our roof groaning under the weight of about the same amount of snow. The precipitation turned to rain and suddenly that mass of fluffy white stuff turned into a leaden mass, threatening to collapse our house. I climbed up on the roof and shoveled like a madman that night. I filled the entire space between my Ballard area house and the garage with snow. I was able to walk from one to the other! While I was shoveling the garage roof my neighbor’s garage collapsed.

While I’m sure our house is better constructed than my old one in Seattle, it is better safe than sorry.

In my adventurous youth I was an alpinist. I still have all my equipment so I pulled some of it out and suited up for the task. My old Choinard harness doesn’t fit anymore… I’m no longer the trim 195 lb kid anymore… I’m a pear-shaped middle aged man. 🙁 The “Static Point Utility Belt” I had my friend Jeff Wright custom-build for me 20 years ago did barely fit, so that became my harness. In the old days these sorts of things were called “swami-belts” … but mine was built specifically for climbing a wonderful big slab here in the Cascades named “Static Point” which was almost a private playground of mine (and a few friends) back in the Mid-to-late 80s. It is all low-angle friction climbing so a shoulder-based rack was impractical, hence my “utility belt”… this was before harnesses started incorporating rack loops.

I dug out my old mountaineering boots and crampons, a snow shovel, and a rope. I had not worn my boots since I summited Mt. Rainier a long, long time ago. I used to do crazy things like climbing in winter… snow, ice, the works. (One of these days I’ll dig out my kodachromes and write up some climbing stories.) I suited up, tossed the rope (in a bag) and the shovel up onto the roof, and started climbing up the ladder. Just about at the top my left crampon seemed to come off, so I came back down. Upon reaching the ground I noted that the crampon hadn’t come off, but my boot sole did!

These are an old pair of Koflach mountaineering boots, IIRC the second generation of plastic boots. They were hailed as being a technological breakthrough that would last forever compared to leather boots. While they certainly were warmer and drier than leather, my Asolo “snowpine” leather telemark boots from the same era certainly lasted longer! I wonder if REI would take these back as defective? 😉

I put on different boots… a tired old pair of Dynafit TourLite ski mountaineering boots, and ascended the ladder once again. I retrieved the rope and wrapped it around the enormous natural bollard that is our chimney, attached myself to it via a set of ancient Jumar acsenders given to me by Ray Smutek. I made a slow circumnavigation of the roof, shoveling off a thick layer of snow all along the edges. The idea is to lighten the load, and hasten melting into the gutters when warmth returns. I was amazed how quickly my brain recalled rope & crampon protocol. The rope wasn’t going to save me if I went off, but it did enable me to perform a classic tension traverse and let me maintain balance as I walked along the edge of the roofline by giving me something to lean away from. This was critical in the moments when large, heavy shovelfuls of snow would dislodge and fall away. The little ‘hip check” maneuver to tip it over the edge would likely send me over the edge after the snow had I not had a force to counter against in another direction.

About halfway through the job I ascended up to the wide flat chimney top to sit down shed a layer of clothing and cool off. I had also forgotten how hot you can get while laboring in cold weather! As I sat there I slapped myself for not bringing a camera up there to document a bit of this for you all. Oh well. One thing I noted while up there… a triggered memory really… was the full-body tension that comes with a certain sound. The trees would occasionally lose their accumulated snow and the “whump – whoosh – whump” sound they make while doing it is exactly the sound a slope makes when it avalanches. To anyone who has travelled in high country in winter, on skis or on foot, knows… and fears, that sound. I was safe however… merely sitting upon the summit or ridges of Mt. Chez Goolsbee.

After I completed the full-roof-circumcision, I went around the newly rebuilt deck and shoveled off the huge piles I had dumped all over it. Man that snow was heavy! Sue even pitched in on this effort after I wore myself out. I was thinking how nice it would be to have the boys here, as they could have done this WHILE I was doing the roof.

As I finished the task I noted water flowing on the driveway and went down to investigate. Sure enough a pipe had burst in the garage. Dammit. This is that garden hose faucet out at the far end of the garage… the one that is subject to freezing. I’d been running a candle under the faucet itself for days now to keep it from freezing. Why had it burst? I shut off the water to the house and undid the faucet from the wall. No obvious issue there, but something was wrong. Time to break into the drywall! Went through it with a hatchet. I decided to just plug that pipe in the wall. I grabbed my plumbing gear from the barn and tried to improvise a plug. I thought I had it sorted out and turned the water back on. A torrent came streaming out of the wall, from higher up than what I had excavated. Turn the water back off and remove the entire bit of (wet) drywall to find a VERY small burst area in the copper pipe about two feet up from the 90° bend where the faucet goes out the wall. My candle had kept the faucet from freezing but could not keep the pipe inside the wall warm. Oh well. It is now after dark and too risky to go into town to try to find the gear to fix this. Thankfully in Sue’s paranoia we’d filled the bathtubs with water to have in case of emergency to run the toilets. We have plenty of drinking water saved in the downstairs fridge too. We’ll be fine… so long as the power stays on.

We ate a dinner of warmed up leftovers and a nice bottle of Tempranillo. Afterwards I went into my home office to write this up, and… the power went out! This was about 7 pm or so. Nothing to do but crawl into bed. Amazing how absolutely dark it is without power.

Around 3 am the power came back on, much to my relief. In the time between 7 pm and now (~4 am) it has snowed another foot!

Thankfully I left the ladder and rope set up should I need to return to the roof for more excavation.

UNstuck, but still immobile.

Yesterday night, when temps plunged below zero here I gave a go at moving the Jetta out of its snowbank. I grabbed a shovel, dug a bit, got in and rocked it out. I ran it up to an area under a big Douglas Fir tree where there was very little snow. The photo above is taken out my bedroom window. I’d love to put it in the barn, but there is just too much snow to navigate my way through to get there. I’ll put a tarp over it today to protect it from the inevitable release of the snow held by the Fir tree. Let’s just hope no branches come down on it.

Meanwhile, I have to suit up in my old ice-climbing gear and shovel off the roof.

Puffy White Clouds

Since I’m snowbound I’m working on my latest bit of professional writing. This one is about the latest over-the-top buzzword in my business “Cloud Computing”. This is a work in-progress, so feel free to comment. Hit “reload” every once in a while… I’m hacking it up and reordering as we speak! 😉

Here is a soundtrack to have going as you read this (thanks Nick!)

Orb – Little Fluffy Clouds
Found at bee mp3 search engine

[Andy Rooney] So what is all this buzz about “cloud computing” anyway? I really do not understand it. [/Andy Rooney]

From what you read and hear in the buzz surrounding cloud computing, it sounds like a model for how to do things that will just steamroller over the whole industry and make everything we’ve built over the past two decades obsolete. It will allow things to scale without effort, at minimal cost! It is an on-demand datacenter with ZERO capital outlay! It slices, dices, and juliennes! But even in the best-case it seems like it can only really solve a small subset of the industry’s needs. In the worst case it will be a punch line for lame jokes a few years from now, much like other over-hyped buzzwords from the past.

To be honest, I had not really thought much about cloud computing until I was asked directly about it. So I sat down, looked at everything that was running inside the facilities I manage, pulled out Occam’s Razor and started slicing. The first cut was on myself, or at least on my perspective. As a user, what would I want to put “out in a cloud”? What sub-set of my data could safely run on top of a completely unknown and amorphous infrastructure? As a provider, how could I make the cloud model work? How could I build the hard assets required to run a “cloud” and survive in the marketplace? At one level, I totally get the concept. It is sexy as hell. Total software abstraction from the hardware layer. Stuff running everywhere and anywhere. In reality though, I can’t see how it can come to fruition in the traditional commercial model of setting up as a service provider and charging users for it. Like a centerfold model in the flesh, without benefit of an army of stylists before the shoot and a heavy dose of Photoshop afterwards, the sexiness wears off fast. Cloud computing has a lot of unrealistic hopes and desires obscuring plenty of flaws, blemishes, and unresolved issues.

As a user, I could not immediately think about any process running that I would want to throw out onto a “cloud”, so I started with the stuff I knew I could never let go of. Mind you, not that I wouldn’t want to let go of it, just that there was always some aspect about it that keeps it from leaving the building.

First on the list is something that is fresh on my mind: Payment Card involved and/or ecommerce systems. We just helped a client survive a rather intense PCI-DSS audit. The auditors have a very clear idea of exactly what they want to see in terms of server infrastructure, software configuration, and network deployment. Deviations from the script are hard to get away with. Paramount to everything is the ability to audit. To see where, when, and how payment card data is used. When they ask “where is X?” You have to point to a specific spot (be it a server, a file system, or a database table) and say “X is right there.” You also have to be able to prove that X has not been altered without record of it, nor has ever left the building in an insecure or unencrypted state. So can any of this be trusted to a cloud? I doubt it. A cloud is amorphous and indistinct. It is layer 7 abstracted from all the lower layers. You can’t audit a cloud. It is virtual. Sure, we all know that it translates to a physical manifestation at some point, but can you touch it? Can you audit, with absolute certainty it’s filesystems, logs, and physical access? Can you be absolutely certain that it is physically secure? Can you be absolutely certain that its virtualized filesystems are not mingled on a physical disk with somebody else’s data? ABSOLUTE CERTAINTY is required for compliance. You can’t find absolute certainty out there in a cloud by definition.

What goes for PCI also goes for all those other Fully-Acronym-Compliant compliance regulations out there. HIPAA, SOX, SAS70, GLBA, etc. No matter what industry you operate in, there is some regulations somewhere that you either have to be compliant with now, or will have to be in the near future. Further it is difficult to fully detach those systems that require compliance with other corporate systems that interact with them.

Additionally as so many IT managers have learned through hard lessons, data retention for legal purposes is also vital these days. At an ISP I dealt with data retention requests from various law enforcement as well as State or Federal courts routinely. In corporate environments issues of civil and contractual liability also play into data retention. This has traditionally been in the realm of email, but can theoretically extend to any and all corporate communications, documentation, applications, and data. Frequently this transforms into third parties wanting physical access to the data, and just as importantly, audit trails of who has access to the data and systems. Here again Cloud Computing isn’t going to fly because it lacks the absolute certainty that auditors and legal systems require.

So if you have to have audit-safe data, cloud computing is out. If you have to live by any retention rules, which cover more and more data types each year, the cloud gets rules out. So is cloud computing just a solution in search of a problem? If it can not really contain core corporate data, what is it good for? Well… Edge cases.

If you Google the term “cloud computing success stories” you get lots of press releases from cloud computing providers and startups, but very few actual success stories. Those that are there are all edge cases. Situations where prototype applications endure fast scaling, such as a Facebook plug-in, or video content. Cloud deployment allows a startup with limited capital to ride somebody else’s infrastructure to scale quickly, but what happens when they need to, in that term that Biz Dev types love so much, “Monetize” it? Once you start down that path you become entangled in regulatory and compliance realms. That startup is going to HAVE to deploy some of their own infrastructure to support that, and revert to some hybrid-mode usage of cloud computing. The cloud can not contain anything “critical”, only things that overwhelm your ability to scale them. Even then, that deployment may only be temporary, until you can build up your own infrastructure. A start-up could use the cloud as a crutch until it could stand on it’s own so to speak.

So in the end, the cloud is a place to put things of little importance. Items of a temporary nature. Much of the Internet can be described as items of little importance, so perhaps there is something to the Cloud concept. The hard part then becomes making it pay. So then from the cloud provider’s perspective, how can you build a successful business on temporary items & users? Every successful Internet business has been built on the concept of reoccurring revenue. Being hit-and-run by a series of resource-hogging customers doesn’t sound like sound business strategy to me.

The old adage is true… There Is No Free Lunch.
Those of us who have built and maintained datacenters know that doing so on a scale required to truly handle anything thrown at them know that doing so is NOT cheap. The bill has to be paid at some point. Wildly popular web apps with no revenue won’t pay the cost of the servers, much less the electricity bill. I can’t see how the cloud providers can spend the cash to build out the infrastructure and then have enough margin in the usage charges to enjoy healthy profits. They will have to keep their usage percentages high to stay ahead of the capital expenditure curve. Just like all the previous iterations of shared computing resources in the past though, as actual usage goes up, performance goes down. So if they are successful in keeping usage high, they’ll have to keep spending more capital to expand and upgrade their infrastructure. This sounds like Sisyphus on roller skates.

I always like to boil down complex concepts to overly simple descriptions. They help clarify so much fuzzy thought. For example I have always said that the definition of a datacenter is “A place where electricity gets transformed into bits, on a very large scale.” Think about it, power goes in, bits come out. The by-product of that large scale process is heat, which plays into the definition a tad, but otherwise that is a datacenter in a nutshell. So let’s boil Cloud Computing down to it’s most basic definition: Cloud Computing is Datacenter-on-demand.

Datacenters, as we know, are capital-intensive places. They are expensive to build, and expensive to run. It is very hard to deliver something so large and unwieldy in an instant to meet sudden demand. Even using modular techniques. Demand fluctuates, and unless you are going to charge usurious rates when demand comes in, you will be burning cash at terrifying rates when demand is down. The fire will continue to burn even when demand is moderate. When demand suddenly scales upward, it is unlikely you can meet it, unless you have phenomenal amounts of unused capacity lying around burning capital. You can not have truly scalable, redundant, reliable datacenter infrastructure at low cost. The capital and return on that capital have to come from somewhere. The lifetime of a datacenter facility averages between 5 and 15 years. The lifetime of a server is even less, 18 to 36 months. No Cloud Provider wants to be a break-even prospect, much less a money-losing one. So how will any of them survive unless they charge their users far more than it costs to build and run their facilities? See the bowl-swirling process trap here awaiting the potential Cloud Computing provider?

Another thing to consider: So when the provider goes tango-uniform what happens to all your data out there in the clouds? It evaporates. Good thing it wasn’t anything critical eh?

The only real successful “Cloud Provider” today is Amazon, with their AWS services, and their current stance actually backs up my viewpoint. If you read their User Agreement “carefully” as they request that you do prior to signing up, it lays out a service that really should not be used for anything critical or sensitive. It is clear that their model is selling unused capacity on their own systems, and while they’ll be as nice as they can while you are a (paying) guest there, their needs come first. With anything from 60 down to 5 days notice they can terminate the bargain, with cause or without. They also state that neither security nor uptime is guaranteed and that they can suspend the service pretty much at any time they wish, and have no liability to their customers whatsoever in that event. This works fine for low-usage stuff, non-critical software infrastructure, and meaningless items of temporary interest… but it will not fly for mission-critical corporate IT functions.

Finally, one thing I think happens often in the business is Buzzword Overlap. People throw the Buzzword du Jour at whatever concept they are trying to sell. The overlap I see a lot is the Cloud-space right now is “Software as a Service” aka “SaaS”. SaaS can use a cloud as it’s underlying infrastructure but SaaS is NOT a “cloud.” So before you start firing up a flaming rebuttal to my thoughts, get out your own mental knife and cut away the SaaS components from your Cloud ones. I feel that SaaS and other online applications have a strong future. I look at the stuff running in the facilities I manage and good portions of it are SaaS delivery of some sort. The whole mobile market and most web applications are SaaS of some sort or another. The SaaS market is in its toddlerhood, having evolved from the previous buzzword “Application Service Provider” … same idea, different name. Google for example is not a cloud provider per se, they are an application (search, video, mail, chat, etc) provider who happens to use cloud technologies to support their applications. You don’t buy compute or datacenter capacity directly from Google, you buy application time online. SaaS has a future.

So what does the future hold for Cloud Computing? I think it that as an underlying technology it makes a lot of sense. Anyone developing software should do it with the assumption that it will run across many machines and many locations. As a business model though? If I were a Venture Capitalist I’d be chasing people out of my office as soon as they used the phrase. I foresee a lot of “Cloud Computing” startups evaporating like their namesake.

-4°F/-20°C

-4°F/-20°C in Arlington, WA right now. Likely colder here at Chez Goolsbee. Easily the coldest weather we’ve ever seen in the decade we’ve lived here.

I’m running a timelapse out the kitchen window. Captured the sunrise, and should get weird when the wind arrives later today!

Meanwhile I’m digging out and stacking firewood in preparation for the inevitable power outage we’ll experience once the wind picks up.

Here is a photo I took of the boys on Thursday before we left for the airport. They called last night and are having fun… skiing with their grandparents.

Stuck

This winter storm has hit us pretty hard. We have about 2 feet of snow at my house. I had to spend the night at work last night… finally came home about 2 pm. The roads were “ok” … mostly packed snow. I made it 62 miles from my office to my driveway. I rolled the car right into the same spot I dug it out of two days before… then thought I should turn it around so I could exit. Boy was that a dumb idea!

I backed up, could not get it oriented properly, then slid off into the deep snow. It sunk up past the door sills and stopped. The body was high-centered on snow… no traction and could not “rock” it at all.

I think I’m stuck here until it thaws. 🙁

Sorry… been a bad few days.

Started mid-week… Got some bad news (I can’t really talk about.)

Then it started snowing. Never a good thing here in the Pacific Northwest.

Then my laptop died. The old G4 I’ve been driving for over 4 years. I installed the 10.5.6 update and it just rolled over and died. Kernel panic, SPOD, you name it. I tried restoring from backup. I tried re-installing. I even “nuked & paved”… SPOD & Kernel Panic. Sigh. It frustrated me so much that after the last failed re-install I applied some “percussive maintenance” and then tossed it across the room. As you can imagine, it did not take that very well.

I awoke this morning at 4, to over a foot of new snow. The Jetta spent the night in the front of the house, uncovered, as Sue’s CRD and the Jag are sleeping in the garage these days. The boys had a 12:40 flight to catch at Sea-Tac, 70 miles away. I started shovelling out the VW from the drift. I started right up despite the cold. I let it warm up while I got the Liberty CRD out and used it to clear a path down the 1/3rd of a mile down to the road for the Jetta. A couple of laps up and down the drive cleared the snow sufficiently. I packed a shovel, some gloves, and boots, along with the boys’ luggage into the car. The boys awakened and fed, we headed off towards the airport around 8. Once down off the hill and onto the freeway the roads were in much better shape. Still snow covered, but plowed, sanded, and well packed. We were able to move along at a good rate. The snow started again in earnest when we reached downtown Seattle. We arrived in a blizzard, with 2 hours to spare, got them checked in, and then I headed off to the Apple Store at Southcenter to pick up a new laptop.

The parking lot was empty, so I had a little fun doing handbrake turns and generally hooning about for a bit to improve my mood.

Grabbed a MacBook Pro. Most people would be thrilled about this. I was just grumpy.

Drove to my office through and around closed freeways, accidents, jack-knifed semis, and idiots.

Spent the rest of my day configuring a new machine and transferring my data to it. I have plenty of backups, and was able to restore an image of my previous laptop to an external hard drive yesterday. Moving to the new machine SHOULD be easy with Migration Assistant? Forget it. Failed twice.

Had to move everything over by hand. Still dealing with the fallout of that. iPhoto is the issue I’m dealing with at the moment. Sigh.

BastionHost Buys Nova Scotia Data Bunker « Data Center Knowledge

Future Home of a Colocation Facility?

BastionHost Buys Nova Scotia Data Bunker « Data Center Knowledge.

I always do a “rollseyes” when I see these “Datacenter in a Cold War Bunker” stories. One because they are just silly when they tout the “can survive a nuclear strike” capabilities… look, if ICBM’s are falling out of the sky, we’ve got much bigger problems than website uptime!

But wait... I need my email!!!

Second, the facilities in question were designed to house PEOPLE, not datacenters. The power & cooling infrastructure is designed to support something like 90 Watts per square foot at MOST. Datacenter these days wants 500 Watts per square foot minimum. Additionally, the infrastructure is all over FORTY YEARS OLD!

Dude, your draining the amps I need to run the cages next door, knock it off!

To relate it to something most of my readers can understand, that is like asking a early or mid-60s race car to be competitive today. First you have to completely restore it, rebuild it with all manner of modern upgrades, then watch as the new cars pass you like you are going backwards.

Sure the James Bond Supervillian image is cool for about 30 seconds. But after that, you have a facility that can never truly compete without dumping cubic tons of money into it.

This market can’t support the “bunker” model unless the grid power available to it is dirt cheap, and you’ve basically gutted the bunker and completely rebuilt it. At that point what do you have that is competitive?

Oh yeah, nuclear strike survival. When that becomes a selling point I’m getting out of this business.