life – Page 53 – chuck.goolsbee.org

A break in the weather, the 65E comes home.

This past week I got a call from the body shop that they were done with the E-type.

The day before they had told me it would be another week and I was unprepared to pick it up since I was in Seattle in the Jetta. Adding to the confusion was the fact that it was a clear day, likely the last we’ll see for quite a while. It was a “now or never” thing if I wanted to get it home without driving in the rain. I got a ride up to Redmond with Damian, picked up the car, and drove it to my office. It gets dark here now around 4:30, and I don’t like driving this car after dark. I grabbed Dan, our new Windows admin, as a ‘dead head” to use I-5’s carpool lane up to Everett where he lives. I think he enjoyed the ride, even though he didn’t have a coat. It was dark by the time I got him to Everett, but I was able to get home a lot faster than if I were alone. I stopped and filled the car up, stopped at NAPA to buy some gasoline stabilizer, and put the E-type away for the winter. Yes, there will be some projects to do with it, and I may take it out for a spin if the weather breaks, but mostly it will hibernate from now until spring.

The dent is gone:

The numberplate on the nose is gone as well. =/

The body shop offered to make me a new one, and I think I’ll take them up on it. I know a lot of folks think they are “ugly”… mostly due to the fact that they are required in the UK, so like all governmental interference, they are seen as intrusive. Here they are not, and you never see an E-type with a big black numberplate over the nose… except mine. I’ve grown to like it over the years… sort of like a mole on a supermodel’s face, it is the flaw that makes the whole that much more desirable. Unfortunately I did not have any idea it was at risk, so I never documented it. I have plenty of photos, but nothing specifically of the numberplate, so now I’m scrambling to reverse-engineer one. The font used is something of a mystery. At first I tried to research what font was used, or find an equivalent. No luck. Then I took an old photo and enlarged it to full size (18″x4”) and started a Beziers Curve drawing of it that could be rendered in PostScript. I haven’t worked in PostScript since the late 80s/very early 90s, but back then I practically dreamed in PostScript – Aldus FreeHand 2.X specifically. It was a fun little project. I still have a copy of Aldus FreeHand 3.1 on my hard drive, and it works in “Classic” (amazing!) Between Photoshop (I’m using an almost current version of that at least) and FreeHand, I was able to render something pretty close. I’ll share results when it comes back from the shop.

Since I left a car in Seattle at my office I rode the train back the next day. I was a full-time train commuter for a while this summer and fall. I liked it, and will start doing it again someday. My only complaint about it is they go too slow, and pause too often. Makes British trains seem efficient by comparison. The ride from Everett to Seattle is truly awesome though. The rails follow the shore of Puget Sound all the way from Everett to the entrance to Salmon Bay in Ballard. From there they follow the route I used to ride on my bike from Ballard to Downtown. This particular morning there was still a break between large weather systems and the whole Puget Sound basin was beneath a clear streak arranged SW-NE following prevailing winds. I had a nice perch on the Sound side of the train, with a fine view of Camano and Whidbey Islands, plus the Kitsap and Olympic peninsulas as the sun came up over the Cascades. The clouds were lit by the rising sun.

You can see the Olympic mountains in the distance, already being swallowed by the next weather system, while the Puget Sound, and the south tip of Whidbey Island remain in darkness below the glowing clouds above.

Russian Roulette …with bombs.

How one software author’s unwise decision ruined my week.

Apologies for the long one, but it explains my lack of writing anything else this week.

Monday evening, as I was getting ready to take my youngest son out for what was likely to be his last Halloween (he’ll be turning 12 in a little over a month and 11 seems to be about the time that “kid stuff” starts losing it’s appeal) “trick or treat” with his friends, somebody pulled a trick on me that ruined my week.

Some history first though: We use a very nice mail server package called “Communigate Pro” by what used to be named “Stalker Software.” Communigate Pro (aka “CGP”) has a reputation for being fast, stable, and scalable. For the most part this has been true for us. We have had some issues with it though over the past four and a half years. We run CGP on several servers, since CGP has been used by several of the web hosting companies we have acquired over the years. The copy of it we bought for ourselves though has been the one that has caused us problems. It runs great for 50 weeks of the year, but for a week in August or September, and a week in December or January, it completely sucks rocks. The only way I can describe it is that interacting with CGP becomes like talking to a starfish.

I watched a show once that well illustrated at least one definition of the word “Relativity.” It showed how nature has made metabolism something of a clock, and that each species operates on a relative clock speed based on their metabolism. If you time-lapse film slow metabolism creatures like starfish, and then adjust the speed up to “match” our metabolic rate… the starfish look very active… zipping about the ocean floor, preying on urchins and other shellfish. Amazing really. Same goes in the other direction, slow down the film of a hummingbird and they start looking like any other bird. I guess to a Hummingbird, a human being looks like a starfish.

Well, for two weeks out of the year our CGP mail server’s metabolism slows to one of a starfish. It works, just at a truly GLACIAL pace. The Server and Operating system are fine (load is low, machine is responsive at the console, shell commands are fine, go figure.) This is obviously frustrating – for both us and our clients. The fact that it comes back like clockwork at certain times of the year is very odd. We eliminated all external causes (traffic, spam, etc) and Stalker support spent hours and hours trying to figure out what was wrong. The only suggestion they could ever come up with was “put a faster filesystem under it.” This error appeared in whatever version of CGP we ran, and I’m pretty sure that we tried them all, starting with 4.0.X, all the way up to 4.2.X (and this week, 4.3.X… but we’ll cover that later) but they all had that odd metabolism time shift appear twice a year.

Putting a faster file system under it usually cleared up the problem. As did switching platforms. We started on FreeBSD, moved to OS X (better threading), then up to OS X Server (on an Xserve); but also we jumped through all sorts of filesystem and bus technology switches, such as IDE, to SCSI, to various RAID setups, to eventually a 2Gb/s FibreChannel RAID array. Last summer when the starfish returned, on a whim (well, not a whim really, more a blind rage and pique of frustration since I wasn’t going to sink any more capital into filesystem improvements!!! Especially since they were seemingly NOT improving the situation!) I told my senior sysadmin to move the CGP directories to the internal IDE drive of the Xserve. Presto! The starfish vanished.

The server was back to it’s responsive, stable state. While I was happy with regards to that, since our clients weren’t angry at us, I was LIVID because all those tens of thousands of dollars we’d spent on hardware was a placebo cure for a real software problem. Stalker (now calling themselves “Communigate Systems”… aka CGS) had no explanation for this, and just sort of slinked away.

There is another significant wrinkle to this story, which explains why I was unable and unwilling to ride Stalker/CGS harder and force the issue into some sort of resolution. In November of 2004, CGS nee Stalker, made significant changes to their software licensing model, and jacked their prices up well over 5.5X their previous levels. Needless to say it was a shock to their customers. Prior to this date, their software was “expensive” but a relatively good value. (IIRC we paid between $8000 and $16,000 for our CGP licenses in 2000 and 2001.) Up until 2004 the core customer for Stalker were Service Providers such as ourselves. CGP had become something of a darling in the Industry press for being a solid performer and a far better value than absurdly over-done and outrageously expensive “Messaging Platforms” such as Lotus Notes and Microsoft Exchange. I guess this attention went to the head of Stalker/CGS’ CEO and founder Vladimir Butenko, and he began transforming CGP into one of those over-done and outrageously expensive “Messaging Platforms”. Hey, in some ways I can’t blame the guy… his core market – ISPs – had gone from niche-market players to a total commodity market with NOBODY making very much money, if any. Just beyond his grasp, and seemingly within reach was a cash-rich “Enterprise Market” with some dominant players showing real weakness. The astounding thing is the way he decided to get there: by actively pissing off their current customers and seeding them with confusion, fear and doubt. The existing customers, all ISPs, schools, and small businesses were angry. Stalker/CGS left no option for a “mail only” (no calendaring, groupware, MAPI support, VOIP support, SIP/PBX functionality, etc) version, and any continued use, other than the VERSION YOU ORIGINALLY BOUGHT would cost you a hefty sum in support and maintenance fees – 18% of purchase price, which in the new scheme was actually what you paid originally! So it was like having to buy your software again every year. Customers were livid, and the sturm and drang on Stalker’s support mailing list was out of control. Stalker’s CEO, Vladimir Butenko defended these new policies with characteristic Russian twisted logic and denial. I don’t know how to say “tough shit” in Russian, but that is what he did, albeit in far more diplomatic terms.

What he didn’t tell anyone at the time was that he ensured compliance with his new licensing scheme and inflated prices by inserting a “time bomb” into Communigate Pro. If your server thought it wasn’t properly licensed, it would cease to run at midnight UTC on some arbitrary date, and then, if re-launched would shut itself down ever 15 or 20 minutes thereafter. No warning. No coherent error code. No reason why. Bang. Boom. Off. Dead.

This was done without any announcement or warning. It add insult to injury, none of us customers had any idea which versions of Communigate Pro had the timebomb code in it or what the dates for explosion were. It was truly “Russian Roulette.”

Up until 2005, the standard refrain from Stalker Tech Support for any issue was “Please Upgrade to the latest version of Communigate Pro.” The support and sales staff frequently touted the benefit of “free upgrades” of their software. You got your value and return on your initial investment by always being able to stay current and get your bug fixes. We had changed versions via upgrade countless times, as we obviously had at least ONE big ugly bug, which unfortunately was never fixed. I don’t recall what version of CGP we were running when the license change was announced, but I knew that in February of 2005, when the (first of what I now assume are going to be many) CGP timebombs exploded we were running a version we weren’t apparently licensed for… despite the fact that we probably upgraded to it two months before while troubleshooting our latest visit from the Communigate Pro Starfish Mode. CGP servers around the globe all blew up at midnight UTC on February 1st 2005, including one of ours. Predictably the CGP support mailing lists, newsgroups etc also exploded with angry, frustrated customers. I called the guy at Stalker who we originally bought the software from and asked him flat out, “OK, tell me exactly what version of CGP we are allowed to run so that this timebomb won’t affect us again.” Bill, my senior sysadmin downgraded us to that version on February 1st, and life went on.

Later in 2005 our CGP Starfish returned, and that is when we tried the “move to internal IDE disk” trick which worked. I had not paid Stalker that hefty price for support and maintenance (or as they ironically call in their emails to me “S&M”) so I was in no position to demand that they admit this “starfish mode” bug exists and fix it. I was stuck at the version we were running for perpetuity. Such is the Kafka-esque world of software licensing. Instead I directed my staff to start evaluating alternatives to Communigate Pro. I didn’t want to be the victim of extortion to pay for the development of features for “Enterprise Customers” that we would NEVER use. Here is a great example: I was on the phone with a guy from Stalker/CGS and he was telling me how great their PBX/SIP/VOIP system was. I asked him “How do our customers call us if the mail server goes down?” I was answered by a very long silence… followed eventually by “Hmmm… never thought of that.” SMTP/POP/IMAP/Webmail… that is ALL I need thank you. So we looked at the expanding pool of products that were filling the void being left by CGP as it acsended to “Enterprise” status. We had narrowed the field to a small handful by last week.

Then we lost at Russian Roulette again.

At 4pm PST on October 31st, which is Midnight UTC, three of the 4 Communigate Pro servers at our facility exploded. Their timebombs went off and they all shut themselves down. My wife had to fill in for me as the Halloween driver (we live in a rural area, so I had planned on taking my son, and a few of his friends into town for trick-or-treating.) I spent the night hunched over my keyboard and on my VOIP phone (thankfully we don’t use Communigate Pro for our VOIP needs!) to my office dealing with the crisis. Based on past events, we very quickly came to the conclusion that it was the infamous Communigate Pro Time Bomb, and not some other issue since it happened at precisely the same time on more than one server, and we were not the only ones it was happening to. (Stalker’s mailing list, which is viewable on the web also was exploding with angry customers.) To get us through the night we rolled the clocks back on the CGP servers, and restarted them. In the morning we started the work of figuring out how to deal with this. I emailed Stalker trying to find out why, when they had told us that THIS version was OK for us, that it still had timebombed. I posted, and replied to other’s postings on the CGP mailing list, but my account was in “moderated” mode, and the moderator was obviously not paying attention (easy to do as that is a significant weakness of the CGP LIST module.) Vladimir Butenko appeared on the list, once again in his twisted Russian logic saying essentially ‘there is no timebomb, and besides you must be stealing my software since your server stopped working.’ Not exactly a confidence or trust building exercise in customer relations there Vlad.

After careful reading of the CGP website, I finally decided that our only course of action was to downgrade to version 4.1.8, which seems to be the last of the “free upgrades” and should run on our license key obtained in 2000. Bill figured he could downgrade the software, and restart the CGP service without causing much disruption to our clients. 4.1.8 went on, we restarted, and suddenly, without warning…

The Starfish Returns!

Our mail server software is once again, moving at the speed of a quaalude-soaked starfish taking a leisurely creep over the ocean floor. It is 7 weeks early, but the starfish is back… with a vengeance!

Great. Just what we need. A software vendor extorting us on one side, and clients angry at us for under-performing software on the other. My loyalty is with my clients, not the bastard that is holding the gun to my head, or the timebomb on my server as the case may be. I rally the staff and roll out a plan; we’ll build a new server from scratch, install a fresh OS and a new install of CGP 4.1.8 on it, move the data over to it and cutover the IP address. Based on our past experience, this should outwit the Starfish!

Thankfully a customer had just decommissioned a very nice Dual CPU/Dual Core Intel server with a built-in Ultra-SCSI RAID system, and we made him an offer on it that he accepted. The only problem with it was the drives inside were low-capacity. Thankfully we have stacks of Sun Stor-Edge Array’s in our backup system that were in an idle state, so we ripped out 6 36GB LVD Ultra-SCSI drives from one and packed them in the server, installed FreeBSD on it, and started rsync on a cross-over cable between it and our production mail server. Oddly enough this went pretty fast, despite CGP in “Starfish Mode” the OS and filesystem is thankfully quite responsive. System load went from 0.10 to 0.34 on the production server while we were syncing… while talking to the Starfish was unbearably slow. For example CGP’s web UI would take 15 minutes to click from page to page.

We cutover to the fresh box at around midnight on Tuesday/Wednesday, and things seemed ‘OK’… instead of talking to a starfish, it felt like talking to a sleepy dog. Movement was perceptible, but not exactly as swift as we had hoped. In past experience “starfish mode” would improve to reasonable performance in the wee hours of the night when the server was under lesser mail load. Since I was staying in my office and had nothing else to do, I vented about this situation to my online friends, discussed via phone with Russ Pagenkopf, the guy I run the Mac-Mgrs list with… ironically running on a Stalker-donated copy of CGP, which also quite ironically had also timebombed! Russ & I decided to cease running CGP on the Mac-Mgrs list server as soon as possible, and once he had it running again I posted to the list about that. I also answered people who were angry on the CGP list about what was going on with us, and some of them relayed to that list what I had said, both to them, and on Mac-Mgrs. The PR backlash at Stalker/CGS was gaining momentum. I think I managed to get about 3 hours of sleep that night.

Sure enough come Wednesday morning east-coast business hours our main server was back to moving like a starfish. I left my staff to handle the angry clients, while I swallowed my anger and called Stalker/CGS for tech support. I didn’t expect much, but luck was on my side and by chance a Director-level employee answered the phone (When our tech support queue gets busy, I pick up the phone too!) I explained our situation with CGP 4.1.8 doing this “glacial slowdown” thing (I haven’t called it “starfish mode” with anyone at Stalker/CGS to date.) I asked him if my long-time contact was there, and he said, “yes, he just walked into the office” so I said to catch up with him since he knew the full history of this almost 5 year old problem and I didn’t have the energy to relate it to him. After a few hours of troubleshooting (it took me 55 minutes just to get to the UI to change a password so Stalker support could access the server) I got a call from them. Three people, all director-level folks at Stalker/CGS were on the phone and making me an offer. They would give us a 90-day License for CGP 4.3.9 to let us load that one up and see if it would fix the “Starfish Mode” bug. I was too exhausted to say anything but “it is worth a try”…. They promised me quotes for extending the 90-day license within a day.

License keys in hand, I woke up Bill, our over-worked and underslept senior sysadmin and had him install the 4.3.9 version on our creeping starfish of a server and restart…. it seemed OK for about 30 secdonds, then immediately tailspun back down to starfish mode once again. It is obvious whatever this bug is, it has never been adequately addressed by Stalker’s coders and remains embedded deep within the current version, and probably in upcoming ones as well. The Stalker support guys were stumped, and fell back into random-mode troubleshooting again, suggesting courses of action which were either impossible due to not being able to perfrom them on such a slow moving system, or stuff they had suggested in the past – which we knew would not work.

I had a plan. It was a total “hail mary” play, but similar stunts had worked for us in the past with the Starfish. Nuke the box we had been running the mailserver on just days before… before the software timebomb exploded. Fresh install of this CGP upgrade, move the data over to it and cutover again. This may sound like what we just tried, and it does. Meanwhile I talked to MY director level guy and said, where ever we are with the proposed new mail system roll-out, hit the gas pedal and get ready to install and ramp it up ASAP! He brought me PO’s for gear and software, and I signed them. I wrote an apology to our clients about the situation, and posted it to our website. I grabbed my laptop and left my office for the first time in almost three days to get some fresh air, and food. I had the laptop as it seems that open wireless networks are everywhere now, so if they needed me at the office I could probably get on AIM or whatnot easily.

Bill finished the install and rsync work, and we cut over to the “old” mailserver around 5 PM PST on Wednesday and….

It worked. The starfish was back in hibernation once again, and the server was behaving “normally.”

I finished up some client communications, and basically passed out on my office couch a few hours later. I slept 12 hours straight.

So, at the moment I have 90 days to get a better mail system rolled out and running. I think we can get that done. We’ll probably build a fresh, old CGP 4.1.8 system to leave any clients that can’t/won’t move to the new system, so we’ll stay in compliance with Stalker/CGS’ looney license scheme, and perpetually avoid the Russian Roulette with Software Timebombs present in CGP 4.2.X and who knows what subsequent versions. We’ll probably NEVER get a satisfactory answer about the causes, or real cures for Communigate Pro’s “Starfish Mode”… but here is my hope:

Someday, it will return. Not to *our* server, but to one of these “Enterprise Customers” that Stalker/CGS so desperately wants to trade their current customers for. Some multi-million dollar CGP “Messaging Platform” cluster installations. They’ll have hundreds of thousands of dollars invested in hardware, and of course CGP software. Their mighty cluster will slow to an inexplicable crawl. They’ll spend massive amounts of time, and eventually money, trying to cure it. Vladimir will log into it and tell them “Put a faster filesystem under it”, so they’ll blow wads and wads of cash at exotic SAN architectures or the like. VP-level guys like me will lose sleep and in-the-trenches guys will loose even more trying to fix the problem of wrestling with a starfish. Then, some geek in the organization will be google-surfing phrases like “CGP slow” or “glacial communigate” and stumble upon this blog entry from who knows how many years past. He’ll pass it up the chain, and somebody will gather up the guts to call me. I chuckle and say “You spent HOW much money to buy this software from these idiots? What, are you NUTS?”

There, I just saved you the phone call.

Slashdot | Price of Power in a Data Center

Interesting read, on a subject I know pretty well. We will likley have to institute power surchages to our colocation customers soon.

I liked this interesting tidbit from the comments section:
Also the street price for a 20A circuit in a datacenter is $200-$300, while the cost of a megabit is $100 or less. So a rack of servers that requires two power circuits and pushes 3Mbps (not an unusual scenario) costs twice as much in power than in bandwidth.

I’ll write more about this subject soon.

The Rains Have Returned

I don’t comment much about weather, but the subject came up in an iChat with a friend on the east coast.

Here in the Pacific Northwest we really only have two seasons, “wet” and “dry”. “Wet” lasts from sometime in September or October, until early July. Dry lasts from early July (usually the 5th or 6th!) until sometime in September or October.

In the 1980s “drought” years “dry” would sometimes last until November. I remember climbing “Outer Space” on the Snow Creek Wall over a weekend in early November around 1987 or so. It was cold at night but very warm… “hot” even during the day. Recently our weather has been unsettled, with either VERY wet years (in 1999 we didn’t really have a “Dry” season… until September. Last winter was the as dry as “wet” can be, with hardly any snow in the mountains and very little rain down here. Oddly enough the past few year’s “wet” started big, with some big October storms … these pictures were taken two years ago today. But then settled into a “very sparsely moist” rather than our usual full-on “wet”.

Well, the “wet” has returned to the Pacific Northwest. October has been more rainy than clear, and quite chilly as well. We had a brief little “Indian Summer” the past two days, mostly sunny temps in the high-60s F. Friday I drove the Jag down to the body shop for the bonnet ding to get repaired, and yesterday was spent hacking back the grass since the sun was out (making hay while the sun shines as it were.) Today however reinforces how brief that nice respite was. Rain, mist, fog, temps in the 40s & 50s F.

It will be this way, relentlessly wet, with only high winds and storms to break the monotony from now until January when the “storm season” ends, and then it will just be plain old rain. You can basically say “Rain, mixed with showers, with a rare sun-break, lows in the mid-30’s, high’s in the mid-50’s” if you were a weatherman from now until March. Sure, we’ll have a few snows sprinkled in there, and of course that one week in January or so when the sun comes out… just to keep us from killing each other. Sometime in April we’ll start to see more sun, and warmer temps, and the Jag will come back out of the barn now and then. Until then, you won’t hear me talk about it other than winter-time projects. (Like my plan to perhaps do something about the radio console once and for all.)

When the “Dry” season returns, I’ll comment about our reward for the crappy weather we put up with around here, until then, I’ll try not to say much about weather.

WVO Filtering Setup

I finally photographed the home-brew Diesel setup I built:

WVO Filter

The waste veggie oil goes in the top barrel. The sawed-in-half gas can acts as a funnel. The oil comes to me via the white 5 gallon buckets on the left, so they don’t pour into 2″ holes without a funnel. The gas can sits very well in the filler bung, so I won’t spill… too much. The top barrel has a bung in the side, through which I have a 3/4″ ID gasoline hose, which goes through a stop-valve, to a 15-30 micron filter, through another stop-valve, through a 5-15 micron filter, through the final stop-valve to the bottom barrel. The bottom barrel is equipped with a nice 10 gallons per minute hand pump. The whole setup is airtight, and is kept from freeze damage (thankfully only a slight possibility here in the Pacific Northwest) with one of those pipe-warming cords that winds its way from the top to the bottom. You will note the bottom barrel has a blanket around it for extra insulation. If we get a significant cold snap I’ll have to supplement the heat with a light bulb or something.

This whole section of the barn was built by the previous owner specifically to store Diesel fuel. The shelf there had four large fuel storage tanks on it when we first viewed the house. The guy owned a logging company and the barn was his workshop for the trucks and equipment. The floor below it is not on the concrete slab, but it is filled with gravel and oil-absorbent stuff. Pretty cool.

I only have about 5 gallons in the upper tank right now. It should start flowing on its own via gravity once I have about 20 gallons in the upper tank. Running the hand pump will also provide suction back through the system to boost the filtering. Can’t wait to get it running at capacity.

How I spent my 42nd Birthday. 10/5/05

Yesterday’s entry ended: “I imagined a nice short day at work, followed by some analog therapy of fixing my car, and a nice drive home with the top down.”

If this were a movie those words would have an ominous sound track playing behind them. Since it is just a blog, you’ll have to imagine it. (I won’t ever put some midi track in my HTML… If I do you have permission to come shoot me in the head.)

Instead my meeting schedule got completely rearranged, and after the digital.forest traditional birthday cake in the kitchen, instead of running over to NAPA to do my alternator swap, I ended up on the second longest conference call of my life. My title says “VP, Technical Operations” but in reality I was playing Contract Lawyer… slogging through the legal language of a vendor agreement several hundred pages thick. (Don’t wait for the midi track, just shoot me now!)

I finally escaped from the office at 5:15 PM, looking at a 60+ mile, Seattle rush-hour commute, with lowering skies, in a leaky 40 year old car with a suspect electrical system. Great.

And of course, it was my birthday! The wife & kids are waiting at home to celebrate.

The car starts right up (still has that new battery smell) and I zip down the hill to NAPA. They have my alternator, but there is a glitch. I had expected a two-piece fan/pulley section and instead it has a separate fan and pulley. Of course the Jag has this oddball grooved belt, and after I did my alternator mod last year some knowledgeable guys told me I could build a “groove” with washers (semi-brilliant!) No problem, I figured I could just swap my modified fan & pulley from the old alternator and be fine. (time for you to cue that ominous sound track again.)

Insert Problem #1. The bolt on the end of the alternator shaft is metric, and an oddball metric size at that: 22mm. That is slightly smaller than the 7/8th inch socket I have in my portable toolbox in the Jag’s boot. No problem I’ll buy one from NAPA!

Insert Problem #2. The largest socket NAPA has is a 19mm. I bring it to the parts guy and tell him that. He fishes around for a socket to fit it. The best he can find is 7/8ths. However he has a power torque wrench and I just have hand tools. I figure I’ll let him remove the pulleys. (I learned all about transfer of liability in my earlier conference call!) He manages to get the pulleys off and we swap them.

While popping the pulley from the old alternator I discovered what had caused it to fail. As I picked it up off the ground a diode that had fallen out was lying on the pavement(!) It had obviously came off internally and having it sitting vertically had caused it to fall out. Go figure.

Insert Problem #3. The new alternator has a slightly different housing than the old one. The shaft is surrounded by a raised sort of bridge, and it leaves no room for the bolt at the end once my fat grooved pulley is slid over it. I had this problem before when I modified the first Hitachi I put on this car. I fixed that by cutting the bushing down a bit and using JB-Weld (the choice of mechanical kludges everywhere!) to stick the fan to the bushing to center it. The bushing from the old alternator was twice as thick as it should have been.

Insert The Solution That Saved My Birthday!

The NAPA guy didn’t have any cutting tools, (and probably didn’t want to risk any further transfer of liability), so he sent me, and the bushing next door to a truck repair place. I jogged over to System Seven and found a mechanic. I explained the situation briefly and he volunteered to cut the bushing in half. He did warn me that it may not be perfect, and I assured him that I wasn’t seeking perfection, just something good enough to get me home. He put the bushing in a vice and expertly sliced it in two, right where I had showed him to cut. KZOK happened to be playing on their shop stereo. I mentioned that the car he was helping me fix was just photographed for KZOK’s calendar, so I’d have to get him a copy when it was published later in the year. I introduced myself and he told me his name was Todd. I plan on calling Steve Slayton tonight and making sure he plays something on KZOK dedicated to Todd and the guys at the System Seven Repair for saving my ass, my birthday, and my Jaguar. Something rockingly awesome from the early 60’s, just like the E-type… Dick Dale’s “Miserlou” would be good.

The bushing in hand, I sprint back over to the NAPA (where my car was sitting, bonnet & boot up, my tools & valuables everywhere, and a half-eaten office birthday cake sitting in the passenger seat!) and pop the fan back on. It fits PERFECT! The fan has maybe 1.5mm of clearance from the housing thanks to Todd’ precision cutting, and the pulley has more than adequate clearance to put on the bolt. I put the whole thing together and bring it out to the car. I’ve removed and re-installed alternators on this car so many times I swear i could do it in my sleep… like a well-drilled soldier cleaning his rifle. The new Hitachi goes on and I tighten the belt. Remove all the tools, re-connect the battery, hop in and hit the starter button. The moment of truth…

The noise of a Jaguar XK engine is a truly wonderful sound. The deep rumble of the long-stroke big-bore six, the clatter of high notes from the dual overhead cam cylinder head, mixed with the wind of triple carbs and the whine of the timing chain and V-belt. It is often called “Sir William’s Sixth Symphony” in reference to Sir William Lyons, founder, heart and soul of Jaguar Cars Ltd. It burbles with pleasing menace at idle and emits a spine tingling roar when driven in anger. Sir William’s Sixth greeted my ears, but it was the sight of the Lucas Ammeter… yes The Prince Of Darkness himself! That Lucas Ammeter on the dash indicating a hard swing to the left. No words in the English language can describe the immense pleasure I felt seeing that needle swing towards the “C”, indicating that the Prince Of Darkness was showing current from the alternator to the battery. “Joy” doesn’t do it. “Relief” doesn’t either. Even “heights of climactic delight and waves of multi-orgasmic pleasure” don’t do the feeling justice.

Let’s just put it this way: I was feeling good!

I took the box and paperwork back into the NAPA to get my core charge credited and probably did not even touch the ground while making that 20 yard stroll. As the NAPA guy was doing the credit card work I went into their washroom and made an attempt to clean my very dirty hands and arms of car grease. Even clean cars get you dirty while working on them… go figure. As I finished up the paperwork I saw Todd outside having a look at the car and he waved to me. Finished, I went out and started packing up the car. I told Todd more about the history of it, and thanked him profusely again for allowing me to make it home on my birthday. He was happy to have helped out and graciously accepted my thanks. I fired up the car and headed home. It got dark shortly after, but I felt confident running with my lights, and watched with immense satisfaction that Lucas gauge showing a positive flow of electricity once again.

I hate driving this car at night, but I didn’t have a choice. At least it was late enough now that the freeway was merely crowded, instead of stop & go. At least it never rained, and it wasn’t too cold to run top-down.

I finally arrived at home around 8pm, and was immediately tossed into the Jetta and driven to our local Mexican restaurant for a traditional Carne Asada and Marguerita dinner. Christopher drove so I downed two stiff ‘Ritas and got to wear the birthday Sombrero. (I’ll post the pic as soon as I scan the polaroid.)

All’s well that ends well.

Sorry I didn’t take any pictures. I usually stock my blog with plenty, but I was just so focussed on getting the car FIXED that it never even occurred to me to stop and document anything by camera. That should be an indication of how stressed I was…

How I spent (the day before) my 42nd Birthday. 10/4/05

My 1965 Jaguar E-type was picked to be photographed for the 2006 KZOK Classic Car Calendar, and the photo shoot was scheduled for October 4th. The shoot took most of the day to setup and perform, so I scheduled myself to have the car in Seattle the day before, and return home to Arlington the day after – which happens to be October 5th, my birthday. As anyone who owns old cars will attest, they don’t make them like they used to, they make them better. There is a reason they don’t have mechanics at every gas station anymore, cars are just another major appliance nowadays. They just work (most of the time.) Not so old cars. There is always something that requires attention. That said, the effort is more than worth it as these old cars have a style and presence that todays econo-boxes and glorified pickup/SUV/truck/wagons can NEVER match.

The Jaguar just went through a major engine rebuild and is still in the “break-in” period, so it has been off the road for the past year. I spent the weekend prior to the photo shoot attending to little things to make it roadworthy for the excursion. For example, I replaced the brake light switch. Unlike the 1960s when small brake lights were a testament to style, drivers today are too busy yakking on cell phones, sipping lattes, watching their on-board DVD systems, and radioing commands down from the bridge of their Exxon Valdez-sized SUVs to pay attention to small cars with miniscule brake lights in front of them. They depend on the presence of a single, high mounted LARGE RED LIGHT in front of their eyes to remember to brake. No such light exists on the early 60’s XKE. In fact it had no lights at all. While out for a run to the parts store a guy in a brand new Ford pickup backed into my Jaguar’s “bonnet”… so now my gorgeous car has a dent in it!

bang!

Above: What happens when drivers of big vehicles don’t look in their mirrors very carefully. Thankfully I saw him and was backing away as fast as I could! It could have been MUCH worse.

The icing on the cake is finding the car’s battery with 10 volts of charge the day before the trip to Seattle. I take it out of the car and run the trickle charger overnight before the trip.

The trip down was uneventful, but my eyes noted the Lucas Ammeter on the dashboard never wavering, which planted a seed of doubt in my mind. Monday passed at work, and I bedded down on my office couch, eager for morning and photographic glory.

The car started fine in the morning and I navigated my way from my office near Boeing Field north towards the photo studio on Queen Anne Hill. As I made my way through SoDo past the Starbuck’s building that seed of doubt burst forth into a sprout that was rapidly reaching full bloom. All my gauges started to waver and I knew that my car’s electrical system was about to die. A block before I was to ramp onto the Alaska Way Viaduct I made a quick right, and looped back south on 4th Avenue. I spotted a Big-O tire store and a Shell station. The car expired as I came even with the gas station and I coasted to a stop in front of a pump. I filled it up, pushed it into a parking space and walked across the street to buy a battery. I knew that the battery was not the problem, but it certainly would get me to the shoot if I bought a new one. Bought, installed, and I was on my way.

I arrive at the shoot a mere 10 minutes late.

The Art Director arrived shortly after me and had me park in the middle of the parking lot while he spent at least 20 minutes looking at the car from every possible angle to find the one he wanted.

After he picked the angle (passenger-side, 3/4 rear shot) I had to back the car into the studio. I scraped exhaust on the way in… and had to find some items (paving stones) to drive over to raise the car through the door. Took a few wiggles to place the car “just right”…

I used to work in the Advertising business, so it was like being teleported back in time. In fact I met this particular photographer 10 years ago. I was partly responsible for building the first large-scale digital photo studio in the Pacific Northwest back in 1992/3 for my former employer, The Bon Marche’s (now Macy’s) Advertising Dept. He toured it a couple of years later when his studio was considering going digital. Small world.

There was supposed to be a professional detailer crew there to prep the car. They didn’t show up so I grabbed some rags and did as good a cleanup job as I could. The car was pretty dirty from the rainy ride down from Arlington to Seattle.

The crew showed a lot of respect for the car. Anytime they needed anything done to the car, they had me do it. But once they started shooting, I stood back and stayed out of their way.

It took easily two hours to light the set and car just right. The light color with the dark interior made for a challenge. They bounced light off the wall and ceiling to light up the interior and pointed a spot light at the dash. A long black curtain just off camera to the right provided a nice “horizon” reflection in the car. They took a bunch of test shots along the way, and I ended up suggesting a couple of changes (changing the gearshift position and dropping the brake handle out of sight… it looked odd peeking up over the door.)

The car looked *great*… better than it ever has!

Oh... my...

That is a shot from the pro photog’s camera, a 22 megapixel Sinar. It isn’t a full-res shot (obviously) and I took the liberty of doing a quick photoshop retouch on it (removing the background bits mostly.)

My sole complaint about it is the exhaust/bumper position. Otherwise, it is stunning.

The model arrived late (they are always late) and then, once they had her hair and makeup done she had to leave(!)… (to go pick up her kid.) So we took a break for an early lunch. The photographer spent the rest of the break picking my brain about network configuration and web/FTP server setup. I was more than happy to help him out (especially if I could get some high-res copies of his work!)

Once shooting began it went really fast… maybe an hour’s worth of work. I just stood back and watched, trying to stay out of the way. The Art Director is in charge and the rest of us are just there to provide him with tools for the final product.

Here is a sample shot of how the final MIGHT look:

They shot at least 100 different poses/positions, so we won’t know until the calendar comes out in December.

Once done, we all whipped out our own cameras and took some pictures of ourselves with the car… The AD, Photog & me stood in with the model and her kid for a shot from the “big” cam. Here is why you rarely see me in FRONT of a camera:

The little girl was dying to climb in, so I let her sit in the car. Oddly enough she didn’t want to sit behind the wheel. Go figure.

Here are all my photos, complete with the occasional caption.

I shot without a flash so I wouldn’t mess up their work, so forgive the occasional… OK, FREQUENT… blurs.

Can’t want to see it in print. The only other car of interest (to me at least) among the Muscle and Custom Rods in the calendar will be a ’62 Maserati Sebring.

I drive back to my office, and stop at a NAPA a few blocks away to pick up a new alternator. I knew my problem was that my car was not properly charging the battery, so it had to be the alternator. Last year I replaced the original Lucas (aka “The Prince of Darkness”) alternator with a Hitachi, which is 85% cheaper, and supposedly more reliable. The only modifications required were swapping the pulley and fan, and wiring around the Lucas (PRINCE OF DARKNESS!) voltage regulator. NAPA didn’t have the part but would get it in tomorrow. Wednesday. The 5th. My 42nd Birthday.

I imagined a nice short day at work, followed by some analog therapy of fixing my car, and a nice drive home with the top down.