chuck.goolsbee.org  goolsbee.org, serving useless content from an undisclosed location since 1997

December 31, 2006

Time for another round of: Name That Car!

Filed under: Car Photo Of The Day,Cars — chuck goolsbee @ 9:15 pm

Nice paint job eh? 😉

December 30, 2006

The Truth About Cars | German Speed Limits: I Can’t Drive 155?

Filed under: Cars,rants — chuck goolsbee @ 2:10 pm

The Truth About Cars | German Speed Limits: I Can’t Drive 155?

I recall being in the back of a big Benz cab, going from the Munich airport into the city (a very long drive)… I was behind the driver, with my co-worker opposite me. It was his first trip to Munich, but I had been there many times before. I was just looking out the window, enjoying the scenery when I turned to say something to him. I stopped speaking when I noted his eyeballs were as big as saucers. He was staring at the dashboard in front of the cab driver with a look of fear. I glanced over the driver’s shoulder and he had the big S-class barge floating along at well over 200 Kp/h.

I just smiled and said to my friend: “Welcome to Germany.”

Up until that moment, I had no sensation of speed at all… just another cab ride on the Autobahn.

I also agree that the 80-110 MPH zone on most restricted access highways is quite comfortable, and would be achievable here in the USA if they made getting (and keeping) a driver’s license more stringent than it is now. My son is 16 and I’ve been helping him learn, but the testing – at least in my state – is laughable. 20 questions, easily half of which are concerned with fines and DUI, and very little about actual driving.

Here, take it yourself!

That is an embarrassment and pretty much sums up why Americans drive the way they do.

–chuck

RatcliffeBlog—Mitch’s Open Notebook

Filed under: Uncategorized — chuck goolsbee @ 11:07 am

RatcliffeBlog—Mitch’s Open Notebook
“Then, there has been an absurd line of reporting about Saddam’s dedication to Joe Stalin. Now, he may have been an ardent admirer of another murderous monster, but when commentators appear on air suggesting that Saddam, the egomaniacal despot, had his statues made to look more like Stalin than himself, you have to wonder about the sanity of the producers. Saddam’s interest in Stalin as a model is an meaningful anecdote, but not the important explanation of Saddam’s rise to power.Why is there not coverage of the meeting between Donald Rumsfeld and Saddam in 1983, when the United States and Iraq agreed they shared many common interests? This was, of course, after the genocide for which he was sentenced to die. Joe Stalin had less direct influence on Saddam than the Reagan Administration and Don Rumsfeld, who cleared the way for sales of weapons to Iraq throughout the 1980s. President Reagan released a national security directive (NSDD 139) that codified our support of the Saddam Hussein regime in 1984.Saddam was a monster, but let’s be realistic about this. He was our monster, not Stalin’s, not Russia’s, not even Islam’s monster. Saddam was an instrument of U.S. policy toward Iran while he cemented his power in Iraq and, like many strongmen we’ve supported in the past, it backfired.

I read that on Mitch Ratcliffe’s blog this morning and it rung so very true. I remember those days… I was in my 20’s and in college, reading in Time & Newsweek how Iraq was or friend and bulwark against the Iranian regime.

The Iranians are still there, we’re sinking into a quagmire in Iraq, and we just provided the “insurgents” with a martyr.

And the best part: George W. Bush will retire and leave the whole mess… “his legacy” to some other poor bastard to try and clean up. What do you want to bet the GOP moves to control Congress and cede the White House to the Dems for ’08. Turn the next guy into “Carter II, the Sequel.”

sigh… I hate politics.

December 29, 2006

Honored again

Filed under: Cars — chuck goolsbee @ 3:24 pm

I took the 65E out for a short drive today. The weather has been dry enough for the past few days to clear the roads. The sun was out for a short bit this morning, but replaced by high overcast by midday. It has been three months since the Jaguar has moved so I figured it was time to limber up the XK and give it some exercise. I ran into town, top down of course, gloved and hatted and teeth chattering. Went about 5 miles out, then back home. Everything worked fine, and it was a joy to hear Sir William’s Sixth Symphony once again!

When I sat back down at my desk my RSS feed in Safari told me I had a bunch of reading to catch up on. Autoblog, being its usual prolific self had several new articles up. One of them was nominations for the “Reader Ride of the Year“.. and there was my Jaguar!

I read the comments and was humbled by the praise for the car. I really am a lucky guy to be its current caretaker.

We’ll see if it gets picked. Go vote now!

BTW I managed to get connected via Autoblog, with a local Seattle Lotus Elise owner, and we’ve agreed to swap drives in each other’s cars come spring. I can’t wait.

Car Photo Of The Day… stuck in the Museum theme

Filed under: Car Photo Of The Day,Cars — chuck goolsbee @ 11:06 am

Well, “stuck” may be the wrong word. I have so many good photos from this particular museum that I could stay on this track for months.

The car in front is pretty dam obvious, but can anyone name the one in the background?

You can claim the crown as the “Ultimate Car Geek” if you can name the fuzzy, out of focus slice of a car barely visible in front of the nose of the one in the background!

December 26, 2006

The spammers keep getting more clever.

Filed under: digital.forest,Technology — chuck goolsbee @ 7:37 pm

One of the most important duties I have at digital.forest is reading the “abuse@” mail address. I have allocated just about every other “front line” task to members of my staff, but not this one. In so many ways I am no longer a “geek”… my day-to-day duties are more inline with my title (I’m an Operations VP) than performing actual, technical tasks. I assist the Sales dept, and the CEO, and leave the tactical management of the technical staff to my “second in command”… so I manage him, and our Network Manager (both of whom are awesome BTW) and remain confident that they have the rest in-hand. The lone exception is dealing with our reputation as a good network neighbor.

We are a colocation facility foremost, and a webhosting provider secondarily. As such we are at a fixed location, both physically in terms of our facilities, and virtually in terms of our Autonomous System Number and our IP address ranges (which are 11739 and 216.168.32/19 respectively.) It is very important to us to keep our good reputation among our network peers… as such I’ve never delegated the duty of monitoring the abuse@forest.net address to anyone else. Mind you, I frequently delegate the task of investigation, or of swinging the clue-by-four at our clients should they do something stupid, but I wouldn’t dream of slipping the ultimate responsibility of reading the inbound complaints downstream. I’ve been doing it since the day I arrived here.

Mostly the abuse address provides entertainment. People who can’t read mail headers, or worst of all, can’t figure out how to unsubscribe themselves to a mailing list they were competent enough to subscribe to (and whose headers, AND footers have easy-to-click URLs for the task!) let me chuckle at the average-or-below intelligence of the typical Internet user. Occasionally there is a real client who does something really stupid and mass-mails people, and I get to handle the backdrafts of anger. But mostly it is handling automated notices of compromised colocated servers, and deleting a lot of spam (since the abuse@ address is listed in the WHOIS databases… so it gets spammed a LOT.)

Occasionally though, we get a puzzle. Late last week I received a complaint about a spam, that REALLY looked like it came right off one of our mail servers. I responded to the complainer, thanking them for the head’s up, and started sifting through the logs to see if I could find out how this mail was sent from our network. The domain belonged to a webhosting client; one we had purchased along with a major acquisition from two years ago. The spam in question was obviously from a forged address, but the domain was valid. I logged into the mail server used by that domain and confirmed the lack of an account matching the spam. But there it was, in the headers, a “Received: from…” that matched the server, our IP, etc. Here is the header info:


Return-path: pollingsuppression's@(removed).com
Envelope-to: mike@(removed)
Delivery-date: Thu, 21 Dec 2006 09:50:43 +0000
Received: from host86-144-187-151.range86-144.btcentralplus.com ([86.144.187.151] helo=api.home)
by node-2.minx.net.uk with esmtp (Exim 4.60)
(envelope-from pollingsuppression's@(removed).com)
id 1GxKZP-0004Sv-VZ
for mike@(removed); Thu, 21 Dec 2006 09:50:43 +0000
Received: from 216.168.37.122 (HELO mail.(removed).com)
by (removed).net with esmtp (B1EIM*(?(-/ .O<8)
id 64ER30-)H,QXG-RQ
for mike@(removed).net; Thu, 21 Dec 2006 09:45:03 +0000
From: "Matilda Vaughan" pollingsuppression's@(removed).com
To: mike @(removed).net
Subject: It's Matilda
Date: Thu, 21 Dec 2006 09:45:03 +0000
Message-ID: <01c724e4$b0665b70$6c822ecf@pollingsuppression's
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Office Outlook, Build 11.0.6353
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
Thread-Index: Aca6Q1?9:;:65*.=Z++3*K(R+W54O==
X-Antivirus: avast! (VPS 0661-0, 12/20/2006), Outbound message
X-Antivirus-Status: Clean
X-MINX-Orig-IP: 86.144.187.151
X-Spam-Score: -0.7 (/)
X-Spam-Level: /
X-Antivirus: AVG for E-mail 7.5.430 [268.15.23/591]
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=iso-8859-2

To the unititiated, you read “Received: from…” headers bottom to top in order to establish the path of the mail from server to server. (looking at this one now, with what I have learned subsequently I see a couple of big red flags that I missed originally, but they are only obvious in hindsight… more on that later.) It appears as if this mail left one of our mailservers (216.168.37.122) then went on to the final recipient.

I checked the logs and grepped (a sort of search/filter tool for those that don’t speak geek) for the forged “from” address. I did find it, but NOT from an outbound mail. Widening the search a bit I noted the domain in question appeared to be under a large-scale directory harvest, or “dictionary attack”… meaning that a LOT of mail was coming from all over the place, all to a series of possible mail addresses… the point of which was to determine which accounts are valid, and which are not. We use an external service (Postini) to both protect our mailservers from this sort of attack, and protect our customers from being buried in spam. This domain however was NOT protected by Postini.

We have been testing a product lately as a possible alternative to Postini, namely a Barracuda Networks “spam firewall”. We had just stopped using it as an outbound filter and I saw a chance to test it for inbound. Here was a perfect test, and apparent harvest attack! What a nice way to give it a workout! So I created a new A record in the domain in question, setup the barracuda to handle the inbound, then pointed the domain’s MX record at the barracuda. It would take a while for the changes to distribute through the DNS infrastructure and really start working, but this was the Friday before Christmas… I had other things to worry about. I left work trying to imagine how all the above was linked together… and what sort of exploit had this spammer found that would allow them to successfully spoof their way into our mailserver to send these spams. My extensive log sifting had not turned up any instance of mail from that domain – matching the header info (timestamps, message-IDs, from addresses, etc) actually being sent by our mailservers. Perplexing.

Today (Tuesday) I returned to work from the holiday weekend, and found another one of these spam complaints, which pretty much looked identical in profile to the one above. Here is the header from that one.


Return-Path: shopkeeper'sregimented@(removed).com
Received: from your-sz6x6sefxo.rochester.rr.com
(cpe-66-67-45-66.rochester.res.rr.com [66.67.45.66])
by host44.swh.bellsouth.net (8.13.1/8.13.1) with ESMTP id
kBO1GCSZ015798
for dawn@(removed).com; Sat, 23 Dec 2006 20:16:12 -0500
Received: from 216.168.32.228 (HELO mx.(removed).com)
by (removed).com with esmtp (T,@M6M(4J)* 7N9M*)
id 255;5---H2;*-0(
for dawn@(removed).com; Sun, 24 Dec 2006 01:15:36 +0300
From: "Terrie Sewell" shopkeeper'sregimented@(removed).com
To: dawn@(removed).com
Subject: Terrie
Date: Sun, 24 Dec 2006 01:15:36 +0300
Message-ID: <01c726f9$049e1690$6c822ecf@shopkeeper'sregimented
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-2"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Office Outlook, Build 11.0.6353
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869
Thread-Index: Aca6Q?I434I<99,75VS4/LE8B.2B==
X-Spam-Checker-Version: SpamAssassin 3.0.5 (2005-11-28) on ls44
X-Spam-Level: *
X-Spam-Status: No, score=1.8 required=2.0 tests=AWL,FORGED_RCVD_HELO,
RCVD_IN_NJABL_DUL,RCVD_IN_SORBS_DUL autolearn=no version=3.0.5
X-UIDL: ]V=!!=< &#!%:@"!TP""!

Baffled, I brought in a second pair of eyes, namely one of our senior sysadmins, Bill Dickson. Bill really knows his way around mail and DNS systems and if anyone could figure it out, he could. We both poked around simultaneously, with me listening to him on my phone headset while he did all the same searches and filters that I did last week. Like me, he was coming up empty.

We finally resorted to sending mails to each other, using accounts on those very same mail servers to compare "known good" headers with the ones from the reported spams. We really needed to see for ourselves HOW that this stuff was coming off our servers, and why we could not find it in the logs. We mailed to ourselves, both internally and to external accounts, and compared the resulting headers with the spams.

Finally we came to the inescapable conclusion that the received headers were also forged, at least the ones that referred to our servers.

It is the pefect Red Herring. Those of us who deal with this stuff have long ago learned to distrust "easily forged" headers such as "From:", but until now we have assumed that "Received: from..." were truth. In this case they are, at least partially. The next ones in line above are truth, but the ones naming our network are forged. How do we know this?

Look at the first one:

Received: from 216.168.37.122 (HELO mail.(removed).com)
by (removed).net with esmtp (B1EIM*(?(-/ .O<8)
id 64ER30-)H,QXG-RQ
for mike@(removed).net; Thu, 21 Dec 2006 09:45:03 +0000

Theoretically this is written by the remote server that received it from ours. It is looking back along the path and noting where it came from, and logging the SMTP transaction (the HELO). The BIG clue that we missed is that while the IP address 216.168.37.122 is the right one for that server, the NAME it calls itself to the remote server (mail.domain.com) is wrong. In reality it would have called itself “palm.forest.net” … not the client’s domain name.

HOW the spammer is forging this so cleverly is by doing an MX lookup on the domain they are spoofing. How we figured this out is after we had changed their DNS to point their inbound mail at our test Barracuda server, the spoofed name changed too!

Received: from 216.168.32.228 (HELO mx.(removed).com)
by (removed).com with esmtp (T,@M6M(4J)* 7N9M*)
id 255;5---H2;*-0(
for dawn@(removed).com; Sun, 24 Dec 2006 01:15:36 +0300

I had created the A record “mx.domain.com” last week and here it was showing up in the “Received: from…” headers. There is NO WAY the mail would have gone OUTBOUND from that Barracuda.. it was now set to only handle INBOUND mail.

So the spammers’ mail sending computer just works like this:
1. Make up a random account name for a valid, but spoofed domain name
2. do an MX lookup on that domain
3. forge a very credible “Received: from…” header that includes the proper IP and name for that domains’ server
4. send spam

Abuse reports will be sent to the ISP hosting the domain, and the actual spam source is hidden deeper in the headers. The actual sending machine is still visible, it just appears to be a relaying mail server in the deliver chain! Most likely these are compromised Windows computers on broadband networks, in this case on British Telecom’s DSL network:
host86-144-187-151.range86-144.btcentralplus.com ([86.144.187.151]
and Roadrunner’s cable network:
your-sz6x6sefxo.rochester.rr.com (cpe-66-67-45-66.rochester.res.rr.com [66.67.45.66])

Knowing this now, a glance at the headers shows many other errors that I should have spotted earlier, such as the fact that our server is listed at the absolute first “Received: from…” target, and the actual MUA is missing. That is only possible if a user sends from a webmail session, but those are tagged differently and that tag is missing. But needless to say, I sniffed the red herring and followed that trail. Goodness knows the vast majority of automated spam reporting and lookup systems will do the same. In hindsight the “dictionary attack” I saw on the mail server was nothing of the sort. It was backscatter from all the bounces generated by this spammer, sending to invalid addresses. I do not know how long spammers have been forging “Received: from…” headers (this is the first time I’ve run into it) but it just goes to show how clever they are at both evading spam blocks, AND covering their own tracks.

How long before spammers embed spamassassin spamscores in an attempt to bypass filtering?

Perhaps a better question: How long before spammers kill email? They are literally polluting the ecosystem they live in… the very golden egg laying goose. How could so clever a people be so suicidal?

December 24, 2006

Happy Christmas

Filed under: life — chuck goolsbee @ 9:29 pm

To all my readers (all half-dozen or so of you!) I also wish you all a happy and prosperous 2007!

We’re spending a quiet holiday at home. Traditional dinner tomorrow night. Tuesday the rest of the family is heading south to Oregon to visit relatives. I’ll be working. I hope to use my spare time to wrap up my series of rants I started well over two months ago. I still owe you the answer to the Maxim Gorky riddle, which I promise before year’s end.

2007 looks like it will be an adventurous year for me and those close to me. I’m heading down to Macworld Expo in a few weeks to speak at the IT conference. Subject matter will be technical aspects of blogging (John Welch asked me to sit on a panel), and the session I developed with John, Julian Koh, and Shaun Redmond entitled “Total Network Awareness, which is focussed on one of my favorite subjects (though I never talk about it here), network monitoring systems. Should be fun.

The family are once again heading out to Colorado for our annual visit to my parents, along with the entire scattered Goolsbee clan – my siblings, their spouses and kids. They even have 802.11 now at mom & dad’s house! (whoo hoo! no more dialup!)

In February Christopher heads down to the southern hemisphere, to spend a a few months in Chile as an Exchange Student. He’ll be back in July (I’m thinking of cashing in the frequent flyer miles to head down there to pick him up… maybe get some MORE skiing in!) More on that later.

Summer of course brings the Jaguar out from hibernation and all sorts of fun… stay tuned.

–chuck

Older Posts »

Powered by WordPress