Rising costs, fixed prices.

Pondering pricing in a post-economic downturn era.

One of the services we offer at digital.forest is data backup. We have four backup servers that run scripts to backup data from “clients” (that is, servers, owned by clients of digital.forest, running “client software” for the backup server. Got it?) Like all services here, we haven’t changed the price of backup in well over 4 years.

Right before the “dot com crash” (which wasn’t a “dot com crash”, but I’ll explain my views on that some other time) we actually performed a large-scale review of all of our offerings, what the competition was charging – and started adjusting our prices accordingly. At that time, we were a small-scale operation with a set of niche offerings. The only price hike we managed to complete prior to the economy’s turn was on FileMaker hosting. We are the largest FileMaker database hosting operation in existence. At one time it was a growing business, but between 2000 and 2004 it seriously stagnated… more due to FileMaker Inc (FMI) taking far too long to rev FMP 5 (again, I’ll have to leave my views on FMP and FMI for another post.) We saw a lot of our clients migrate to PHP/mySQL solutions from Lasso-or-CDML/FMP solutions. Hosting FMP databases is a very expensive business to run since it requires more resources – more software and servers per customer that just about any Internet database offering I can think of. Odd considering that it is considered a “low end” database solution. So it made sense to raise our prices, especially since FMI kept raising theirs. If I recall correctly we raised them about 10-15%, but only lost about 1% of our clients due to the price hike. That was an interesting exercise in Capitalism. Too bad the economy, other database products, and FMI’s slow work on what eventually became FMP 7 managed to wipe out 40% of our FMP hosting business over the next four years.

Thankfully other offerings filled the gap. Server colocation became a significant part of our business. We had built a pretty nice little datacenter by 2000. It was small, but had almost everything you would expect to find in a large-scale industrial datacenter, just on a small scale. It was basically some converted office space in Bothell, but we had a great backup power system, and multiple fiber lines coming into the building. We were an autonomous network with BGP4 connectivity to several major Internet “backbone” providers (I hate that term, but I’ll use it here for simplicity.)

In 2001/2002 when large colo providers were going down every week, or consolidating datacenters, we went from being viewed as “risky because we’re small” to being “safe because we’re small.” Another thing that happened at the same time, and continued well into 2004, is that prices plummeted. Webhosting rates fell by 60% or so, and server colocation fell through the floor to unsustainable rates. I remember in 2000 Exodus charged anywhere from $4000 to $8000 a month for a single rack. We charged $2000, which was “cheap.” Within two years the “big boys” (which in Seattle meant only InterNAP and a few remaining operators) were practically giving rackspace away. I remember losing an 8-rack deal to InterNAP in early 2003 when they lowballed the price to something insane like $250 a rack. It was obvious they were floating on investment capital, had a big huge brand new (but mostly empty) facility to fill, and knew that any revenue was better than no revenue. We have never been big enough to operate like that. Our colo prices have come down though, right along with the rest of the industry. No, you can’t buy a rack from us for $250, but we have gone from being a “value priced” provider to being about the same as everyone else, if not a little high. I’m OK with not being the cheapest, mostly because we offer what so few providers can’t, personal service. We are a niche player, not a commodity one.

Today we are still here, still growing, and overall doing pretty good. We moved into a new facility (ironically one built by a failed competitor) and now actually do have a top-tier facility in every way. Unfortunately the costs of operation have grown at the same rate as our growth, and we have basically kept our level of profitability all along (if you were to pool our total profit over the past three years you could buy a small Korean sedan.) We at least are marginally profitable, unlike so many in our industry. We’ve done it by taking advantage of every cost savings we could find (in bandwidth, equipment, etc.) and keeping the rising costs (electricity, storage, people, etc.) as under control as we could.

So our prices have either stayed where they were in 2000, or in many cases, gone down. One price that has been frozen is data backup. Back in 2000 we charged $30 a month for data backup. Back in 2000 your average web server had maybe 250 megabytes of data, with 20 megs of that changing on a daily basis (usually database dumps.) We were running a VXA tape library with a 15-tape capacity, and our other two backup machines ran single drive AIT tapes. So at $30 a month we were covering the cost of the tape autoloader and probably making a buck or two per client until the cost of the library was covered. I doubt it ever was because by 2002 we had to start backing up to hard disks. Why? There just was not enough time in a night to backup to tape anymore. Our backup window kept increasing until we were backing up during non-night hours. When our backup software started supporting backups to HDDs we jumped on it and started buying the biggest disks we could (at that time around 100GB) and using them like tape – chew them up and throw them out. When drives got bigger, we bought bigger drives – 120GB, 180GB, 200GB, 250GB. Of course, so did our customers, so we were rarely able to stay ahead of the time/capacity curve.

Apple shipped their XRAID drive array a couple of years ago, and we have purchased a few since to add to our arsenal of backup and storage devices. We sell space on one for clients, but use the others for backup media.

About three months ago I cried “uncle”… Here we are, spending tens of thousands of dollars to maintain a service we are making a few thousand dollars a year on. We’ve fallen into a similar trap our competitors did when they dropped colocation prices in 2002… only this time we didn’t raise our prices to at least match the cost of the service provided.

We are using close to 6TB of storage, and backups now run 7/24. Any pause for a data restore puts us in a position where we play catch-up for several days. Clients complain about missed backups (your server too slow? sorry, we have to skip you); clients complain about backups happening during business hours (OK, we can put you in the special “nighttime” script, but no guarantee that we can back you up every night); clients complain about the time it takes to back them up (let’s do the math… three 250GB volumes of mostly uncompressed and non-compressible data, over a network at around 250-300 MB per minute… that is almost two days!)

The client who has a small server with a few hundred MB of data? They are still paying a reasonable data backup price at $30 a month. The client with more than 50GB of data (and we have some with >TB of data) THEY are getting way more service @ $30 a month than they can imagine, even when we skip them or miss them entirely a few times a week.

It is obvious that we have to implement a pay-for-what-you-use data backup system, and that is what we are about to do next month. It could not come soon enough for me.

The Rains Have Returned

I don’t comment much about weather, but the subject came up in an iChat with a friend on the east coast.

Here in the Pacific Northwest we really only have two seasons, “wet” and “dry”. “Wet” lasts from sometime in September or October, until early July. Dry lasts from early July (usually the 5th or 6th!) until sometime in September or October.

In the 1980s “drought” years “dry” would sometimes last until November. I remember climbing “Outer Space” on the Snow Creek Wall over a weekend in early November around 1987 or so. It was cold at night but very warm… “hot” even during the day. Recently our weather has been unsettled, with either VERY wet years (in 1999 we didn’t really have a “Dry” season… until September. Last winter was the as dry as “wet” can be, with hardly any snow in the mountains and very little rain down here. Oddly enough the past few year’s “wet” started big, with some big October storms … these pictures were taken two years ago today. But then settled into a “very sparsely moist” rather than our usual full-on “wet”.

Well, the “wet” has returned to the Pacific Northwest. October has been more rainy than clear, and quite chilly as well. We had a brief little “Indian Summer” the past two days, mostly sunny temps in the high-60s F. Friday I drove the Jag down to the body shop for the bonnet ding to get repaired, and yesterday was spent hacking back the grass since the sun was out (making hay while the sun shines as it were.) Today however reinforces how brief that nice respite was. Rain, mist, fog, temps in the 40s & 50s F.

It will be this way, relentlessly wet, with only high winds and storms to break the monotony from now until January when the “storm season” ends, and then it will just be plain old rain. You can basically say “Rain, mixed with showers, with a rare sun-break, lows in the mid-30’s, high’s in the mid-50’s” if you were a weatherman from now until March. Sure, we’ll have a few snows sprinkled in there, and of course that one week in January or so when the sun comes out… just to keep us from killing each other. Sometime in April we’ll start to see more sun, and warmer temps, and the Jag will come back out of the barn now and then. Until then, you won’t hear me talk about it other than winter-time projects. (Like my plan to perhaps do something about the radio console once and for all.)

When the “Dry” season returns, I’ll comment about our reward for the crappy weather we put up with around here, until then, I’ll try not to say much about weather.

Stop me somebody…

before I buy this.

It is close by. It is cheap. It is legendary for reliability – one was the world record holder for mileage: 1.9 million miles(!). It is of course a Diesel. Not exactly the 300SD or 300SDL I’ve been looking for, but still. I bet it does 0-60 in at least 30 seconds! (downhill, with a tailwind!)

Update 10/23/05: I managed to show some self-restraint and didn’t bid. Final price wasn’t too bad. I did some reading on the model and it was the first really successful Merceded-Benz Diesel for export. It has ZERO collectible value though.

WVO Filtering Setup

I finally photographed the home-brew Diesel setup I built:

WVO Filter

The waste veggie oil goes in the top barrel. The sawed-in-half gas can acts as a funnel. The oil comes to me via the white 5 gallon buckets on the left, so they don’t pour into 2″ holes without a funnel. The gas can sits very well in the filler bung, so I won’t spill… too much. The top barrel has a bung in the side, through which I have a 3/4″ ID gasoline hose, which goes through a stop-valve, to a 15-30 micron filter, through another stop-valve, through a 5-15 micron filter, through the final stop-valve to the bottom barrel. The bottom barrel is equipped with a nice 10 gallons per minute hand pump. The whole setup is airtight, and is kept from freeze damage (thankfully only a slight possibility here in the Pacific Northwest) with one of those pipe-warming cords that winds its way from the top to the bottom. You will note the bottom barrel has a blanket around it for extra insulation. If we get a significant cold snap I’ll have to supplement the heat with a light bulb or something.

This whole section of the barn was built by the previous owner specifically to store Diesel fuel. The shelf there had four large fuel storage tanks on it when we first viewed the house. The guy owned a logging company and the barn was his workshop for the trucks and equipment. The floor below it is not on the concrete slab, but it is filled with gravel and oil-absorbent stuff. Pretty cool.

I only have about 5 gallons in the upper tank right now. It should start flowing on its own via gravity once I have about 20 gallons in the upper tank. Running the hand pump will also provide suction back through the system to boost the filtering. Can’t wait to get it running at capacity.

Stickin’ it to the Man by burning my own oil.

Some late night ramblings about Diesel cars and my home-brew fuel setup:

I’m halfway done building my fuel making setup out in the barn. I’ve been getting WVO-based fuel from a friend of mine all summer, while I slowly collected the stuff to build my own. (WVO = Waste Vegetable Oil… i.e. used french fry oil.) In the Spring I gave John as much of the left over Diesel from the old Bothell Genset as he could carry. I fueled my car out of it from January on, then bought/borrowed as many gas cans as I could find and filled them with the fuel. There was still close to 50 gallons left a week before we sold the genset, so I had John take the rest. Once the gas can supply ran out, I got filtered WVO from John. This I mixed 50/50 with pump-bought Diesel. The car runs great on it, and I get over 50 MPG. Not bad for effectively cutting my fuel cost in half! John just took a new job, so he connected me up with the Burger joint where he gets his free oil near his old job. I should have my two-barrel filtering setup done tomorrow, at which point I’ll be in the fuel business… personal consumption only of course. Just pick up 5-10 gallons a week from the burger place, and make my own.

The Jetta is the fourth Diesel car I have owned. I much prefer Diesels, especially turbo-Diesels to gasoline cars for everyday driving. They are reliable, and very economical. They don’t produce greenhouse or CO2 gasses. Yes, they are a bit “sooty” but modern CDI and TDI engines do an amazing job at reducing the smoke – mostly with VERY efficient injections systems and EGR systems (Exhaust Gas Recirulation.)

I guess now that gasoline is at $3.00 a gallon the rest of America is suddenly interested in Diesel cars again. I’ve been casually shopping for a new or used VW or Mercedes Diesel for almost a year, and in the last three months they have gotten virtually impossible to find or buy. I passed on some really nice MB 300 2.5, and 300sdl cars last winter. I should have grabbed them when I had them close at hand. Too bad the ONLY choices for new Diesels are either VW or MB, … or a gigantic full-sized pickup truck. Fuel costs have been rising steadily for four straight years but did Detroit pull their heads out of their SUV asses? Hell no.

VW is the lone seller of a full line of Diesels… from the Beetle all the way up to the Passat have a TDI option. MB only offers an E Class Diesel now in the US I think. I don’t make enough money to fork over big bucks for a new Mercedes though.

Anyway, once my fuel setup is done I’ll photograph it and post it here.

No Comment, just comments requested

Yeah, I know that this is a big week for technology… especially Apple. Unfortunately I don’t have any comment because I’m too busy working in the technology business to have any time to comment at the moment. =)

I may be looking for some geek help next week though. Sue left her gig at the Skagit County Public Defender and is now a consulting Attorney for a Defender’s association. She’s also looking to pick up some private practice in Snohomish, and maybe Skagit counties. She’s looking at putting an ad in the yellow pages, but I was thinking that whipping up a quick website and buying some Google Ad Words might be a better use of her money. Especially since the phone directory people want like $500 a month(!) and it seems like with Google you are getting something more tangible in terms of traffic for your revenue.

Especially since what she specializes in… Minor in Possesion, DUI, etc. If she was doing estates, civil contracts, etc a phone book ad might be smarter… I don’t know.

I never look in the phone book anymore… I look online. I am a geek though, so maybe I am not the right test group for the experiment.