Well, the "sales letter" style homepage I tried out for CaRP last month was no Willy Loman, but a statistical analysis of its conversion ratio (page views to downloads of the free version) showed that it didn't perform as well as the page that I'm using right now, and while I don't have stats to back this up, I think it didn't sell the commercial versions of the software as well either.

Here's how the story played out: early last month, I ditched my old CaRP homepage design for a "sales letter" type page. It had all the elements--a narrative approach, answers to possible objections, customer testimonials, a photograph of yours truly, the money back guarantee, and so on. At first, I tracked it's conversion ratio manually and made little tweaks every once in a while. Honestly, it didn't perform that badly--perhaps as well as the old version of the page.

Later in the month, I threw together some scripts to enable me to run split tests of two versions of the page and get a proper statistical analysis of the results. I created a pared down version of the page that drastically shortened the narrative section of the pitch, and let the two run for a few days. When a visitor arrived at the page, the system randomly selected which page to show them, and then for 24 hours, continued to show the same page to anyone using that same IP address.

The statistics showed the pared down version performing just barely better than the long version, so I axed the long version and replaced it with a complete remake of the page based partially on the old version of the page, but taking a few elements from the sales letter. I ran that with the pared down sales letter for a few days, and the stats showed the non-sales letter page performing better. And at the same time (though I can't statistically prove any connection), my income jumped back up a notch.

So the sales letter is dead, and I'm running two slightly different variants of the new page...

What lessons have I learned from the experience? 1) No single approach works for every market--you have to test and find out what works for you. 2) To compare two sales pages, use split testing with proper statistical analysis.

I may release a product--perhaps a free download--based on the code I wrote for myself at some point...if I can find the time to package it up nicely.