A/B Split Testing code? --

[-- TW]

I've known about the term for many, many years. But I've also heard it discussed among many (most?) internet marketers I've talked to. So, I thought sure it'd be a cinch to find ready-made code on this site.

I do know about the 'disconnect' between the techie world and the marketing world. -- but I didn't know the 'divide' was so pronounced as to make a basic *function* like testing headlines (on page 3 of any mrktng book) be invisible to *all* the techies on this site.

That was a shocker to me, that's all.

I know most sites don't even have a headline, even though having an effective headline is also on page 3 of any mrktng book.

But, the very next step after the 'epiphany' of the importance of having a headline, is to test one headline against another.

That's where a/b split testing comes in.

In print media it takes the form of the ad in every other newspaper rolling off the presses having a different headline + different (trackable) address or phone number (for example).

But, even now, as I'm typing this, it's hard to believe any of this is news to anyone.

Cheers!

-- TW

[Twey]

I do know about the 'disconnect' between the techie world and the marketing world. -- but I didn't know the 'divide' was so pronounced as to make a basic *function* like testing headlines (on page 3 of any mrktng book) be invisible to *all* the techies on this site.

It's cultural -- I think a lot of techs would prefer to know as little as possible about marketing techniques, which we regard as attempts to get users to buy something that doesn't stand under its own weight.

This is, of course, a generalisation, and thus somewhat inaccurate, but that's the stereotype.

[-- TW]

Don't get me started.

What it seems some people don't realize is... everyone *is* selling something.

'Sales' + 'marketing' really isn't the devil's work.

Everyone is doing it, whether they know it (or admit it), or not.

The only non-superfluous 'prefessions' in the world are the ones you'd see at an Amish village -- after that, every other job in the world is not really a necessity.

It's a house of cards with one unneccessary 'job' stacked on top of another.

In order to keep everyone + the economy afloat, there must be sales + marketing involved.

That's where devil-workshop-mongers like me come in

But seriously, it's sometimes amusing how maligned mrktng + advertising is -- even though everyone's ability to earn a living would vanish without it.

Part of the problem is, most consumers (what I call merketEEs) mistakenly believe they would behave in exactly the same way (buy the same things) with, or without marketing/advertising.

This is incorrect + quite easy to prove it's incorrect. But still, most people believe it.

Cheers!

-- TW

[jscheuer1]

'Sales' + 'marketing' really isn't the devil's work.

Everyone is doing it, whether they know it (or admit it), or not.

Can you say 'megalomaniacal', or just plain psychotic?

What you say actually is true within a certain subset of humanity, I may even qualify for membership, that is not what I take issue with.

To assume that everyone is like you whether they know it or not is arrogant, possibly naive, and insulting at best.

Still really beside the point. Did:

http://www.dynamicdrive.com/forums/s...65&postcount=6

answer your question or not?

[djr33]

It's amusing that you come here asking for help, then explain these concepts to us. If you already know the answer, then why ask?
If that's because you don't know how to make the script yourself, then I'd suggest making it clear what you want with the script.

"A/B" could have a lot of possible meanings. For example, a mathematician would perhaps think the script should use the Pythagorean theorem to calculate the shortest distance between two points.

And even now, after reading those articles, it is still quite vague.

What a lot of non-coders don't seem to understand about code is its basic nature. Code doesn't "do stuff", but rather performs very specific operations in order to reach a certain goal. Knowing the commands is of course important, but knowing how to accomplish the task is the most important part.
Even if I did write a script that did everything you need, it might not work on your server. What I wrote was in PHP. Does your server support PHP? Etc.

I think this concept is pretty stupid unless you have a large site. "Testing" which one works is also testing which one fails, so you'll be losing as much traffic as you gain, though, eventually, in theory, gaining more if you use the "better" system, but that assumes a number of factors, mostly a huge test subject size.

Considering the impact of a headline or a different background color, this might affect, at BEST, maybe 10% of the users (and I think that thought is ridiculous). So, if you have 100 people buying stuff from your site in a week, then you have 55 sales from the "better" way and 45 sales from the "worse" way, do you really use that data? Any good scientist knows that you need to be repeat the experiment. Maybe they just bought it because those people happened to want the product more. Maybe more people got linked to version A than version B... etc.

A good headline won't convince me to buy anything I don't want in the first place, anyway. Then again, I suppose some people are gullible.
But, looks like you are too, with those fancy headlines on the sites. You do realize they are probably using the same techniques to trick people into using their software, right?

[boogyman]

A good headline won't convince me to buy anything I don't want in the first place, anyway. Then again, I suppose some people are gullible.

I agree whole heartedly; and yes some people are really gullible. This past weekend it was time for me to get a new phone as my plan just ran out and my phone quite busted an bruised. Anyway, while I was waiting my turn, I saw this guy being insistent on getting this one phone that compared to the one that the salesman was trying to offer him ($100 cheaper) and had more features, kept on that initial phone that he was on about. He was ranting and raving about how all the big guys at his office had this other phone and that he HAD to have it. I couldn't help but laugh, and wonder what was so special about this other phone, so when they went away I looked at the features and cost and service warranties. But hey, some people just have their mind set on something and they want what they want, regardless of other options

[-- TW]

It seems there are two different 'counters' here.

I stepped up to the 'coding' counter and aksed about a/b split testing code.

I gladly admit I know nothing about coding.

Then, possibly inadvertently, you stepped up to the counter marked 'marketing.'

Please, please -- when you make statements like, 'the headline may affect 10% or less of the readers,' watch what you say. You are making wild guesses based on your opinion.

It is quite common for one headline to yield 5-10x the response rate of another. Sometimes the difference can be 20-30x difference. This is the exact reason to want to use a/b split testing!

Smart marketers do not rely on opinion. They rely on science. These differences in headline effectiveness are provable and repeatable -- and very real. Not smoke + mirrors.

The economy would crumble almost instantly without ads + marketing.

None of this depends on people being 'gulible.' -- ok, some of it does ;-)

Before any of you attempt to man the marketing 'counter,' may I suggest you read up on the subject.

As I've said, all of this is easily provable... scientifically. What the public believes is the case marketing-wise is often 180 degrees out of phase.

BTW, one of the first things marketers learn is -- nothing sells successfully 'of its own weight.'

I admit, good things SHOULD sell better than crappy things, just because they are better.

But that is not so.

Unfortunately or fortunatley, effective marketing is never based on what SHOULD work, only on what DOES work.

Marketing + advertising is the only thing that makes the difference between what sells + what doesn't.

Which explains why crappy things often outsell good things -- better marketing.

For better or worse, the headline is one of the KEY FACTORS that determines what sells + what doesn't.

Books have been written on the subject -- it's not theory.

Again, all easily provable -- scientifically.

-- TW

PS: A good place to start reading up on this is Scientific Advertising Methods by John Caples -- written in the 60's, I believe.

PPS: Why do good copywriters charge tens of thousands of dollars for a single ad or sales letters (sometimes 100's of 1,000s of $$)? I'll answer that one -- It is because their sales letters, etc. waaaaay outperform (no, not a difference of 10% -- a difference of 10, 20, 30, 400 TIMES) what anoher, less talented person would write. Same product/service, same price, same everything. Just different WORDS used. As you can imagine, the people who SPEND these big bux on copywriters want to see the PROOF that the outrageous fees are WORTH IT -- this they do with testing, testing, testing -- with provable, repeatable results.

There's no sense in arguing about all this -- it just IS. ---- which is the whole reason to want a/b split testing!!

I admit -- it SHOULDN'T be that way ---- but it IS that way. It has nothing to do with how 'gulible' people are.

[djr33]

You are making wild guesses based on your opinion.

Opinions generated from logic (we are programmers, so generally our logic has some merit), economics/social sciences in HS/college, and common sense.

The difference really depends on the extremity of the difference between the A and B versions; if it's just a headline on the same site, it will make a much smaller impact than an entirely different marketing campaign sitewide (or broader, such as TV commercials, etc.). From your description, it's all embedded in a single site, with just a few details changed, that customers might not even notice.

written in the 60's

In response to this, I'll link you to the new AMC series on the subject, Mad Men. I think you'll enjoy it. Doesn't really interest me. The commercials sum it up... lots of BS, though, well directed BS

It has nothing to do with how 'gulible' people are.

I assume you've heard of supply and demand. Yeah...
Use your advertising to get the demand, and the system works, you make money, etc.

If everyone was not gullible, things would not work like this. People would simply buy the best product for the money, each time, not the flashiest.

For better or worse, the headline is one of the KEY FACTORS that determines what sells + what doesn't.

Sure. And that is plural, factors. So, let's say there are 3 factors. Let's just, for fun, bump up the number I gave to 33% influence of the title. Out of 100 people, that would still be a fairly insignificant change, let's say 65 of 100 of the sales are from the better headline. Now you've still lost a number of the possible sales from the "bad" one, and waste a week, or more, waiting to test the possible results.

And, really, this all comes down to whether you are willing to play with 100 customers. If so, that means you have a lot of traffic, so why don't you just pay for one of the real packages, if you are making such a profit?

In a large scale scenario, this makes sense, I agree. But as any scientist would tell you, as I said, you need to verify the results with several trials. In a small scale scenario, it just doesn't work as well, and there are too many random factors that might make it seem one way. And I'm assuming that you aren't dealing with such a large scale scenario as you are looking for a free solution.

Frequently, companies use test groups to try out a new idea, in a controlled environment, rather than just randomly tossing it out to the whole population. (But of course that test group would still need to be large enough to matter.)

BTW, one of the first things marketers learn is -- nothing sells successfully 'of its own weight.'

Fair enough, but you can't sell empty boxes either. This split A/B testing isn't some crazy miracle. That I can guarantee. Of course the companies making it want you to think that, and it's working, and there is some merit in the theories, but, again, only on a very large scale.

Before any of you attempt to man the marketing 'counter,' may I suggest you read up on the subject.

Then why did you assume that everyone knows what "A/B marketing" is?

Anyway, in regard to your original question, which seems to have been forgotten, did the code I supplied above work?

[Twey]

A good headline won't convince me to buy anything I don't want in the first place, anyway. Then again, I suppose some people are gullible.

No, but if you're looking for something -- say a new 'phone -- you're more likely to buy one that's well-advertised if you haven't toured the market looking for one. It's sad but true that a lot of people have a lemming-esque tendency to "just want one that works" and pick up the first thing that grabs their eye. Another circumstance is when you're technically unsavvy -- if you're not sure if 512MB or 15GB is bigger, you're quite likely to take the best advice you can get, even if it is obviously biased.

I think this concept is pretty stupid unless you have a large site. "Testing" which one works is also testing which one fails, so you'll be losing as much traffic as you gain, though, eventually, in theory, gaining more if you use the "better" system, but that assumes a number of factors, mostly a huge test subject size.

Not to mention confusing the heck out of your users, as I mentioned above, when they go to your site one day, find an article they like, then come back the next day to find everything's moved around and the article no longer has the same heading or picture. HTTP is a stateless protocol: there's no sure-fire way to determine if a user performing one request is the same as the user performing another. Cookies get deleted; IPs change. The best you can manage is an approximation like the one offered above.

Fair enough, but you can't sell empty boxes either.

Oh, yes you can. The vapourware industry is very profitable these days.

[-- TW]

I didn't assume that *every*one here knew what a/b split testing is -- I just assumed that *some*one here knew it.

As I said, the initial search on this site yielded nothing which was very surprsing -- and still is. Especially when it's something so standard.

I don't know yet if the code given will work. Thanks though.

Maybe I should just buy the ready-made software.

As for whether a headline can make that much difference -- yes it can. And testing it beforehand saves someone from throwing good money after bad.

Yes -- even if it means 'wasting' some potential customers during the test.

But again, this is all standard stuff. The most basic of all marketing practices.

-- TW