Bing It On Google

Microsoft has launched a new website called Bing It On and an accompanying ad campaign touting Bing’s superior search results. Microsoft is asking people to try their searches at BingItOn.com where Google and Bing results for the same query are shown side by side. When organic search results are displayed without any Google or Bing branding, Microsoft claims that Bing beats Google 2 to 1 in blind comparisons. Wow, sounds a lot like the Pepsi Challenge campaign from the 80′s! Same flawed test, same results favoring the underdog.

Being a search geek, it was fun for me to punch in queries and compare, especially for our clients search terms that I’m intimately familiar with. And not to mention it’s a cool way to check non-personalized rankings on Google and Bing in one place. We have SEOmoz and proxies for that but it’s cool to see actual results side by side. It’s also kind of ironic that, not so long ago, Google accused Bing of copying search results from Google.
This leads me to ask the question, “It’s a clever campaign but is it any more than that?” Well, the media attention and the fact that I’m taking time to write about it will likely lead to a bump in market share for a short time. I’m sure that bump won’t be from this post, I definitely don’t have anything like the Colbert Bump. The campaign has gotten a lot of attention and a lot of people seem to be talking about it. But I would venture to say that the results of this “experiment”  are flawed in the same way that The Pepsi Challenge was and I don’t see a lot of people talking about this aspect of it.

The results of this “experiment” are flawed in the same way that The Pepsi Challenge was and I don’t see a lot of people talking about this…

The Cola Wars

So let’s back up for a few minutes (or decades) and talk about The Pepsi Challenge. The Pepsi Challenge was a blind taste test where Coke drinkers were served Coke and Pepsi in unmarked containers and asked which they preferred. The blind taste tests showed Americans preferred Pepsi by a hefty margin. It’s rumored that Coke, even though they’ll probably never admit it, formulated “New Coke” to taste more like Pepsi and did their own taste tests until they had a formula that beat out Pepsi. In 1985, New Coke was released and the classic Coke ceased to exist. 77 days later Coca-Cola retreated and brought Classic Coke back. There was a huge backlash and it’s often called one of the largest marketing failures of all time.
Malcolm Gladwell outlined in his book Blink why he thinks New Coke failed even though the taste tests indicated it would be a success. His first point was that it was a “sip” test. So people preferred the sweeter, lighter flavor of Pepsi in a sip test but if they sat down to drink a whole can they might have preferred the less sweet flavor of Coca-Cola. As a Coke drinker, I would agree with this to some extent. I definitely prefer Coke because I think Pepsi is too sweet, especially if you drink it out of a 50 gallon drum or a Big Gulp.

I Picked Coke in the Pepsi Challenge

But I think there’s far more to it than just taste. I believe a lot of the taste has to do with the whole experience and what you associate with the brand.  For instance, when I find Coca-Cola in glass bottles at Costco, I do a happy dance and proceed to have the helpful Costco employee hop on his forklift, pick up a pallet and load it on top of my car. I’ve convinced myself that Coke tastes better out of a glass bottle than it does out of a can or plastic bottle. I’ve also decided that these bottles imported all the way from Mexico to  the Granite State taste better and are better for me because they are made with real sugar instead of some high fructose frankin syrup created in a laboratory out of surplus corn. In reality, non of this is probably true and I probably couldn’t tell the difference in a blind taste test. So why do I prefer Coke out of a bottle? I think it’s mostly nostalgia. It brings back memories of childhood, returning those tall worn bottles back to the grocery store to be filled again, or my trips to Brazil when I took breaks from drinking cheap beers on the beach and had an ice cold Coke in a glass bottle. It’s the packaging, it’s color, it’s the logo and it’s everything I personally associate with the brand.

Enough About Cola, What Does This Have to Do with Search?

Okay, so three long paragraphs about soda, pop, cola or whatever you call it, but what the heck does this have to do with search? It has a lot to do with branding. Google has a brand that makes people think of  a new and innovative company that does no evil (although I would disagree with the evil part these days). The Microsoft brand brings up thoughts of blue screens of death, Zune, an awful Windows phone I was dumb enough to buy not so many years ago and many slowly progressing or outdated products. Rebranding to Bing was a good move but they are starting with a new brand that doesn’t mean anything to most people except for it’s association with Microsoft.
Even if Bing can show people prefer their organic search results over Google’s in a blind test, they’ve got a lot more work to do before they can edge out the 800 pound gorilla.

Why the Blind Comparison is Flawed

Here are the three major flaws I see that the Pepsi Challenge and Bing It On have in common:

• As Malcolm Gladwell pointed out in Blink, it’s a “Sip” test. People do a few made up searches but don’t actually use Bing in real life situations, refining their searches to get closer to the answer they are looking for. This makes it an unreliable indicator.
• It’s not representative of the actual search experience. It excludes all the ads and all the results returned from structured data like weather, bacon scores, shopping results, calculators and much more. On top of that paid ads are occupying more space than ever and garnering more clicks. Quality ads can actually help users find what they are looking for and are a big part of the search experience.
• The test doesn’t take into account the inference that brand has on the search experience. It’s hard to measure or place a finger on it but it’s there and has a real impact on search behavior.

Conclusion

So flash forward in the Cola Wars and Pepsi is still #2, or #3 if you take into account that Diet Coke bumped them out of second place in 2011, and Coke is the undisputed leader by a very wide margin. And for the Search Wars, Google is the undisputed champion with 67% of market share, while Bing with 16% is gaining small amounts of market share but taking it from Yahoo not Google (source: comScore). I don’t think the executives at Google are sweating bullets and developing the “New Google” with an all new algorithm that people prefer in blind tests.

It’s good to have a healthy rivalry and it encourages innovation that will ultimately lead to better search experiences. I’d take a lesson from Coke though, and I wouldn’t put too much faith in a blind test because there’s a lot more to it than that.