WolframAlpha beats Google for Results and Usability but not Brand

In 2005, here at Bunnyfoot, we carried out an eye tracking usability study; it showed that 79% of people were able to find the 2003 UK gross domestic product using Google.
We carried out a similar eyetracking study in May 2009 using Bunnyfoot’s Mass User Testing approach and found that this had dropped to 37%.
We also compared the performance of Google to the new WolframAlpha search engine where 100% of people got the correct answer. This result is worrying for Google for two reasons:

• Google’s algorithms have got better in the intervening years; despite there being significantly more pages indexed on Google in 2009 compared to 2005 Google returns fewer results for the same search string; “gross domestic product UK 2003”. Given more pages to return results from and better algorithms it ‘should’ be easier to find information, not harder.
• The general level of people’s Internet experience and expertise has increased since the original study – people ‘should’ be more successful, not less.

WolframAlpha also outperforms Google on three key measures of usability; effectiveness and efficiency and satisfaction. However, the strength of the Google brand dominated WolframAlpha with 100% of users saying that they would recommend using Google to a friend with only 77% saying they would recommend WolframAlpha.
The study is by no means comprehensive; it is based on a single search query and one that favours WolframAlpha’s approach to knowledge management/search, but is does pose an interesting question:
Can Google’s search dominance be beaten by better results and usability or is the brand so strong that people will stay loyal no matter how good the competition gets?