How Google tested Google Instant

Google's John Boyd (standing) and LaDawn Jentzch, user experience researchers at Google, explain to CNET's Tom Krazit how Google's usability lab works, as seen from the observation room.

(Credit: James Martin/CNET)

MOUNTAIN VIEW, Calif.--In a world of data-obsessed number-crunching engineers, Google's John Boyd is the people person.
Boyd is responsible for testing user-experience changes to Google Search, the company's most important product. While Google is famous for obsessing over statistical differences in user clicks between one shade of blue versus another, Boyd's team focuses on studying how real people interact with products under development inside Google through the company's usability lab.
This mission took on great importance as Google prepared to make perhaps the biggest change to its search experience it had ever contemplated: Google Instant. Google surveyed 160 people--divided equally between Googlers and the general public--as it developed "Google Psychic," the internal code name for what would become Google Instant.
Over the course of several weeks, Google continued to tweak Instant in front of new testers until it was finally confident in the product. It claims those users became fans: of the 160 people who tested the product, just one said they didn't plan on using Google with the Instant feature turned on, Boyd said, which he called "unheard of in lab testing."
So just how does Google make sure its new ideas are ready for the real world? CNET got a tour of the company's usability labs to find out.
Watching what they eat
Google, like just about every technology company, employs a bevy of eager and captive testers--employees--when getting ready to roll out a new product. However, there are clear limits to what "dogfooding" (as the process is known) can predict about how the general public will receive a product, especially at a company like Google where employees are selected in part because they are "outliers" compared to the general population, Boyd said.
"The user research team is instrumental in understanding how users interact with the design we're creating," said Irene Au, director of user experience across all of the Google-branded product efforts, which excludes things like Android and YouTube.


Read more: http://news.cnet.com/8301-30684_3-20019652-265.html?tag=topTechContentWrap;editorPicks#ixzz12lVEZyWj