cu Reed Snider

Chromecast Setup Usability Study

Testing the Google Chromecast open-box setup: A software and hardware in-home usability study
April 2016

The User Friendly team was a project for a testing and assessments course at Bentley University in the Human Factors and Information Design MS program. We gathered 10 participants and visited their homes to study how they would set up the Google Chromecast in their home entertainment systems.

My role in the project was to moderate multiple in-home usability tests and assist in analyzing the data to help provide recommendations for product improvements. I created mockups to provide a visual representation of the recommended design changes for both the physical product of the Google Chromecast as well as the accompanying hardware.

In summary, we found out that the tutorial video for the Google Chromecast needed to be shown much earlier in the software setup as participants struggled to search the web for help when at a stopping point. Something that helps to describe the problems with the sample population we studied is that many adults do not know what an HDMI port is and often do not know where it is located on a television. This entire study was conducted with females who typically do not set up electronic devices in their home. Our group chose to visit the homes of 10 participants to observe the setup process in-person.

We learned about contextual observation, incorporating both ethnography and usability testing user research. This study helped me gain insight into what must be considered in assessing a physical product that combines with software.

Long-form final presentation on slideshare>>     


The Problem

    Google believes that users can set up their Chromecast device on their own, without external support. Their aim is to make the setup process fast and intuitive, ensuring a high level of product usability and a low likelihood of product returns. Google seeks to understand and evaluate the current setup process for a target user group of individuals between the ages of 45-80 who typically do not set up electronic devices in their home, or who often need help when setting up such devices.

    Study Goals

    • Assess the ease and intuitiveness of the self-service setup process for the Chromecast device
    • Find points of user error and frustration
    • Determine opportunities for design improvements in both the hardware and software
    • Find points of user error and frustration
    • Determine opportunities for design improvements in both the hardware and software


Workflow phases for the Chromecast setup process

Participants were excited to use the Chromecast and were impressed by its functionality. Several participants expressed an interest in purchasing one following the usability study.

"“It's made by Google, so it’s probably easy to setup. I can probably get it done in 10 minutes.” - P10""

However, the Chromecast setup process proved to be highly challenging for the target user group. None of the participants were able to complete the setup on their own (i.e. without assistance from an evaluator or a family member), which conflicts with a key business goal. Additionally, the setup process received a very low SUS score of 28 out of 100. If Google wishes to attract and retain customers in this user group, they must take steps to improve the ease and intuitiveness of the setup process.


Research Process

Each participant completed:

  • A background questionnaire to verify their information
  • A pre-test interview to assess prior knowledge of the Chromecast
  • Series of tasks, each of which was followed by a set of questions
  • System Usability Scale (SUS) test
  • A post-test interview
    • We consolidated our findings as a team, discussed duplicative issues, and arrived at a consensus on how to definine our findings and designate severity ratings.

      Here is a complete overview of the findings about the product that were uncovered during the study.

      What I learned

      This study was a great chance to observe people in their home environment to take in all the complexities that may have not been otherwise found in a typical lab setting. The interruptions from children and spouses was an unanticipated hurdle we had to overcome.

      Looking back, I would have liked to include a wider range of participants in terms of socioeconomic status and age as well as including more males. However, the experience helped me discover what type of preparation and research methods are best for hardware and software products like the Google Translate device and application. I learned about analyzing data in a group setting, utilizing tools such as Excel in combination with communications tools like Hangouts to stay connected throughout the project.

      We learned about contextual observation, incorporating both ethnography and usability testing user research. This study helped me gain insight into what must be considered in assessing a physical product that combines with software.

      Highlights
      • Working with a combination Physical/Virtual product
      • Testing products with "non-tech-savvy" users
      • Creating hardware and software design solution mockups

Reed Snider contact

This website was coded by Reed Snider