When testing the usability of the products we design, we’re looking for definitive answers to the various assumptions and hypotheses we have made up to that point. We seek this validation from probable users who confirm whether pieces such as navigation, content, and interactions work together to make for a successful experience. By evaluating task-based actions of 5-10 users, we can detect trends in the overall comfort and ease of use. However, when communicating these trends, particularly to our clients, it can be hard to summarize this through a series of task success and failure rates. This is where the System Usability Scale (SUS) comes in handy.
What is the SUS?
The SUS provides us with a single usability score. Our clients use this number as a summary of the testing rather than cobbling together a series of random observations. Conceptualized in 1986 by engineer John Brooke, the SUS quickly went viral and has since become the unofficial “official” standard for measuring usability. The beauty of the scale is that it is extremely flexible: it can be applied to really any kind of product, from a mobile app to automobiles.
Often you’ll see the words “quick & dirty” used to describe the SUS. It is essentially a variation of the Likert Scale in which users are given a set of 10 statements and are asked to rate them on a scale from 1 (strongly disagree) to 5 (strongly agree) (1).
- I think that I would like to use this system frequently.
- I found the system unnecessarily complex.
- I thought the system was easy to use.
- I think that I would need the support of a technical person to be able to use this system.
- I found the various functions in this system were well integrated.
- I thought there was too much inconsistency in this system.
- I would imagine that most people would learn to use this system very quickly.
- I found the system very cumbersome to use.
- I felt very confident using the system.
- I needed to learn a lot of things before I could get going with this system.
The surveyed results are then quantified into a score of 40 and multiplied by 2.5 to become a score out of 100. Generally, anything above a 68 is considered usable. Easy enough, right?
The Challenges of the SUS
We love the simplicity of the SUS but have noticed some potential pain points. Acknowledging these issues, especially during the testing results summary, is critical.
Challenge #1: Finding the right time to implement the SUS.
Solution: Since the SUS is meant to measure the usability of a fully realized product, the prototype should be in or nearing its final stages before administering the test. If it’s given too soon, users will respond only to the system in front of them – not how they think the final product will eventually look.
Challenge #2: Confusing the users with the words and phrasing of statements.
Solution: We have noticed some users struggling to understand the statements as they are originally worded. Some of the terms are above a certain reading level (“cumbersome” from statement #7 for instance). This is not only going to affect the rating but also could place our users in an embarrassing situation. The phrases also alternate back and forth between positive and negative leaning. While this format is intentionally utilized to prevent what is known as acquiescence bias (2), manipulating our users is not a direction we are comfortable going. By adjusting the template – for example, making each sentence positive and replacing large words with simple language – the user’s brain will not have to work as hard to understand each phrase, leading to ease of completing the test.
Is it safe to customize such a well-known and beloved format? We think so.
Because the SUS, by nature, is “quick and dirty” in revealing conclusions, the structure can and should be adjustable. Even founder John Brooke is open to adapting the SUS. In a 25-year retrospective, he wrote:
“If I was starting out again, I might do things differently … I would have to decide whether user errors outweigh response biases. As it is, ‘SUS Classic’ now has a mass of research to support it, but there’s always the option of using an ‘all positive SUS’ if you worry about user errors.” (3)
Challenge #3: Placing too much emphasis on the SUS score.
Solution: While the SUS score is a useful tool for clients to understand the test results, it is important to remember that it is not the only takeaway from the test. Rather, the score should serve as a motivator to proceed with clearly defined actionable next steps.
We’re always looking for ways to improve our testing process. By strategizing when we use the SUS, how we present it to users and summarize it to our clients, we strive to get the most meaningful results possible.