Quantifying the User Experience: Practical Statistics for User Research

Because the total area must add up to 100% under the curve, we can express a z-score of 1.28 as being higher than 90% of values or less than 10% of values

UI problems, typically organized into lists, have names, a description, and often a severity rating that takes into account the observed problem frequency and its impact on the user.

User research is the systematic study of the goals, needs, and capabilities of users so as to specify design, construction, or improvement of tools to benefit how users work and live.

There are generally two types of usability tests: finding and fixing usability problems (formative tests) and describing the usability of an application using metrics (summative tests). The terms formative and summative come from education (Scriven, 1967) where they are used in a similar way to describe tests of student learning (formative—providin

Chapters 6 and 7 contain a thorough discussion of power and computing sample sizes to control Type II errors.

From an analytical perspective, a useful way to organize UI problems is to associate them with the users who encountered them, as shown in Table 2.1.

Don’t let the size of your sample (even if you have as few as 2–5 users) preclude you from using statistics to quantify your data and inform your design decisions.

The larger our sample size, the smaller we would expect the standard error to be and the less we’d expect our sample mean to differ from the population mean.

Task time, number of usability problems, and rating-scale data are quantitative.