With null hypothesis testing, all it takes is sufficient evidence (instead of definitive proof) that a 0 difference between means isn’t likely and you can operate as if at least some difference is true.

The more discrete the data, the larger the required sample size for the same level of precision

you’ll usually use different statistical tests for continuous versus discrete data

Errors provide excellent diagnostic information on why users are failing tasks and, where possible, are mapped to UI problems. Errors can also be analyzed as binary measures: the user either encountered an error (1 = yes) or did not (0 = no).

You can also use the Excel function =GEOMEAN()

the sample of users you measure represents the population

Sample statistics estimate unknown population parameters.

discrete data can usually be preceded by the phrase “number of …”—for

One reason for the confusion between sample size and representativeness is that if your population is composed of, say, 10 distinct groups and you have a sample of 5, then there aren’t enough people in the sample to have a representative from all 10 groups.