Why do we calculate confidence intervals

Imagine that you are trying to find out how many Canadians have taken at least two weeks of vacation in the past year. You could ask every Canadian about his or her vacation schedule to get the answer, but this would be expensive and time consuming. To save time and money, you would probably survey a smaller group of Canadians. However, your finding may be different from the actual value if you had surveyed the whole population. That is, it would be an estimate.

Each time you repeat the survey, you would likely get slightly different results. Commonly, when researchers present this type of estimate, they will put a confidence interval CI around it.

The CI is a range of values, above and below a finding, in which the actual value is likely to fall. The confidence interval represents the accuracy or precision of an estimate. We often see CIs in newspapers when the results of polls are released. An example from the Globe and Mail newspaper regarding the mayoral race in Toronto read, "52 per cent [of survey respondents] said they would have voted for Mr. Miller if the election had been held last week. The margin of error is plus or minus 4.

Finally, we plug in the mean, which is This is unfortunate. If the true population mean were as high as Oh, dear. That would be Here are the confidence intervals for this sample, for some typical levels of risk:. You can see that, as we reduce the risk, we increase the confidence level and end up with a wider confidence interval—and in this example, also have an increasing level of depression about that launch date.

Have you come across Six Sigma, the quality improvement program that Motorola originated, which is now popular in many manufacturing companies? They wanted to be very, very sure that they knew the risk of manufacturing poor-quality products and chose a confidence level of Increase the sample size.

The more data in your sample, the smaller your confidence interval. The mean for all of the participants is This is true, but only if the sample is a random sample.

If we took a random sample of just runners, we would get a narrow confidence interval. But would that be a good target to have?

Figure 5 shows a set of data that is quite typical for user experience: a peak at low values—for example, task times—then a long tail with a few values that are much higher.

The mean is But how useful is the mean? But what if we were working in minutes instead of seconds? Many people would indeed notice if a task that they anticipated taking less than an hour actually took over two hours. Also, look at the way most of the values pile up at the shorter end. Those users ought to be happy—the time it took them to complete the task is much shorter than the advertised time. So our colleagues who are managing system performance are likely to be far more interested in the most frequent value, which is the mode, than in the mean.

The confidence interval for the mean helps you to estimate the true population mean and lets you avoid the additional effort that gathering a lot of extra data would require. You can compare the confidence interval you calculated with the target you were aiming for. Once you have worked out what level of risk you are willing to accept, confidence intervals for the mean are easy to calculate. Thanks for writing this article, Caroline. They should understand the limitations of their data and generate findings that are not more precise than the data-collection methods allow.

Nice work! You need a sample size of at least 30 for the CLT to apply. So unfortunately, there are two things you need to be able to validly construct a confidence interval around a sample mean:. This means our small usability test samples of are not enough for this sort of quantitative analysis. On being an inexperienced quant researcher: Definitely! On the sample size issue: Oops. Must go back to the stats books again.

He also had a calculator and a detailed explanation of how it works, for small-sample—less than —confidence intervals. Best, Caroline. When it comes to confidence intervals, the smaller the better! This is because we have a smaller range of values our population mean could lie within. When time and money are tight in user research, sometimes we do have to rely on smaller sample sizes. However, by calculating the confidence intervals around any data we collect, we have additional information about the likely values we are trying to estimate.

Confidence intervals, although they may not seem it, are there to help! They make your data analyses richer and give you more from the metrics you captured and help you to make more informed decisions about your research questions.

Start listening to users and collaborating with stakeholders now! UserZoom is a registered trademark. Discover what's next on the horizon for digital experience: Join our webinar The Experience Apex Register now.

UserZoom GO Make quick decisions with a platform for remote user interviews and usability testing. UX Research capabilities. Quick design iteration Continuously test to keep up with the pace of design and development. User-focused product decisions Get experience insights to inform a variety of stakeholder questions and make bigger bets. Digital experience measurement Get metrics business leaders need by tracking digital experience progress over time or relative to competitors.

UserZoom See what's included in every subscription plan. UserZoom GO Compare options and start a free trial. Newsroom Get the latest from UserZoom. Our company Learn more about UserZoom's values and mission. Our experts Meet the great minds behind our research and training.

Our customers Learn from our customers' stories. Careers Join our team and help drive UX forward. Join our panel Get paid to share your feedback. Integrations Centralize all your customer and research data Partners Partner with UserZoom to improve business impact. UX library Build your UX knowledge with our blog-posts, ebooks, reports and webinars.