Calculating confidence intervals around sample based survey estimates
The LFS survey data is based on a sample of the population. Estimates derived from the survey are therefore subject to sampling variability because a different sample of individuals could have produced slightly different results.
The precision of a survey estimate can be assessed by calculating a confidence interval. A confidence interval provides a range of plausible values for the true population value (that is, the value that would have been obtained if the entire population had been surveyed), assuming no bias is present.
HSE presents 95% confidence intervals for LFS-based estimates. If the same survey were repeated many times and a 95% confidence interval calculated from each sample, around 95% of those intervals would be expected to contain the true population value. A 95% confidence interval is approximately equal to the estimate plus or minus 1.96 standard errors, where the standard error reflects the sampling variability of the estimate.
Confidence intervals reflect sampling variability only and do not capture other sources of uncertainty, such as non-response bias or measurement error.
For estimates based on small numbers, the lower confidence limit may be calculated as a negative value. As negative values are not meaningful for these measures, lower negative confidence intervals are truncated to zero.
Reliability thresholds
Precision generally decreases as the number of sample cases contributing to an estimate becomes smaller. Estimates based on small sample sizes are therefore subject to greater sampling variability and wider confidence intervals.
- For prevalence and incidence estimates based on fewer than 30 sample cases, confidence intervals should be quoted in preference to the central estimate or rate. In published tables, a note is included alongside these estimates to indicate that they are based on a small sample size.
- For working days lost estimates and rates based on fewer than 40 sample cases reporting time off work, confidence intervals should be quoted in preference to the central estimate or rate. In published tables, a note is similarly included alongside these estimates.
- Estimates based on fewer than 20 sample cases are not published.
Calculating confidence intervals around sample based survey estimates of change
Change between two periods is calculated as the difference between the corresponding estimates.
As with annual estimates, estimates of change are subject to sampling variability. Where two independent estimates are compared, the standard error of the estimate of change is calculated as:
Standard error (estimate1 − estimate2) = √(variance(estimate1) + variance(estimate2))
A 95% confidence interval for the change is then calculated using the estimated change and its standard error.
If the 95% confidence interval for the change excludes zero, the change is considered statistically significant at the 5% level. All tests of statistical significance of LFS estimates used in HSE publications are conducted at the 5% significance level, unless otherwise stated.
Improving the precision of sample estimates by pooling annual data
Reliability can be improved by increasing the sample size contributing to an estimate. Whilst the annual LFS sample size is fixed, data from multiple survey years can be combined to produce estimates representing an average across the pooled period.
HSE generally presents estimates by demographic and employment-related characteristics as three-year averages. Estimates by detailed occupation and industry, where sample numbers are often lower, are also typically presented as five-year averages.
The measures of workplace injury, work-related ill health and working days lost are calculated using the same methods as for annual estimates, but are based on pooled data covering multiple years rather than a single survey year.