Design | Assumptions | Measure of interindividual response variance† | Limitations |
Uncontrolled designs (one group pre–post design) | |||
1. Single premeasurement and postmeasurement |
| Variance of observed change scores. | Cannot establish if observed change or its variance is attributable to treatment. |
2. Multiple premeasurements and postmeasurements |
| Variance of the of the observed change score minus the sum of the average within subject prevariance and postvariance. Can be estimated using classic ANOVA or mixed model. | May be able to remove variance due to measurement error and day-to-day variability but still cannot establish if the estimated interindividual response variability would occur without the intervention. Multiple assessments required. |
3. Longitudinal with repeated measurements spread over time |
| Estimated variance of random slopes as estimated from a linear mixed model. | If linear (or other) model is correct then measurement error and day-to-day variance can be removed but still cannot establish if average change or variance of change is caused by treatment. Multiple assessments required. |
Control group designs (parallel RCT comparing intervention(s) to control) | |||
4. Single premeasurement and postmeasurement |
| Variance of the observed change in the intervention arm minus variance of the observed change in the control arm. | Relies on strong untestable assumptions. Difference in variation between training and control groups is neither necessary nor sufficient for subject-by-training interaction to be present. |
5. Multiple premeasurements and postmeasurements |
| Variance of the of the observed change score minus the sum of the average within subject pre and post variances. Can be estimated using classic ANOVA or mixed model. | Relies on model assumptions. Multiple assessments required. |
6. Longitudinal with repeated measurements spread over time |
| Estimated variance of random slopes as estimated from a linear mixed model. | Relies on model assumptions. Multiple assessments required. |
Other designs | |||
7. Crossover study with multiple intervention and control periods |
| Mixed linear model. In theory, the mixed effects model can isolate the true interindividual response variability for this design. | Costly, may require extensive washout periods, difficult to retain participants over entire study, potential carry-over effects may invalidate results. |
8. External reliability studies |
| Subtract error variance estimated externally from total variance of change observed in current study. | Error estimates from external study may not accurately reflect current study. |
9. Internal reliability substudy |
| Subtract internal estimate of error variance from total variance of change. | Fairly complicated analysis required. Assumes a particular components of variance model. |
*Expanded from table 3 in Hecksteden et al.23
†Take the square root of the individual response variance to obtain SD of individual response (SD_{IR}).
ANOVA, analysis of variance; RCT, randomised controlled trials.