Explain why effect size is more informative than statistical significance when evaluating a program's impact on student outcomes.

• A. Effect size indicates practical magnitude of change, not sample size; significance alone may be due to large samples.
• B. Statistical significance is always sufficient.
• C. A large sample makes any effect seem important, regardless of magnitude.
• D. Effect size is irrelevant to practical decisions.

Answer: (A)

The key idea is focusing on how big the program’s impact is, not just whether that impact is statistically detectable. Statistical significance tells you the probability that the observed difference could occur by chance, given the sample size and variability. It can be shaped by how many students you studied: with a very large sample, even tiny, practically irrelevant changes can look statistically significant.

Effect size, on the other hand, measures the magnitude of the change in a standardized way, independent of sample size. It answers the question of practical importance: is the improvement large enough to matter in classrooms and to justify adopting the program? By looking at the size of the effect, you can compare results across studies and contexts and make more informed decisions about scaling, resources, and policy.

So the best choice highlights that effect size conveys the practical magnitude of change, while significance alone can be driven by sample size and may not reflect real-world impact.