Comparing Treatments Using Randomized Experiments Practice Test

A principal randomly assigned 40 classrooms to two ways of organizing a 10-minute daily reading time for one month: Treatment A (silent independent reading) and Treatment B (teacher read-aloud). Twenty classrooms were randomly assigned to each method. The outcome was the mean increase in a reading fluency score (points) from pretest to posttest. Treatment A had a mean increase of 12.3 points and Treatment B had a mean increase of 10.9 points, so the observed difference in means was $12.3-10.9=1.4$ points (A − B). Under “no treatment effect,” the principal shuffled the treatment labels across classrooms 2000 times and recalculated the difference each time. In the simulations, 52 of the 2000 shuffled differences were at least as large as 1.4 (A − B).

Which conclusion is most reasonable about the treatment effect?

A library tested two reminder methods for returning books on time. They randomly assigned 50 patrons to Treatment A (text reminder) and 50 patrons to Treatment B (email reminder). The outcome was whether the book was returned on time. Treatment A had 41 on-time returns ($0.82$) and Treatment B had 33 on-time returns ($0.66$). The observed difference in proportions was $0.82-0.66=0.16$ (A − B). A randomization test was performed by shuffling the on-time/late outcomes among all 100 patrons 4000 times under “no treatment effect.” In the 4000 shuffles, 12 produced a difference in proportions of at least $+0.16$ (A − B).

Which conclusion is most reasonable about the treatment effect (A − B)?