Third-grade teacher Annie Long grappled with a new problem when her students reviewed the online practice test to prepare for the Common Core-aligned assessments they will be taking in a few weeks.

The last decade has seen a massive shift toward automated testing practices, which wiped out the manual handling of a lot of rote tasks. But the pervasiveness of this change has been so quick that ...