Jim Manzi’s Uncontrolled

 此新書或可一讀
請點進去看Amazon的書評



Tampering or Experiments?
National Review Online
Statistical quality control guru W. Edwards Deming used the term “tampering” to describe this process of introducing changes without rigorously evaluating ...

Tampering or Experiments?

By Reihan Salam

May 29, 2012 12:04 P.M.

Comments

0

In a review of Jim Manzi’s excellent new book Uncontrolled, Arnold Kling offers the following aside:

I was once seated at a dinner table next to an official of the Department of Education involved in education research. I made an impassioned plea for more controlled experiments in education. The official responded by asking, “Would you want your child to be the subject of an experiment?” At this, my jaw dropped, and I sputtered, “They do it to my children all the time! They constantly introduce curriculum changes, scheduling changes, and changes in teacher methods. They just don’t bother to evaluate whether or not it works.”
Statistical quality control guru W. Edwards Deming used the term “tampering” to describe this process of introducing changes without rigorously evaluating results. Tampering and experiments are two ways of disturbing the status quo. But only experiments are designed with the intent of producing reliable measurements of success or failure.
Like my dinner companion, most policymakers view experiments as at best costly and at worst immoral. Even though tampering is just as bad, if not worse, it somehow escapes such criticisms.

I often encounter arguments against decentralized trial-and-error. The alternative to trial-and-error, however, is centralized trial-and-error, in which errors are far more costly and far more difficult to reverse. But it is very easy to mask these costs: all you need to do is fail to engage in the rigorous evaluation of results, as Kling suggests.
On a separate note, an education professor recently made the case that choice-based reform in New York city has exacerbated the advantages of savvy, affluent parents. She makes a number of entirely legitimate points. Yet it is important to reckon with the counterfactual, e.g., had New York city not embraced (decidedly imperfect) choice-based reform, would we have a more egalitarian, high-performing educational system? Or would we have somewhat more middle-class flight? And if we had the latter, would this have materially improved outcomes for students in New York city public schools — or would it have eroded the tax base, undermined the political investment of middle-class voters in the school system, and contributed to a general deterioration of the quality of life in New York city? This seems like a question worth asking.