Addressing Authority: Complex Causality and the Path from Social Science to Policy

One of the differences between the social sciences and most natural sciences is the unavailability of controlled experiments as a device for testing  hypotheses. The empirical basis for the social sciences consists almost entirely of observations of the past, almost always made for some extra-scientific purpose. Inevitably, alternative explanations of the same facts can coexist for a long time. If they are policy-relevant, controversy about policy recommendations can also persist. In the past three decades there has been a systematic effort to conduct experimental tests of social programs, though not usually of scientific hypotheses. The problems connected with self-selection are avoided by random assignment of members of the eligible population to experimental and control groups. The experimental group is exposed to the program being tested; controls are exposed to whatever society offers generally. Both groups must be followed for a long time--five years at a minimum, though longer is better--and relevant differences measured. Such experiments have made it possible to detect nice-sounding policies that have no statistically significant effect in practice. It is a common finding that, even when a social program turns out to have a statistically significant effect, the effect is discouragingly small. Why is this? The most likely reason is that the social policies that anyone will pay to test in this expensive way are aimed at serious and complex social pathologies. They have multiple interacting causes. Even a well-designed remedial policy will affect only a few of the underlying causal pathways. The effects, while real, are not dramatic. Is anything to be done about this? Obviously it is important to invent better causal models and design sharper experimental tests. It may be more immediately important to teach the political process that even small gains against complex social problems are valuable, and that dramatic "solutions" are, except in rare circumstances, simply not available.