Weight of Evidence Frameworks in the Regulatory Context: A Conceptual Comparison

Environmental regulations seek to protect the public from health risks
that are too rare to be observed directly but nonetheless deemed of
public concern, so decisions must be made by drawing inference from
observable but only indirectly relevant effects in animal testing at
high doses.  Such information is frequently contradictory and seldom
dispositive, yet inaction because of suspended judgment or action on
imperfect information have their own negative impacts on public good.
The process of inferring what we wish to know from what we can know is
critical to sound and defensible evaluation of the bearing of scientific
data.  A weight-of-evidence approach is required.  I review several
approaches - rules-based systems, evidence-based toxicology, expert
judgment elicitation, and structured hypothetico-deductive processes -
and gauge their comparative utility in supporting public-health
regulatory decision-making.  Rules-based approaches can be consistent
and operational, but risk codification of conventional wisdom and
succeed only to the degree that generally sound inference is built into
the rules.  Evidence-based toxicology promises rigor, but in the
frequent case of underdetermined systems, it provides poor basis for
sound choices.  Expert judgment is good at synthesis of diverse lines of
evidence, but it is nontransparent and invites criticism of choice of
judges.  Hypothetico-deductive systems are complex and require
case-specific assembly of arguments, but promise a means to judge the
relative credence to be accorded differing interpretations - with
different consequences for regulatory decisions - in a way that is
transparent and encourages open and productive discussion of how
inferences relate to the evidence at hand.  A key question is where in
the regulatory decision-making process, and in whose hands, the
evaluation of uncertainty of inference and the consideration of
possibilities, plausibilities, and soundness of inferences should
reside.