We attempt to clarify, and suggest how to avoid, several serious
misunderstandings about and fallacies of causal inference in
experimental and observational research. These issues concern some
of the most basic advantages and disadvantages of each basic
research design. Problems include improper use of hypothesis tests
for covariate balance between the treated and control groups, and
the consequences of using randomization, blocking before
randomization, and matching after treatment assignment to achieve
covariate balance. Applied researchers in a wide range of
scientific disciplines seem to fall prey to one or more of these
fallacies, and as a result make suboptimal design or analysis
choices. To clarify these points, we derive a new four-part
decomposition of the key estimation errors in making causal
inferences. We then show how this decomposition can help scholars
from different experimental and observational research traditions
better understand each other's inferential problems and attempted
solutions.
|