Clarity on how much uncertainty there is about the logic behind any intervention is important to define the appropriate scale of a project. If uncertainty is high, it may be crucial to carry out a small pilot intervention before scaling up. This reduces the risk of wasting resources or having unintended consequences. In these situations of uncertainty, it is more valuable to design a project that incorporates a robust experimental design, so that we can actually learn lessons for the future.
The idea is that we want to avoid carrying out interventions where we have weak evidence and a weak experimental design. This is what we call “guessing”. Equally, if there is already a robust evidence base, then maybe the extra cost of robust experimental design isn't worth it - everything has a tradeoff. If we can become more confident about the evidence base or design the project such that it can be evaluated as an experiment, then we can make sure that we are either generating impact, learning a lesson, or both if appropriate.
We have come up with a simple conceptual framework to capture this interaction between the strength of the evidence base and the need for a robust experimental design. This has helped us improve our own evidence use and monitoring processes, and we hope it may be helpful for others too.
