Evidence based medicine is a term we use (or more correctly, throw around) when we are trying to assure the validity that what medicine we are giving has been proven by science to work. Actually this is a definition I just made up as a summary of best explanation I can find to describe this term. I don’t think there is a definite definition for it as it depends who you are talking to and what they are trying to prove. I tend to think since science is constantly learning, claiming evidence based can be a tenuous term but a valuable one nonetheless when used correctly – that evidence shows it works.

The overall judge of this is statistical analysis, a gift really, that allows us to state in one sentence if the results we are looking at mean anything. Statistical analysis is an elementary type of math that allows you to determine easily if what you saw in an experiment can be applied to everyone. It is fascinating to see in action. In other words, the sample of people you drew for the experiment from everyone on the planet gives a result. It is statistic’s job to evaluate whether or not the results you have observed have a small enough range in values based on your sample size to be representative of the larger population. This range in values is referred to as Standard Deviation and gives a measure of how far from the mean all of your readings deviate. This is why an experiment with a large number of study subjects giving results close to the same reading gives a more robust ability to come to a conclusion that it represents the entire population. Conversely, readings all over the board may mean statistically you cannot come to a firm conclusion, and a small sample size in the experiment means you really need to be on your game and have vary little deviation in results to arrive at a strong conclusion.

Observe the above graph that outlines a typical bell curve in readings in an experiment. The goal is to keep all readings as close to the mean as possible to claim that there is a statistical effect. Outside of this range of SD+-1 readings happen that broaden the SD to +-2 and can reduce the certainty of your results. Note the word certainty, we can narrow down the certainty that the conclusion is true but we can’t prove beyond a shadow of a doubt that it is. The biggest beef with self described strict evidence based only medicine subscribers is that a Type 1 error or false positive will result: we have claimed a difference between placebo and drug when in fact there isn’t one. They really don’t care about the Type 2 errors or false negatives when a small study claims there is no difference when in fact there is one because that rarely leads to the therapy being used.

So a larger sample size should have the power to detect smaller differences that are statistically significant between placebo and drug group. When we are looking for a benefit like pain relief with topically applied medication, small differences are not what we are looking for so a big difference is not too difficult to find with a small sample size. Increasing the sample size will help to increase the power of the test and make it more sensitive meaning we avoid type 2 errors more. We make all tests stronger when we repeat the experiment and find similar results, even if that test has a small number of test subjects. Going back to the pain relief in cancer patients, where placebo effect might be there (the palliative care Doctor says if the placebo effect does that then we should bottle that and sell it), a sizeable improvement in pain relief with a small sample size (N of 1) is required to really show the pain creams work, and repeated application of the cream with the same results backs this up.

So what about those patients that fall on the outside of this bell curve. Those in the centre seem to respond well to the drug where as a population sample drawn on the left or right of this graph seems to have non-responders or low responders. How do we treat them? Perhaps the geographical area where the study was done had a result that the larger population didn’t. Small sample size studies are more able to find subtle changes in a medication’s response to a population group that may be lost in a large sample size.

Overall, evidence based medicine is the gold standard and small studies are quite unique in contributing to our overall understanding of the puzzle when applied correctly, just as any large study needs to be evaluated for its strength in arriving at the conclusion it did.