A Theoretical Aside
This post is a followup to my last post concerning false positive findings in animal research. After that post I participated in quite a few conversations that suggested that often times the younger generation of researchers (of which I am a member), often seem to have a fundamental misconception of what constitutes a valid, testable hypothesis and what does not.
Now I think this is a critical topic to cover simply because it is not always straightforward. For example: A research question may be a hypothesis, but not necessarily. A theory is not a hypothesis, but is often necessary to generate hypotheses. An idea, no matter how good, may or may not be testable. What is a null hypotheses anyway? See?, confusing.
When I say a hypothesis, I mean a simple, clear, and above all else testable question accompanied by a clearly predicted outcome. A theory is an overarching mental structure that is used to describe data and predict outcomes for experiments. Personally I think that a researcher, to be successful, must have a theory that guides the overall direction of their research, but it is not necessary that they use this theory indiscriminately to describe everything. Every study, however, must have a clear hypothesis that is easily tested using the methods reported.
Examples of great theories in my opinion (regardless whether I agree with the assumptions of the theory or not): The Hippocampus as a Cognitive Map Link, multiple trace theory Link, hippocampus dependent conjunctive/configural processing Link, and the attribute theory of brain function Link note, this last one is Ray Kesner’s and my theory, so I am a bit biased
A null hypothesis is a statistical idea meaning that multiple distributions of data actually come from a single population-and will not be covered in this post.
I have been very fortunate in my scientific life for having started my research career in Ray Kesner’s laboratory at the University of Utah. The reason I consider myself lucky to have a lasting research collaboration with Ray is that he has raised the skill of developing simple experiments to answer clear questions to a new level within behavioral neuroscience/experimental psychology.
When I joined Ray’s lab he had recently begun work testing the predictions of computational models of hippocampus functions, particularly the models from Edmund Rolls and Alessandro Treves Link, Link. Ray was not the first by any means to try to test these predictions, almost a decade has passed since the models had been published. The novel approach to answer these questions was actually quite simple (albeit difficult to perform in practice); that was to actually disrupt function in one hippocampus subregion while leaving the others intact. This was accomplished primarily through ablative lesion studies wherein either the dentate gyrus, the CA3, or CA1 subregions of the hippocampus were removed using excitotoxins and function of the animal was then evaluated.
The lesion method was effective for answering these questions because the questions that were being asked were extremely simple (in form, not in actually testing them). For the dentate gyrus, the hypothesis was that if the dentate gyrus was nonfunctional, then rodents would be unable to tell similar locations apart, due to an inability to overcome spatial interference. If the CA3 subregion was nonfunctional, then rodents would have an inability to rapidly associate stimulus and spatial locations, be unable to recall patterns based on partial information, and would show learning deficits with short intervals between study and test phases. If the CA1 subregion were nonfunctional, then the rodent would be unable to process temporal relationships among stimuli and show deficits for remembering learned information or learning over temporal gaps.
I know these sound like almost ludicrously simple hypotheses, particularly with the vast research on hippocampus function. But that is exactly the point. These hypotheses are simple, to the point, and most importantly, testable. Special care was always taken to avoid overly general definitions like learning and memory, since they did not provide outcomes that were testable.
What made these hypotheses from Ray so rich was the fact that the specific hypotheses all research questions suggested that as interference among stimuli increased, performance would decrease as a function (e.g., spatial cues overlapping between locations so it is hard to tell them apart or things happen in time very close together and are thus hard to keep straight). By looking at this function in control animals and comparing with lesioned animals, then one may be able to see the different contributions of hippocampus subregions for learning and memory task performance.
Now to the Point
My point with all of this is that often times we as scientists are bedazzled by flashy new techniques that give pretty pictures, be it PET, fMRI, Brainbow, Optogenetics, or anything that may be invented while I write this post. I have nothing against these methods, in fact Optogenetics seems like a nice way to combine lesion studies with elevated activity studies without having to use drugs. However, like all new techniques, the exerted tons are only as good as the questions being asked.
As an example, I have been discussing pattern separations no pattern completion with colleagues, but the definitions are unclear. To grossly oversimplify, pattern separation is the ability of the brain to take incoming information and store it in a way that emphasizes differences, thus reducing interference among very similar stimuli. Pattern completion is the ability of an organism to recall an entire memory from only a partial cue to guide recall (for a much longer, more thorough definite, see the links provided above).
These definitions are clear enough that one can make a hypothesis that is relatively straightforward to test. Unfortunately, the majority of the research into these processes simplify to false positive = pattern separation and generalizing across stimulus = pattern completion. This is a problem because these definitions and the resulting hypotheses are not testing anything that can be clearly defined. In many studies, the hypotheses are vaguely states as spatial pattern separation is important for memory function, and thus if the animals show intact memory and do not generalize across contexts, then they have intact spatial pattern separation. Failure to discriminate contexts is then assumed to be a spatial pattern separation, rather than general spatial memory deficit. In reality, these studies are simply testing general memory, and are thus interpretable in any number of ways other than as the authors intended. This is a problem in most of the studies using transgenic mice at present, as there is no modulation of interference, just a same/different judgment required to discriminate very different contexts.
However, if the authors choose to state that their hypothesis thusly we believe the dentate gyrus mediates spatial pattern separation, therefore if we require that arts overcome spatial interference to perform the task, then rats with a dentate gyrus lesion will show impaired performance as a function of increasing interference, whereas control rats will not show similar performance decreases, there are clear, unambiguous measures that can be evaluated.
Overall, younger scientists need to learn, or be taught by our advisors and mentors, how to ask simple, clear questions. In this time of difficulty in obtaining funding, it is unreasonable to design experiments that involve throwing everything in lab at a problem and wring like the solution was planned. It is okay to, as a collaborator of mine says, throw the spaghetti at the wall and see what sticks. The key is to then design a clear experiment to evaluate why it stuck, not to proclaim tot he world that the problem is now solved based on a shot in the dark.
So, since this is a long post, I will end with another TL;DR message. Clear hypotheses are good. A good, clear hypothesis should lend itself to straightforward studies and analysis. A good hypothesis prevents scientists from having to dance around their data or provide a Procrustean discussion to wedge the findings into their theory.
p>Additional reading “A computational theory of hippocampal function, and empirical tests of the theory.” By Edmund Rolls and Raymond P. Kesner from 2006 in Progress in Neurobiology.