This essay is not about breakfast, although breakfast is, I am told by the morning news anchors, the most important meal of the day. Instead of accepting that "there is research evidence" for the importance of breakfast, however, I usually just have a very large cup of coffee. I know there are studies saying that I should eat a balanced breakfast --eggs perhaps -- but there are also studies that say it does not make any difference at all.
Most health research is like that, actually. Collect up studies on the value of most health interventions (whether they involve medications, procedures, or meals) and you will find that some studies show positive effects, some show negative effects, and some show effects that are close to zero. For a long time, I have believed that I cannot trust any one study because most individual health research studies are probably false. That is, their findings do not reveal what, in nature, is true.
This was not originally my idea. Dr. John Ioannidis, Professor of Medicine, Health Research and Policy at Stanford University, and their director of the Meta-Research Innovation Center has published some interesting papers on this topic. They have catchy titles like: "Why Most Published Research Findings Are False" and "How to Make More Published Research True." The problem with research studies is that they are subject to all kinds of limitations and biases; there are many factors that affect whether a given research finding is true. These factors range from the way the study is designed (a researcher can make the most miniscule effect "significant" by using enough subjects) to the degree to which the researcher has a vested interest (financial, professional, etc.) in the results.
When news reporters tell us about the "latest research study," the single study results they report are bound to be wrong. Days later we might even be told about a new finding without mention of what was reported a few days earlier although this is the nature of news. The fact that individual studies often disagree with each other and tend to range widely in their results might not concern news reporters at all, but should be something that consumers of health information take seriously.
A few decades ago, we were told, unequivocally, that eggs were bad for you because they are high in cholesterol. After a few years, however, a number of studies found that eggs are not so bad after all, as long as we don't eat too many of them (how many was unclear). Later studies found that eggs are really good for people. Again, the studies varied. Some studies had many subjects; others had few. Different kinds of eggs were used in the different studies. Some studies examined the effects of egg consumption on cholesterol (the good, the bad, and the total), which may or may not matter, and on heart attacks, which probably do.
When clinical recommendations are made based on the results of the most recent study (or the largest, or the one done by professors from the most elite universities) the pendulum swings (good for you, bad for you, good for you) depending upon the results of a single investigation.
One way to approach the truth is to instead examine the entire collection of studies done on a particular topic. This is a very time-intensive method; it is a major statistical review called meta-analysis. Meta-analyses of egg consumption have found no danger, and perhaps even some benefit, to eating eggs. The same thing has recently happened with coffee. Drinking a lot of coffee was once considered a "vice," but now coffee is considered clearly healthy, and coffee drinking has benefits in a variety of medical conditions. It is associated with a decreased risk for type II diabetes, gallstones, Parkinson's disease, heart disease, and stroke; it even seems to protect the liver. Coffee drinking is associated with decreased mortality from all causes.
For both eggs and coffee, researchers and clinicians have used meta-analysis to examine all the research that has been done to date, and they combine the results statistically. The need for this approach is particularly acute in public health and medicine, which are disciplines that generate too much information to manage easily. And, the consequences of not finding the truth are fairly high. In medicine, over two million articles are published every year! When medical researchers do more than one study (which they should do), the results are bound to differ from one study to the next, making it difficult to summarize a body of research and offer a correct clinical recommendation (like what to have for breakfast). A broader and more objective view of research is available from meta-analysis because the “landscape” (or distribution of results) can be examined in detail. Meta analysis can answer some important questions like these: Are certain findings more likely with certain study characteristics? Are results different for different populations? Do some researchers find positive effects while others do not?
News reporters tend not to distinguish between individual studies and meta-analyses, but consumers of health information should make it a point to understand the differences. Headlines might capture our attention, but as consumers of health information, we need to distinguish between headlines worth listening to and those that are a just lot of noise. Whether a reported finding comes from a meta-analysis study or from an individual study is a good place to start. Meta-analysis results are not perfect, but they do move us closer to the truth.
I just Googled "meta-analysis breakfast" and found that there is not yet a clear direction in the research. Some studies are showing that weight is managed better with a healthy breakfast; others suggest that the calories from breakfast simply raise the total of daily intake. There has not been enough research to effectively meta-analyze. Health research is sometimes like that, of course. When there is not enough evidence to make a decision, and the issue is critical (like which medications are safest or whether a surgery tends to have good outcomes), we hope that meta-analysts like Dr. Ioannidis and his team are working hard at major statistical reviews for the benefit of all of us. If the issue is not so urgent we can decide on our own and hope for the best until meta-analyses tell us otherwise. Our own decisions are probably as good as the advice of any one study. So, tomorrow's breakfast, for me, will likely be a very large coffee --and if I am hungry, probably an egg.
Most health research is like that, actually. Collect up studies on the value of most health interventions (whether they involve medications, procedures, or meals) and you will find that some studies show positive effects, some show negative effects, and some show effects that are close to zero. For a long time, I have believed that I cannot trust any one study because most individual health research studies are probably false. That is, their findings do not reveal what, in nature, is true.
This was not originally my idea. Dr. John Ioannidis, Professor of Medicine, Health Research and Policy at Stanford University, and their director of the Meta-Research Innovation Center has published some interesting papers on this topic. They have catchy titles like: "Why Most Published Research Findings Are False" and "How to Make More Published Research True." The problem with research studies is that they are subject to all kinds of limitations and biases; there are many factors that affect whether a given research finding is true. These factors range from the way the study is designed (a researcher can make the most miniscule effect "significant" by using enough subjects) to the degree to which the researcher has a vested interest (financial, professional, etc.) in the results.
When news reporters tell us about the "latest research study," the single study results they report are bound to be wrong. Days later we might even be told about a new finding without mention of what was reported a few days earlier although this is the nature of news. The fact that individual studies often disagree with each other and tend to range widely in their results might not concern news reporters at all, but should be something that consumers of health information take seriously.
A few decades ago, we were told, unequivocally, that eggs were bad for you because they are high in cholesterol. After a few years, however, a number of studies found that eggs are not so bad after all, as long as we don't eat too many of them (how many was unclear). Later studies found that eggs are really good for people. Again, the studies varied. Some studies had many subjects; others had few. Different kinds of eggs were used in the different studies. Some studies examined the effects of egg consumption on cholesterol (the good, the bad, and the total), which may or may not matter, and on heart attacks, which probably do.
When clinical recommendations are made based on the results of the most recent study (or the largest, or the one done by professors from the most elite universities) the pendulum swings (good for you, bad for you, good for you) depending upon the results of a single investigation.
One way to approach the truth is to instead examine the entire collection of studies done on a particular topic. This is a very time-intensive method; it is a major statistical review called meta-analysis. Meta-analyses of egg consumption have found no danger, and perhaps even some benefit, to eating eggs. The same thing has recently happened with coffee. Drinking a lot of coffee was once considered a "vice," but now coffee is considered clearly healthy, and coffee drinking has benefits in a variety of medical conditions. It is associated with a decreased risk for type II diabetes, gallstones, Parkinson's disease, heart disease, and stroke; it even seems to protect the liver. Coffee drinking is associated with decreased mortality from all causes.
For both eggs and coffee, researchers and clinicians have used meta-analysis to examine all the research that has been done to date, and they combine the results statistically. The need for this approach is particularly acute in public health and medicine, which are disciplines that generate too much information to manage easily. And, the consequences of not finding the truth are fairly high. In medicine, over two million articles are published every year! When medical researchers do more than one study (which they should do), the results are bound to differ from one study to the next, making it difficult to summarize a body of research and offer a correct clinical recommendation (like what to have for breakfast). A broader and more objective view of research is available from meta-analysis because the “landscape” (or distribution of results) can be examined in detail. Meta analysis can answer some important questions like these: Are certain findings more likely with certain study characteristics? Are results different for different populations? Do some researchers find positive effects while others do not?
News reporters tend not to distinguish between individual studies and meta-analyses, but consumers of health information should make it a point to understand the differences. Headlines might capture our attention, but as consumers of health information, we need to distinguish between headlines worth listening to and those that are a just lot of noise. Whether a reported finding comes from a meta-analysis study or from an individual study is a good place to start. Meta-analysis results are not perfect, but they do move us closer to the truth.
I just Googled "meta-analysis breakfast" and found that there is not yet a clear direction in the research. Some studies are showing that weight is managed better with a healthy breakfast; others suggest that the calories from breakfast simply raise the total of daily intake. There has not been enough research to effectively meta-analyze. Health research is sometimes like that, of course. When there is not enough evidence to make a decision, and the issue is critical (like which medications are safest or whether a surgery tends to have good outcomes), we hope that meta-analysts like Dr. Ioannidis and his team are working hard at major statistical reviews for the benefit of all of us. If the issue is not so urgent we can decide on our own and hope for the best until meta-analyses tell us otherwise. Our own decisions are probably as good as the advice of any one study. So, tomorrow's breakfast, for me, will likely be a very large coffee --and if I am hungry, probably an egg.