One day the eggs are packed with cholesterol. The next day, they are safe for people with excess cholesterol. Studies follow one another with often contradictory results: enough to cast doubt on the quality of research in nutrition. Here’s what to check before changing your diet for nothing.
Just about all foods, and the nutrients they contain, have been scrutinized by researchers or companies who are trying to find benefits – or unhealthy effects – on our bodies. Some of these studies are reflected in the media or social networks, but not always with the necessary nuances. How to sort?
To begin with, there are several basic questions that ordinary people can ask about a study: Rumor detector had spoken about it here and here, in the context of the pandemic. However, all these questions remain valid for all scientific studies, whatever the discipline. But when it comes to nutrition, there are a few that need to be taken more seriously.
1. When did the study date ?
The media often provide little information on the context in which a study takes place: do the results confirm or refute the conclusions of one or more previous research? The absence of this context can cause the importance of the study to be overstated.
You can’t ask everyone to search a database of scientific studies to see if other researchers have explored the same subject. On the other hand, we can check when the study was dated: if it is a few months or a few years old, and if the journalist or the YouTuber who told us about it does not seem to have asked other experts what they had to say about it since this publication, it’s a bad sign.
It is normal for a study that confirms a popular belief (“milk is good for bones!”) Or goes against what was taken for granted (“chocolate is good for you!”) draws attention media and social media audience. It is therefore even more important to ask whether this is the very first study to reach this conclusion, or if, since its publication, other researchers have not brought downs or criticisms.
If, in fact, the results call into question established scientific knowledge, it is preferable to wait for new studies to confirm them before giving credence to them.
2. How many people participated in the study ?
Large studies, with thousands of participants, often provide more reliable results than small studies with as few as 10 people. The number of participants is information that should feature prominently in a story.
Meta-analyzes, which combine the results of several individual studies, increase sample size and statistical power. However, some meta-analyzes group studies with extremely different populations and methods and this variability reduces their power, which should be mentioned by the authors, and relayed by the popular articles.
Here again, it is not uncommon to see, in nutrition, studies that have been blown out, when they were carried out on groups much too small for any conclusions to be drawn.
3. Was the study performed on animals or humans? ?
It is inevitable that the question will arise in health, where any drug will first have to be tested on animals. But it also arises in food: nutrients and food are regularly tested on animals. Consider saturated fats that promote obesity and diabetes in mice, sugar that increases their anxiety, or antioxidants and other foods known to be anti-cancer that were first tested in animals.
Except that to better understand how these foods and nutrients affect human health, sooner or later they must be studied in humans. Tests done on animals cannot systematically be applied to humans due to physiological differences (reactions to chemicals, susceptibility to viruses, etc.). In addition, doses administered to animals may be different from those given to humans.
4. Does the methodology used reflect normal usage? ?
Studies on foods or their components (vitamins, minerals, etc.) often involve doses that are impossible to consume in reality. When one would have to eat four or six cups of broccoli or blueberries every day to reap the advertised benefits, it’s hard to think that the study’s conclusions apply!
You can’t expect a story to always have the space to go into all of these details. But researchers and the media reporting the study should clarify whether it was about a food or these components. For example, were participants to eat blueberries, a blueberry juice extract, or a capsule of an antioxidant in blueberries?
5. What are the limits of the study ?
When publishing their study, authors are required to cite what they consider to be the weak points of their work. For example, a high number of dropouts among participants in long-term studies requiring strict diets. Or, results that have not yet been confirmed. Or a weakness in the way data is collected. Or the age of the participants, since the effects of a food or a nutrient on the body can differ depending on whether you are more or less young. The report that recounts this study should echo such limitations, where they exist. And there almost always is.
6. What type of study is it ?
When it comes to testing a drug, we know that “double-blind” studies are the most solid. One group receives the treatment we want to test and the other, a placebo. No one knows which group he belongs to. The results are then compared for each group.
In nutrition, such studies are often impractical due to the high cost, the difficulty of maintaining the same diet over a long period, and potential ethical issues. And it is almost impossible to control a person’s diet in a real life setting. Imagine, for example, subjecting 1,000 people to a vegetarian diet and as many to a red meat diet for 20 years to compare the health impacts. What is more, it would be impossible to hide from the participants what they are eating. Not to mention the difficulty in distinguishing the effect of a single food from the rest of the diet.
Often, therefore, nutrition researchers conduct what are called observational studies. We bring together thousands of participants who are given a medical examination and fill out a form on their lifestyle and diet. Researchers observe participants and assess their health regularly for a few months, but most often for years, even up to 15 years. These large cohorts make it possible to cross-reference thousands of food variables with thousands of effects on health. Researchers can thus associate eating habits with states of health. On the other hand, they can hardly demonstrate a cause and effect relationship between a food and a change in health.
Despite these limits, observational, or epidemiological, studies remain the best way to conduct nutritional studies. They allow us to identify avenues for reflection and new avenues of research.
7. Who finances the study ?
Do the financial partners of the study work in the same sector of activity as the product studied? The food or agrifood industry funds a lot of research. There may then be interference in the choice of research question, selection of data, interpretation of results and how to explain them to the public.
In 2007, an analysis of over 100 studies funded by industry revealed that the source of funding was closely related to the findings of the study. It is therefore important that the researchers, as well as those who popularize the study, mention this source of funding. This research should not necessarily be put aside, but analyzed with the risk of bias in mind.
8. How are the results presented in the media? ?
Nutritionists and nutrition researchers often criticize journalists for not reading the studies and presenting the results out of context. The media too often content themselves with publishing information contained in press releases issued by universities and research centers – which necessarily focus on the most encouraging or discouraging elements of the study – without taking the time to read the studies to assess their strengths and weaknesses.
This is what happened with a literature paper who sparked controversy in 2019 when he was told that we should not limit the consumption of red meat. However, by reading the methodology, it was found that the authors estimated the consumption of red meat in the population at two or three servings per week. In the end, their recommendation to maintain the consumption of red meat therefore corresponded to that of the World Health Organization and Canada’s food guide, which is to limit their consumption of red meat to two or three servings per week. A contextualization which is very important …
This article is part of the column of Rumor detector, click here for other texts.