From Denise's blog...
"An exhaustive analysis of the raw data from the China Project by
educator and freelance writer Denise Minger [41] shows that Campbellfailed to take into account other disease-causing variables (increased
Hepatitis B and schistosomiasis infection and rates, industrial work
hazards, etc.) that tend to cluster in higher-cholesterol counties in
the China Study. Campbell also omitted data showing a higher
correlation between wheat flour intake and many diseases (notably
coronary heart disease, cervical cancer, hypertension and stroke) than
with animal protein intake."
Read more...
The following is from an Epidemiologist that refutes Denise Mingers China Study Claims due to incorrect data analysis...
OH MY. By request of beautiful Freelee, I've taken a look at Denise's analysis. I'm an epidemiologist, and on top of that my research focuses on cancer (not that this makes me completely infallible, but at least I feel equipped to provide an informed critique of her statistical capability). Dr. Campbell was certainly gracious in his response to criticism, but I cannot be so kind. Denise is incredibly naive in her crude analysis of the raw data. She uses correlations and ecologic comparisons to draw conclusions about relationships between diet and outcome (cancer, cardiovascular disease, etc.). WRONG WRONG WRONG!!
A correlation does not an association make.
And, as epidemiologists, our studies are intended to determine associations between exposures and disease. (Yes, there are special methods to determine actual causes of disease, but for most of us, associations will do.) See point 1 below for more on this.
Denise, while meticulous, went through a series of exercises only to:
1) Provide a series of correlations, which honestly, is just the FIRST STEP of any good statistical analysis. Let me explain in a nutshell - a correlation is a linear (assumes a "straight-line" relationship - but not all things are related in this manner), unadjusted (does not account for multiple factors that could potentially confound the relationship between an exposure, like diet, and outcome, like cancer), and non-directional (it does not say if one caused the other or the other way around). An association, on the other hand, is generally adjusted for potential confounding factors and - if a study is properly conducted - gives us an idea of temporality or direction. While we certainly look at correlations between all factors (i.e. between the exposure, potential confounding factors, and the outcome), typically more complex modeling of the data ensues so that multiple factors can be accounted for when investigating the relationship between an exposure and the outcome.
2) Much of her conclusions are drawn from purely ecologic data - that is, data that is in aggregate - such as evaluating total cholesterol and colorectal cancer (as Denise does). Sure, it can be informative, but it doesn't tell us anything about some of the other factors that might be related to cholesterol and colorectal cancer. And while she does perform a stratified analysis (stratifying on schistomiasis), which is a form of "adjustment for confounding"), it still does not take into account other possible confounders and still only tells us about general patterns, but nothing of individual-level associations. Furthermore, she doesn't present results for regions with schistomiasis. What if there was also little correlation between cholesterol and colorectal cancer in these regions? There might be other factors unaccounted for.
Ecologic studies are considered to be at the bottom of the "epidemiologic study totem pole." And we can NOT draw individual-level conclusions from them, i.e. we cannot say that an individual who consumes less fat will, on average, be protected from breast cancer (even if that's true, we cannot draw this conclusion from an ecologic study - there's even a term for it: "ecologic fallacy").
OK, my disclaimer: I'm an epidemiologist, and yes, scientists are NOT objective (I'll say it: I'm an ardent veggie with a happy veggie family). Hell, science is not objective. I mean, you could be given a blob of numbers that mean nothing. It takes some context, interpretation, and data processing to make anything meaningful out of those numbers. Yes, scientists can be biased, and so can the studies they conduct, and the analysis of those studies. But good scientists do the best they can, are open about their methods, and fair when discussing their results. I applaud Dr. Campbell for making his raw data available - very few scientists do this. I will be totally honest and say I have not read "The China Study" (I guess I feel it'd just be preaching to the choir, but I think I will read it now...). But I know enough to know that Denise's analysis was crude at best and completely wrong at worst. No card-carrying epidemiologist would EVER be able to publish her results and draw the conclusions that she does.
I've posted the following comment on Denise's blog (which, was there for a few minutes, and now when I go back to the site, it is mysteriously not there anymore...):
Your analysis is completely OVER-SIMPLIFIED. Every good epidemiologist/statistician will tell you that a correlation does NOT equal an association. By running a series of correlations, you’ve merely pointed out linear, non-directional, and unadjusted relationships between two factors. I suggest you pick up a basic biostatistics book, download a free copy of “R” (an open-source statistical software program), and learn how to analyze data properly. I’m a PhD cancer epidemiologist, and would be happy to help you do this properly. While I’m impressed by your crude, and – at best – preliminary analyses, it is quite irresponsible of you to draw conclusions based on these results alone. At the very least, you need to model the data using regression analyses so that you can account for multiple factors at one time.
** Updated to include an example from Denise's analysis rather than my original example of fat consumption and breast cancer.
I just realized that there's still some trail left about "fat consumption and breast cancer". I should clarify. Denise looked at cholesterol level in each Chinese county and the corresponding incidence rates of colorectal cancer in that region (this is what makes it "ecologic" - each dot represents a county). But the statement still stands - we can't make individual-level conclusions about cholesterol, colorectal cancer, and schistomiasis.
I also just want to add that when she refers to "statistical significance", all that's being tested is the "null hypothesis" that there is no correlation (i.e. correlation = 0). it is not testing whether an exposure is or is not arisk factor for the outcome, even though Denise uses this term loosely.
I also just want to add that when she refers to "statistical significance", all that's being tested is the "null hypothesis" that there is no correlation (i.e. correlation = 0). it is not testing whether an exposure is or is not arisk factor for the outcome, even though Denise uses this term loosely.
Replies
this poped up on fb. its a reply by campbell to minger : http://www.tcolincampbell.org/fileadmin/Presentation/finalmingercri...
*******
Impressive. Nice analysis and DAGs. However, I’d like to know what kind of regression model he used – linear regression? If so, it would be important to check that your outcome variable is normally distributed (this is separate from using a nonparametric method to determine the variance and p-values for the beta coefficients).
Are there other factors that could be included in the DAG? Probably.
This is just one analysis for one disease, and points to the strong confounding effect of schistosomiasis. However, is schistosomiasis really a big issue in the North America?? Not really: http://www.cdc.gov/ncidod/dpd/parasites/schistosomiasis/factsht_sch...
So, in a model that is generalizable to the US/Canadian population, the DAG shouldn’t include schistosomiasis. This is precisely one of the points Campbell makes in his response to Denise’s original analysis – context is important!
BTW, which data was used? Oxford data or hand-keyed data from the monograph? If Ms. Minger is to remain open about her methods, then she should probably start using the data that is freely available to all, so that comparable analyses can be repeated by anyone.
this concept seems to repeatedly fall on deaf ears.
BTW, which data was used? Oxford data or hand-keyed data from the monograph?
ok so that's part of the confusion then - denise has been using data that is not the same as what we have access to? "hand-keyed"??? that's about as raw as raw data can get and would certainly require scrutinizing and likely considerable correcting of input errors. interesting that these matters are starting to surface after the big brohahas have died down.
in friendship,
prad
------
That was part of my point, sorry it wasn’t clearer: we might not be able to generalize to the US/Canadian population based on the China survey data alone. But “The China Study” wasn’t just based on the survey data.
You’re right, we absolutely can’t ignore schistosomiasis, especially in this dataset, as Ned’s analysis made evident.
I do think the analysis excluded a potentially important confounder – fiber. Perhaps one might also want to include wheat based on Ms. Minger’s findings. So, while the analysis was quite well done, and the proposed causal pathways nicely laid out in DAG form, I think the model can still be improved upon (as with many things in epidemiologic research).
BTW, I thought Ms. Minger had already performed multiple regression analyses and chosen not to post them? So why is someone else repeating her analyses and posting them instead? It would be best to see these results from Ms. Minger herself.
i haven't really looked at ned's stuff in detail yet, but i did read his comment about campbell's data being good (though he doesn't provide explanation as to why he thinks so). his argument may be a good one for our group to examine more closely than healthycritique has done.
in friendship,
prad
there is a saying that the more one knows, the more one finds out how little one knows.
on the other hand, the less one knows, the more one seems to need to make it sound like knowledge and expertise amount to zilch.
the internet has an 'equalizing' effect - which, while is generally a good thing, tends unfortunately to be misapplied to make things appear equal, when they are often not - and scientific evidence so often is made to take a backseat to personal belief.
in such situations, it is good to remind the 'believer' what yogananda's guru once told him:
"the universe doesn't wait for the sanction of our beliefs"
in friendship,
prad
-
1
-
2
-
3
of 3 Next