Medical Forum / General / General / January 2004
Health studies don't prove cause and effect
|
|
Thread rating:  |
Duke of Hazard - 07 Jan 2004 16:43 GMT As an engineer I do not understand why journals and the media publish health studies that do not demonstrate cause and effect. The typical study goes something like this:
Researchers found that those who:
------ exercised regularly --or-- ------ ate fruits and vegetables
throughout life were less likely to have contracted xyz disease than those who did not.
One can not make such type of conclusions without first proving there is a relationship between the variables. The conclusions are as absurd as saying "people who eat chocolate are less likely to contract xyz disease". While the statement may be true for the sample population , it does not establish there is any relationship between chocolate and xyz disease.
So my question is why do respectable medical journals and the media continue to publish these types of pseudo-science studies?
DP - 07 Jan 2004 17:16 GMT agree completely! many researchers and 'researchers' respectively in medicine, esp. in epidemiology lack the ability for logical reasoning... media? they live from that kind of garbage :-( brdp the researcher
> As an engineer I do not understand why journals and the media publish > health studies that do not demonstrate cause and effect. The typical [quoted text clipped - 18 lines] > So my question is why do respectable medical journals and the media > continue to publish these types of pseudo-science studies? Mark London - 07 Jan 2004 17:17 GMT It's not as simple as that. They usually compare their findings with the average population, or better yet, a 2nd group of people, which they also monitor to compare to. Plus, the bigger the number of people in the group, the more likely the results are true. Of course, some studies are designed better than others, so repeatable of a study is necessary to confirm the original study, and this is often done. Sometimes it turns out that it's not repeatable.
Also, you have to read the actual article to determine just what was involved in making the conclusion. The summaries you see in the media don't reflect the true design of the study, and the statistics involved in making the conclusion. The media often simplifies (and sometimes distort) the conclusions.
In a previous article, faraz_hussain@yahoo.com (Duke of Hazard) wrote: ->As an engineer I do not understand why journals and the media publish ->health studies that do not demonstrate cause and effect. The typical ->study goes something like this: -> ->Researchers found that those who: -> ->------ exercised regularly ->--or-- ->------ ate fruits and vegetables -> ->throughout life were less likely to have contracted xyz disease than ->those who did not. -> ->One can not make such type of conclusions without first proving there ->is a relationship between the variables. The conclusions are as absurd ->as saying "people who eat chocolate are less likely to contract xyz ->disease". While the statement may be true for the sample population , ->it does not establish there is any relationship between chocolate and ->xyz disease. -> ->So my question is why do respectable medical journals and the media ->continue to publish these types of pseudo-science studies?
David Rind - 08 Jan 2004 01:47 GMT > As an engineer I do not understand why journals and the media publish > health studies that do not demonstrate cause and effect. The typical [quoted text clipped - 18 lines] > So my question is why do respectable medical journals and the media > continue to publish these types of pseudo-science studies? As an engineer, you probably encounter people who know nothing about engineering and yet think that they have simple solutions to problems you know are hard. Fields that have existed for a long time typically have a lot of expertise that may or may not come through in media reports. Yet making sneering comments about fields you do not really understand seems to be a common pastime of many groups.
Medical researchers know perfectly well that (as people in the biz quote constantly) "association does not prove causation". They do all sorts of clever things to try to adjust for this, some of which are more believable than others. However, when dealing with humans, you often cannot run experiments to answer questions. How, exactly, would you suggest doing an experiment to figure out whether exercising for twenty years reduces the risk of heart disease?
Furthermore, you likely completely believe (and rightly so) some things that have only been proven because of associations. No one has ever done an experiment in humans that shows that smoking causes lung cancer. Yet the evidence from observational studies of the sort that you are suggesting are not worthy of publication overwhelmingly demonstrates causation.
You picked an interesting example in particular though with exercise. There are many reasons to worry that in particular with exercise causation may run in the other direction. Healthier people are able to exercise more, and so high levels of exercise may be a marker for health rather than a cause of health. This is extremely hard to tease out of observational studies.
 Signature David Rind drind@caregroup.harvard.edu
Carey Gregory - 08 Jan 2004 03:17 GMT >As an engineer I do not understand why journals and the media publish >health studies that do not demonstrate cause and effect. The typical >study goes something like this: > *snip* David Rind addressed your question much better than I could, but I have to point out that what the media publishes is almost always hyped up, misinterpreted, grandiose summaries of otherwise reasonable research. Next time you see medical research reported in the media, go read the actual journal articles and I think you'll almost always find valid science with appropriately restrained conclusions.
LawsonE - 11 Jan 2004 02:34 GMT "Carey Gregory" <tiredofspam123@comcast.net> wrote in message
> Next > time you see medical research reported in the media, go read the actual > journal articles and I think you'll almost always find valid science with > appropriately restrained conclusions. Depends on the field and what is being reported: if a study supports the status quo beliefs, methodology isn't as important a factor in publication as when a study challenges the standard model.
In behavioral sciences, this problem is so prevelant that as much as 50% of the studies reporting "no effect" lack sufficient statistical power to find an effect if there is one (assuming a .5 standard deviation difference between experimental and control groups).
Jacob Cohen's _Statistical Power in the Behavioral Sciences_ goes into this in great detail.
My favorite place where this shows up is my own field of interest: meditation research. The number of lousy, poorly controlled, tiny studies that find "no difference" between relaxation and meditation, or between two or more different forms of meditation, is astounding. It has become the standard model in the field with an assumption that any study that finds otherwise is somehow flawed or biased, even though it is absolutely trivial to prove (in the logical/mathematical sense) that this standard model simply cannot be supported by the research cited by the vast majority of scientists to support their belief.
Steven O. - 08 Jan 2004 04:15 GMT I haven't read any of these medical studies myself, and as others have pointed out, media coverage is necessarily simplified and condensed. That said, it does seem clear to me that it's worthwhile to pursue studies which suggest associations.
First, let's keep our logic straight: Just because I do not have proof of a cause and effect relationship between two events -- that is, just because I don't have precise, detailed knowledge of the molecular chain of events and energy transfers that may link cause A to result B -- does not mean that causation does not exist.
Second, association gives us ground to suspect that a causal relation does exist, and so serves as a guideline to suggest further research.
Third, the reality is that biochemical systems are very complex. Our present state of science, for all its vast sophistication, is still dwarfed by the complexity of living systems. It may be that, at our present point in history, the *possibility* of causality -- inferred based on association -- may be the best level of knowledge we are able to achieve. I'd rather have partial, tentative, preliminary indications of plausible truths, rather than having no guidelines at all.
If anything, my gripe with medical science is the opposite. Too, often, medicine refuses to consider a possible causal chain unless it's been validated by formal studies. The result is that noteworthy anecdotal evidence is often dismissed. I've been having high blood pressure problems. My doctor put me on a medicine for it, and I noticed my pulse shot way up. I checked the Web, and found anecdotal evidence (accounts from people in newsgroups) that this medicine occasionally, if rarely, does cause rapid pulse. But since the side effect is apparently rare, it's not noted in the literature for the drug. Given that we each have a unique body chemistry, it should certainly be considered that a drug might not show a side effect in a controlled study of 1000 people, but might show a side effect when used in a general population of 100,000 people. (And of course, occasionally approved drugs are withdrawn when dangerous side effects are discovered in general use.)
Anyway, the point is, sometimes knowledge is less than perfect, but still constitutes useful knowledge all the same.
Steve O.
>As an engineer I do not understand why journals and the media publish >health studies that do not demonstrate cause and effect. The typical [quoted text clipped - 18 lines] >So my question is why do respectable medical journals and the media >continue to publish these types of pseudo-science studies? Standard Antiflame Disclaimer: Please don't flame me. I may actually *be* an idiot, but even idiots have feelings.
Duke of Hazard - 08 Jan 2004 17:27 GMT Let me compare back to engineering. Using the health study approach, I could argue that bridges made of steel are less likely to break compared to those made of wood. Why? Because historically and statiscally steel bridges have a better track record than wooden bridges.
However, any decent bridge engineer knows they can build a wooden bridge that is 10X stronger than a steel bridge made of thin and undersized beam sections! This is because they understand the relationship between loads, forces, stresses, and material properties. Detailed mathematics , physics and chemistry can explain what the relationships are.
However, in medicine it seems the health studies really have not identified the association between exercise and heart disease or smoking and lung cancer (as poster David Rind pointed out). They seem content with simply observing that some association may exist which is just as primitive as saying "all bridges made of steel are less likely to break than wooden bridges".
I realize that any theory has at its foundation statements which are accepted as true without being proven. You cannot build a theory starting with nothing.It just seems that other sciences are able to successfully dig a lot deeper into the foundations than medicine is able to. What do you think? Or is Steve correct in that medicine just knows too little?
JG - 08 Jan 2004 20:48 GMT "Duke of Hazard" <faraz_hussain@yahoo.com> wrote...
> Let me compare back to engineering. Using the health study approach, I > could argue that bridges made of steel are less likely to break [quoted text clipped - 22 lines] > able to. What do you think? Or is Steve correct in that medicine just > knows too little? Actually, the epidemiologic studies you criticize can and have identified "associations." As David implied, various methods have been developed to do this. As he also asked, don't you think that biomedical scientists have been wrestling with this problem for many years? If you read the statistics section of such studies, you will see the types of multivariate analysis usually employed. The question then becomes whether there is a causal connection. Contrary to what you implied, we often do have plausible causal connections from laboratory studies and/or deductions from known biochemical or physiologic relationships. What typically happens is that such analyses find significant independent connections between A and B in a number of studies, and laboratory experiments provide a plausible causal link. It's a debatable point when the evidence is good enough to accept the relationship, but this is so for many things in many sciences.
JA Golczewski, Ph.D. http://users.rcn.com/jigo/jg.HTM Updates, free book on health and life-extension
Duke of Hazard - 09 Jan 2004 05:01 GMT Yes, I see your point. I think it's best for me to read some of these studies directly instead of relying on the media summary. Are they available on the internet?
JG - 09 Jan 2004 13:55 GMT > Yes, I see your point. I think it's best for me to read some of these > studies directly instead of relying on the media summary. Are they > available on the internet? Abstracts of just about any study published are available on Medline (which is publicly accessible and free):
http://www.ncbi.nlm.nih.gov/PubMed/
These do not contain all the details or the statistical analysis. You can order the full paper for a price through that site. Full text of some papers is available through links.
JA Golczewski, Ph.D. http://users.rcn.com/jigo/jg.HTM Updates, free book on health and life-extension
Carey Gregory - 09 Jan 2004 02:18 GMT >I realize that any theory has at its foundation statements which are >accepted as true without being proven. You cannot build a theory >starting with nothing.It just seems that other sciences are able to >successfully dig a lot deeper into the foundations than medicine is >able to. What do you think? Or is Steve correct in that medicine just >knows too little? Of course they're able to dig deeper and of course medicine knows too little. Isn't that patently obvious? You can build models of inanimate structures and subject them to laboratory testing to your heart's content. You can't do that with living creatures, not even simple ones.
The most complex structure ever envisioned by engineers is a triviality compared to the simplest of living creatures.
LawsonE - 11 Jan 2004 02:47 GMT > >I realize that any theory has at its foundation statements which are > >accepted as true without being proven. You cannot build a theory [quoted text clipped - 10 lines] > The most complex structure ever envisioned by engineers is a triviality > compared to the simplest of living creatures. Google "Science of Complexity" for more discussion of these issues.
A real-life example might highlight the issue: it is perfectly trivial to build a simple broom balancing apparatus that uses a neural network or genetic algorithm approach to drive the balancing mechanism. A bright high schooler could easily design and build one these days using hobbyist equipment and home-grown software. On the other hand, using a pre-ANN approach would probably be nearly impossible for all but the most talented engineers using rather sophisticated and expensive technology.
The point is, the high schooler's AI-based solution would be very difficult, if not impossible, to describe completely using standard engineering approaches, while a life sciences approach can describe it in terms of a simplified model of learning and a software-based implementation of this model.
And that's a exceedingly trivial example. Now try to model, using any method, how a cell metabolizes a specific quantity of chemical, keeping in mind that no static model will adequately encompass all aspects of the cell that might impact what you are studying. Now model a system of interacting cells (organ). Now model a system of interacting systems of interacting cells (human being).
PF Riley - 10 Jan 2004 05:17 GMT >Second, association gives us ground to suspect that a causal relation >does exist, and so serves as a guideline to suggest further research. This is a very important point. One idea behind doing retrospective "correlation studies" is to see if it would be a waste of time or not to do a prospective "causation" study.
PF
|
|
|