>> I have prepared an article on the misuse of statistics in medical
>> research. If the subject is of interest to you or if you work with
>> cancer, you might be interested to read it. Your opinion would be
>> appreciated.
<snip excellent response>
> Finally how do you -know- that milk has nothing to do with cervical
> cancer. (Maybe it's not the milk itself but stuff the cow was fed that
> gets into it?) When I was at school my maths teacher insisted that
> "obviously" is not proof, and if the only reason you can offer is that the
> result is obvious, then it is probably wrong.
Thanks, Tim.
Mary
I have great difficulty understanding the logic of your posting: on
the one hand, you give the examples of clearly dishonest behavior of
pharmaceutical industry, on the other hand, you believe that "... the
researchers already have pretty good evidence for usefulness of their
product before going to double-blind trials". Why? Dishonest people
do dishonest things. One thing is no more dishonest that the other.
In the case of heart medications discussed in the article, I show that
the researchers have had no proof whatsoever. Indeed, they claim that
their medication helps at best 1 (ONE!) patient per 50 (FIFTY!)
treated. This kind of percentage becomes statistically significant
only when the sample is at least 5,000 patients. Convincing?
I do not get the logic of your
> In the analysis, you often refer to the normal distribution as
> being an inappropriate model, but it seems to be a matter of
> faith, I do not see any reason given, or how a different model
> affects the validity of the formula for (eg) confidence interval.
You are a scientist, "faith" belongs to religion; science is based on
facts. No distribution is taken "on faith": there are rigorous
methods of model validation. I believe, I do give enough explanation.
For example, when I say that age does not follow normal distribution,
no explanation is needed: age can not be negative. In general, for a
good medication, no statistical model is applicable: everybody is
cured. The worse is the medication, the closer to some statistical
model the situation becomes. The purpose of my article was criticism;
no statistical model can save fictitious medication.
You write:
> Giving a patient an additional treatment for a very small but
> statistically significant benefit may seem pointless, but really
> it is a matter of risk analysis, if there are no drawbacks, then
> why not get the benefit?
You missed the main argument of my article: the advertised benefit is
a fiction. A medication is something, which affects all or, at least,
majority of humans. If it affects one out of fifty, then a proper
question is WHY? I posed this question to authors of the studies. I
never got any answer, except "we agree to disagree", which proves that
they are not stupid, but rather dishonest. Did you read the article
about prayer? Do you take the benefit of a prayer seriously (when a
patient does not know someone is praying for him)?
Crooks in pharmaceutical industry are praying on human misery. They
take long-developing deadly diseases and claim that they found
medication which slows the process. Take such terrible diseases as
Alzheimer or MS. They often take 10 years or more to kill people. You
can take any junk and declare that it slows process, say, by one year.
You will never be caught in your lie, because different people
progress differently. The only way to show it conclusively, is to
have the same person live twice, which is not possible. In the
meantime, they rob people of billions of dollars.
About higher education being a risk factor. Do you by chance have
numbers, indicating, how much more often educated women are diagnosed,
compared to uneducated ones? I have a pretty good idea, why it
happens: educated women take better care of themselves, visiting
doctors more often, which leads to greater number of false positives.
If I am right, then higher education would be a risk factor for many
other illnesses. Do you wish to check my hypothesis?
Now about benefits of red wine, harmfulness of milk, etc. There was a
Chinese study of over 30,000 patient, which "showed" extreme
usefulness of vitamin E, then there was a Canadian study, which showed
a non-statistically significant harmfulness of vitamin E vis-a-vis
heart disease; and usual reaction: more studies are needed. The same
happened with birth control pills: it was deemed to increase the risk
of cancer, latest study "showed" that it actually protects from
cancer, and again, more studies are needed. Remember the study which
"showed" that an aspirin can prevent heart attack in men? Well,
recent publication in New Engtand J. of Medicine "shows" that it does
not help men, but helps women, in addition, now it prevents strokes;
further studies are needed.
I use "showed" in quotation marks, because they actually showed
nothing. The truth is: no more studies are needed, this is all
garbage science. If you take two groups of people, no matter how
large, and one group is doing A and another group is doing B and then
you discover a statistically significant difference in some rare
event, it DOES NOT MEAN that A or B have anything to do with it: human
beings are much more complicated than that. Statistics only tells us
that two groups are different, it does NOT tell us why. Any
scientific result should be REPRODUCIBLE, if it is not, it is not a
scientific result, and no more studies are needed.
Fabrikant
Tim Jackson - 24 Mar 2005 10:55 GMT
> I have great difficulty understanding the logic of your posting: on
> the one hand, you give the examples of clearly dishonest behavior of
> pharmaceutical industry, on the other hand, you believe that "... the
> researchers already have pretty good evidence for usefulness of their
> product before going to double-blind trials". Why? Dishonest people
> do dishonest things. One thing is no more dishonest that the other.
Whether they are honest or not, they are legally required to conduct
double-blind clinical trials. In order to get the trial approved, they
have to present evidence that the product or method is likely to be
effective.
This is the law in most 1st-world countries, my beliefs about honesty do
not come into it, but if you want to know, I suspect that like in most
walks of life, the pharmaceutical industry mostly works around
principles most of us can accept, but sometimes, and perhaps too often,
it oversteps the mark, and sometimes overenthusiastic individuals do
something illegal, with or without malice.
> In the case of heart medications discussed in the article, I show that
> the researchers have had no proof whatsoever. Indeed, they claim that
[quoted text clipped - 11 lines]
> You are a scientist, "faith" belongs to religion; science is based on
> facts.
My point exactly. Yet you appear to expect your readers to take these
statements (of the invalidity of statistical models) on faith.
> No distribution is taken "on faith": there are rigorous
> methods of model validation. I believe, I do give enough explanation.
[quoted text clipped - 4 lines]
> model the situation becomes. The purpose of my article was criticism;
> no statistical model can save fictitious medication.
I don't think anyone claims that age follows a normal distribution. That
would clearly be daft. However, for example, age of death can be said
to be approximately normally distributed about the mean life expectancy.
For most purposes this will work, but as I discussed in another thread,
if you apply it say to cancer deaths only (in that case to try to work
out a cost/benefit analysis for chemotherapy), then the distribution
becomes somewhat skewed and you need to estimate how much the skew
affects the mathematical results. It doesn't mean the results are not
useful, but they have to be used with care.
> You write:
>
[quoted text clipped - 7 lines]
> majority of humans. If it affects one out of fifty, then a proper
> question is WHY?
It's like wearing seatbelts. It benefits less than one in 1000
travellers but we all do it because we can't predict which ones are
going to be in an accident, and for those few, the benefit is large.
By your logic we shouldn't bother with seatbelts or airbags or crash
helmets, or have lifejackets and oxygen masks on aircraft because the
vast majority will not need them.
Or is the advertised benefit of safety devices a fiction too?
I posed this question to authors of the studies. I
> never got any answer, except "we agree to disagree", which proves that
> they are not stupid, but rather dishonest.
Again it may be true but it is not proven. They could be just bored
with arguing with someone who demands that certain premises with which
they don't agree, be taken on faith. I agree to disagree on many things
with my Moslem neighbours, but that doesn't mean I think they are
dishonest, or stupid, I have great respect for them.
> Did you read the article
> about prayer? Do you take the benefit of a prayer seriously (when a
> patient does not know someone is praying for him)?
No. As an atheist I find that hard to believe.
> Crooks in pharmaceutical industry are praying on human misery. They
> take long-developing deadly diseases and claim that they found
[quoted text clipped - 5 lines]
> have the same person live twice, which is not possible. In the
> meantime, they rob people of billions of dollars.
I dare say. And the same industry also produce medications which
prolong many of our lives and make the unpleasant parts of life more
bearable. For example I am very grateful to the said industry for such
things as slow-release morphine which made my wife's death from cancer
so much more comfortable. You are in danger of throwing the baby out
with the bathwater.
> About higher education being a risk factor. Do you by chance have
> numbers, indicating, how much more often educated women are diagnosed,
[quoted text clipped - 3 lines]
> If I am right, then higher education would be a risk factor for many
> other illnesses. Do you wish to check my hypothesis?
I would not be at all surprised if it were true. But I do not see that
the knowledge leads us anywhere useful.
> Now about benefits of red wine, harmfulness of milk, etc. There was a
> Chinese study of over 30,000 patient, which "showed" extreme
[quoted text clipped - 8 lines]
> not help men, but helps women, in addition, now it prevents strokes;
> further studies are needed.
I don't know about the particular studies you mention, but in general I
find that if I read a news headline that a study "shows drug protects
from cancer", then go and read the actual study I find the author very
much more circumspect about his claims, and very careful to say only
what the statistics does actually show. If not, peer review should
reject the paper, although I know that doesn't always work.
Most of the problem you perceive (which I agree exists) I think actually
results from journalistic hype of the results for a good headline.
> I use "showed" in quotation marks, because they actually showed
> nothing. The truth is: no more studies are needed, this is all
[quoted text clipped - 6 lines]
> scientific result should be REPRODUCIBLE, if it is not, it is not a
> scientific result, and no more studies are needed.
"This is not logical Captain." How could you possibly reproduce a
result -without- doing more studies?
These studies are not identical, if two studies appear to contradict
each other then what needs to be done is to check for evidence of errors
or bias and to figure out in what way the studies differ and so what
factors may have led to the difference. It may well be that both are
valid despite the contradictory headlines.
The only way the results can be shown to be reproducible or not is by
doing another identical study. If a result is not reproducible there
are still lessons to be learnt, we need to find out why the result was
wrong and avoid making the same mistake again.
I don't believe that most of these researchers are either incompetent or
deliberately setting out to mislead, although inevitably a minority
will. Even crooks produce useful information - look at Hendrik Schon's
research in organic electronics - he faked his results but others have
since built real science on this dubious foundation after finding out
what could and couldn't be reproduced.

Signature
Tim Jackson
Mary Fisher - 24 Mar 2005 12:46 GMT
This is a fascinating discourse and I'm not capable of contributing to it -
Tim says it all so eloquently.
> I don't believe that most of these researchers are either incompetent or
> deliberately setting out to mislead, although inevitably a minority will.
> Even crooks produce useful information - look at Hendrik Schon's research
> in organic electronics - he faked his results but others have since built
> real science on this dubious foundation after finding out what could and
> couldn't be reproduced.
This last strikes a chord with me. Over my life I've seen very many
solutions for medical and quasi medical problems which have been superseded
in a long or short time. I've had a lot of children. When I had the first I
was given advice by my mother and mother in law about how we were brought
up, I was also given advice by the hospital staff, GP and health visitor, as
well as friends. Books on the subject had other advice. All claimed to have
the support of evidence and all believed that they had the patient's best
interest at heart.
When I had my second child, less than two years later, everything had
changed. It would be hard to imagine that there could be so many
authoritative opinions on the same processes.
The third - 17 months later - brought even more new and improved advice.
And so it went on.
By this time I'd made up my own mind. As we have to do in all spheres of
life.
This is all anecdotal but something I feel strongly about because I see the
same history of misapplied and sometimes cruelly applied 'science' in the
field of health. Today's reported magic bullet could well be tomorrow's
threat to health.
Anyone who thinks/he has the ultimate answer to any one problem is
misguided, I believe. There are going to be differences in responses between
individuals to any one process (including drugs). The only common cure for
all ills is death.
Sorry to ramble ...
Mary