Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion Groups
General
GeneralCardiologyVisionDentistryPharmacyLaboratoryNutritionAlternative
Diseases and Disorders
AIDSAlzheimer'sArthritisAsthmaCancerBreast CancerDiabetesEpilepsyGlaucomaHepatitisHerpesLupusProstate BPHProstate CancerProstatitisSinusitisTinnitus

Medical Forum / Diseases and Disorders / Breast Cancer / March 2005

Tip: Looking for answers? Try searching our database.

Misuse of statistics in medical research

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Fabrikant - 02 Mar 2005 00:31 GMT
I have prepared an article on the misuse of statistics in medical
research. If the subject is of interest to you or if you work with
cancer, you might be interested to read it. Your opinion would be
appreciated.

To read the article, click here:
http://www.geocities.com/benny_patrick/stats.html

Fabrikant
Tim Jackson - 02 Mar 2005 10:09 GMT
> I have prepared an article on the misuse of statistics in medical
> research. If the subject is of interest to you or if you work with
[quoted text clipped - 5 lines]
>
> Fabrikant

I am as much angered by the abuse of statistics as the next man,
probably more so, but I can't go along with the conclusions here.  I am
no apologist for the pharmaceutical industry and believe that it is
guilty of some serious abuses, for example the creation of new syndromes
in order to sell drugs to 'cure' them, or using hard-sell techniques to
distort doctors clinical judgement, but don't agree that they have been
correctly identified here.

In the analysis, you often refer to the normal distribution as being an
inappropriate model, but this seems to be a matter of faith, I don't see
any reasons given, any explanation of what distribution does apply, or
how the different model affects the validity of the formula for (eg)
confidence interval.

Of the conclusions, of course for a new medicine one hopes that the
researchers already have pretty good evidence for the usefulness of
their product before going to double-blind trials, but no way can I
agree that medicines or techniques should be approved for mass treatment
-without- such a trial.  It has often happened that apparently effective
medications have problems which only show up when applied to a wider
range of the population.  A drug that might be very effective in a
handful of volunteers can have serious problems when applied to the
elderly or children, or when used in combination with other therapies,
or it might show up a small risk of very serious side effects.

Giving a patient an additional treatment for a very small but
statistically significant benefit or chance of benefit may seem
pointless, but really it is a matter of risk analysis, if there are no
drawbacks (including cost, inconvenience etc.) then why not get the
benefit?

Often these small-benefit analyses do not refer to new medications but
to changes in method. For example a recent trial compared giving
chemotherapy every three weeks to giving a smaller dose weekly.  This is
not additional treatment but an either-or situation.  Not only does this
sort of research optimise the efficacy of the treatment, but it also
points out any trend, showing researchers where further benefits might
be obtained.

The same principle applies to drug research. If a drug gives a small but
significant benefit then it is a potential area for research.  For
example, Clodronate is a bisphosphonate which helps bone regrowth.  It
was not very effective and of pretty limited use itself, but over the
last decade or so it pointed the way to a series of newer drugs like
Zometa which are hundreds of times more effective

I feel sure your concern over risk factors is mostly a matter of
semantics.  I don't believe that in identifying higher education as a
risk factor for breast cancer the ACS was suggesting that it should be
avoided, although I accept that I have often seen such statements
mis-interpreted by journalists.  Surely what they were saying was that
women with higher education appear to have a higher risk of breast
cancer, for whatever reason, and so should perhaps take more care over
checks etc.

Likewise the apparent benefits of (eg) drinking red wine should not be
simply ignored.  The fact that some groups live longer or avoid certain
diseases of course does not imply that -changing- your lifestyle to
match certain identifiable attributes of theirs, such as the wine
drinking, will have any benefit, but it does indicate there is a
potential health benefit to be had somewhere in their lifestyle or heredity.

While you may feel that with cervical cancer risk at 1 in 250 it is
small enough that doubling it can be ignored, but you still look both
ways before crossing the road even though your risk of being road-kill
is even smaller.  You are ignoring the possibility of prolonging
something like half a million women's lives in the US alone.

Finally how do you -know- that milk has nothing to do with cervical
cancer.  (Maybe it's not the milk itself but stuff the cow was fed that
gets into it?)  When I was at school my maths teacher insisted that
"obviously" is not proof, and if the only reason you can offer is that
the result is obvious, then it is probably wrong.

Signature

Tim Jackson

-Who needs to pay Microsoft?  Free Open Source Software-
www.mozilla.org       FIREFOX Web Browser,  THUNDERBIRD Email/News
www.openoffice.org      Office Suite

Mary Fisher - 02 Mar 2005 13:48 GMT
>> I have prepared an article on the misuse of statistics in medical
>> research. If the subject is of interest to you or if you work with
>> cancer, you might be interested to read it. Your opinion would be
>> appreciated.

<snip excellent response>

> Finally how do you -know- that milk has nothing to do with cervical
> cancer.  (Maybe it's not the milk itself but stuff the cow was fed that
> gets into it?)  When I was at school my maths teacher insisted that
> "obviously" is not proof, and if the only reason you can offer is that the
> result is obvious, then it is probably wrong.

Thanks, Tim.

Mary
Fabrikant - 23 Mar 2005 22:38 GMT
I have great difficulty understanding the logic of your posting: on
the one hand, you give the examples of clearly dishonest behavior of
pharmaceutical industry, on the other hand, you believe that "... the
researchers already have pretty good evidence for usefulness of their
product before going to double-blind trials".  Why?  Dishonest people
do dishonest things.  One thing is no more dishonest that the other.

In the case of heart medications discussed in the article, I show that
the researchers have had no proof whatsoever.  Indeed, they claim that
their medication helps at best 1 (ONE!) patient per 50 (FIFTY!)
treated.  This kind of percentage becomes statistically significant
only when the sample is at least 5,000 patients.  Convincing?

I do not get the logic of your

>    In the analysis, you often refer to the normal distribution as
>    being an inappropriate model, but it seems to be a matter of
>    faith, I do not see any reason given, or how a different model
>    affects the validity of the formula for (eg) confidence interval.

You are a scientist, "faith" belongs to religion; science is based on
facts.  No distribution is taken "on faith": there are rigorous
methods of model validation.  I believe, I do give enough explanation.
For example, when I say that age does not follow normal distribution,
no explanation is needed: age can not be negative.  In general, for a
good medication, no statistical model is applicable: everybody is
cured.  The worse is the medication, the closer to some statistical
model the situation becomes.  The purpose of my article was criticism;
no statistical model can save fictitious medication.

You write:

> Giving a patient an additional treatment for a very small but
>    statistically significant benefit may seem pointless, but really
>    it is a matter of risk analysis, if there are no drawbacks, then
>    why not get the benefit?

You missed the main argument of my article: the advertised benefit is
a fiction.  A medication is something, which affects all or, at least,
majority of humans.  If it affects one out of fifty, then a proper
question is WHY?  I posed this question to authors of the studies. I
never got any answer, except "we agree to disagree", which proves that
they are not stupid, but rather dishonest.  Did you read the article
about prayer?  Do you take the benefit of a prayer seriously (when a
patient does not know someone is praying for him)?

Crooks in pharmaceutical industry are praying on human misery.  They
take long-developing deadly diseases and claim that they found
medication which slows the process.  Take such terrible diseases as
Alzheimer or MS. They often take 10 years or more to kill people.  You
can take any junk and declare that it slows process, say, by one year.
You will never be caught in your lie, because different people
progress differently.  The only way to show it conclusively, is to
have the same person live twice, which is not possible.  In the
meantime, they rob people of billions of dollars.

About higher education being a risk factor.  Do you by chance have
numbers, indicating, how much more often educated women are diagnosed,
compared to uneducated ones?  I have a pretty good idea, why it
happens: educated women take better care of themselves, visiting
doctors more often, which leads to greater number of false positives.
If I am right, then higher education would be a risk factor for many
other illnesses.  Do you wish to check my hypothesis?

Now about benefits of red wine, harmfulness of milk, etc.  There was a
Chinese study of over 30,000 patient, which "showed" extreme
usefulness of vitamin E, then there was a Canadian study, which showed
a non-statistically significant harmfulness of vitamin E vis-a-vis
heart disease; and usual reaction: more studies are needed.  The same
happened with birth control pills: it was deemed to increase the risk
of cancer, latest study "showed" that it actually protects from
cancer, and again, more studies are needed.  Remember the study which
"showed" that an aspirin can prevent heart attack in men?  Well,
recent publication in New Engtand J. of Medicine "shows" that it does
not help men, but helps women, in addition, now it prevents strokes;
further studies are needed.

I use "showed" in quotation marks, because they actually showed
nothing.  The truth is: no more studies are needed, this is all
garbage science.  If you take two groups of people, no matter how
large, and one group is doing A and another group is doing B and then
you discover a statistically significant difference in some rare
event, it DOES NOT MEAN that A or B have anything to do with it: human
beings are much more complicated than that.  Statistics only tells us
that two groups are different, it does NOT tell us why.  Any
scientific result should be REPRODUCIBLE, if it is not, it is not a
scientific result, and no more studies are needed.

Fabrikant
Tim Jackson - 24 Mar 2005 10:55 GMT
> I have great difficulty understanding the logic of your posting: on
> the one hand, you give the examples of clearly dishonest behavior of
> pharmaceutical industry, on the other hand, you believe that "... the
> researchers already have pretty good evidence for usefulness of their
> product before going to double-blind trials".  Why?  Dishonest people
> do dishonest things.  One thing is no more dishonest that the other.

Whether they are honest or not, they are legally required to conduct
double-blind clinical trials.  In order to get the trial approved, they
have to present evidence that the product or method is likely to be
effective.

This is the law in most 1st-world countries, my beliefs about honesty do
not come into it, but if you want to know, I suspect that like in most
walks of life, the pharmaceutical industry mostly works around
principles  most of us can accept, but sometimes, and perhaps too often,
it oversteps the mark, and sometimes overenthusiastic individuals do
something illegal, with or without malice.

> In the case of heart medications discussed in the article, I show that
> the researchers have had no proof whatsoever.  Indeed, they claim that
[quoted text clipped - 11 lines]
> You are a scientist, "faith" belongs to religion; science is based on
> facts.  
My point exactly. Yet you appear to expect your readers to take these
statements (of the invalidity of statistical models) on faith.

> No distribution is taken "on faith": there are rigorous
> methods of model validation.  I believe, I do give enough explanation.
[quoted text clipped - 4 lines]
> model the situation becomes.  The purpose of my article was criticism;
> no statistical model can save fictitious medication.

I don't think anyone claims that age follows a normal distribution. That
would clearly be daft.  However, for example, age of death can be said
to be approximately normally distributed about the mean life expectancy.

For most purposes this will work, but as I discussed in another thread,
if you apply it say to cancer deaths only (in that case to try to work
out a cost/benefit analysis for chemotherapy), then the distribution
becomes somewhat skewed and you need to estimate how much the skew
affects the mathematical results.  It doesn't mean the results are not
useful, but they have to be used with care.

> You write:
>
[quoted text clipped - 7 lines]
> majority of humans.  If it affects one out of fifty, then a proper
> question is WHY?  

It's like wearing seatbelts.  It benefits less than one in 1000
travellers but we all do it because we can't predict which ones are
going to be in an accident, and for those few, the benefit is large.

By your logic we shouldn't bother with seatbelts or airbags or crash
helmets, or have lifejackets and oxygen masks on aircraft because the
vast majority will not need them.

Or is the advertised benefit of safety devices a fiction too?

I posed this question to authors of the studies. I
> never got any answer, except "we agree to disagree", which proves that
> they are not stupid, but rather dishonest.  

Again it may be true but it is not proven.  They could be just bored
with arguing with someone who demands that certain premises with which
they don't agree, be taken on faith.  I agree to disagree on many things
with my Moslem neighbours, but that doesn't mean I think they are
dishonest, or stupid, I have great respect for them.

> Did you read the article
> about prayer?  Do you take the benefit of a prayer seriously (when a
> patient does not know someone is praying for him)?

No.  As an atheist I find that hard to believe.

> Crooks in pharmaceutical industry are praying on human misery.  They
> take long-developing deadly diseases and claim that they found
[quoted text clipped - 5 lines]
> have the same person live twice, which is not possible.  In the
> meantime, they rob people of billions of dollars.

I dare say.  And the same industry also produce medications which
prolong many of our lives and make the unpleasant parts of life more
bearable.  For example I am very grateful to the said industry for such
things as slow-release morphine which made my wife's death from cancer
so much more comfortable.  You are in danger of throwing the baby out
with the bathwater.

> About higher education being a risk factor.  Do you by chance have
> numbers, indicating, how much more often educated women are diagnosed,
[quoted text clipped - 3 lines]
> If I am right, then higher education would be a risk factor for many
> other illnesses.  Do you wish to check my hypothesis?

I would not be at all surprised if it were true.  But I do not see that
the knowledge leads us anywhere useful.

> Now about benefits of red wine, harmfulness of milk, etc.  There was a
> Chinese study of over 30,000 patient, which "showed" extreme
[quoted text clipped - 8 lines]
> not help men, but helps women, in addition, now it prevents strokes;
> further studies are needed.

I don't know about the particular studies you mention, but in general I
find that if I read a news headline that a study "shows drug protects
from cancer", then go and read the actual study I find the author very
much more circumspect about his claims, and very careful to say only
what the statistics does actually show. If not, peer review should
reject the paper, although I know that doesn't always work.

Most of the problem you perceive (which I agree exists) I think actually
results from journalistic hype of the results for a good headline.

> I use "showed" in quotation marks, because they actually showed
> nothing.  The truth is: no more studies are needed, this is all
[quoted text clipped - 6 lines]
> scientific result should be REPRODUCIBLE, if it is not, it is not a
> scientific result, and no more studies are needed.

"This is not logical Captain."  How could you possibly reproduce a
result -without- doing more studies?

These studies are not identical, if two studies appear to contradict
each other then what needs to be done is to check for evidence of errors
or bias and to figure out in what way the studies differ and so what
factors may have led to the difference.  It may well be that both are
valid despite the contradictory headlines.

The only way the results can be shown to be reproducible or not is by
doing another identical study.  If a result is not reproducible there
are still lessons to be learnt, we need to find out why the result was
wrong and avoid making the same mistake again.

I don't believe that most of these researchers are either incompetent or
deliberately setting out to mislead, although inevitably a minority
will.  Even crooks produce useful information - look at Hendrik Schon's
research in organic electronics - he faked his results but others have
since built real science on this dubious foundation after finding out
what could and couldn't be reproduced.

Signature

Tim Jackson

Mary Fisher - 24 Mar 2005 12:46 GMT
This is a fascinating discourse and I'm not capable of contributing to it -
Tim says it all so eloquently.

> I don't believe that most of these researchers are either incompetent or
> deliberately setting out to mislead, although inevitably a minority will.
> Even crooks produce useful information - look at Hendrik Schon's research
> in organic electronics - he faked his results but others have since built
> real science on this dubious foundation after finding out what could and
> couldn't be reproduced.

This last strikes a chord with me. Over my life I've seen very many
solutions for medical and quasi medical problems which have been superseded
in a long or short time. I've had a lot of children. When I had the first I
was given advice by my mother and mother in law about how we were brought
up, I was also given advice by the hospital staff, GP and health visitor, as
well as friends. Books on the subject had other advice. All claimed to have
the support of evidence and all believed that they had the patient's best
interest at heart.

When I had my second child, less than two years later, everything had
changed. It would be hard to imagine that there could be so many
authoritative opinions on the same processes.

The third - 17 months later - brought even more new and improved advice.

And so it went on.

By this time I'd made up my own mind. As we have to do in all spheres of
life.

This is all anecdotal but something I feel strongly about because I see the
same history of misapplied and sometimes cruelly applied 'science' in the
field of health. Today's reported magic bullet could well be tomorrow's
threat to health.

Anyone who thinks/he has the ultimate answer to any one problem is
misguided, I believe. There are going to be differences in responses between
individuals to any one process (including drugs). The only common cure for
all ills is death.

Sorry to ramble ...

Mary
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.