It seems to me I heard somewhere that Jim Chinnis wrote in article
<bph111tjvtstbig9uadcc7etv4ctdghhqb@4ax.com>:

[Trimming the content substantially]

[. . .]

[. . .]

I just want to thank all three of you for having the ability to work
through issues like this for the benefit of the entire group and for
taking the time to do it.  Considering the apathy or, often, the
antipathy toward things scientific in US society these days, the
importance of this sort of conversation can't be overstated, IMO.  I
would have to do considerable brush-up and then supplement my meager
knowledge of statistics with additional study, but I can follow the gist
of the arguments and would like to think any interested reader could do
the same.

It saddens me, and I'm sure others as well, that science seems to have
become simply one more elective way to know the world and conduct a
life, on an even par with dogma, myth, folklore, and even fiction (Da
Vinci, anyone?).  

David Rind <drind@caregroup.harvard.edu> wrote in part:

Very nice. I used to shorten this for students beginning a course
in statistics and experimental design by saying that a hypothesis
test is done to see if the data are "surprising." That is, given
that the null hypothesis is true, what is the probability that the
data would occur?

Unfortunately, the distinction between the probability of the data
and the probability of the hypothesis is of enormous importance.
But many people get it wrong. Even authors of published papers get
it wrong.

(I thought you were an MD. Maybe I'm wrong? Haha.)
--
Jim Chinnis   Warrenton, Virginia, USA

No, this isn't the point I'm making, and it's a hard discussion to have
via the Web. I'll try to briefly clarify in two different ways.

1) Imagine running 20 beautifully designed randomized trials to see
whether eating bananas decreases the risk of TIAs. In the way we
currently define statistically significant, we would expect one of those
20 studies to "show" that bananas decrease the risk of TIAs purely due
to random chance. Let's say the p-value calculated in that study was
0.045. Given that you had no expectation that bananas would decrease the
risk of TIAs (not to mention 19 other negative studies), you should not
conclude that the probability that bananas decreased the risk of TIAs in
that study was 95.5%. The probability is close to 100% that bananas had
nothing to do with the decrease in TIAs in that study and that you were
seeing a statistical blip. In the other direction, imagine that instead
of bananas you were studying aspirin and got that same p-value of 0.045.
You would be wrong to conclude that based on that study the likelihood
that aspirin reduces the risk of TIAs is only 95.5%. Given all the prior
evidence that aspirin works, the probability that it worked in that
study was nearly 100%, no matter what the p-value was.

2) Think of p-values the way you think of sensitivity. That is,
sensitivity is the thing you are generally able to know about a
diagnostic test (probability that the test will be positive in someone
who has the disease), but positive predictive value is what you really
care about clinically (probability that if someone has a positive test
that they have the disease). In the same way, p-values tell you the
probability in the wrong direction -- they tell you the probability that
if nothing were going on (null hypothesis) that you would have seen the
result found in the trial. They do not tell you what the probability is
given the result found in the trial that something is going on (real
effect).

Again, remember that the way we calculate p-values, we expect a value
below 0.05 in one of twenty studies where nothing is happening. So when
someone studies 20 homeopathic remedies and one of them has finds a
p-value below 0.05, we don't want to conclude that there is better than
a 95% chance that the homeopathic preparation really worked in that
study. On the contrary, we want to recognize that the p-value does not
translate in any direct way into the probability that the homeopathic
remedy really worked.

need to object to the above interpretation of p values. Assuming the
study was done correctly and pravachol really has no effect on TIAs, we

would only have expected to see a result as extreme as the result they
saw about 5.1% of the time. This is not the same as saying that there
is
94.9% chance that pravachol really does decrease TIAs. You can't know
that from the study. It depends, among other things, on what the prior
probability was that pravachol decreased TIAs before the study was
done.
P values deal with probabilities under the null hypothesis (no effect)
and there is no simple way to get from there to the probability you
want
(that the effect seen is real). <<

COMMENT:

Interesting point, and one that needs discussion.  I have to say I
wouldn't go that far. The whole point of a "study done correctly" is
you've designed a study in which the null hypothesis (no effect, the
two groups are the same in risk) and your best alternate hypothesis
(the drug causes risk difference between groups) are far and away the
only "reasonable" hypotheses that exist. Which is the same as saying
that it is "perverse" (though always possible) to entertain additional
more-complicated hypotheses. Thus, with a (mostly) excluded middle, the
probability of rejection of the null is at least close (minus minor
contributions for perverse and complicated possibilities)  to the
simple converse probability that you should accept as "causal," the
single extra perturbing factor which you've carefully additionally
supplied as variable (rather than blame random chance for the
differences you see).
Yes, it's true that "affirming the consequent" is a no-no in deductive
logic, in and of itself. But again, well-designed studies involve extra
premises about how things work causally, and how the universe behaves
mechanically, which allow such conclusions from a sort of  logical
enthymeme. And yes, these mechanistic premises include prior
probability estimates, and do involve calling your shots beforehand, so
that you are not cherry-picking post-hoc statistical deviations. But
there are methods to avoid this. PROSPECT, for example, was designed to
look at vascular and neurological sequellae, including TIAs, in the
older group of people being studied, and the possibility that statins
influenced TIAs as tertiary endpoint was taken into account in the
initial study design, which was separately published in 1999, at the
end of the enrollment, but before the 3 year treatment phase began (Am
J Cardio 84:1192, 1999). So they called their shots here, and aren't
just massaging post hoc data in 2002.
Consider the alternative of skepticism! How far are you going to go?
As we all know from Hume, inductive conclusions about causality always
defy all formal logic, and always completely rely on causal premises
which can, of themselves, always be challenged ad-hoc. Should we then
do so? One can go all the way to rejecting causality completely, but
one still has to drive to work, and what do you do with the gas pedal
and the brake?  Skepticism of this magnitude would leave us not only
not knowing anything for sure (not a problem) but would leave us
without any method of even estimating *confidence* in causal
conclusions of any sort, without full Bayesian analysis. And how often
do we do that? When you say "P values deal with probabilities under the
null hypothesis (no effect) and there is no simple way to get from
there to the probability you want (that the effect seen is real)" I
think you're wrong! There IS a simple way, and that's to design good
controlled blinded randomized prospective studies in which you can be
sure (minus probability of a few unlikely wild alternatives, ie
"reasonably sure") that any statistically significant deviations from
the null *are due to* your own singular and well-controlled meddling.
It's not complicated. People were doing good scientific studies and
inferring correct causal mechanisms long before Bayesian reasoning
became the rage. Radios worked, medical care worked, airplanes flew,
bridges stayed up. It was all done on p values. Sometimes it was even
done without THEM.

If your point is that the probability of statins influencing TIAs in
this large randomized blinded controlled well-powered study isn't
*exactly* the converse of the p value for rejection of the null that
they don't, but is a bit less due to other very improbable scenarios, I
can concede your point, and agree. I am properly chastized and
shouldn't have used the precise 1-p number, but noted it as an
estimate. Otherwise, I think you're being overly contrary. If you were
*really* that epistemologically contrary in facing life, you wouldn't
be able to function at all.  

SBH

I agree with most of what Dr. Harris wrote in his post, but feel the
need to object to the above interpretation of p values. Assuming the
study was done correctly and pravachol really has no effect on TIAs, we
would only have expected to see a result as extreme as the result they
saw about 5.1% of the time. This is not the same as saying that there is
94.9% chance that pravachol really does decrease TIAs. You can't know
that from the study. It depends, among other things, on what the prior
probability was that pravachol decreased TIAs before the study was done.

P values deal with probabilities under the null hypothesis (no effect)
and there is no simple way to get from there to the probability you want
(that the effect seen is real).

cardiovascular disease who experience benefit from statins to nonfatal
stroke, which may lead to improvements in cognition that may help to
balance out harms to cognition from other mechanisms. Although there
are trends toward increases in fatal stroke with statins in most of the

large statin trials, those who have died cannot complete cognitive
surveys.<<

COMMENT:

Sorry, but this is a bogus argument, unless you name the studies. In
the one study you DO discuss (PROSPER), there was NO possiblity that a
difference between recognized nonfatal stroke or stroke influenced the
cognitive outcome, because recognized stroke of any kind brought the
treatment and trial to a halt for the individual patient, so no
cognitive tests were done after that. All cognitive tests reported are
prior to ANY stroke endpoint. They show no difference between treatment
and placebo groups across more than 3 years of on-treatment testing
done every 3 months, including the last set of tests before treatment
end. This is NOT consistant with any negative (or positive) cognitive
effect in this population. There was no difference between groups on
the last on-treatment cognitive test (comparing groups) or the
second-to-baseline test. Both groups decline from baseline to end, but
they decline in cognitive function at the same rate.

statins in most of the large statin trials, those who have died cannot
complete cognitive surveys.<<

COMMENT

NO. It's no use trying to make anything of a trend unless it's a trend
in large numbers of people. The other reason this argument won't run
even if testing had been done after stroke, is that when you're talking
about fatal stroke, you're talking about tiny numbers.  PROSPER (for
example) is a study of 5800 people, and there were 14 (placebo) vs 22
(Pravachol) fatal strokes. Those extra 8 fatal strokes are not
significant (p = .19) and in any case, cognitive testing NOT done on 6
dead people is certainly not going to influence any mean difference in
cognitive testing in 5800  people, even if they WERE doing testing in
non-fatal stroke patients (which they weren't), and the 8 dead stroke
patients all came from this group. Come on. Bad inferential crap like
this is why the net is called the net of a million lies. You can
sometimes get away with this stuff if nobody wants to do the work to
look the studies up, but sometimes you run afoul of people like me, who
will. So stop it.

trial with its sole focus on the elderly population. <<

True enough. It may well be that pravachol and other statins have
significant anti-stroke effect only in other well-selected groups. None
was seen in this trial, though effect on TIAs barely missed
significance (p = 0.051 = 94.9% chance that pravachol really did
decrease TIAs in the trial).

attacks and nonfatal strokes was actually matched by a similar number
of increased fatal strokes.<<

COMMENT:

Baloney-- that's quite wrong. The number of reduced transient ischemic
attacks was 77 (drug) vs 102 (placebo), a difference of 25, which blows
away differences in the stroke numbers (the totals for stroke plus TIA
in this study I note have been mis-done in the table, for they do not
add up to the stated drg/placebo 204/212, difference of 8, but are
actually 212/233, difference of 21). As for total strokes they were 135
(drug) vs 131 (difference of 4 in favor of placebo), which splits up
into non-fatal 116 (drug) v 119 (difference of 3 in favor of drug) and
fatal 22 (drug) vs 14 (difference of 8 in favor of placebo). These
number don't quite add up, either (there's one missing person), but
it's clear that the differences in fatal stroke numbers are too small
to decide that they simply came out of one group and went to the other.
Certainly they did NOT come out of the TIA group, for the difference of
25 there is reduced merely to 21 if you add in the total stroke
numbers. None of the stroke differences are significant, so nothing can
be said about this, either way.

participants considered for enrollment were placed on simvastatin, and
those who were not fully compliant were dropped from the study.
Participants who perceived problems on the drug, including cognitive
problems, may have dropped the study themselves or skipped pills
intentionally. In addition, participants who developed memory problems
may have had trouble remembering to take the pills even if they did not

recognize deterioration in cognitive function. This run-in process may
have excluded participants who developed cognitive problems on the
drug, selecting only those who did not experience problems. Over
one-third of those who were interested in enrolling were excluded
following this compliance run-in.<<

COMMENT:  This is an interesting hypothesis, that all the people would
had cognitive problems with statins were selected out in the first 6
weeks, and went out with the 1252 people given statin who didn't meet
inclusion criteria or who refused to participate. But anybody who
advances this argument for PROSPER had better accept the concomitant
conclusion, which is that if you *don't* have problems with statins in
6 weeks, THEN you won't have any for at least 3 years. Which is what
was then seen in the radomized 5804 people who went on to the next arm
of the trial. You can't just hypothesize parts of explanations you
like, but ignore the obligatory parts of the same hypotheses you don't.

measured in people who have experienced fatal stroke), benefits by
statins for cognitive function in those in whom a stroke was averted
might be expected.<<

COMMENT:
Nonsense, for reasons carefully explained above. The PROSPER trial
measured cognition before stroke, and also the number differences
between non-fatal and fatal stroke are non-significant.  In any case,
some hypothetical raising of mean cognitive scores by killing stroke
victims is far too small to affect scoring of cognitive function in
populations of patients 360 times larger than the number of excess
stroke deaths.

SBH

What happens to those of us who took them for years is quite different
from what happens in clincial trials. Frankie's husband for example
would have been deemed a raging success 6 months out, two years out;
and he is not alone. We hear from hundreds; we get corroboration from
their physicians and the pattern is the same. Over and over and over.
No; anecdote is not as good as fact. But it certainly raises question
in any reasonable person.

statins, stroke and cognition:
http://www.geriatrictimes.com/g040618.html

Second, the large trials enrolled people at high risk for
cardiovascular disease who experience benefit from statins to nonfatal
stroke, which may lead to improvements in cognition that may help to
balance out harms to cognition from other mechanisms. Although there
are trends toward increases in fatal stroke with statins in most of the
large statin trials, those who have died cannot complete cognitive
surveys. The impact on total number of strokes was unaffected in the
PROSPER trial with its sole focus on the elderly population. In the
PROSPER trial, the number of reduced transient ischemic attacks and
nonfatal strokes was actually matched by a similar number of increased
fatal strokes.

Finally, the HPS used what is termed an "active run-in." For six weeks,
participants considered for enrollment were placed on simvastatin, and
those who were not fully compliant were dropped from the study.
Participants who perceived problems on the drug, including cognitive
problems, may have dropped the study themselves or skipped pills
intentionally. In addition, participants who developed memory problems
may have had trouble remembering to take the pills even if they did not
recognize deterioration in cognitive function. This run-in process may
have excluded participants who developed cognitive problems on the
drug, selecting only those who did not experience problems. Over
one-third of those who were interested in enrolling were excluded
following this compliance run-in.

Because statins reduce nonfatal stroke (and cognition is obviously not
measured in people who have experienced fatal stroke), benefits by
statins for cognitive function in those in whom a stroke was averted
might be expected. It must be emphasized that the randomized trial
evidence has, to date, uniformly failed to show cognitive benefits by
statins and has supported no effect or frank and significant harm to
cognitive function.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Frisoni GB, Fratiglioni L, Fastbom J et al. (1999), Mortality in
nondemented subjects with cognitive impairment: the influence of
health-related factors. Am J Epidemiol 150(10):1031-1044.

Golomb BA, Kane T, Dimsdale JA (2004), Severe irritability associated
with statin cholesterol-lowering drugs. QJM 97(4):229-235.

Golomb BA, Yang E, Denenberg J, Criqui M (2003), Statin-associated
adverse events. P95. Presented at the 43rd Annual Conference on
Cardiovascu

King DS, Jones DW, Wofford MR et al. (2001), Cognitive impairment
associated with atorvastatin. Presented at the American College of
Clinical Pharmacy Spring Practice and Research Forum. Salt Lake City;
April 22-25.

King DS, Wilburn AJ, Wofford MR et al. (2003), Cognitive impairment
associated with atorvastatin and simvastatin. Pharmacotherapy
23(12):1663-1667.

Korten AE, Jorm AF, Jiao Z et al. (1999), Health, cognitive, and
psychosocial factors as predictors of mortality in an elderly community
sample. J Epidemiol Community Health 53(2):83-88.

The article makes it clear that there are conflicting studies: some
show
no protection against dementia, others do. Obviously, much more
research
is needed.

Who wouldn't hope that there *might* be some benefit from statins
against
this most debilitating disease, alzheimers? <<

COMMENT:

While we're visiting this subject, let us note that Alzheimer's is
responsible for maybe half of dementia only. A large fraction of the
other half is caused by mini-strokes and vascular disease. And of
course, there's a 20 to 30% overlap of people who have both problems.

Of these, Alzheirmer's is the process I would LEAST expect statins to
interfere with. They might, but they might not.  However, statins have
already show impressive anti-stroke capability, even in people with
normal cholesterol levels. So if statins do not work in slowing or
preventing progression of Alzheimer's, this in no way means we've ruled
out their role in preventing "dementia."

SBH

Another misleading subject line.....

The article makes it clear that there are conflicting studies: some show
no protection against dementia, others do. Obviously, much more research
is needed.

Who wouldn't hope that there *might* be some benefit from statins against
this most debilitating disease, alzheimers?

L.

Many of us who have been exposed first-hand to the devastating cognitive
adverse effects of statins have been tremendously skeptical of the "Can
statins prevent Alz?????" headlines, which appeared at a time that
conveniently offset articles in the popular media that exposed the memory
loss caused by statins.

We doubters also questioned how the studies would differentiate between Alz
and statin-induced memory loss.

As it turns out, this latest study shows that statins do NOT prevent
Alzheimer's:

     Statins Don't Protect Against Dementia: Study
     http://today.reuters.co.uk/news/newsArticle.aspx?type=healthNews&storyID=2005-02
-10T211401Z_01_B371082_RTRIDST_0_HEALTH-STATINS-DEMENTIA-DC.XML
     Reuters.uk, UK - 5 hours ago
     NEW YORK (Reuters Health) - The use of cholesterol-lowering drugs
belonging to the statin family, such as Lipitor or Pravacol, does not seem
to have any effect ...

     Statins Don't Protect Against Dementia: Study
     http://www.reuters.com/newsArticle.jhtml?type=healthNews&storyID=7598600
     Reuters - 5 hours ago
     NEW YORK (Reuters Health) - The use of cholesterol-lowering drugs
belonging to the statin family, such as Lipitor or Pravacol, does not seem
to have any effect ...

     Statins Don't Protect Against Dementia: Study
     http://abcnews.go.com/Health/wireStory?id=488976
     ABC News - 5 hours ago
     Feb 10, 2005 - NEW YORK (Reuters Health) - The use of
cholesterol-lowering drugs belonging to the statin family, such as Lipitor
or Pravacol, does not seem ...

Statins Don't Protect Against Dementia: Study
Thu Feb 10, 2005 9:15 PM GMT

NEW YORK (Reuters Health) - The use of cholesterol-lowering drugs belonging
to the statin family, such as Lipitor or Pravacol, does not seem to have any
effect on the risk of dementia or Alzheimer's disease, according to findings
from a new study.

This supports the results of another study, but run counter other study
findings that have linked statin use with a reduced risk of dementia.

The current study involved elderly residents living in Cache County, Utah,
who were evaluated for statin use and dementia between 1995 and 1997 and
then again between 1998 and 2000.

Dr. John C. S. Breitner, from the VA Puget Sound Health Care System in
Seattle, and colleagues report their findings in the Archives of General
Psychiatry.

Of the 4,895 subjects evaluated at the initial assessment, 355 had dementia,
including 200 with Alzheimer's disease. In this analysis, statin use was
associated with a 56-percent reduction in risk of dementia.

During 3-year follow-up, 185 of 3308 at-risk survivors were diagnosed with
dementia, including 104 with Alzheimer's disease. In this analysis, statin
use at the start of the study or at follow-up had no effect on the risk of
dementia or Alzheimer's disease.

One explanation for the different findings could be that after dementia sets
in, patients may simply be less likely to use statins, along with other
drugs.

Studies with sufficient statistical power are needed to assess the effect of
statin use on dementia risk, the authors note. "Until such research is able
to demonstrate more promising results, however, we suggest that costly
randomized trials of statins are premature."

SOURCE: Archives of General Psychiatry, February 2005.