Wednesday, March 05, 2008

Puzzling Research Interpretations



You may remember an earlier study which found an elevated breast cancer risk for women who had taken hormone replacement therapy at menopause, compared to a control group who took a placebo. Now a follow-up study suggests that the higher breast cancer risk remains, even after the women had ceased taking estrogen and progestin.

I'm unhappy with some of the interpretations that this study has been given. First there is the fact that many of the findings were not statistically significant was downplayed in the reports of the study. For example:

But the researchers were surprised to find that the overall risk of cancer was 24 percent higher in women who took hormones compared to those who an inert placebo: 281 of those who had taken hormones developed some type of cancer, compared to 218 in the placebo group. That appeared to have been driven by a 27 percent increased risk of breast cancer, although that difference did not meet a test of statistical significance. There were 79 breast cancers in hormone group, compared to 60 in the placebo group.

And for another example of the same problem:

Future papers will provide additional analysis of the cancer trends in the study, the investigators said. During the three years women stopped taking hormones, there was some suggestion that their breast cancer risk began to drop from peak levels, but the overall risk remained about the same. The breast cancer data weren't statistically significant, suggesting chance could play a role, but researchers say the trends are credible because they are consistent with previous research.

Other data on cancer risk also failed to reach statistical significance. For instance, there was a troubling suggestion that lung cancer risk was slightly higher among former hormone users, but that trend could also be due to chance.

It was only after researchers combined all the data from various types of cancers that they were able to show a statistically significant difference between the former hormone users and those who had used placebos during the study.

Why am I troubled by this? Because, strictly speaking, results that fail to reach statistical significance cannot be used to prop up the favored hypothesis. What would the interpretation have been if some statistically non-significant results had gone in the other direction, i.e. showing higher cancer rates in the group that never took hormones?

Oh wait. Some results did go that way:

The researchers found that the annualized event rates for the outcome �all cancer� was higher during the postintervention follow-up for the CEE plus MPA group (1.56 percent per year [n = 281]) than the placebo group (1.26 percent per year [n = 218]). This reflects a greater risk of invasive breast cancer and other cancers in the CEE plus MPA group; the rates of colorectal cancer did not differ significantly between the two groups; rates of endometrial cancer were lower in the CEE plus MPA group. Though risk of breast cancer remained elevated during the follow-up, the risk was less than that experienced towards the end of the trial period.

Or to translate some of that into plainer English: The experimental group had lower colon cancer rates earlier but those lower rates rose in the three years after the treatment was stopped to match the rates in the control group. Also, the rates of endometrial cancer were lower in the group which had taken the hormones. Of course, these differences may well have been statistically non-significant. But so was the breast cancer finding.

I cannot help feeling that some sort of a bandwagon effect is operating here. Now that hormone replacement therapy is viewed as something dangerous all the findings are interpreted within that framework.

Then there is this panic-button pressing bit:

And the millions of women who have taken the hormones should be monitored closely for cancer, especially breast cancer, she said.

"The important message is women really need to make sure they continue getting their mammograms," Stefanick said.

It remains unclear how long the increased risk persists, she said, and researchers have continued following the women to try to answer that crucial question.

"This says, 'You're not quite safe yet, but let's hope you'll be safe soon,'" said Stefanick. It is also unclear whether women who took the hormone combination for shorter periods or took estrogen alone face similar ongoing risks.

Quite safe from what? From ever getting breast cancer? That is a very poor way of framing the issues. Yes, the experimental group had 79 breast cancer cases. But the control group didn't have zero cases, it had 60 cases. Moreover, each of those groups consisted of thousands of women.

And that bit about mammography being important for the women who have taken hormone replacement therapy. Isn't it every bit as important for those who have not?

Someone should write a good explanation about the concept of risks and about all the different types of risks we take (knowingly or unknowingly) every day.