A conventional oncology clinician or a typical patient deciding whether to step out of the box and try a “complementary” treatment got a strong message from multiple major media outlets the past three weeks. A New York Times subheading was representative: “People who used herbs, acupuncture and other complementary treatments tended to die earlier than those who didn’t.”

In short, get back in the box.

The flurry of media accounts followed publication of an observational study from Yale researchers in JAMA Oncology entitled “Complementary Medicine, Refusal of Cancer Therapy, and Survival among Cancer Patients with Curable Cancers.” The authors, led by Skyler Johnson, MD , suggested a causal relationship between use of “complementary medicine” and shortened life span among cancer patients.

The story broke amidst completion of review work on among nearly 70 submissions for a Special Focus Issue on Integrative Oncology for JACM – Paradigm, Practice and Policy Advancing Integrative Health (The Journal of Alternative and Complementary Medicine) for which Lynda Balneaves, PhD, RN is serving as a guest co-editor. Balneaves is the president of the internationally influential Society for Integrative Oncology (SIO).  I took off my hat as JACM editor-in-chief and asked if I might interview her on the JAMA Oncology column. [An overview article on the JAMA Oncology piece is here.]

The timing of the publication was fascinating, from the perspective of the growing respect for using complementary therapies in integrative oncology. The SIO had just last year partnered with the Journal of the National Cancer Institute for a special issue on the emerging field. American Society for Clinical Oncology (ASCO) had just months earlier endorsed SIO’s breast cancer guidelines that advise on use of many of these complementary practices with breast cancer patients. The ASCO Post had chosen to publish a column from SIO’s immediate past president Jun Mao, MD in which he called for reimbursement of many of these complementary therapies. What gives?

The following is extracted from a much longer interview with Balneaves. It is published following her review.

Weeks: You’ve read the paper. General comments?

Balneaves: I will say first that we always encourage people to conduct research in complementary medicine. It’s important to take advantage of databases, to get more population-based perspectives. We do wish there were more databases with standardized assessments and measurement of complementary therapy use and health outcomes for these therapies.

Weeks: The lack of databases that respect the public’s use of complementary and alternative therapies always creates garbage in/garbage out problems for population-based studies.

Balneaves: In addition, in this research there are some significant methodological issues and operational definition issues that are not well addressed. The authors use a 1998 definition of “complementary” as “unproven cancer treatment administered by non-medical personnel.” This is from an article when the field first hit. Today, a good deal of complementary medicine is being offered by conventionally trained and regulated and licensed practitioners – many in mainstream institutions. These are medical personnel. And with the research over the past two decades, many of the complementary practices have moved beyond “unproven.” It’s disappointing that they did not draw on the published definition of “integrative oncology” that was recently published by SIO and would have helped inform this study.

Weeks: One might have hoped that the paper’s review process would have picked up these – I guess they might be considered “oversights”.

Balneaves: Many of the authors’ references are also old. They are clearly not following the field. This is something that unfortunately may happen when there is a lack of collaboration with people experienced in the field. It would have been good had they brought someone with expertise in integrative oncology onto their research team.

Weeks: What are the issues, then.

Balneaves: They confound “alternative” with “complementary.”  It’s a significant misclassification bias. If a person is delaying or refusing care recommended by their conventional practitioners, they are choosing an “alternative.”  And the authors note that when they control for those using complementary therapies who delay or refuse, the difference goes away. Another significant problem is with the title. It would have been better portrayed as outcomes from people who delay or refuse care. And they haven’t looked at any differences between people who don’t use complementary therapies and delay or refuse care, and those who do use it and delay or refuse.

Weeks: The numbers are just plain odd. The article looks very compelling at the outset as the authors say it is based on a cohort of nearly 2-million patients in the National Cancer Database. Then suddenly they are only looking at just 248 people who met their criteria for using what they called “complementary.”

Balneaves: Yes, we know that 40% to 80% of people with cancers try complementary therapies. The authors even acknowledge this. We also know many people never tell their doctors – so the database would not pick these up. But how do we get from those percentages of actual use down to the .01% that they based this off? This is a very small, select sample that probably does not represent the average cancer patient using complementary therapies. It sets up a huge potential for bias. You would have hoped reviewers would have raised questions about this.

Weeks: So a huge set of those classified as not using complementary care actually were – and the authors knew it. Nor do they seem to have sorted for those who have simply chosen to forego chemo or radiation for quality of life reasons – deciding that they’d rather have a little shorter life without chemo than a longer one hairless and with a disturbed sense of taste and who knows what other compromising issues. I would imagine that those who choose complementary medicine may have a view about life that might dispose them more in this direction. It wouldn’t, of course, be the complementary medicine that was shortening their life.

Balneaves:  It’s not entirely clear if the authors included those who may have chosen to not use a conventional treatment for quality of life reasons. They excluded people that were in palliative care. Really, we would benefit from a thorough review from a statistician. [Note that following Balneaves’ suggestion, I procured such a review. Those comments are published here. ADD LINK]

Weeks: Do you think there was anti-complementary medicine prejudice driving this paper?

Balneaves: Everyone comes to research with biases. We all come with certain assumptions. What is important when we do our research is that we try to control them. Do I see biases in how this research was framed? The answer is yes. But this does not mean I want to discredit it in its entirety.

Weeks: I find myself wondering – especially giving the media uptake of this message and the chilling impact that might have on practitioners who may be considering recommending complementary therapies, or patients who are, whether JAMA Oncology should be asked to retract the article.

Balneaves: It’s a good question. Retractions are often focused on methodological issues and errors or on violations of ethics. I believe there is a major misclassification bias. But is that reason enough? Some would argue that’s not sufficient cause for retraction. I did seriously wonder why JAMA Oncology took the paper, there are so many significant problems. What I’d like to see at least is a published dialogue and have the authors respond.

Weeks: Any additional comments?

Balneaves: Honestly, this has disappointed me. It’s too bad that the authors didn’t just connect with researchers knowledgeable in integrative oncology before doing this work.  There are a lot of us out there. We need more of the kind of understanding that a paper like this could provide. But we’ve got to get the questions and methodology right. [See effort to engage Johnson in a dialogue here.}

Weeks: It would be useful if JAMA Oncology realized that academic medicine in North America today is strewn with useful integrative oncology “peers” – like you! – to whom they can turn to get them right. You are kind. I think it is a damn shame JAMA Oncology visited this paper and the resulting media on the general public.

____________________________________________