Dear Senators Baucus and Grassley:
To me, as the publisher of one of the most widely read publications in orthopedics, the most important issue facing our industry right now is the lack of fair and objective discourse regarding the role of physicians as researchers and consultants.
I’m referring to something foundational, vital and so very fragile—innovation and the future of medicine.
I think you’d agree that without objective information—rigorously tested in the clinical setting, double checked by peers and fairly presented in publications—we will never find the common ground upon which to build better patient outcomes, reduce waste and innovate products, procedures and systems.
Without accurate information, how can physicians built trust with patients; or payers build trust with hospitals; or hospitals build trust with the companies that supply them?
Your October 25, 2012 report titled Staff Report on Medtronic’s Influence on Infuse Clinical Studies set the cause of disseminating accurate information back by repeating and expanding inaccurate information and faulty conclusions that were first put into print by Eugene Carragee, M.D., in a special edition of The Spine Journal (TSJ) in June 2011.
The faulty conclusions were that Infuse researchers were influenced by corporate payments to under report adverse events associated with the use of InFuse.
Almost 14 months ago, we double checked some of Dr. Carragee’s data and found that he had omitted key data points and engaged in a form of data aggregation that is guaranteed to convey inaccurate information and conclusions. We called it a methodological error.
On September 13, 2011 we published the results of our review of Dr. Carragee’s work in Orthopedics This Week. That article as well as a follow-on analysis is included as an attachment to this open letter.
The surprising and distressing mistakes we uncovered by fact-checking (for example: looking up his citations to see if they conformed to his conclusions, calling the researchers and asking them if Dr. Carragee’s claims of payments occurred) have not been corrected or researched by TSJ.
As you might imagine, we were disheartened to read in the committee’s staff report that Dr. Carragee’s flawed study was the basis for a large portion of that work product.
Here, in summary form, are the errors we uncovered in Dr. Carragee’s research.
Omission of Facts
We have two examples. In Carragee et al.’s study titled, A critical review of recombinant human bone morphogenic protein-2 trials in spine surgery the authors presented two tables (page 473, page 488) listing 13 studies. One study listed was the 2006 Dimar et al. study. Carragee et al. stated that there were no adverse events reported—“none reported” was what he wrote. In fact Dimar included a table of adverse events from using rhBMP-2 and to show that he really did, we reproduced the table in our analysis.
When we asked Dr. Carragee about the omission he said: “We never stated that the authors did not reported any complications at all, we stated specifically and explicitly that the authors did not attribute any complication whatsoever to rhBMP-2.”
But as we demonstrated by publishing the table in our analysis and quoting Dr.Carragee directly from his study, Dr. Carragee’s response to us was factually wrong. Dr. Carragee wrote with respect to the Dimar study and complications associated with BMP-2 the following words: “none reported.” What does “none reported” mean? Furthermore, Dimar’s table clearly states that the complications are from “rhBMP-2” (quote from the Dimar study table).
Example Number Two
The second example concerns the Boden et al. study (The Spine Journal 11 (2011) 471–491 475 study with an instrumented ICBG arm, a non-instrumented rhBMP-2 arm, and an instrumented rhBMP-2 arm).
In Dr. Carragee’s critical review of the Boden study, he omitted key data which would have argued against his conclusions.
In 2002, Drs. Scott Boden, James Kang, Harvinder Sandhu and John Heller published the results of a study using rhBMP-2 titled, Use of Recombinant Human Bone Morphogenetic Protein-2 to Achieve Posterolateral Lumbar Spine Fusion in Humans. Its purpose was to see if the carrier for rhBMP-2 could induce consistent radiographic spine fusion in humans.
There were three arms to the 2002 Boden study:
- rhBMP-2 with instruments
- Iliac crest bone graft (ICBG) with instruments
- rhBMP-2 without instruments
Dr. Carragee mentioned the results from only two arms—not the third. This is vitally important since that third arm provided evidence that contradicted Dr. Carragee’s conclusions.
After reviewing the first two arms of Boden’s study, Dr. Carragee said, “there was some indication of possible adverse events associated with rhBMP-2” and that “such an effect might have been related to the known pro-inflammatory properties of rhBMP-2.”
One of the patients in the Boden et al. study had leg pain. One patient.
Was that leg pain, as Dr. Carragee suggests, from rhBMP-2?
There are many potential causes of post-operative leg pain (or inflammation for that matter) following spine fusion surgery. How does a researcher eliminate variables so that the product being tested, in this case rhBMP-2, is either confirmed or eliminated as the cause of the inflammation?
The answer is to test rhBMP-2 alone and see if those patients showed any signs of inflammation.
In other words, the third arm of the Boden study (which Dr. Carragee omitted from his critique).
And did the patients in the third arm of the study, the rhBMP-2 arm, show any signs of inflammation? No they didn’t. In fact, in the Boden study, four times as many patients received rhBMP-2 as had their bone harvested from the iliac crest. That means four times the chances for inflammation.
Here is a direct quote from the Boden study (page 2667): “The decreases (improvement) in the leg pain score at the most recent follow-up assessment, as compared with preoperative values, were statistically significant only for the BMP-2 only group (-9.9 +/- 1.9: p=0.001), and the differences among groups were statistically significant (p=0.042). In the autograft/TSRH group, the mean bone graft donor site pain score decreased from 16.0 +/- 0.7 at the time of hospital to 5.2 +/- 2.3 at the most recent follow-up assessment.”
Let us repeat the key point in that last paragraph. Leg pain, (the exact issue that Dr. Carragee claimed was likely due to rhBMP-2 triggered inflammation) was LOWER in the patients with rhBMP-2 only.
The bar charts Dr. Carragee used in his study for the Boden study show only two bars. One bar for the ICBG/TSRH patients. The second bar is for the rhBMP-2/TSRH patients. The third bar is missing.
Why?
Data Used Out of Context
One of the most dramatic data points used in the Carragee study was the list of payments made to authors of the 13 studies under review. The total amounts were astonishing. To quote from the Carragee study, “Authors of nearly all the studies had financial ties with the manufacturer of rhBMP-2, with various compensations ranging to more than 26 million dollars/study.”
$26 million per study?
When we fact-checked Dr. Carragee’s numbers we found that he was wrong.
To arrive at $26 million, Dr. Carragee and his co-authors looked up ANY payment made for ANY thing at ANY time over the course of 15 years—which of course was years after the studies in question were conducted and published.
We called one of the researchers who’d been highlighted by Dr.Carragee in his critique, Scott Boden, M.D., and asked about the payments. “I didn’t get paid anything by Medtronic to do the study, ” he told us. But Dr. Carragee listed in a table the number $21 million beside Boden’s study. According to Dr. Boden, he did not receive any payment at all much less the $21 million listed in Dr. Carragee’s report prior to or coincident with the study. In fact, Dr. Boden felt that the work he performed was adequately compensated by his salary from Emory University.
Furthermore, when Medtronic offered Dr. Boden stock based compensation to conduct this study, he refused it.
How big a methodological error is it to count data from ANY source applied to the research subject for ANY reason over the course of 15 years as the cause for a particular 2-year effect (the rhBMP studies often followed their subjects over two years)?
Instead of U.S. dollars, what if the item being measured was doses of a new drug? What peer review journal would have accepted such clearly specious logic as the basis for ANY conclusions about such a drug?
Intellectual Dishonesty
We have since checked with many of the authors mentioned by Dr. Carragee in his critique and all of them told us that they had not received any compensation for conducting the studies questioned in the Carragee 2011 critique. Furthermore, when we looked into the timing and purpose of payments, there was literally zero connection to the 13 studies Dr. Carragee critiqued.
There are two words which describe this type of research—intellectual dishonesty.
Driving the Wedge Between Companies and Physician Researchers
What is happening because of Dr. Carragee’s work product and now this staff report is that a massive wedge is being hammered between medical companies and physician researchers/consultants.
Separating the physician researcher from industry is a serious mistake.
We all understand the problem you are hoping to address—namely to wring bias out of the clinical research process. And all serious people in medicine agree.
But using bias to fight bias is wrong. It must be challenged.
Otherwise we lose the ability to engage in a fair and honest discourse about the role of the physician researcher and industry.
The future of U.S. medicine hangs in the balance.
Sincerely yours,
Robin R. Young
