Under Carragee The Spine Journal Lives Dangerously

The Spine Journal’s (TSJ) recent review of 13 rhBMP-2 studies, (June 2011, The Year of Living Dangerously) has pulled the publication into a morass of methodological errors and unapologetic bias. In short, living dangerously. While our list of errors and apparent bias is by no means exhaustive, it is large enough to create a growing sense of deep unease.

With Eugene J. Carragee M.D., Editor in Chief of TSJ and a member of the faculty at Stanford’s University School of Medicine leading the way, the entire June issue of the publication was dedicated to a critical review of a group of early studies of rhBMP-2. After the review, the editors not only repudiated those studies, but linked payments to the authors of those studies from the study’s sponsor, Medtronic, Inc. as the explanation for the alleged flaws. Carragee and his fellow authors then embarked on an aggressive PR campaign to convince regulators, CMS, and the general public that both patients and physicians who used rhBMP-2 had been “living dangerously” over the past couple of decades.

When we reviewed Carragee’s study, we, however, discovered a number of deeply troubling mistakes, omissions, and what appears to us to have been a systematic pattern of intellectual dishonesty.

There are three specific areas which cause us the most concern:

1. Omissions of facts which had the potential to change the conclusions of TSJ’s study.

2. Data used out of context.

3. Errors in logic which, in turn, impugned the integrity of dozens of researchers. This, in our view, created a clear appearance of intellectual dishonesty.

Yale University Review

Yale University has launched an independent and highly detailed review of the entire body of studies regarding rhBMP-2, including the studies which were critically reviewed by TSJ. 

Based on our own limited review, we believe the Yale University study has the potential to (embarrassingly for the editors, authors, and organization supporting TSJ), reverse many, perhaps a majority of the conclusions that were reached by Carragee et al.

How could this happen? We believe it is because, under Carragee’s leadership, TSJ descended into advocacy journalism to promote a particular point of view and engaged in the three areas we mention above and below.

Omission of facts

We have two examples. 

In the Carragee et al.’s study titled, A critical review of recombinant human bone morphogenic protein-2 trials in spine surgery: emerging safety concerns and lessons learned, the authors presented two tables (one on page 473 and the other on page 488) which listed each of the 13 studies which they said were flawed. Specifically with regard to the 2006 Dimar et al. study, TSJ stated that there were no adverse events reported. The exact words used by Carragee et al. were, “none reported.” Below we have copied and pasted a table from the Dimar study that specifically lists complications.

Here is a section of the table on Page 473 of the Carragee study which mentions the Dimar study.

Table 1 – Original industry-sponsored rhBMP-2 clinical studies and reported adverse event rates because of rhBMP-2

Here is a section of the table on Page 488 of the Carragee study which also mentions the Dimar study.

OTW emailed Dr. Carragee and asked him about the Dimar study and the list of complications that were, in fact, in the study. Here is his response:

“We never stated that the authors did not report any complications at all, we stated specifically and explicitly that the authors did not attribute any complication whatsoever to rhBMP-2.”

In our reading of his article he actually DID state that Dimar did not report any complications at all. What does “none reported” mean? Furthermore, if you look at the table from the Dimar study, the heading over the column says “rhBMP-2” and the other column heading says “complication.”

The second example concerns the first study listed in the table—the Boden et al. study (The Spine Journal 11 (2011) 471–491 475 study with an instrumented ICBG arm, a non-instrumented rhBMP-2 arm, and an instrumented rhBMP-2 arm).

In 2002, Drs. Scott Boden, James Kang, Harvinder Sandhu, and John Heller published the results of a study using rhBMP-2 titled, Use of Recombinant Human Bone Morphogenetic Protein-2 to Achieve Posterolateral Lumbar Spine Fusion in Humans. The purpose of the study was to see if the carrier which was used successfully in rhesus monkeys could induce consistent radiographic spine fusion in humans. 

There were three arms to the study:

1. rhBMP-2 with instruments

2. Iliac crest bone graft (ICBG) with instruments

3. rhBMP-2 without instruments

In Carragee’s review of the Boden study, apparently there were only two arms—the rhBMP-2 with instruments and the ICBG with instruments. Carragee’s conclusions after critically reviewing two of Boden’s three arms of the study were that “there was some indication of possible adverse events associated with rhBMP-2” and that “such an effect might have been related to the known pro-inflammatory properties of rhBMP-2.”

Sure enough, one of the patients in the Boden et al. study had leg pain. One patient. 

Could that be, as Carragee suggests, from rhBMP-2? 

That’s why Boden used the third arm of the study. That third arm was rhBMP-2 alone. There are many potential causes of post-operative leg pain (or inflammation for that matter) following spine fusion surgery. 

How does a researcher eliminate variables so that the product being tested, in this case rhBMP-2, is either confirmed or eliminated as the cause of the inflammation?

Of course, use rhBMP-2 alone and see if those patients showed any signs of inflammation. The third arm, in other words.

So what did Carragee say about the third arm? He said nothing. Radio silence.

And did the patients in the third arm of the study, the rhBMP-2 arm, show any signs of inflammation? No they did not. In fact, in the Boden study, four times as many patients received rhBMP as had their bone harvested from the iliac crest. That means four times the chances for inflammation.

Here is a direct quote from page 2667 of the Boden study (which, by the way, was the Volvo Award winning paper in 2002):

“The decreases (improvement) in the leg pain score at the most recent follow-up assessment, as compared with preoperative values, were statistically significant only for the BMP-2 only group (-9.9 +/- 1.9: p=0.001), and the differences among groups were statistically significant (p=0.042). In the autograft/TSRH group, the mean bone graft donor site pain score decreased from 16.0 +/- 0.7 at the time of hospital to 5.2 +/- 2.3 at the most recent follow-up assessment.”

Source: Boden Study

Here we reproduce two tables from the Boden study. The tables clearly show that the rhBMP-2 only patients registered LESS back and leg pain. Where is the rhBMP-2 induced inflammation?

Let us repeat the key point in that last paragraph. Leg pain, (the exact issue that Carragee claimed was likely due to rhBMP-2 triggered inflammation) was LOWER in the patients with rhBMP-2 only.

Had Carragee included the third arm, the rhBMP-2 only arm, in his study, the data would not have supported his conclusions. Amazingly, Carragee omitted this clearly material data from his study. 

The bar charts Carragee used in his study for the Boden study show only two bars. One bar for the ICBG/TSRH patients and one bar for the rhBMP-2/TSRH patients. The third bar is missing.

Why? 

Data Used Out of Context

When a researcher uses data out of context, does that qualify as methodological error? 

One of the most dramatic data points used in the Carragee study was the list of payments made to authors of the 13 studies under review. The total amounts were astonishing. To quote from Carragee’s study, “Authors of nearly all the studies had financial ties with the manufacturer of rhBMP-2, with various compensations ranging to more than 26 million dollars/study.” 

$26 million per study?

We checked the numbers. Turns out that Carragee’s characterization is wrong. In fact, it is entirely false. 

To arrive at $26 million, Carragee and his co-authors looked up ANY payment made for ANY thing at ANY time over the course of 15 years—which of course was about 90% of the time AFTER the studies in question were conducted and published. In short, virtually all of the $26 million was NOT tied to the studies.

We called Dr. Boden and asked about the payments. “I didn’t get paid anything by Medtronic to do the study, ” he told us. But Carragee listed in a table the number of $21 million beside your study. According to Boden, he figured his salary from his institution (not Medtronic) was his pay for conducting the study. We also heard (not from Boden) that he had REFUSED stock-based compensation prior to and after conducting this study. To be fair, when Boden visited Medtronic headquarters as a consultant, he did charge an hourly consultant’s fee. But nothing, nada, zip for conducting the study. He also received payments (many years after that study) for intellectual property licensed to Medtronic NOT related to either the subject of the study or the current InFuse product.

How big a methodological error is it to count data from ANY source applied to the research subject for ANY reason over the course of 15 years as the cause for a particular two-year effect (the rhBMP studies often followed their subjects over two years)?

Instead of U.S. dollars, what if the item being measured was doses of a new drug? What peer review journal would have accepted such clearly specious logic as the basis for ANY conclusions about such a drug? 

WE would have thought that the answer would be “none, ” until we read and double checked the June 2011 issue of TSJ.

Intellectual Dishonesty

In the two studies we reviewed, we found material information that had been omitted from the Carragee study and gives the strong appearance of intellectual dishonesty. Accusing the authors of, in effect, spinning the results of their studies for pay from Medtronic, which Carragee and his co-authors are clearly doing, is a serious charge.

That charge impugns the integrity of each and every individual researcher whose name is associated with those studies. 

But, when individual authors are examined at the time these studies were conducted, a very different picture emerges, as we saw with Boden. 

TSJ and the authors of the June 2011 issue engaged, we believe, in advocacy journalism which created the appearance if not the fact of intellectual dishonesty. The expanding body of evidence includes, but is not limited to, omitting material information which would have changed the study’s conclusions as well as failing to apply standard and appropriate standards of research to the question of financial ties to the supplier of rhBMP-2 and the real issue of bias.

Bottom line, we think that Carragee et al are guilty of pulling The Spine Journal into a morass of methodological errors and unapologetic bias. In short, living dangerously.