Stanford University’s Dr. Eugene Carragee joined a long, distinguished line of academicians when he published his controversial June 2011 issue of The Spine Journal (TSJ). Under his direction, the Journal dedicated its entire issue to seeking out and correcting perceived flaws in the early studies of rhBMP-2—better known as InFuse.
Carragee and his colleagues also raised the issue of bias in those early studies and directed readers to consider the role of industry sponsorship of those studies as well as payments made directly to the investigators of those studies.
Orthopedics This Week found that Carragee and his colleagues had omitted key information from their study which could arguably have changed the author’s conclusions. (See Under Carragee The Spine Journal Lives Dangerously: Orthopedics This Week; September 13, 2011)
Confirming Stereotypes
Stepping back from the specific problems that have emerged from Carragee’s June TSJ study and considering more broadly the meaning of Carragee’s study, we are reminded of other famous and accomplished academicians who also fell into the intellectual sand trap that now holds Carragee.
We’ll start with the eminent Dr. Samuel George Morton, professor of anatomy at the University of Pennsylvania and author of the then standard anatomy text book. Dr. Morton was interested in conducting a study regarding the sources of intelligence in humans. Dr. Morton’s hypothesis was that the differences he noticed between different human beings were due to a difference in species, not variability within a species. He was particularly interested in the concept of polygenism, which is the theory of multiple racial creations.
Dr. Morton then set as his dependent variable, human intelligence. He set as his independent variable, the dimensions of the human cranium. As a prelude to the big study, Dr. Morton inspected the skulls of three mummies from ancient Egyptian catacombs and concluded that the dimensions of the Caucasian skull were different three thousand years ago from the dimensions of the Negro skull. Since the Bible pegged Noah’s Ark as having occurred one thousand years ago and since Noah was Caucasian, the Negro must therefore be a distinct species from the Caucasian—reasoned Dr. Morton.
Morton’s hypothesis, it is important to note, was consistent with the best scientific thinking in the U.S. in 1839.
Dr. Morton measured thousands of skulls and reported in his classic work Crania Americana that the Caucasians had the biggest brains, averaging 87 cubic inches (1, 426 cc), Indians were in the middle with an average of 82 cubic inches (1, 344 cc) and Negroes had the smallest brains with an average of 78 cubic inches (1, 278 cc). Morton believed that the skulls of each race were different because a wise creator had created each race and positioned them in separate homelands to dwell in from the beginning.
Two separate studies, one conducted in 1988 and the other in 2011, show that Morton’s work was, in so far as his measurements of the craniums were concerned, accurate. Dr. Morton’s study was careful, rigorous and his data was strong. As the professor of anatomy at the University of Pennsylvania—at the time the leading medical school in the United States—Dr. Morton’s study was widely praised. The anthropologist Alex Hrdicka called Morton “the father of American physical anthropology.”
In retrospect, Dr. Morton made an elementary mistake. He failed to isolate his independent variables. Had he controlled for every independent variable except cranium size (like education level, socio-economic factors and standardized intelligence tests) , he would have had a study that might well have discovered the relationship (or lack thereof) between head size and intelligence.
Instead, Dr. Morton’s flawed study gave scientific rigor and academic standing to institutional racism and its logical variants— slavery, oppression and, in our lifetimes, a tragically “separate but equal” educational system.
Harvard’s Stephen Jay Gould Joins the Fraternity
Harvard University’s Stephen Jay Gould was a lifelong civil rights advocate and spoke out frequently against cultural oppression in all its forms. In 1978 Dr. Gould wrote a paper which was later expanded into his monograph: The Mismeasure of Man (1981), where he asserted that Dr. Samuel George Morton had selectively reported data, manipulated sample compositions, made analytical errors, and mismeasured skulls in order to support his prejudicial views on intelligence differences between different populations.
Dr. Gould’s paper confirmed the then prevailing wisdom in 1981 that the United States had been treating different population segments differently based on race and that there had been pseudoscience used in the service of racism and sexism. Dr. Gould’s study was very popular and received fawning and uncritical attention in the mass media.
In 2011, six anthropologists in a paper titled, “The Mismeasure of Science: Stephen Jay Gould versus Samuel George Morton on Skulls and Bias” agreed that there was bias—but that the bias had come from Gould, who failed to examine, let alone remeasure, the crania to determine Morton’s level of accuracy.
Morton’s measurements were, in fact, spot on.
But who is going to challenge Harvard University’s eminent and powerful Stephen Jay Gould when he presents a scientific paper which claims that a study from 1839 had measurement error? Everyone knows Morton was racist and Harvard is confirming it.
Well, someone checked the data. Drs. Lewis, DeGusta, Meyer, Monge, Mann and Holloway remeasured the original Morton craniums and 33 years after Gould’s famous study was published in Science, showed conclusively that it was Gould who was biased.
Columbia University Joins the Fraternity
Columbia University’s Sherry Glied, Ph.D. professor of Health Policy and Management, who is currently on leave to serve as the Assistant Secretary for Planning and Evaluation at the U.S. Department of Health and Human Services (HHS), conducted a study to determine, in part, whether hip surgeon fees contributed to higher health care spending in the U.S. Her co-author for the study was Miriam Laugesen. Ph.D.
Their conclusion? Higher fees paid to orthopedic surgeons for performing hip replacement surgery in the U.S. is a “main driver” of higher health care spending in the U.S.
While picky skeptics may argue that individual orthopedic surgeons would have to be collecting $40 million annually to sway the trillion dollar U.S. healthcare system, there was a more urgent problem with the Columbia University study—they used the wrong data.
Drs. Glied and Laugesen based their study on data from the Medicare Physician Fee Schedule which is the maximum allowable orthopedic surgeon payment—not the actual paid. The figure for the actual amount paid is to be found in the CMS (Centers for Medicare and Medicaid Services) Carrier Standard Analytical Files.
How far off were the Columbia University (one of whom is now a policy maker at HHS) professors? About 67% or $1, 600 versus the actual $960. Turns out, contrary to an increasingly popular stereotype, U.S. hip surgeons are likely paid LESS per procedure than their counterparts in Australia, Germany or the UK. Yes, U.S. hip surgeons are paid less than their colleagues in socialist medicine UK.
But because the Columbia University study was consistent with today’s prevailing stereotype of an overcharging, overpaid, industry sponsored orthopedic surgeon, no one challenged the stereotype.
As we saw with Morton and Gould, when a powerful academic institution lends its standing to confirm pre-existing stereotypes, no one checks. No one challenges. No one dares say that the emperor has no pants, no shirt, and no shame.
Carragee Joins the Fraternity
Much as Morton, Gould and others wrote about subjects which, in their times, were accepted wisdom, so too, we think, did Carragee issued the June 2011 The Spine Journal into a similarly stereotyped environment. Carragee quite purposefully we think, jumped into the middle of the popular narrative that spine surgeons perform too much surgery, seek to curry favor with manufacturers and rely on clinical studies that have been influenced by implant and drug companies.
As we saw with the previous examples, presenting scientifically flawed studies under the imprimatur of institutions like Harvard University, University of Pennsylvania, Columbia University and, now, Stanford University, but which confirm existing stereotypes, practically guarantees minimal critical review.
Take for example the following quote by Carragee from his June 2011 editorial in The Spine Journal. “In the case of rhBMP-2, 13-peer-reviewed articles by industry-sponsored authors did not report a single adverse event associated with rhBMP-2.” (Emphasis added.)
We checked and contrary to Carragee’s assertion, we found specific and numerous examples of cases where the authors of those same studies did report adverse events associated with rhBMP-2 and in one case (Dimar et al. study of 53 patients treated with rhBMP2) published a table that listed numerous adverse events associated with using rhBMP2.
It is hard to miss a table that lists adverse events.
What Happened to The Spine Journal’s Peer Review Process?
How did The Spine Journal’s peer review process miss these errors in fact—but also, we think, errors in intellectual judgment and scientific method.
(Specifically, in our review of Carragee et al. review of the 13 early rhBMP2 studies we found that Carragee, like Morton 170 years ago, failed to isolate his independent variables. Carragee and his fellow authors created a kind of gumbo of unrelated data points and then presented them as if they showed, at a minimum, correlation between the allegedly flawed studies and investigator fees or royalties.)
What was The Spine Journal’s peer review process? Did it comport with the process that the publisher, Elsevier, states is standard operating procedure? Did it comport with the process that peer review journals generally adopt in order to squeeze out bad science and bias?
Elsevier, the publisher of The Spine Journal, says; “Peer review (known as refereeing in some areas) is an essential dividing line for judging what is scientific and what is speculation. The process screens article submissions and requires that authors meet the standards of their discipline and achieve scientific objectivity. This means that science is more than just another opinion.”
As described by Elsevier, the National Institutes of Health, the American Medical Association and numerous other commentators, the typical medical peer review process is supposed to function as follows:
-
The peer review process begins when a journal’s editor sends copies of an author’s work to two or three researchers or scholars who are experts in the field (known as “referees” or “reviewers”) to evaluate.
-
The reviewers read and evaluate the author’s work, return their evaluation of the author’s work to the editor, noting weaknesses or problems along with suggestions for improvement.
-
Eventually, though not immediately, most of the reviewer’s comments are seen by the author. Frequently, reviewers are anonymous and independent in order to avoid cronyism and to encourage unvarnished criticism.
-
The editor, who is usually familiar if not also an expert in the subject matter of the manuscript (although typically not in as much depth as the referees, who are specialists), then evaluates the referees’ comments, her or his own opinion of the manuscript, and the context of the scope of the journal or level of the book and readership, before passing a decision back to the author(s), usually with the referees’ comments.
When it works, the peer review process ensures that authors do not disseminate irrelevant findings, unwarranted claims, unacceptable interpretations, and personal views.
When it does not work (and peer review failures do occur—as for example when a peer-reviewed article contains obvious fundamental errors that undermine at least one of its main conclusions) then the main procedure to deal with those failures is publishing letters to the editor.
Of course, in the case of The Spine Journal, the editor was also the author.
Key questions that need answering are: Did Carragee select his own special editor for the manuscript that he submitted to The Spine Journal? Did Carragee select the peer reviewers who reviewed his manuscript for the June 2011 issue? Did Carragee have any role in the editorial or peer review process that ultimately resulted in June 2011 special edition of The Spine Journal?
Stay tuned.
