Back in August, we, along with most other media in the U.S., reported on U.S. News & World Report’s 2016-2017 ranking of top orthopedic hospitals in America.
Coming in fourth on that ortho list was Rush University Medical Center in Chicago, Illinois. The 677-bed general medical and surgical center was ranked nationally in nine adult specialties. It was also high-performing in three adult specialties. However, the center received the lowest ranking possible for patient safety.
What?
Rush has consistently scored high on patient safety measure in other rankings. In fact, Rush has received nine consecutive “A” grades for safety from the Leapfrog Group, putting it among fewer than 100 hospitals in the country that have received the highest possible grade each time since Leapfrog began issuing the semi-annual ratings in 2012.
It just didn’t add up. So Rush’s quality team made up of Tom Webb, Bala Hota, M.D., MPH, and Omar Lateef, DO, dug into the U.S. News’ data and methodology.
“When we compared the data that U.S. News used for our hospital to our own internal data, we found big differences, ” says Hota, Rush’s chief research informatics officer.
In an article published in the October 2016 issue of The Joint Commission Journal on Quality and Patient Safety, written by the Rush team and titled, “Consumer Rankings and Health Care: Toward Validation and Transparency, ” they write that they discovered that the U.S. News’s data displayed many more patient safety events like pressure ulcers than the hospital had tracked. Hota found that the data had included patient health issues that were present in patients prior to admission to Rush.
They wondered what else was wrong and dug further.
They found that Rush was not the only hospital with discrepancies in data. False-positive event rates were common among high-transfer and high-volume hospitals. Hota and colleagues concluded that more transparency and validation is needed for consumer-based benchmarking methods. In response to these findings and concerns raised by others, U.S. News has reportedly made changes to its methodology and data sources in 2016.
In an accompanying editorial, “The Quality Measurement Crisis: An Urgent Need for Methodological Standards and Transparency, ” David Shahian, M.D., Elizabeth Mort, M.D., MPH, and Peter Pronovost, M.D., Ph. D, wrote, “Just as health care providers have ethical and moral responsibilities to the public they serve, rating organizations and journalists that grade providers have similar obligations—in their case, to ensure measure validity and methodological transparency.”
Rating Systems too “Complex”
Lateef and Hota, and the quality team believe that right now the rules used by hospital rating systems are too complex for most consumers to navigate. They are urging U.S. News and other hospital rating systems to be more transparent about how they rate hospital quality, so that individual hospitals can validate the accuracy of their scores and work with the rating organization to adjust the scores as appropriate.
“The companies that are producing these ratings have to be more transparent, ” Hota says. “Their methodology should be reproducible and easily understood.”
In the meantime, Lateef says Rush will continue to measure, track, and share patient safety data rigorously, and use that date to drive improvements to make Rush’s care even safer. “We consider quality to be the most important part of patient care. We believe in being transparent about quality so we can continue marching towards improvement.”
