If just a couple of the billionaires giving away their money these days were to offer funds to orthopedic training programs, there just may be a high flying virtual reality simulator in every school. But alas, we must await the trillionaires club.
Ann Van Heest, M.D., is a hand surgeon at the University Minnesota, and has been the Residency Program Director for 12 years. She states, “The ACGME (Accreditation Council for Graduate Medical Education) announced six areas of core competencies for orthopedists in 2001. While there is no specific core competency for technical skills, they do fall under the competency of ‘patient care.’ We orthopedic surgeons spend about 50% of our time in the OR, but as it has been for so long, when residents are assessed on motor skills the comments are not specific (‘She has great hands’ or ‘He can’t operate.’)”
Traditionally, tests in the surgical specialties are knowledge based…but being able to perform in the OR is a completely different issue.
Not awaiting a commandment from any governing body, Dr. Van Heest and her team took the initiative and added technical skills as an area of resident evaluation. “We established an evaluation scale of 1-5 with 5 being outstanding. While initially we just used the scale on rotation reviews, the faculty wanted to expand that to include a high stakes test. Using data from the American Board of Orthopaedic Surgery on the 25 most common procedures, we selected the top three upper extremity procedures in order to develop a skills test. The test involves three stations, one for carpal tunnel release, one for distal radius plating, and another for trigger finger release. After each station there is a debriefing, which is really where the trainees learn the most. The faculty members go around to each station, have the students open things up again, and then immediately address any issues. This test is given each year starting in the second year of residency, thus making it possible for students to improve over time.”
Dr. Van Heest has a front row seat to the learning process. She notes, “My research has shown that you can predict that someone will fail on the technical skills from how well they did on the knowledge test; however, if someone does well on the knowledge test it does not necessarily mean that they will do well on the technical skills portion. Yes, residents must have a baseline of knowledge, but it’s not enough to only test knowledge—which is what we’ve been doing for years.”
Now, thanks to a visionary company, ToLTech, and the American Academy of Orthopaedic Surgeons (AAOS), some orthopedic residents can train in a high tech, high touch environment. Dr. Van Heest says, “AAOS has partnered with ToLTech, a company that has developed a knee arthroscopy simulator with haptics (meaning that surgeons can have tactile feedback as they operate). At present the company is validating its use in residencies, with our program being one of the test sites. These machines, which are costly, measure residents’ ability to do knee arthroscopy on a simulator. We have a group of residents who perform the surgery on a live patient and also have a control group that does not have a knee simulator. The data is still out, however.”
“But a less expensive option for orthopedists in training is also now available. “General surgery has adopted the requirement that all residents do a simulator program on the basic skills of laparoscopic surgery. These training ‘boxes’ cost a mere $300 per box and thanks to a grant, they are affordable and available for every surgical training program in the country. Residents train on six tasks that correlate to things they do in surgery; this way, they learn the basic skills of laparoscopy before they do it in the OR.”
My colleagues and I have obtained an Innovation Grant from the American Orthopaedic Association (AOA), and have taken the same principles from the general surgery assessment and developed something similar for arthroscopy in orthopedics. The important thing is that it is not joint-specific…the goal is to teach the residents how to use the equipment, with a focus on navigation and object manipulation.
“At this point we have developed the prototype and are beginning to work with residents to get them to a more advanced level on ‘the box.’ In the spring we will do a retest…after the residents have undergone significant training.”
Larry Marsh, M.D., a professor at the University of Iowa, and former Chair of the AAOS Evaluation Committee, was also awarded an AOA Innovation Grant. “For several years I have worked with the AOA and the Council of Orthopaedic Residency Directors (CORD) to examine better ways to assess physician training. For four to five years committees of these organizations have worked on developing assessment tools in the six core competencies, but in the last year we have tried to create an assessment tool for surgical skills, something that falls under the ‘patient care’ core competency. Our efforts are also clearly ones that will be welcomed by residents…they rate obtaining technical skills as one of the most important things that they need to learn.”
And if young orthopedic trainees confer with their colleagues in other specialties, they might find the need to play some catch-up when it comes to manual skills training. Dr. Marsh: “Orthopedics is somewhat behind in both teaching and assessing technical skills, in part due to the complexity and range of procedures. We are literally teaching hundreds of different surgical procedures, in contrast to general surgery, where they ‘contend’ with only a relatively smaller number of procedures. In addition, perhaps orthopedists have not pushed for dedicated time for motor skills training because we are so busy that we have not taken the time to step back and engage in this detailed training. Compare our situation with that of general surgery, which even back in the mid ‘90s had publications assessing the use of simple questionnaires and scales to evaluate motor skills. We are just starting this process now and actually, we have used information from general surgery as a springboard to help develop our own tools.”
The residents at the University of Iowa are already benefitting from the work of Dr. Marsh and other committee members.
“The Assessment Tools Subcommittee of CORD has developed a seven question form that is now loaded into our hospital wide electronic system. Starting with the questions from the general surgery form, we then tailored the questions to orthopedics, essentially creating a tool that is meant to assess the basic skills that are common to hundreds of procedures. Some of these include, ‘Was the resident well prepared? Did he or she know how to prep and drape the patient? How were their hand movements and dissecting abilities?’ etc.
“On each of these measures the residents are rated in comparison to their peers.”
“We think that the questions, which each have a four-point scale, can be used to detect resident outliers who might need additional training. The scale was designed to have no ‘in between’ options, i.e., the raters can choose between two ‘satisfactory’ options (highly skilled or skilled) and two unsatisfactory options (less skilled or beginner). We have just begun using the form and are aiming to have faculty do three evaluations per resident per rotation. The questionnaire was designed to be easy and quick and to not interfere with surgeons’ daily routines. By the June 2011 meeting of the AOA we should have a good idea as to its utility.”
In the push toward simulated environments, says Dr. Marsh, the highest level work is that previously mentioned by Dr. Van Heest. “This technology allows residents to be in a computer environment simulating knee arthroscopy looking at a screen with their hands on the handles. They are viewing the inside of a knee, moving the handles and using the haptics in the system (actually feeling things inside the virtual knee). With this level of sophistication comes an elaborate scoring scale. I don’t, however, think that ten years from now we’ll have a virtual reality simulator in all orthopedic training programs. But the future is in better assessment and feedback regarding manual skills. Just as we now have mandated faculty assessments of resident performance in other competencies, in several years we will have mandated, formal assessment of motor skills.”
With new simulation and assessment tools future orthopedists can drive better patient outcomes the old fashioned way—with better surgical skills.
