Researchers from Johns Hopkins and Harvard recently set out to determine if the Surgical Training and Educational Platform (STEP) developed by the American Society for Surgery of the Hand (ASSH) is useful in differentiating psychomotor skill level among intermediate-level orthopedic trainees.
Their work, “Do Resident Surgical Volumes and Level of Training Correlate with Improved Performance on Psychomotor Skills Tasks: Construct Validity Testing of an ASSH Training Platform (STEP)?” was published in the January-March 2021 edition of The Journal of Bone and Joint Surgery Open Access.
Co-author Dawn LaPorte, M.D., an orthopedic hand surgeon who is vice chairman of education at Johns Hopkins, explained to OTW why this study on the construct validity of STEP was necessary. “Since the implementation of national work hour restrictions in residency training, there has been an increased focus on surgical simulation—resident and fellow education outside of the operating room—to focus on improving skills and increasing trainees repetitions without compromising patient safety. The ASSH Simulation Task Force developed STEP in an effort to create a comprehensive psychomotor training and assessment tool for the fundamental skills of hand surgery.”
“The initial study of STEP—published in the Journal of Hand Surgery—demonstrated that board-certified hand surgeons significantly outperform novice trainees (interns) in all simulation tasks. The impetus for the current study was to better assess construct validity of STEP by testing its ability to differentiate psychomotor skill level between junior and senior residents and to evaluate for correlation between resident surgical case volume and their performance on the tasks.”
Taking 30 residents from two orthopedic training programs, the researchers looked at 8 different tasks:
- lag screw fixation
- depth of plunge during bicortical drilling
- flexor tendon repair
- phalangeal fracture pinning
- central axis scaphoid fixation
- full-thickness skin graft harvest
- microsurgery, and
- wrist arthroscopy.
The 15 junior (PGY 2-3) residents who participated had completed 0-1 hand surgery rotations; the 15 senior residents (PGY 4-5) had completed 2 to 3 rotations.
STEP Good at Distinguishing Between Resident Levels
“The most important results,” stated Dr. LaPorte to OTW, “were that we found total STEP score correlated with total task-related case volumes as well as months in training and number of hand rotations. These findings support that STEP can differentiate between the proficiency of junior and senior residents; support construct validity of STEP in assessing progression of surgical skill through residency.”
“An important next step would be to study correlation between STEP scoring and performance in the operating room—transfer validity. (We/the ASSH Simulation Task Force look forward to investigating that as well).”
“The impetus for this study was to further establish the validity of this low-cost, low fidelity training platform that can be widely disseminated to all training programs in America. It is only through broad acceptance of the same training platform that we can have objective data to assess our trainees’ psychomotor skills. There is a third study that is currently under review for publication from my group that details that increased practice on this training platform does correlate to improved scoring on STEP. The ASSH simulation task force led by Dr. Ranjan Gupta is working to collect even more data to establish a more robust scoring system and to establish transfer validity of the STEP platform.”
