Advertisement

Ten-Hour Simulation Training Improved the Suturing Performance of Medical Students

Open AccessPublished:January 04, 2022DOI:https://doi.org/10.1016/j.avsg.2021.12.076

      Abstract

      Purpose

      We evaluated the effectiveness of an original simulation training system in improving the suturing performance of medical students using a previously developed web application for scoring suturing performance.

      Methods

      Medical students were recruited for this study and trained on vascular graft anastomosis. Prosthetic grafts were anastomosed and evaluated after orientation, and after 1 hr and 10 hr after training. Vascular surgeons were recruited as controls. Using a previously developed web application, suturing performance was evaluated on the basis of procedural time, coefficient of variation of bite (length of a stitch across the graft), coefficient of variation of pitch (interval between stitches), and skewness (symmetry of the angles between stitches).

      Results

      Forty-eight medical students and 10 vascular surgeons were recruited. After 1 hr of training, only the students’ procedural time improved. After 10 hr of training, all scores improved compared with those in the first trial, and all students’ scores except procedural time were statistically similar to those of the vascular surgeons.

      Conclusions

      Ten-hour training improved all factors, including bite, pitch, skewness, and time. Our simple and inexpensive training system and web application for calculating anastomosis scores can be a useful open educational resource.

      Keywords

      Abbreviations:

      OPRS (Operative Performance Rating System), CV (Coefficient of variation)

      INTRODUCTION

      Surgical skills and techniques have traditionally been passed down from experienced surgeons to beginners, and there are almost no objective methods to evaluate training effectiveness. Procedural time has been used as a gold standard for objective evaluation
      • Nakata BN
      • Cavalini W
      • Bonin EA
      • et al.
      Impact of continuous training through distributed practice for acquisition of minimally invasive surgical skills.
      • Morris M
      • Caskey RC
      • Mitchell M
      • et al.
      Surgical skills training restructured for the 21st century.
      • Admire JR
      • Pounds LL
      • Adams AJ
      • et al.
      Development and construct validity of a low-fidelity training platform for driving large and small suture needles.
      . The Operative Performance Rating System (OPRS), which is not necessarily objective, has been one of the most widely used systems for evaluating surgical performance
      • Davis CR
      • Toll EC
      • Bates AS
      • et al.
      Surgical and procedural skills training at medical school – a national review.
      • Shen Z
      • Yang F
      • Gao P
      • et al.
      A novel clinical-simulated suture education for basic surgical skill: suture on the biological tissue fixed on standardized patient evaluated with objective structured assessment of technical skill (OSATS) tools.
      • Atesok K
      • Satava RM
      • Marsh JL
      • et al.
      Measuring surgical skills in simulation-based training.
      • Hu Y
      • Kim H
      • Mahmutovic A
      • et al.
      Verification of accurate technical insight: a prerequisite for self-directed surgical training.
      .
      Anastomosis of grafts or blood vessels is one of the most basic and familiar procedures for vascular and cardiovascular surgeons; therefore, to assess the effectiveness of our suture training, we focused on the thread arrangement in anastomoses. We had previously determined that bite, pitch, and skewness were associated with suturing symmetry and confirmed that bite, pitch, procedural time, and OPRS were strongly correlated
      • Nitta J
      • Akai T
      • Miyahara K
      • et al.
      Original homebuilt off-the-job training system for vascular surgeons: system analysis and assessment.
      .
      Since 2017, we have recruited medical students for a simulation training course including a 1-hr-long experience with prosthetic vascular graft anastomoses during their clinical rotation in our department. The optimized training methodology included instruction by vascular surgeons and use of established equipment, including prosthetic vascular grafts, sutures, and surgical instruments. In addition, an open access web application (https://ojt-suture-eval-240602.firebaseapp.com) was developed that calculated the performance scores for anastomosis within several seconds, allowing easy evaluation of training outcomes
      • Miyahara K
      • Hoshina K
      • Akai T
      • et al.
      Development of a web application that evaluates suture performance in off-the-job training.
      .
      In this study, we evaluated the ability of medical students (novices) and vascular surgeons (specialists) to perform the anastomosis procedure and verified the effectiveness of our original training system in improving the suturing performance of medical students.

      METHODS

      Study Protocol

      The study protocol was approved by the Institutional Research Ethics Committee of the University of Tokyo Hospital (No. 11567). Participants were recruited from medical students at the University of Tokyo rotating on clinical rotation in the vascular service for 3 weeks. They had no surgical experience and provided informed consent to participate in this study. First, the medical students attended an orientation program wherein they were given instructions on how to anastomose grafts and to handle surgical devices including needle holders, forceps, and tweezers. Then, they anastomosed a large (14–20 mm in diameter) artificial graft, fixed on a cork board with pushpins, using a double-armed 3–0 polypropylene suture in an end-to-end manner. Next, they were trained by the vascular surgeons using our training system, as previously described
      • Nitta J
      • Akai T
      • Miyahara K
      • et al.
      Original homebuilt off-the-job training system for vascular surgeons: system analysis and assessment.
      . In brief, a simple anastomosis was prepared on a flat table, and the degree of difficulty was increased by adding a small (2–8 mm in diameter) tube/graft anastomosis to be sutured using the thinner suture or performing the maneuver at a deep site such as the bottom of a plastic pot of various depths. The rim of the pot limited the movement of the trainees’ arms. Such difficult anastomosis training is expected to develop the skill levels of the students. The participants chose the length of training from 1 to 10 hr. During the training, vascular surgeons with more than 8 years of clinical experience (2 with 9–10 years, 2 with 11–15 years, and 3 with more than 15 years) instructed and directed the students in line with the components of OPRS: correctness of needle angle and orientation throughout the anastomosis, fluency of movements using appropriate force, careful tissue handling, and clear economy of motion. Finally, their techniques after completion of the training were compared with their initial attempt of anastomosis of the artificial grafts on the table (Fig. 1A).
      Fig 1
      Fig. 1A. Study protocol. The evaluation of the anastomoses (first trial, second trial and third trial): imaging and scoring with the web application (left). The outline of the simulation training (right). Figure 1B. Factors for evaluation.
      Vascular surgeons with more than 5 years of clinical experience were recruited as the control group. The experienced surgeons performed the same anastomosis as the students, and their scores were compared.

      Evaluation of Anastomosis

      The anastomosed grafts were cut open (after undoing the last knot) to expose the entire circumference and placed between transparent acrylic plates. 2-dimensional images of the front and back were captured and uploaded to our web application for manual stitch-line tracing and anastomosis evaluation, as previously described (Fig. 1A)
      • Miyahara K
      • Hoshina K
      • Akai T
      • et al.
      Development of a web application that evaluates suture performance in off-the-job training.
      .
      Performance scores included the coefficient of variation (CV) of bite (length of a stitch across the graft), CV of pitch (interval between stitches), and skewness (symmetry of the angle among stitches). The total score was defined as the norm of the vectors of the CV of bite, CV of pitch, and skewness: ({(bite)2 + (pitch)2 + (skewness)2}1/2) (Fig. 1B)
      • Nitta J
      • Akai T
      • Miyahara K
      • et al.
      Original homebuilt off-the-job training system for vascular surgeons: system analysis and assessment.
      • Miyahara K
      • Hoshina K
      • Akai T
      • et al.
      Development of a web application that evaluates suture performance in off-the-job training.
      • Pewsey A.
      The large-sample joint distribution of key circular statistics.
      . The score for time (minutes/stitch) was determined as the total procedural time divided by the number of stitches.

      Statistical Analysis

      JMP® Pro 15.1.0 (SAS Institute Inc., Cary, NC, USA) was used for statistical analyses. Continuous variables were expressed as mean ± standard deviation. Group differences were evaluated using the unpaired t-test for continuous variables. Statistical significance (P) was set at < 0.01.

      RESULTS

      Study Cohort

      Forty-eight medical students [73% male, age 24 (23–30)] were recruited between 2019 and 2021. We selected 3 time points: first trial, after the orientation and before the training (n = 48); second trial, 1 h after the training (n = 40 [73% male, age 24 (23–25)); and third trial, 10 h after the training (n = 16 [63% male, age 24 (23–25)]). In addition, 10 vascular surgeons were recruited as the control group.

      Scores At Each Time Point

      The calculated scores (bite, pitch, skewness, total score, time, and time × total score) are shown in Table I and Figure 2. There was no statistical difference between the bites of the medical students after the orientation and those of the vascular surgeons. After 1 hr of training, only the procedural time improved. After 10 hr of training, all scores improved compared with those in the first trial.
      Table IComparison of the scores for the first, second, and third trials of the medical students and the vascular surgeons’ scores
      First trialP-value (vs. VS)Second trial (1 h
      hours after the simulation training.
      )
      P-value (vs. first trial)P-value (vs. VS)Third trial (10 h
      hours after the simulation training.
      )
      P-value (vs. first trial)P-value (vs. VS)VS
      1. Bite0.26 ± 0.070.030.25 ± 0.070.380.110.19 ± 0.04< 0.010.340.21 ± 0.04
      2. Pitch0.28 ± 0.06< 0.010.26 ± 0.050.09< 0.010.19 ± 0.03< 0.010.540.20 ± 0.03
      3. Skewness−0.18 ± 0.13< 0.01−0.15 ± 0.120.300.01−0.06 ± 0.04< 0.010.80−0.04 ± 0.06
      4. Total score0.46 ± 0.11< 0.010.42 ± 0.090.08< 0.010.29 ± 0.03< 0.010.580.31 ± 0.02
      5. Time1.82 ± 0.50< 0.011.36 ± 0.37< 0.01< 0.010.91 ± 0.32< 0.01< 0.010.43 ± 0.07
      6. Total Score × Time0.82 ± 0.31< 0.010.58 ± 0.24< 0.01< 0.010.25 ± 0.07< 0.010.240.13 ± 0.02
      VS, vascular surgeons.
      low asterisk hours after the simulation training.
      Fig 2
      Fig. 2Comparison of the scores for the first, second, and third trials of the medical students and the vascular surgeons’ scores.
      After 10 hr of training, all scores except the medical students’ procedural times were statistically similar to those of the vascular surgeons. The total score multiplied by the time for all trials was presented as a scatter plot, demonstrating the improvement according to the training time (Fig. 3). Scores of 4 of the medical students were in the 99% confidence ellipse of the vascular surgeons.
      Fig 3
      Fig. 3Scatter plot of scores of medical students (first trial, second trial and third trial) and vascular surgeons. The 99% confidence ellipse of the vascular surgeons is shown. Vertical Axis: Total score. Horizontal Axis: Time.

      Discussion

      The present study showed that our simulation training system improved the suturing skills of medical students based on objective scoring of graft anastomoses. The procedural time was shortened after 1 hr of training. After 10 hr of training, all scores improved, and some were comparable with those of vascular surgeons (Fig. 3). The difference among the trials is presented as a scatter plot (Fig. 4). After 10 hr, most of the students showed improvement in score and time, while after 1 hr, one-fourth of the students did not show improvement in score or time. As we previously reported, there was a strong correlation between the distribution of procedural speed and the quality, and the total evaluation of the skill (OPRS); the improved scores and the shortened time can indicate the advancements in the suturing techniques of the trainees
      • Nitta J
      • Akai T
      • Miyahara K
      • et al.
      Original homebuilt off-the-job training system for vascular surgeons: system analysis and assessment.
      . Those results can help determine the appropriate level of the surgical education in terms of the time required for practical training. Although the suturing performance reflected only one aspect of surgical skills, the objective outcomes derived from our training system would be useful for medical education in any institute, region, or country. In addition, the free application is open to the public and the training system is inexpensive, allowing worldwide use by surgeons.
      Fig 4
      Fig. 4Scatter plot of the difference between the first and second trials and the difference between the first and third trials. Vertical Axis: Total score. Horizontal Axis: Time.
      Since the Japanese Board of Cardiovascular Surgery mandated 30 hr of off-the-job training (i.e., training outside of working hours) for new cardiovascular specialist applicants in 2017, all institutes and cardiovascular and vascular surgeons have started designing their own training systems. When we started to develop a training system, we found the quality of the existing training, including some expensive simulators and commercialized kits, had not been scientifically evaluated. We, thus, successfully developed a simple and inexpensive training system whose prototype was previously introduced with objective evidence
      • Nitta J
      • Akai T
      • Miyahara K
      • et al.
      Original homebuilt off-the-job training system for vascular surgeons: system analysis and assessment.
      . Our training system can fulfill a part or whole of the mandatory 30 hr training, according to the hours the trainees decide to spend. In this training system, we focused on the suture line of graft anastomosis and analyzed the workmanship and symmetry of each thread. Although a more symmetric anastomoses may not be necessarily ideal or ensure less leakage in clinical practice, it can indicate the ability to handle the needle and the skill of a surgeon. There were several practical limitations to the prototype system. First, the segmentation method of the sutured anastomosed line was time-consuming owing to multiple processes: removal of the artifacts, extraction of the stitch lines, labeling, hue conversion, and saturation. Second, 2-dimensional images of the front half-circle suture line of the graft placed between transparent acrylic plates might not reflect the entire suture. Lastly, in addition to bite and pitch, other factors might represent the symmetry of the suture.
      For the current study, we used an improved version of our original web application that used manual tracing of the stitch line and displayed the scores within several seconds. The suture lines were segmented by using OpenCV (https://opencv.org/), and the scores were computed using NumPy (https://numpy.org/) and SciPy (https://www.scipy. org/). In addition, a new factor, skewness, was added to reflect the symmetry in the curved suture line. Finally, to evaluate the entire suture, we cut open the ringed sutured graft and analyzed both the front and back.
      We used this improved system for medical students for 2 reasons: to increase their interest in vascular and cardiovascular surgery and to establish the skill level of novices to evaluate the training results. In Japan, the number of medical students aspiring to be a surgeon has been decreasing for decades. The statistics performed by the Ministry of Health, Labor and Welfare of Japan in 2018 showed a gradual decrease in the total number of surgeons and the percentage of surgeons aged 40 years or younger. Using this training, we hope to stimulate the interest of medical students in surgical procedures by providing them with a simulation of clinical experience. Fortunately, according to interviews after their rotation through our department, the medical students provided a positive response due to their first experience using surgical devices, the ability to perform difficult procedures that have not yet been included in the educational program, and the immediate feedback from the application. These novices with similar skills have little experience using surgical devices, which is different from the resident cohort whose surgical experience varies widely.
      We gradually added the difficult scenarios to the simulation training; however, the degree of added difficulty was trainer-dependent and 10 hr was a very short period to fully learn the techniques. In real open abdominal aortic aneurysm repair, we face challenges such as securing the operational field, changing needles with one hand, and flipping the graft or aortic wall with the needle tip when forceps cannot be used. However, we were able to simulate such situations by suturing at the bottom of the plastic pot. A limitation of this study was that not all the simulation training steps affected the outcomes.
      There were some additional limitations to the present study. The total score was calculated as the norm of the vectors of the CV of bite, CV of pitch, and skewness. The coefficient for each factor should be appropriately optimized. Other experimental system to confirm the quality of anastomosis such as a leak test might be required for setting the appropriate coefficients. In addition, the difference in each student's interest might affect the results because there was a decrease in the number of the trainees with time. Moreover, the quality of education might have differed between trainers and students. However, the first author was in attendance with almost all students in this study and routinely provided instruction with our video (https://youtube/0sKEfOPChLY) and a simple explanation of the procedure. In fact, all medical students were engrossed in the anastomosis procedure, which resulted in the trainers teaching the students several technical tips per procedure.

      CONCLUSION

      We evaluated the effectiveness of our original simulation training system on suturing performance for graft anastomosis among medical students. 1-hr training improved procedural time, and 10-hr training improved all factors, including bite, pitch, skewness, and time. The total scores of some students were comparable to those of vascular surgeons. Our simple and inexpensive training system and web application for calculating anastomosis scores could be useful open educational resource.

      Author contributions

      Study conception: KHa, KHo; Data collection: KHa, ST, KM, MS, MM, TI, TT; Analysis: KHa, KHo; Investigation: KHo; Writing: KHa, KHo; Critical review and revision: all authors; Final approval of the article: all authors; Accountability for all aspects of the work: all authors.

      Declaration of Competing Interest

      None.
      The authors thank the vascular surgeons in our department (Takafumi Akai, Shinya Okata, Takashi Endo, and Ryohei Maeno) for participating in the vascular surgeon group.

      REFERENCES

        • Nakata BN
        • Cavalini W
        • Bonin EA
        • et al.
        Impact of continuous training through distributed practice for acquisition of minimally invasive surgical skills.
        Surg Endosc. 2017; 31: 4051-4057https://doi.org/10.1007/s00464-017-5451-y
        • Morris M
        • Caskey RC
        • Mitchell M
        • et al.
        Surgical skills training restructured for the 21st century.
        J Surg Res. 2012; 177: 33-36https://doi.org/10.1016/j.jss.2012.03.060
        • Admire JR
        • Pounds LL
        • Adams AJ
        • et al.
        Development and construct validity of a low-fidelity training platform for driving large and small suture needles.
        J Surg Educ. 2015; 72: 387-393https://doi.org/10.1016/j.jsurg.2014.10.009
        • Davis CR
        • Toll EC
        • Bates AS
        • et al.
        Surgical and procedural skills training at medical school – a national review.
        Int J Surg. 2014; 12: 877-882https://doi.org/10.1016/j.ijsu.2014.05.069
        • Shen Z
        • Yang F
        • Gao P
        • et al.
        A novel clinical-simulated suture education for basic surgical skill: suture on the biological tissue fixed on standardized patient evaluated with objective structured assessment of technical skill (OSATS) tools.
        J Invest Surg. 2018; 31: 333-339https://doi.org/10.1080/08941939.2017.1319994
        • Atesok K
        • Satava RM
        • Marsh JL
        • et al.
        Measuring surgical skills in simulation-based training.
        J Am Acad Orthop Surg. 2017; 25: 665-672https://doi.org/10.5435/jaaos-d-16-00253
        • Hu Y
        • Kim H
        • Mahmutovic A
        • et al.
        Verification of accurate technical insight: a prerequisite for self-directed surgical training.
        Adv Health Sci Educ Theory Pract. 2015; 20: 181-191https://doi.org/10.1007/s10459-014-9519-3
        • Nitta J
        • Akai T
        • Miyahara K
        • et al.
        Original homebuilt off-the-job training system for vascular surgeons: system analysis and assessment.
        Ann Vasc Dis. 2018; 11: 525-530https://doi.org/10.3400/avd.oa.18-00075
        • Miyahara K
        • Hoshina K
        • Akai T
        • et al.
        Development of a web application that evaluates suture performance in off-the-job training.
        Ann Vasc Dis. 2020; 13: 52-55https://doi.org/10.3400/avd.oa.19-00108
        • Pewsey A.
        The large-sample joint distribution of key circular statistics.
        Metrika. 2004; 60: 25-32https://doi.org/10.1007/s001840300294