切换至 "中华医学电子期刊资源库"

中华腔镜外科杂志(电子版) ›› 2024, Vol. 17 ›› Issue (03) : 129 -134. doi: 10.3877/cma.j.issn.1674-6899.2024.03.001

专家论坛

人工智能视觉图像分割在腔镜外科教学中的应用
李梦阳1, 张恩犁1, 吴俊杰1, 赵之明1,()   
  1. 1. 100853 北京,中国人民解放军总医院第一医学中心肝胆胰外科医学部
  • 收稿日期:2024-04-28 出版日期:2024-06-30
  • 通信作者: 赵之明

Application of artificial intelligence visual image segmentation in surgical training

Mengyang Li1, Enli Zhang1, Junjie Wu1, Zhiming Zhao1,()   

  1. 1. Faculty of Hepatopancreatobiliary Surgery, The First Medical Center of Chinese People′s Liberation Army (PLA) General Hospital, Beijing 100853, China
  • Received:2024-04-28 Published:2024-06-30
  • Corresponding author: Zhiming Zhao
引用本文:

李梦阳, 张恩犁, 吴俊杰, 赵之明. 人工智能视觉图像分割在腔镜外科教学中的应用[J]. 中华腔镜外科杂志(电子版), 2024, 17(03): 129-134.

Mengyang Li, Enli Zhang, Junjie Wu, Zhiming Zhao. Application of artificial intelligence visual image segmentation in surgical training[J]. Chinese Journal of Laparoscopic Surgery(Electronic Edition), 2024, 17(03): 129-134.

随着科技手段的发展和外科模式的改变,外科教学的方式也随之进步。人工智能的深度介入将持续影响医疗行业变革。"数据"是人工智能探索的基础,腔镜外科手术产生的海量视频、图像数据高度契合人工智能模型开发的需求。人工智能视觉分割是机器学习进行情景感知的基础,目前已实现了术中自动化手术器械的分割识别、部分术式的手术流程识别和关键手术区域识别。在进一步提升准确性后,手术视频的全自动识别和标注将有助于低年资外科医师高效自主学习手术。这或可改变传统外科的教学模式。

Surgical training has progressed with the development of medical technology. The in-depth intervention of artificial intelligence will continue to influence the medical industry. "Data" is the basis for AI exploration, and the massive video and image data generated by laparoscopic surgery is highly suitable for the development of AI models. Artificial intelligence visual segmentation is the basis of machine learning for situational awareness, and has already achieved the segmentation and identification of automated intraoperative surgical instruments, the identification of surgical processes in some surgical procedures, and the identification of key surgical areas. With further improvement in accuracy, fully automated recognition and annotation of surgical videos will help junior surgeons learn surgery efficiently and autonomously. This may change the traditional teaching mode of surgery.

图1 SurgNet模型自动分割手术器械及术中重要血管注:A.无标注原始图像;B. SurgNet自动化分割和标注血管及手术器械
表1 手术图像分割数据集信息
1
Maier-Hein L, Vedula SS, Speidel S, et al. Surgical data science for next-generation interventions[J]. Nat Biomed Eng, 2017, 1(9):691-696.
2
胡明根,陈况,马奔,等. 微创外科时代肝胆外科医师教学培养模式的探讨[J/CD]. 中华腔镜外科杂志(电子版), 2020, 13(2):65-68.
3
Bailey RW, Imbembo AL, Zucker KA. Establishment of a laparoscopic cholecystectomy training program[J]. Am Surg, 1991, 57(4):231-236.
4
Madion MP, Kastenmeier A, Goldblatt MI, et al. Robotic surgery training curricula: prevalence, perceptions, and educational experiences in general surgery residency programs[J]. Surg Endosc, 2022, 36(9):6638-6646.
5
Bjerrum F, Thomsen ASS, Nayahangan LJ, et al. Surgical simulation: current practices and future perspectives for technical skills training[J]. Med Teach, 2018, 40(7):668-675.
6
Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis[J]. JAMA, 2011, 306(9):978-988.
7
Chen G, Jin S, Xia Q, et al. Insight into the history and trends of surgical simulation training in education: a bibliometric analysis[J]. Int J Surg, 2023, 109(8):2204-2213.
8
Costello DM, Huntington I, Burke G, et al. A review of simulation training and new 3D computer-generated synthetic organs for robotic surgery education[J]. J Robot Surg, 2022, 16(4):749-763.
9
Aydin A, Raison N, Khan MS, et al. Simulation-based training and assessment in urological surgery[J]. Nat Rev Urol, 2016, 13(9):503-519.
10
Chen IA, Ghazi A, Sridhar A, et al. Evolving robotic surgery training and improving patient safety, with the integration of novel technologies[J]. World J Urol, 2021, 39(8):2883-2893.
11
Zheng, S. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. in computer vision and pattern recognition (2021).
12
Shvets A, Rakhlin A, Kalinin AA, et al. Automatic instrument segmentation in robot-assisted surgery using deep learning[C]. 2018.
13
Jin Y, Cheng K, Dou Q, et al. Incorporating temporal prior from motion flow for instrument segmentation in minimally invasive surgery video[C]//International Conference on Medical Image Computing and Computor-Assisted Intervention. Springer, Cham, 2019.
14
Ni ZL, Bian GB, Hou ZG, et al.Attention-guided lightweight network for real-time segmentation of robotic surgical instruments[C].IEEE, 2020.
15
Shen W, Wang Y, Liu M, et al. Branch aggregation attention network for robotic surgical instrument segmentation[J]. IEEE Trans Med Imaging, 2023, 42(11):3408-3419.
16
Madad Zadeh S, Francois T, Calvet L, et al. SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology[J]. Surg Endosc, 2020, 34(12):5377-5383.
17
Kumazu Y, Kobayashi N, Kitamura N, et al. Automated segmentation by deep learning of loose connective tissue fibers to define safe dissection planes in robot-assisted gastrectomy[J]. Sci Rep, 2021, 11(1):21198.
18
Koo B, Robu MR, Allam M, et al. Automatic, global registration in laparoscopic liver surgery[J]. Int J Comput Assist Radiol Surg, 2022, 17(1):167-176.
19
Une N, Kobayashi S, Kitaguchi D, et al. Intraoperative artificial intelligence system identifying liver vessels in laparoscopic liver resection: a retrospective experimental study[J]. Surg Endosc, 2024, 38(2):1088-1095.
20
Kitaguchi D, Takeshita N, Matsuzaki H, et al. Real-time vascular anatomical image navigation for laparoscopic surgery: experimental study[J]. Surg Endosc, 2022, 36(8):6105-6112.
21
Chen J, Li M, Han H, et al. Surgnet: self-supervised pretraining with semantic consistency for vessel and instrument segmentation in surgical images[J]. IEEE Trans Med Imaging, 2024, 43(4):1513-1525.
22
Bodenstedt S, Allan M, Agustinos A, et al.Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery[C]. 2018.
23
Allan, M.Shvets, A.Kurmann, T.Zhang, Z. & Azizian, M. 2017 Robotic Instrument Segmentation Challenge. (2019).
24
Allan M, Kondo S, Bodenstedt S, et al. 2018 Robotic Scene Segmentation Challenge[C]. 2021.
25
Roβ T, Reinke A, Full PM, et al. Comparative validation of multi-instance instrument segmentation in endoscopy: results of the ROBUST-MIS 2019 challenge[J]. Med Image Anal, 2021, 70:101920.
26
Pfeiffer M, Funke I, Robu MR, et al. Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation[C]. 2019.
27
Jha D, Ali S, Emanuelsen K, et al. Kvasir-Instrument: diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy[C]//International Conference on Multimedia Modeling. Springer, Cham, 2021.
28
Maier-Hein L, Mersmann S, Kondermann D, et al. Can masses of non-experts train highly accurate image classifiers? A crowdsourcing approach to instrument segmentation in laparoscopic images[J]. Med Image Comput Comput Assist Interv, 2014, 17(Pt 2):438-445.
29
Bouget D, Benenson R, Omran M, et al. Detecting surgical tools by modelling local appearance and global shape[J]. IEEE Trans Med Imaging, 2015, 34(12):2603-2617.
30
Hong WY, Kao CL, Kuo YH, et al. CholecSeg8k: A semantic segmentation dataset for laparoscopic cholecystectomy based on cholec80[C]. 2020.
31
Grammatikopoulou, M.,et al. CaDIS: Cataract dataset for surgical RGB-image segmentation. 71 (2021).
32
Carstens M, Rinner FM, Bodenstedt S, et al. The dresden surgical anatomy dataset for abdominal organ segmentation in surgical data science[J]. Sci Data, 2023, 10(1):3.
33
Madani A, Namazi B, Altieri MS, et al. Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy[J]. Ann Surg, 2022, 276(2):363-369.
34
Mascagni P, Vardazaryan A, Alapatt D, et al. Artificial intelligence for surgical safety: automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning[J]. Ann Surg, 2022, 275(5):955-961.
35
Liu Y, Zhao S, Zhang G, et al. Multilevel effective surgical workflow recognition in robotic left lateral sectionectomy with deep learning: experimental research[J]. Int J Surg, 2023, 109(10):2941-2952.
36
Kiyasseh D, Ma R, Haque TF, et al. A vision transformer for decoding surgeon activity from surgical videos[J]. Nat Biomed Eng, 2023, 7(6):780-796.
37
Le Roy B, Ozgur E, Koo B, et al. Augmented reality guidance in laparoscopic hepatectomy with deformable semi-automatic computed tomography alignment (with video)[J]. J Visc Surg, 2019, 156(3):261-262.
38
Tukra S, Marcus HJ, Giannarou S. See-through vision with unsupervised scene occlusion reconstruction[J]. IEEE Trans Pattern Anal Mach Intell, 2022, 44(7):3779-3790.
[1] 李洋, 蔡金玉, 党晓智, 常婉英, 巨艳, 高毅, 宋宏萍. 基于深度学习的乳腺超声应变弹性图像生成模型的应用研究[J]. 中华医学超声杂志(电子版), 2024, 21(06): 563-570.
[2] 曹雨欣, 毛卓君, 梁嘉赫, 伊江浦, 张泽凯, 马文帅, 陈云涛, 李晓倩, 张宇新, 曹铁生, 袁丽君. 3D打印心脏模型在模拟左心耳封堵术临床教学中的应用价值[J]. 中华医学超声杂志(电子版), 2024, 21(06): 602-607.
[3] 杨敬武, 周美君, 陈雨凡, 李素淑, 何燕妮, 崔楠, 刘红梅. 人工智能超声结合品管圈活动对低年资超声医师甲状腺结节风险评估能力的作用[J]. 中华医学超声杂志(电子版), 2024, 21(05): 522-526.
[4] 胡森焱, 徐冬, 方健, 谢冬冬, 王财庆. ICG荧光显影Laennec膜入路腹腔镜解剖性肝切除的临床研究[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 513-516.
[5] 高金红, 陈玉梅, 郭韵. 基于King互动达标理论的心理疏导在腹腔镜肝癌切除术患者的应用效果分析[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 517-520.
[6] 林巧, 周丽. RFA联合LAH术治疗原发性肝癌并门静脉癌栓的临床效果分析[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 521-524.
[7] 姜宁宁, 蔺艳丽, 陈惠明. 胆总管结石患者腹腔镜胆总管探查术中应用“三针法”缝合方案的临床研究[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 531-534.
[8] 杨小栎, 李万富, 马柱, 马兰, 郑义, 付晓丽, 王晶. 钳式针一步法在小儿腹腔镜疝囊高位结扎术中的应用效果[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 535-538.
[9] 周丽, 林巧. 腹腔镜TME与开腹手术治疗原发性直肠癌对患者近远期疗效及生存率的影响比较[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 544-546.
[10] 李阳, 董峰, 曾立鹏. 局部进展期直肠癌新辅助治疗后腹腔镜TaTME与TME中的对比研究[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 547-550.
[11] 兰运升, 赵梓竣. 腹腔镜下高位与低位结扎肠系膜下动脉对低位直肠癌患者的影响[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 551-553.
[12] 李猛, 姜腊, 董磊, 吴情, 贾犇黎. 腹腔镜胃袖状切除术治疗肥胖合并2型糖尿病及脂肪胰的临床研究[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 554-557.
[13] 崔宏帅, 冯丽明, 东维玲, 韩博. 腹腔镜右半结肠癌D3根治术+IGLN清扫术治疗局部进展期结肠肝曲癌的临床效果研究[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 566-569.
[14] 丁志翔, 于鹏, 段绍斌. 血浆BRAF基因检测对腹腔镜右半结肠癌D3根治术中行幽门淋巴结清扫的指导价值[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 570-573.
[15] 王维花, 王楠, 乔庆, 罗红. 完全腹腔镜右半结肠癌切除术两种腔内消化道重建方案对比研究[J]. 中华普外科手术学杂志(电子版), 2024, 18(05): 574-577.
阅读次数
全文


摘要