SEGCRIP: Segmentation-Based Dynamic Cropping of Endoscopic Videos to Address Label Leakage In Surgical tool Detection

In recent times, surgical data science has emerged as an important research discipline in interventional healthcare. There are many potential applications for analysing endoscopic surgical videos using machine learning (ML) techniques such as surgical tool classification, action recognition, and tissue segmentation. However, the efficacy of ML algorithms to learn robust features drastically deteriorates when models are trained on noise-affected endoscopic videos. Appropriate data augmentation for endoscopic videos is thus crucial to ensure robust ML training. To this end, we empirically demonstrate the presence of label leakage when surgical tool classification is performed naively and present textit{SegCrop}, a dynamic U-Net model with an integrated attention mechanism to dynamically crop the arbitrary field of view (FoV) in endoscopic surgical videos to remove spurious label-related information from the training data. Our proposed method eliminates label leakage in surgical videos through dynamic cropping while achieving 99\% accuracy as compared to (manually cropped) ground truth.