Toward augmented reality in laparoscopic liver surgery using electromagnetic tracking: a clinical feasibility study

Abstract Background Augmented reality (AR) has the potential to enhance intraoperative visualization, orientation and precision in laparoscopic liver surgery by combining preoperative imaging data with surgical reality. Accurate registration and motion compensation remain significant challenges for clinical implementation. This study aims to translate electromagnetic (EM)-tracked navigation to the laparoscopic setting, incorporate AR and assess clinical feasibility and accuracy. Methods The open navigation workflow was adapted by implementing EM tracking for a pointer, laparoscopic ultrasound (LUS) probe, and laparoscope. Four AR visualization modes were developed and tested ex vivo on a liver phantom; seven hepatobiliary surgeons rated essential anatomical structures and the preferred display modes. A prospective feasibility study was conducted in patients undergoing laparoscopic liver resection. An EM sensor was attached to the liver surface, landmark-based registration was performed using tracked LUS and pointer, and AR overlays were displayed intraoperatively. Outcomes included technical feasibility of the surgical workflow, registration accuracy and intraoperative AR performance. Results The translation of the EM navigation workflow to the laparoscopic setting yielded robust tracking performance across all laparoscopic instruments. Surgeons rated the tumor, hepatic veins, and portal veins as the most important structures for AR visualization and preferred displays emphasizing a resection plane with target structures and applying depth shading to improve depth perception. The complete workflow (i.e., sensor attachment, registration and AR visualization) was achieved in 13 of 14 procedures without device-related complications. Mean target registration error was 6.3 ± 3.8 mm and registration took an average of 11 ± 7 min. Conclusions AR navigation during laparoscopic liver surgery using EM tracking is feasible and enables organ tracking including local motion compensation. This workflow supports intraoperative orientation and tumor localization in minimally invasive resections potentially improving clinical outcome.