Authors: Vivek Buch, MD; Brian Park, MD; Kobie Mensah-Brown; Jimmy Germini; Peter Madsen, MD; Francis Quattrone; Christopher Morley; Osamah Choudhry; James Schuster, MD, PhD; H. Chen, MD (Philadelphia, PA)


As surgeons, we operate in a three-dimensional (3D) environment, yet our imaging information is presented in two dimensions. The use of augmented reality (AR) to overlay anatomical models in actual 3D space is an active frontier of cutting-edge innovation. Our team has been working to develop and employ a pipeline for intraoperative AR technology to visualize, interact with, and register holographic projections of patient anatomy during spinal fixation surgery. In this initial stage, we are evaluating clinical feasibility and accuracy of 3D holographic visualization.


Imaging-based 3D volumes can be generated by two approaches: surface rendering in which the model is derived from a semi-automatic segmentation process of tissue interfaces; or volume rendering where the entire imaging volume is rendered into a 3D model. Using intraoperative O-arm imaging, we used Microsoft HoloLens (Redmond, WA) to compare surface (5 patients) and volume (3 patients) rendering techniques to generate 3D holographic models of patient anatomy. We utilized a custom, radiopaque fiducial system to manually register the models onto the patients.


Surface rendering enabled generation of low density, high fidelity 3D models of posterior bony elements from an intraoperative O-arm scan though required significant model generation times (38+/-11mins). Volume rendering generated rapid, high density, perfect fidelity 3D models (7+/-3mins). However, hologram stability was an issue with the high-density data from this latter approach. In one patient for each group, accuracy measurements were taken after manual registration. Surface rendering achieved a 0.7cm Euclidean distance error while volume rendering achieved 1.1-3.1cm error due to hologram instability and headset parameters.


Though still in early development, intraoperative AR presents a unique opportunity to enhance how surgeons interface with operative anatomy. We are iterating improvements in efficiency of holographic surface renderings, stability of volume renderings, and integrating automated registration to enable feasibility and industry-standard accuracy.