A Mixed-Reality System for Breast Surgical Planning
We have developed a mixed-reality system that projects a 3D “hologram” of images from a preoperative breast MRI onto a patient using the Microsoft HoloLens. One quarter of women who undergo breast lumpectomy to treat early-stage breast cancer in the United States undergo a repeat surgery due to concerns that residual tumor was left behind. This has led to a significant increase in women choosing mastectomy operations in the United States. We aim to reduce the number of repeated surgeries by displaying medical images of the tumor directly on the patient while the surgeon is planning the operation, which we believe will improve surgeons’ ability to determine tumor location and extent. We are conducting a pilot study in patients with palpable tumors that tests a surgeon’s ability to accurately identify the tumor during surgical planning via mixed-reality visualization compared to “cognitive fusion” (the standard method of referring to 2D medical images on an external computer monitor) and palpation (the ground truth).
Perkins SL, Lin MA, Srinivasan S, Wheeler AJ, Hargreaves BA, Daniel BL. A mixed-reality system for breast surgical planning. 2017 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Nantes, France, 2017, pp. 269-274, doi: 10.1109/ISMAR-Adjunct.2017.92.
A prototype of the mixed-reality system for breast surgical planning. The surgeon wears a Microsoft HoloLens, which projects a 3D hologram of images from the preoperative MRI onto the patient’s breast via marker-based tracking, and shows the tumor location and shape (green).
Tumor markings made in two example patients (pink = cognitive fusion, blue = HoloLens, black = palpation). a) In this patient, the HoloLens marking is displaced partly due to discrepancies in patient arm position between the MRI and the surgical position. b) In this patient, the palpating surgeon initially marked a lesion in a different region of the breast, which may have been benign. This example shows how important it is for surgeons to have access to additional technologies to identify malignant tumors.
Suba Srinivasan is an alumnus of the BMR group