Visualization

Mixed reality is a powerful tool for 3D visualization of medical imaging data.  It allows users to view and interact with anatomical structures in 3D, and also enables collaboration, as multiple people can view the same virtual content simultaneously.  Specific project areas are detailed below.

Augmented Reality Visualization in Microscopic Surgery

Middle and inner ear procedures target hearing loss, infections, and tumors of the temporal bone and lateral skull base. Despite the advances in surgical techniques, these procedures remain challenging due to limited haptic and visual feedback. We developed an augmented reality stereo microscope system that improves operative safety by allowing the 3D visualization of anatomical structures from preoperative computed tomography (CT) scans on real intraoperative microscope video feed. Our calibration workflow achieved a registration error of 0.11 ± 0.06 mm in 2D and 0.98 ± 0.13 mm in 3D. In addition, we overlaid a 3D CT model on the microscope feed of a 3D resin printed model of a segmented temporal bone. The system exhibited small latency and good registration accuracy. The AR process introduces enhanced visualization of the surgical field while allowing depth perception.

 

Thoracic Surgical Planning

Lung nodule localization can be particularly difficult during minimally invasive thoracic surgery. Mixed reality can assist surgeons in planning their approach to localizing and resecting small lung nodules by allowing them to view and interact with medical imaging data effectively and more intuitively in 3D.  We have developed a mixed-reality visualization tool employing the first-generation Microsoft HoloLens that allows the user to (1) view a patient’s original preoperative imaging, (2) manipulate a 3D rendering of that patient’s chest anatomy, and (3) simulate lung deflation and surgical instrument placement. 

Volumetric MRI Viewers

While the MR volumes and images show the inside of the human body, visualization and image evaluation of these images is done separately on flat displays. This separation between the display and the human body can make it difficult for students to learn human anatomy and for medical professionals to simultaneously evaluate interior and exterior features and how they relate to each other. In our research we are developing different virtual rendering methods to visualize medical imaging data on mixed reality displays and test how these technologies compare against conventional 2D visualization methods.  Example applications include the brain and kidneys.

Relevant Publication Highlights

Director

Bruce Daniel
Professor of Radiology (Body Imaging) and, by courtesy, of Bioengineering

Co-Director

Brian A. Hargreaves
Professor of Radiology (Radiological Sciences Laboratory) and, by courtesy, of Electrical Engineering and of Bioengineering

Contact Information

Lucas Center for Imaging
1201 Welch Rd P170
Stanford, CA 94305-5488

Directions to Lucas Ctr