Abstract

The operating microscope plays a central role in middle and inner ear procedures that involve working within tightly confined spaces under limited exposure. Augmented reality (AR) may improve surgical guidance by combining preoperative computed tomography (CT) imaging that can provide precise anatomical information, with intraoperative microscope video feed. With current technology, the operator must manually interact with the AR interface using a computer. The latter poses a disruption in the surgical flow and is suboptimal for maintaining the sterility of the operating environment. The purpose of this study was to implement and evaluate free-hand interaction concepts leveraging hand tracking and gesture recognition as an attempt to reduce the disruption during surgery and improve human-computer interaction.An electromagnetically tracked surgical microscope was calibrated using a custom 3D printed calibration board. This allowed the augmentation of the microscope feed with segmented preoperative CT-derived virtual models. Ultraleap's Leap Motion Controller 2 was coupled to the microscope and used to implement hand-tracking capabilities. End-user feedback was gathered from a surgeon during development. Finally, users were asked to complete tasks that involved interacting with the virtual models, aligning them to physical targets, and adjusting the AR visualization.Following observations and user feedback, we upgraded the functionalities of the hand interaction system. User feedback showed the users' preference for the new interaction concepts that provided minimal disruption of the surgical workflow and more intuitive interaction with the virtual content.We integrated hand interaction concepts, typically used with head-mounted displays (HMDs), into a surgical stereo microscope system intended for AR in otologic microsurgery. The concepts presented in this study demonstrated a more favorable approach to human-computer interaction in a surgical context. They hold potential for a more efficient execution of surgical tasks under microscopic AR guidance.

View details for DOI 10.1007/s11548-024-03224-w

View details for PubMedID 39162975

View details for PubMedCentralID 4710572

Abstract

Analyzing anatomic shapes of tissues and organs is pivotal for accurate disease diagnostics and clinical decision-making. One prominent disease that depends on anatomic shape analysis is osteoarthritis, which affects 30 million Americans. To advance osteoarthritis diagnostics and prognostics, we introduce ShapeMed-Knee, a 3D shape dataset with 9,376 high-resolution, medical-imaging-based 3D shapes of both femur bone and cartilage. Besides data, ShapeMed-Knee includes two benchmarks for assessing reconstruction accuracy and five clinical prediction tasks that assess the utility of learned shape representations. Leveraging ShapeMed-Knee, we develop and evaluate a novel hybrid explicit-implicit neural shape model which achieves up to 40% better reconstruction accuracy than a statistical shape model and implicit neural shape model. Our hybrid models achieve state-of-the-art performance for preserving cartilage biomarkers; they're also the first models to successfully predict localized structural features of osteoarthritis, outperforming shape models and convolutional neural networks applied to raw magnetic resonance images and segmentations. The ShapeMed-Knee dataset provides medical evaluations to reconstruct multiple anatomic surfaces and embed meaningful disease-specific information. ShapeMed-Knee reduces barriers to applying 3D modeling in medicine, and our benchmarks highlight that advancements in 3D modeling can enhance the diagnosis and risk stratification for complex diseases. The dataset, code, and benchmarks will be made freely accessible.

View details for DOI 10.1101/2024.05.06.24306965

View details for PubMedID 38766040

Abstract

Cartilage T2 can detect joints at risk of developing osteoarthritis. The quantitative double-echo steady state (qDESS) sequence is attractive for knee cartilage T2 mapping because of its acquisition time of under 5 minutes. Understanding the reproducibility errors associated with qDESS T2 is essential to profiling the technical performance of this biomarker.To examine the combined acquisition and segmentation reproducibility of knee cartilage qDESS T2 using two different regional analysis schemes: 1) manual segmentation of subregions loaded during common activities and 2) automatic subregional segmentation.Prospective.11 uninjured participants (age: 28 ± 3 years; 8 (73%) female).3-T, qDESS.Test-retest T2 maps were acquired twice on the same day and with a 1-week interval between scans. For each acquisition, average cartilage T2 was calculated in four manually segmented regions encompassing tibiofemoral contact areas during common activities and 12 automatically segmented regions from the deep-learning open-source framework for musculoskeletal MRI analysis (DOSMA) encompassing medial and lateral anterior, central, and posterior tibiofemoral regions. Test-retest T2 values from matching regions were used to evaluate reproducibility.Coefficients of variation (%CV), root-mean-square-average-CV (%RMSA-CV), and intraclass correlation coefficients (ICCs) assessed test-retest T2 reproducibility. The median of test-retest standard deviations was used for T2 precision. Bland-Altman (BA) analyses examined test-retest biases. The smallest detectable difference (SDD) was defined as the BA limit of agreement of largest magnitude. Significance was accepted for P < 0.05.All cartilage regions across both segmentation schemes demonstrated intraday and interday qDESS T2 CVs and RMSA-CVs of ≤5%. T2 ICC values >0.75 were observed in the majority of regions but were more variable in interday tibial comparisons. Test-retest T2 precision was <1.3 msec. The T2 SDD was 3.8 msec.Excellent CV and RMSA-CV reproducibility may suggest that qDESS T2 increases or decreases >5% (3.8 msec) could represent changes to cartilage composition.Stage 2.

View details for DOI 10.1002/jmri.29431

View details for PubMedID 38703134

View details for Web of Science ID 001280544200514

Abstract

Middle and inner ear procedures target hearing loss, infections, and tumors of the temporal bone and lateral skull base. Despite the advances in surgical techniques, these procedures remain challenging due to limited haptic and visual feedback. Augmented reality (AR) may improve operative safety by allowing the 3D visualization of anatomical structures from preoperative computed tomography (CT) scans on real intraoperative microscope video feed. The purpose of this work was to develop a real-time CT-augmented stereo microscope system using camera calibration and electromagnetic (EM) tracking.A 3D printed and electromagnetically tracked calibration board was used to compute the intrinsic and extrinsic parameters of the surgical stereo microscope. These parameters were used to establish a transformation between the EM tracker coordinate system and the stereo microscope image space such that any tracked 3D point can be projected onto the left and right images of the microscope video stream. This allowed the augmentation of the microscope feed of a 3D printed temporal bone with its corresponding CT-derived virtual model. Finally, the calibration board was also used for evaluating the accuracy of the calibration.We evaluated the accuracy of the system by calculating the registration error (RE) in 2D and 3D in a microsurgical laboratory setting. Our calibration workflow achieved a RE of 0.11 ± 0.06 mm in 2D and 0.98 ± 0.13 mm in 3D. In addition, we overlaid a 3D CT model on the microscope feed of a 3D resin printed model of a segmented temporal bone. The system exhibited small latency and good registration accuracy.We present the calibration of an electromagnetically tracked surgical stereo microscope for augmented reality visualization. The calibration method achieved accuracy within a range suitable for otologic procedures. The AR process introduces enhanced visualization of the surgical field while allowing depth perception.

View details for DOI 10.1007/s11548-023-02980-5

View details for PubMedID 37450175

View details for PubMedCentralID 4634572