Kathryn Simmons Stamey Professor


  • Real-time Detection of Bladder Cancer Using Augmented Cystoscopy with Deep Learning: a Pilot Study. Journal of endourology Chang, T. C., Shkolyar, E., Del Giudice, F., Eminaga, O., Lee, T., Laurie, M., Seufert, C., Jia, X., Mach, K. E., Xing, L., Liao, J. C. 2023


    Detection of bladder tumors under white light cystoscopy (WLC) is challenging yet impactful on treatment outcomes. Artificial intelligence (AI) holds the potential to improve tumor detection; however, its application in the real-time setting remains unexplored. AI has been applied to previously recorded images for post hoc analysis. In this study, we evaluate the feasibility of real-time AI integration during clinic cystoscopy and transurethral resection of bladder tumor (TURBT) on live, streaming video.Patients undergoing clinic flexible cystoscopy and TURBT were prospectively enrolled. A real-time alert device system (real-time CystoNet) was developed and integrated with standard cystoscopy towers. Streaming videos were processed in real time to display alert boxes in sync with live cystoscopy. The per-frame diagnostic accuracy was measured.Real-time CystoNet was successfully integrated in the operating room during TURBT and clinic cystoscopy in 50 consecutive patients. There were 55 procedures that met the inclusion criteria for analysis including 21 clinic cystoscopies and 34 TURBTs. For clinic cystoscopy, real-time CystoNet achieved per-frame tumor specificity of 98.8% with a median error rate of 3.6% (range: 0 - 47%) frames per cystoscopy. For TURBT, the per-frame tumor sensitivity was 52.9% and the per-frame tumor specificity was 95.4% with an error rate of 16.7% for cases with pathologically confirmed bladder cancers.The current pilot study demonstrates the feasibility of using a real-time AI system (real-time CystoNet) during cystoscopy and TURBT to generate active feedback to the surgeon. Further optimization of CystoNet for real-time cystoscopy dynamics may allow for clinically useful AI-augmented cystoscopy.

    View details for DOI 10.1089/end.2023.0056

    View details for PubMedID 37432899

  • A magnetic hydrogel for the efficient retrieval of kidney stone fragments during ureteroscopy. Nature communications Ge, T. J., Roquero, D. M., Holton, G. H., Mach, K. E., Prado, K., Lau, H., Jensen, K., Chang, T. C., Conti, S., Sheth, K., Wang, S. X., Liao, J. C. 2023; 14 (1): 3711


    Only 60-75% of conventional kidney stone surgeries achieve complete stone-free status. Up to 30% of patients with residual fragments <2 mm in size experience subsequent stone-related complications. Here we demonstrate a stone retrieval technology in which fragments are rendered magnetizable with a magnetic hydrogel so that they can be easily retrieved with a simple magnetic tool. The magnetic hydrogel facilitates robust in vitro capture of stone fragments of clinically relevant sizes and compositions. The hydrogel components exhibit no cytotoxicity in cell culture and only superficial effects on ex vivo human urothelium and in vivo mouse bladders. Furthermore, the hydrogel demonstrates antimicrobial activity against common uropathogens on par with that of common antibiotics. By enabling the efficient retrieval of kidney stone fragments, our method can lead to improved stone-free rates and patient outcomes.

    View details for DOI 10.1038/s41467-023-38936-1

    View details for PubMedID 37349287

    View details for PubMedCentralID 5853829

  • Single-cell pathogen diagnostics for combating antibiotic resistance NATURE REVIEWS METHODS PRIMERS Li, H., Hsieh, K., Wong, P., Mach, K. E., Liao, J. C., Wang, T. 2023; 3 (1)
  • Bladder cancer risk stratification using a urinary mRNA biomarker panel - A path towards cystoscopy triaging. Urologic oncology Shkolyar, E., Zhao, Q., Mach, K. E., Teslovich, N. C., Lee, T. J., Cox, S., Skinner, E. C., Lu, Y., Liao, J. C. 2021


    OBJECTIVES: The risk of bladder cancer (BCa) diagnosis and recurrence necessitates cystoscopy. Improved risk stratification may inform personalized triage and surveillance strategies. We aim to develop a urinary mRNA biomarker panel for risk stratification in patients undergoing BCa screening and surveillance.METHODS AND MATERIALS: Urine samples were collected from patients undergoing cystoscopy for BCa screening or surveillance. In patients who underwent transurethral resection of bladder tumor, urine samples were categorized based on tumor histopathology, size, and focality. Subjects with intermediate and high-risk BCa based on American Urological Association (AUA) guideline for non-muscle invasive bladder cancer were classified as "increased-risk"; those with no cancer and AUA low-risk BCa were classified as "low-risk". Urine was evaluated for ROBO1, WNT5A, CDC42BPB, ABL1, CRH, IGF2, ANXA10, and UPK1B expression. A diagnostic model to detect "increased-risk" BCa was created using forward logistic regression analysis of cycle threshold values. Model validation was performed with ten-fold cross-validation. Sensitivity and specificity for detection of "increased-risk" BCa was determined and net benefit analysis performed.RESULTS: Urine samples (n = 257) were collected from 177 patients (95 screening, 76 surveillance, 6 both). There were 65 diagnoses of BCa (12 low, 22 intermediate, 31 high risk). ROBO1, CRH, and IGF2 expression correlated with "increased-risk" disease yielding sensitivity of 92.5% (95% CI, 84.9%-98.1%) and specificity of 73.5% (95% CI, 67.7-79.9%). The overall calculated standardized net benefit of the model was 0.81 (95%CI, 0.71-0.90).CONCLUSIONS: A 3-marker urinary mRNA panel allows for non-invasive identification of "increased-risk" BCa and with further validation may prove to be a tool to reduce the need for cystoscopies in low-risk patients.

    View details for DOI 10.1016/j.urolonc.2021.02.011

    View details for PubMedID 33766467

  • Droplet-Based Single-Cell Measurements of 16S rRNA Enable Integrated Bacteria Identification and Pheno-Molecular Antimicrobial Susceptibility Testing from Clinical Samples in 30 min ADVANCED SCIENCE Kaushik, A. M., Hsieh, K., Mach, K. E., Lewis, S., Puleo, C. M., Carroll, K. C., Liao, J. C., Wang, T. 2021
  • CD47-targeted Near-Infrared Photoimmunotherapy for Human Bladder Cancer. Clinical cancer research : an official journal of the American Association for Cancer Research Kiss, B., van den Berg, N. S., Ertsey, R., McKenna, K., Mach, K. E., Zhang, C. A., Volkmer, J., Weissman, I. L., Rosenthal, E. L., Liao, J. C. 2019


    PURPOSE: Near-infrared photoimmunotherapy (NIR-PIT) is a localized molecular cancer therapy combining a photosensitizer-conjugated monoclonal antibody and light energy. CD47 is an innate immune checkpoint widely expressed on bladder cancer cells but absent from luminal normal urothelium. Targeting CD47 for NIR-PIT has the potential to selectively induce cancer cell death and minimize damage to normal urothelium.EXPERIMENTAL DESIGN: The cytotoxic effect of NIR-PIT with anti-CD47-IR700 was investigated in human bladder cancer cell lines and primary human bladder cancer cells derived from fresh surgical samples. Phagocytosis assays were performed to evaluate macrophage activity after NIR-PIT. Anti-CD47-IR700 was administered to murine xenograft tumor models of human bladder cancer for in vivo molecular imaging and NIR-PIT.RESULTS: Cytotoxicity in cell lines and primary bladder cancer cells significantly increased in a light-dose dependent manner with CD47-targeted NIR-PIT. Phagocytosis of cancer cells significantly increased with NIR-PIT compared to antibody alone (p=0.0002). In vivo fluorescence intensity of anti-CD47-IR700 in tumors reached a peak 24-hour post injection and was detectable for at least 14 days. After a single round of CD47-targeted NIR-PIT, treated animals showed significantly slower tumor growth compared to controls (p<0.0001). Repeated CD47-targeted NIR-PIT treatment further slowed tumor growth (p=0.0104) and improved survival compared to controls.CONCLUSION: CD47-targeted NIR-PIT increased direct cancer cell death and phagocytosis resulting in inhibited tumor growth and improved survival in a murine xenograft model of human bladder cancer.

    View details for PubMedID 30890547

  • Augmented Bladder Tumor Detection Using Deep Learning. European urology Shkolyar, E. n., Jia, X. n., Chang, T. C., Trivedi, D. n., Mach, K. E., Meng, M. Q., Xing, L. n., Liao, J. C. 2019


    Adequate tumor detection is critical in complete transurethral resection of bladder tumor (TURBT) to reduce cancer recurrence, but up to 20% of bladder tumors are missed by standard white light cystoscopy. Deep learning augmented cystoscopy may improve tumor localization, intraoperative navigation, and surgical resection of bladder cancer. We aimed to develop a deep learning algorithm for augmented cystoscopic detection of bladder cancer. Patients undergoing cystoscopy/TURBT were recruited and white light videos were recorded. Video frames containing histologically confirmed papillary urothelial carcinoma were selected and manually annotated. We constructed CystoNet, an image analysis platform based on convolutional neural networks, for automated bladder tumor detection using a development dataset of 95 patients for algorithm training and five patients for testing. Diagnostic performance of CystoNet was validated prospectively in an additional 54 patients. In the validation dataset, per-frame sensitivity and specificity were 90.9% (95% confidence interval [CI], 90.3-91.6%) and 98.6% (95% CI, 98.5-98.8%), respectively. Per-tumor sensitivity was 90.9% (95% CI, 90.3-91.6%). CystoNet detected 39 of 41 papillary and three of three flat bladder cancers. With high sensitivity and specificity, CystoNet may improve the diagnostic yield of cystoscopy and efficacy of TURBT. PATIENT SUMMARY: Conventional cystoscopy has recognized shortcomings in bladder cancer detection, with implications for recurrence. Cystoscopy augmented with artificial intelligence may improve cancer detection and resection.

    View details for DOI 10.1016/j.eururo.2019.08.032

    View details for PubMedID 31537407

  • Detection and surveillance of bladder cancer using urine tumor DNA. Cancer discovery Dudley, J. C., Schroers-Martin, J. n., Lazzareschi, D. V., Shi, W. Y., Chen, S. B., Esfahani, M. S., Trivedi, D. n., Chabon, J. J., Chaudhuri, A. A., Stehr, H. n., Liu, C. L., Lim, H. n., Costa, H. A., Nabet, B. Y., Sin, M. L., Liao, J. C., Alizadeh, A. A., Diehn, M. n. 2018


    Current regimens for the detection and surveillance of bladder cancer (BLCA) are invasive and have suboptimal sensitivity. Here, we present a novel high-throughput sequencing (HTS) method for detection of urine tumor DNA (utDNA) called utDNA CAPP-Seq (uCAPP-Seq) and apply it to 67 healthy adults and 118 patients with early-stage BLCA who either had urine collected prior to treatment or during surveillance. Using this targeted sequencing approach, we detected a median of 6 mutations per BLCA patient and observed surprisingly frequent mutations of the PLEKHS1 promoter (46%), suggesting these mutations represent a useful biomarker for detection of BLCA. We detected utDNA pre-treatment in 93% of cases using a tumor mutation-informed approach and in 84% when blinded to tumor mutation status, with 96-100% specificity. In the surveillance setting, we detected utDNA in 91% of patients who ultimately recurred, with utDNA detection preceding clinical progression in 92% of cases. uCAPP-Seq outperformed a commonly used ancillary test (UroVysion, p=0.02) and cytology and cystoscopy combined (p is less than or equal to 0.006), detecting 100% of BLCA cases detected by cytology and 82% that cytology missed. Our results indicate that uCAPP-Seq is a promising approach for early detection and surveillance of BLCA.

    View details for PubMedID 30578357

  • New and developing diagnostic technologies for urinary tract infections. Nature reviews. Urology Davenport, M., Mach, K. E., Shortliffe, L. M., Banaei, N., Wang, T., Liao, J. C. 2017


    Timely and accurate identification and determination of the antimicrobial susceptibility of uropathogens is central to the management of UTIs. Urine dipsticks are fast and amenable to point-of-care testing, but do not have adequate diagnostic accuracy or provide microbiological diagnosis. Urine culture with antimicrobial susceptibility testing takes 2-3 days and requires a clinical laboratory. The common use of empirical antibiotics has contributed to the rise of multidrug-resistant organisms, reducing treatment options and increasing costs. In addition to improved antimicrobial stewardship and the development of new antimicrobials, novel diagnostics are needed for timely microbial identification and determination of antimicrobial susceptibilities. New diagnostic platforms, including nucleic acid tests and mass spectrometry, have been approved for clinical use and have improved the speed and accuracy of pathogen identification from primary cultures. Optimization for direct urine testing would reduce the time to diagnosis, yet these technologies do not provide comprehensive information on antimicrobial susceptibility. Emerging technologies including biosensors, microfluidics, and other integrated platforms could improve UTI diagnosis via direct pathogen detection from urine samples, rapid antimicrobial susceptibility testing, and point-of-care testing. Successful development and implementation of these technologies has the potential to usher in an era of precision medicine to improve patient care and public health.

    View details for DOI 10.1038/nrurol.2017.20

    View details for PubMedID 28248946

  • Image-Guided Transurethral Resection of Bladder Tumors - Current Practice and Future Outlooks. Bladder cancer (Amsterdam, Netherlands) Chang, T. C., Marcq, G. n., Kiss, B. n., Trivedi, D. R., Mach, K. E., Liao, J. C. 2017; 3 (3): 149–59


    Transurethral resection of bladder tumor (TURBT) under white light cystoscopy (WLC) is the cornerstone for the diagnosis, removal and local staging of non-muscle invasive bladder cancer (NMIBC). Despite technological improvements over the decades, significant shortcomings remain with WLC for tumor detection, thereby impacting the surgical quality and contributing to tumor recurrence and progression. Enhanced cystoscopy modalities such as blue light cystoscopy (BLC) and narrow band imaging (NBI) aid resections by highlighting tumors that might be missed on WLC. Optical biopsy technologies such as confocal laser endomicroscopy (CLE) and optical coherence tomography (OCT) characterize tissue in real-time to ensure a more thorough resection. New resection techniques, particularly en bloc resection, are actively under investigation to improve the overall quality of resections and aid pathologic interpretation. Moreover, new image processing computer algorithms may improve perioperative planning and longitudinal follow-up. Clinical translation of molecular imaging agents is also on the horizon to improve optical diagnosis of bladder cancer. This review focuses on emerging technologies that can impact the quality of TURBT to improve the overall management of NMIBC.

    View details for PubMedID 28824942

    View details for PubMedCentralID PMC5545914

  • Endoscopic molecular imaging of human bladder cancer using a CD47 antibody. Science translational medicine Pan, Y., Volkmer, J., Mach, K. E., Rouse, R. V., Liu, J., Sahoo, D., Chang, T. C., Metzner, T. J., Kang, L., van de Rijn, M., Skinner, E. C., Gambhir, S. S., Weissman, I. L., Liao, J. C. 2014; 6 (260): 260ra148-?


    A combination of optical imaging technologies with cancer-specific molecular imaging agents is a potentially powerful strategy to improve cancer detection and enable image-guided surgery. Bladder cancer is primarily managed endoscopically by white light cystoscopy with suboptimal diagnostic accuracy. Emerging optical imaging technologies hold great potential for improved diagnostic accuracy but lack imaging agents for molecular specificity. Using fluorescently labeled CD47 antibody (anti-CD47) as molecular imaging agent, we demonstrated consistent identification of bladder cancer with clinical grade fluorescence imaging systems, confocal endomicroscopy, and blue light cystoscopy in fresh surgically removed human bladders. With blue light cystoscopy, the sensitivity and specificity for CD47-targeted imaging were 82.9 and 90.5%, respectively. We detected variants of bladder cancers, which are diagnostic challenges, including carcinoma in situ, residual carcinoma in tumor resection bed, recurrent carcinoma following prior intravesical immunotherapy with Bacillus Calmette-Guérin (BCG), and excluded cancer from benign but suspicious-appearing mucosa. CD47-targeted molecular imaging could improve diagnosis and resection thoroughness for bladder cancer.

    View details for DOI 10.1126/scitranslmed.3009457

    View details for PubMedID 25355698

  • Electronic Documentation of Intraoperative Observation of Cystoscopic Procedures Using the cMDX Information System. JCO clinical cancer informatics Eminaga, O., Lee, T. J., La, V., Breil, B., Xing, L., Liao, J. C. 2024; 8: e2300114


    PURPOSE: Accurate documentation of lesions during transurethral resection of bladder tumors (TURBT) is essential for precise diagnosis, treatment planning, and follow-up care. However, optimizing schematic documentation techniques for bladder lesions has received limited attention.MATERIALS AND METHODS: This prospective observational study used a cMDX-based documentation system that facilitates graphical representation, a lesion-specific questionnaire, and heatmap analysis with a posterization effect. We designed a graphical scheme for bladder covering bladder landmarks to visualize anatomic features and to document the lesion location. The lesion-specific questionnaire was integrated for comprehensive lesion characterization. Finally, spatial analyses were applied to investigate the anatomic distribution patterns of bladder lesions.RESULTS: A total of 97 TURBT cases conducted between 2021 and 2023 were included, identifying 176 lesions. The lesions were distributed in different bladder areas with varying frequencies. The distribution pattern, sorted by frequency, was observed in the following areas: posterior, trigone, lateral right and anterior, and lateral left and dome. Suspicious levels were assigned to the lesions, mostly categorized either as indeterminate or moderate. Lesion size analysis revealed that most lesions fell between 5 and 29 mm.CONCLUSION: The study highlights the potential of schematic documentation techniques for informed decision making, quality assessment, primary research, and secondary data utilization of intraoperative data in the context of TURBT. Integrating cMDX and heatmap analysis provides valuable insights into lesion distribution and characteristics.

    View details for DOI 10.1200/CCI.23.00114

    View details for PubMedID 38484216

  • PolypMixNet: Enhancing semi-supervised polyp segmentation with polyp-aware augmentation. Computers in biology and medicine Jia, X., Shen, Y., Yang, J., Song, R., Zhang, W., Meng, M. Q., Liao, J. C., Xing, L. 2024; 170: 108006


    AI-assisted polyp segmentation in colonoscopy plays a crucial role in enabling prompt diagnosis and treatment of colorectal cancer. However, the lack of sufficient annotated data poses a significant challenge for supervised learning approaches. Existing semi-supervised learning methods also suffer from performance degradation, mainly due to task-specific characteristics, such as class imbalance in polyp segmentation.The purpose of this work is to develop an effective semi-supervised learning framework for accurate polyp segmentation in colonoscopy, addressing limited annotated data and class imbalance challenges.We proposed PolypMixNet, a semi-supervised framework, for colorectal polyp segmentation, utilizing novel augmentation techniques and a Mean Teacher architecture to improve model performance. PolypMixNet introduces the polyp-aware mixup (PolypMix) algorithm and incorporates dual-level consistency regularization. PolypMix addresses the class imbalance in colonoscopy datasets and enhances the diversity of training data. By performing a polyp-aware mixup on unlabeled samples, it generates mixed images with polyp context along with their artificial labels. A polyp-directed soft pseudo-labeling (PDSPL) mechanism was proposed to generate high-quality pseudo labels and eliminate the dilution of lesion features caused by mixup operations. To ensure consistency in the training phase, we introduce the PolypMix prediction consistency (PMPC) loss and PolypMix attention consistency (PMAC) loss, enforcing consistency at both image and feature levels. Code is available at was evaluated on four public colonoscopy datasets, achieving 88.97% Dice and 88.85% mIoU on the benchmark dataset of Kvasir-SEG. In scenarios where the labeled training data is limited to 15%, PolypMixNet outperforms the state-of-the-art semi-supervised approaches with a 2.88-point improvement in Dice. It also shows the ability to reach performance comparable to the fully supervised counterpart. Additionally, we conducted extensive ablation studies to validate the effectiveness of each module and highlight the superiority of our proposed approach.PolypMixNet effectively addresses the challenges posed by limited annotated data and unbalanced class distributions in polyp segmentation. By leveraging unlabeled data and incorporating novel augmentation and consistency regularization techniques, our method achieves state-of-the-art performance. We believe that the insights and contributions presented in this work will pave the way for further advancements in semi-supervised polyp segmentation and inspire future research in the medical imaging domain.

    View details for DOI 10.1016/j.compbiomed.2024.108006

    View details for PubMedID 38325216

  • Ensuring Successful Biomarker Studies in Bladder Preservation Clinical Trials for Non-muscle Invasive Bladder Cancer BLADDER CANCER McConkey, D. J., Baumann, B. C., Greenberg, S., DeGraff, D. J., Delacroix, S. E., Efstathiou, J. A., Foster, J., Groshen, S., Kadel, E. E., Khani, F., Kim, W. Y., Lerner, S. P., Levin, T., Liao, J. C., Milowsky, M. I., Meeks, J. J., Miyamoto, D. T., Mouw, K. W., Pietzak, E. J., Solit, D. B., Sundi, D., Tawab-Amiri, A., West, P. J., Wobker, S. E., Wyatt, A. W., Apolo, A. B., Black, P. C. 2024; 10 (1): 1-8

    View details for DOI 10.3233/BLC-230082

    View details for Web of Science ID 001208510500001

  • Revealing hidden patterns in deep neural network feature space continuum via manifold learning. Nature communications Islam, M. T., Zhou, Z., Ren, H., Khuzani, M. B., Kapp, D., Zou, J., Tian, L., Liao, J. C., Xing, L. 2023; 14 (1): 8506


    Deep neural networks (DNNs) extract thousands to millions of task-specific features during model training for inference and decision-making. While visualizing these features is critical for comprehending the learning process and improving the performance of the DNNs, existing visualization techniques work only for classification tasks. For regressions, the feature points lie on a high dimensional continuum having an inherently complex shape, making a meaningful visualization of the features intractable. Given that the majority of deep learning applications are regression-oriented, developing a conceptual framework and computational method to reliably visualize the regression features is of great significance. Here, we introduce a manifold discovery and analysis (MDA) method for DNN feature visualization, which involves learning the manifold topology associated with the output and target labels of a DNN. MDA leverages the acquired topological information to preserve the local geometry of the feature space manifold and provides insightful visualizations of the DNN features, highlighting the appropriateness, generalizability, and adversarial robustness of a DNN. The performance and advantages of the MDA approach compared to the existing methods are demonstrated in different deep learning applications.

    View details for DOI 10.1038/s41467-023-43958-w

    View details for PubMedID 38129376

    View details for PubMedCentralID 8791835

  • Efficient Augmented Intelligence Framework for Bladder Lesion Detection. JCO clinical cancer informatics Eminaga, O., Lee, T. J., Laurie, M., Ge, T. J., La, V., Long, J., Semjonow, A., Bogemann, M., Lau, H., Shkolyar, E., Xing, L., Liao, J. C. 2023; 7: e2300031


    Development of intelligence systems for bladder lesion detection is cost intensive. An efficient strategy to develop such intelligence solutions is needed.We used four deep learning models (ConvNeXt, PlexusNet, MobileNet, and SwinTransformer) covering a variety of model complexity and efficacy. We trained these models on a previously published educational cystoscopy atlas (n = 312 images) to estimate the ratio between normal and cancer scores and externally validated on cystoscopy videos from 68 cases, with region of interest (ROI) pathologically confirmed to be benign and cancerous bladder lesions (ie, ROI). The performance measurement included specificity and sensitivity at frame level, frame sequence (block) level, and ROI level for each case.Specificity was comparable between four models at frame (range, 30.0%-44.8%) and block levels (56%-67%). Although sensitivity at the frame level (range, 81.4%-88.1%) differed between the models, sensitivity at the block level (100%) and ROI level (100%) was comparable between these models. MobileNet and PlexusNet were computationally more efficient for real-time ROI detection than ConvNeXt and SwinTransformer.Educational cystoscopy atlas and efficient models facilitate the development of real-time intelligence system for bladder lesion detection.

    View details for DOI 10.1200/CCI.23.00031

    View details for PubMedID 37774313

  • Tumor detection under cystoscopy with transformer-augmented deep learning algorithm. Physics in medicine and biology Jia, X., Shkolyar, E., Laurie, M. A., Eminaga, O., Liao, J. C., Xing, L. 2023; 68 (16)


    Objective.Accurate tumor detection is critical in cystoscopy to improve bladder cancer resection and decrease recurrence. Advanced deep learning algorithms hold the potential to improve the performance of standard white-light cystoscopy (WLC) in a noninvasive and cost-effective fashion. The purpose of this work is to develop a cost-effective, transformer-augmented deep learning algorithm for accurate detection of bladder tumors in WLC and to assess its performance on archived patient data.Approach.'CystoNet-T', a deep learning-based bladder tumor detector, was developed with a transformer-augmented pyramidal CNN architecture to improve automated tumor detection of WLC. CystoNet-T incorporated the self-attention mechanism by attaching transformer encoder modules to the pyramidal layers of the feature pyramid network (FPN), and obtained multi-scale activation maps with global features aggregation. Features resulting from context augmentation served as the input to a region-based detector to produce tumor detection predictions. The training set was constructed by 510 WLC frames that were obtained from cystoscopy video sequences acquired from 54 patients. The test set was constructed based on 101 images obtained from WLC sequences of 13 patients.Main results.CystoNet-T was evaluated on the test set with 96.4 F1 and 91.4 AP (Average Precision). This result improved the benchmark of Faster R-CNN and YOLO by 7.3 points in F1 and 3.8 points in AP. The improvement is attributed to the strong ability of global attention of CystoNet-T and better feature learning of the pyramids architecture throughout the training. The model was found to be particularly effective in highlighting the foreground information for precise localization of the true positives while favorably avoiding false alarmsSignificance.We have developed a deep learning algorithm that accurately detects bladder tumors in WLC. Transformer-augmented AI framework promises to aid in clinical decision-making for improved bladder cancer diagnosis and therapeutic guidance.

    View details for DOI 10.1088/1361-6560/ace499

    View details for PubMedID 37548023

  • Long noncoding RNA MALAT1 is dynamically regulated in leader cells during collective cancer invasion. Proceedings of the National Academy of Sciences of the United States of America Zhu, N., Ahmed, M., Li, Y., Liao, J. C., Wong, P. K. 2023; 120 (27): e2305410120


    Cancer cells collectively invade using a leader-follower organization, but the regulation of leader cells during this dynamic process is poorly understood. Using a dual double-stranded locked nucleic acid (LNA) nanobiosensor that tracks long noncoding RNA (lncRNA) dynamics in live single cells, we monitored the spatiotemporal distribution of lncRNA during collective cancer invasion. We show that the lncRNA MALAT1 (metastasis-associated lung adenocarcinoma transcript 1) is dynamically regulated in the invading fronts of cancer cells and patient-derived spheroids. MALAT1 transcripts exhibit distinct abundance, diffusivity, and distribution between leader and follower cells. MALAT1 expression increases when a cancer cell becomes a leader and decreases when the collective migration process stops. Transient knockdown of MALAT1 prevents the formation of leader cells and abolishes the invasion of cancer cells. Taken together, our single-cell analysis suggests that MALAT1 is dynamically regulated in leader cells during collective cancer invasion.

    View details for DOI 10.1073/pnas.2305410120

    View details for PubMedID 37364126

  • Laying the Groundwork for Optimized Surgical Feedback. JAMA network open Shkolyar, E., Pugh, C., Liao, J. C. 2023; 6 (6): e2320465

    View details for DOI 10.1001/jamanetworkopen.2023.20465

    View details for PubMedID 37378988

  • Conceptual Framework and Documentation Standards of Cystoscopic Media Content for Artificial Intelligence. Journal of biomedical informatics Eminaga, O., Jiyong Lee, T., Ge, J., Shkolyar, E., Laurie, M., Long, J., Graham Hockman, L., Liao, J. C. 2023: 104369


    The clinical documentation of cystoscopy includes visual and textual materials. However, the secondary use of visual cystoscopic data for educational and research purposes remains limited due to inefficient data management in routine clinical practice.A conceptual framework was designed to document cystoscopy in a standardized manner with three major sections: data management, annotation management, and utilization management. A Swiss-cheese model was proposed for quality control and root cause analyses. We defined the infrastructure required to implement the framework with respect to FAIR (findable, accessible, interoperable, reusable) principles. We applied two scenarios exemplifying data sharing for research and educational projects to ensure compliance with FAIR principles.The framework was successfully implemented while following FAIR principles. The cystoscopy atlas produced from the framework could be presented in an educational web portal; a total of 68 full-length qualitative videos and corresponding annotation data were sharable for artificial intelligence projects covering frame classification and segmentation problems at case, lesion, and frame levels.Our study shows that the proposed framework facilitates the storage of visual documentation in a standardized manner and enables FAIR data for education and artificial intelligence research.

    View details for DOI 10.1016/j.jbi.2023.104369

    View details for PubMedID 37088456