Medical images play a vital role in disease detection, risk-stratification, and treatment planning. Accurate and timely interpretation of medical images can help improve patient care and reduce deaths and adverse outcomes from deadly diseases like cancer. But, interpretation of medical images is often challenging, due to subtle imaging features, lack of specific expertise (often in rural areas), and wide inter-reader variability. Machine learning and image analysis methods have huge potential in improving medical image interpretation, and in supporting clinicians in disease diagnosis and treatment planning.
We develop machine learning systems to improve medical image interpretation. We specifically investigate how to develop these systems by integrating multimodal medical information (e.g., radiology images, pathology images, clinical variables, omics data) in clinically-relevant ways. Moreover, since deriving accurate labels for medical imaging datasets is challenging due to the requirement of specialized clinical expertise and time, we also investigate how to optimally learn from labeled and unlabeled datasets. The end goal of our research is to clinically deploy machine learning-assisted precision medicine systems to help clinicians in routine clinical practice. Therefore, we particularly emphasize on building generalizable and unbiased systems, conducting multi-reader studies to assess how the automated systems perform relative to clinicians, how clinicians perform when assisted by the automated systems, and how prospective evaluations and clinical trials can be designed to enable translation from the laboratory to the clinic.
Radiology-Pathology Fusion For Cancer Detection and Risk-Stratification
How do we leverage information from pathology images in clinically-relevant ways to improve cancer detection and risk-stratification using radiology images?
(1) Radiology-Pathology Registration -- enables deriving accurate and detailed labels from pathology onto radiology images for training machine learning models.
(2) Multimodal Co-Learning -- enables learning radiology biomarkers that capture pathology features of cancer, and thereby help in improving radiology image interpretation?
Selected Publications:
- Weakly Supervised Registration of Prostate MRI and Histopathology Images, MICCAI, 2021.
- Automated detection of aggressive and indolent prostate cancer on magnetic resonance imaging, Medical Physics, 2021.
- Selective identification and localization of indolent and aggressive prostate cancers via CorrSigNIA: an MRI-pathology correlation and deep learning framework, Medical Image Analysis, 2022.
- Bridging the gap between prostate radiology and pathology through machine learning, Medical Physics, 2022.
- Computational Detection of Extraprostatic Extension of Prostate Cancer on Multiparametric MRI Using Deep Learning, Cancers, 2022.
- MIC-CUSP: Multimodal Image Correlations for Ultrasound-Based Prostate Cancer Detection, MICCAI Workshop, 2023.
- ArtHiFy: artificial histopathology-style features for improving MRI-based prostate cancer detection, SPIE Medical Imaging, 2024.
- RAPHIA: A deep learning pipeline for the registration of MRI and whole-mount histopathology images of the prostate, Computers in Biology and Medicine, 2024.
- Aggressiveness Classification of Clear Cell Renal Cell Carcinoma Using Registration-Independent Radiology-Pathology Correlation Learning, Medical Physics, 2024 (in press).
Multimodal Fusion for Improving Radiology Image Interpretation
How do we integrate information from complementary multimodal modalities (multiple radiology modalities, clinical variables, pathology images, clinical domain knowledge) to improve radiology image interpretation?
Selected Publications:
- Swin Transformer-based affine registration of MRI and ultrasound images of the prostate, SPIE Medical Imaging, 2024.
- Integrating zonal priors and pathomic MRI biomarkers for improved aggressive prostate cancer detection on MRI, SPIE Medical Imaging, 2022.
- MIC-CUSP: Multimodal Image Correlations for Ultrasound-Based Prostate Cancer Detection, MICCAI Workshop, 2023.
- Aggressiveness Classification of Clear Cell Renal Cell Carcinoma Using Registration-Independent Radiology-Pathology Correlation Learning, Medical Physics, 2024 (in press).
Improving Model Performance when Training with Limited Labeled Data
How do we improve model accuracy and generalizability when developing machine learning models for medical images with limited labeled data?
Selected Publications: