Skip to content

Machine Learning for Medical Image Analysis

Multimodal machine learning for medical image analysis

Medical images play a vital role in disease detection, risk-stratification, and treatment planning. Accurate and timely interpretation of medical images can help improve patient care and reduce deaths and adverse outcomes from deadly diseases like cancer. But, interpretation of medical images is often challenging, due to subtle imaging features, lack of specific expertise (often in rural areas), and wide inter-reader variability. Machine learning and image analysis methods have huge potential in improving medical image interpretation, and in supporting clinicians in disease diagnosis and treatment planning.

We develop machine learning systems to improve medical image interpretation. We specifically investigate how to develop these systems by integrating multimodal medical information (e.g., radiology images, pathology images, clinical variables, omics data) in clinically-relevant ways. Moreover, since deriving accurate labels for medical imaging datasets is challenging due to the requirement of specialized clinical expertise and time, we also investigate how to optimally learn from labeled and unlabeled datasets. The end goal of our research is to clinically deploy machine learning-assisted precision medicine systems to help clinicians in routine clinical practice. Therefore, we particularly emphasize on building generalizable and unbiased systems, conducting multi-reader studies to assess how the automated systems perform relative to clinicians, how clinicians perform when assisted by the automated systems, and how prospective evaluations and clinical trials can be designed to enable translation from the laboratory to the clinic.

Radiology-Pathology Fusion For Cancer Detection and Risk-Stratification

How do we leverage information from pathology images in clinically-relevant ways to improve cancer detection and risk-stratification using radiology images?

(1) Radiology-Pathology Registration -- enables deriving accurate and detailed labels from pathology onto radiology images for training machine learning models.

(2) Multimodal Co-Learning -- enables learning radiology biomarkers that capture pathology features of cancer, and thereby help in improving radiology image interpretation?

Selected Publications:

Multimodal Fusion for Improving Radiology Image Interpretation

How do we integrate information from complementary multimodal modalities (multiple radiology modalities, clinical variables, pathology images, clinical domain knowledge) to improve radiology image interpretation?

Selected Publications:

Improving Model Performance when Training with Limited Labeled Data

How do we improve model accuracy and generalizability when developing machine learning models for medical images with limited labeled data?

Selected Publications: