Our laboratory focuses on developing translational image analysis and machine learning methods for healthcare applications. We specifically investigate how to seamlessly integrate and learn from complementary multimodal imaging and non-imaging data for developing these systems.
The two focus areas of our research include: (a) multimodal machine learning for medical image analysis to assist clinicians in early disease detection, characterization and treatment planning, and (b) multimodal machine learning for human behavior analysis to assess and improve behavioral health, patient wellbeing, and clinician-patient interactions.
Multimodal Machine Learning for Medical Image Analysis
Medical images play a vital role in disease detection, risk-stratification, and treatment planning. Accurate and timely interpretation of medical images can help improve patient care and reduce deaths and adverse outcomes from deadly diseases like cancer. But, interpretation of medical images is often challenging, due to subtle imaging features, lack of specific expertise (often in rural areas), and wide inter-reader variability. Machine learning and image analysis methods have huge potential in improving medical image interpretation, and in supporting clinicians in disease diagnosis and treatment planning.
We develop machine learning systems to improve medical image interpretation. We specifically investigate how to develop these systems by integrating multimodal medical information (e.g., radiology images, pathology images, clinical variables, omics data) in clinically-relevant ways. Moreover, since deriving accurate labels for medical imaging datasets is challenging due to the requirement of specialized clinical expertise and time, we also investigate how to optimally learn from labeled and unlabeled datasets. The end goal of our research is to clinically deploy machine learning-assisted precision medicine systems to help clinicians in routine clinical practice. Therefore, we particularly emphasize on building generalizable and unbiased systems, conducting multi-reader studies to assess how the automated systems perform relative to clinicians, how clinicians perform when assisted by the automated systems, and how prospective evaluations and clinical trials can be designed to enable translation from the laboratory to the clinic.
Learn more here.
Multimodal Machine Learning for Human Behavioral Analysis
Behavioral health disorders affect human wellbeing as much as physical health disorders, but are often overlooked until it is too late. How can we automatically estimate symptoms of anxiety and depression and initiate appropriate, timely interventions? How can we develop these behavior estimation systems such that they are privacy-preserving and unobtrusive? Moreover, interactions between clinicians and patients are an important aspect in improving overall patient health and well-being. For example, situational/physical illness-induced anxiety and depression in patients can be alleviated to some extent through trust and rapport in clinician-patient interactions. But, what constitutes a trusting clinician-doctor relationship? Are there behavioral patterns that are correlated with perceived trust and rapport, and thereby with positive or negative treatment outcomes? A third application area for behavior estimation is privacy-critical spaces like hospitals and elderly living spaces. How can we automatically estimate activities of daily living and alert health care workers when an anomaly happens?
Automatic understanding of behavioral health and overall well-being of human beings is challenging due to the intricate nature of human behavior patterns, which are often subtle and non-verbal, dynamically changing, and multimodal. Moreover, automated human behavior estimation systems for healthcare applications need to be unobtrusive and privacy-preserving, so that do not feel uncomfortable/inhibited, with a feeling of being "watched and listened". We investigate how to integrate information from multimodal sensors (e.g., video cameras, range sensors, audio sensors) for automatic human behavior estimation. We specifically focus on building unobtrusive and privacy-preserving systems where occupants do not feel uncomfortable/inhibited, and do not feel like they are being "watched and listened".
Learn more here.
