Advancing Health Research through Multimodal AI

Background

Multimodal AI has the potential to capture the complexity of biomedical and behavioral systems and improve clinical decision‑making, but realizing this promise requires new innovations in data fusion, model training, evaluation, and application. Ethical considerations—including privacy, fairness, accountability, and transparency—must be integrated throughout the entire lifecycle, from data selection and preparation to model deployment, with careful attention to stakeholder needs. Collaboration among researchers, patients, policymakers, the scientific community, and end users is essential to co‑create multimodal AI systems that reflect shared values and real‑world requirements. As AI advances faster than traditional funding and development timelines, more flexible and agile approaches to AI research and implementation are increasingly necessary.

Purpose

The purpose of this program is to develop ethically focused and data-driven multimodal AI approaches to more closely model, interpret, and predict complex biological, behavioral, and health systems and enhance our understanding of health and the ability to detect and treat human diseases.

Program Goals

• Creation of ethics- and data-driven multimodal AI models for use in biomedical, behavioral, and/or clinical fields
• Build portfolio of innovative projects that address systems level biomedical challenges using a co-design approach to multimodal AI that integrate the work of various stakeholder groups as appropriate
• Inform considerations for the appropriate use of multimodal AI and take significant steps towards incorporation of ethical and co-design approach in multimodal AI lifecycle
• Use translational or end use applications that will be identified and used as test cases for testing and evaluation

Expected Program Outputs

Mulitmodal AI Awards

Lead Institution	Sub-awardees
Brigham and Women's Hospital	Massachusetts General Hospital, University of Washington
University of Florida	Indiana University, University at Buffalo
University of Michigan	University of California Los Angeles, Cornell Medicine, Vanderbilt University, University of South Florida
University of Wisconsin-Madison	Marshfield Clinic Research Institute, The Medical College of Wisconsin, The University of Chicago
Northwestern University at Chicago	Cleveland Clinic, Cornell Medicine
University of Pennsylvania
University of Colorado Denver	Cleveland Clinic
University of California Berkeley	UCSF , Stanford University, University of Virginia
Baylor College of Medicine	Credence Management Solutions, UT Health
Stanford University
Mayo Clinic Arizona	Mayo Clinic (Minnesota - Rochester), Arizona State University
University of Texas Health Science Center	UT-Houston, Rice University, UNC-CH, UNC-Charlotte, NC State
University of North Carolina	UC San Diego, Moffitt Cancer Center, Wake Forest, Medical University of South Carolina, UT Houston
University of Pittsburgh	Duke, UT MD Anderson Cancer Center
Stanford University
University of California, San Diego
Emory University	Stanford University, Mayo Clinic Arizona
Children's Hospital of Philadelphia	Columbia University, Boston Children’s Hospital

Research Focus

• Breast Cancer Treatment

• Modeling Of T Cell Therapies

• Outcome Prediction of Critically Ill Patients

• Phenotyping Of Inherited Diseases

• Disease Phenotype Discovery

• Systemic Lupus Detection

• Preeclampsia Risk in Early Pregnancy

• Chronic Kidney Disease

• Mapping Type 1 Diabetes Progression

• Substance Misuse Prevention

• Heart Failure Endotypes

• Frailty & Digestive Diseases

• Chronic Obstructive Pulmonary

• Dermatology

• Congenital Heart Disease

• Prostate Cancer

• Pain Recovery

• Digital Twins for Medical Device Surveillance

Contact Information

Please direct questions about Advancing Health Research through Multimodal AI to [email protected]