Unlocking rich genetic insights through multimodal AI with M-REGLE

Everything from medical specialists with cutting-edge technology to simple smartwatches are generating data on an unprecedented scale. The aggregation of electronic health records, medical imaging, diagnostic tests, genomic data, and even real-time measurements from smartwatches creates a wealth of data for researchers and clinicians to analyze. These diverse data streams often carry unique and overlapping signals, even within the same organ system.

In the cardiovascular system, for example, an electrocardiogram (ECG) measures the heart’s electrical activity, while a photoplethysmogram (PPG) — common in smartwatches — tracks blood volume changes. The co-analysis of these modalities can simultaneously assess both the heart’s electrical system and its pumping efficiency, thus providing a more complete picture of heart health. Integrating these physiological signatures with genetic information from large nation-level biobanks could enable the identification of the genetic underpinnings of disease.

Our earlier work, REGLE, was successful for genetic discovery using health data, but it was designed for a single data type (i.e., the unimodal setting). Alternatively, analyzing each modality separately and then trying to piece together the findings later (what we refer to as U-REGLE or Unimodal REGLE) also might not be the most efficient way. U-REGLE could miss subtle shared information between different modalities. Instead, we hypothesized that jointly modeling these complementary data streams would boost the important biological signals, reduce noise, and lead to more powerful genetic discoveries.

Here we present our recent paper, “Utilizing multimodal AI to improve genetic analyses of cardiovascular traits”, which we published in the American Journal of Human Genetics. We developed a multimodal version of REGLE, called M-REGLE, that allows the analysis of multiple types of clinical data together at once. M-REGLE produces lower reconstruction error, identifies more genetic associations, and outperforms risk scores in predicting cardiac disease compared to its predecessor, U-REGLE.