Multi-modal single-cell foundation models via dynamic token adaptation

Recent advances in applying deep learning in genomics include DNA-language and single-cell foundation models. However, these models take only one data type as input. We introduce dynamic token adaptation and demonstrate how it allows combining these models to predict gene regulation at single-cell level in different genetic contexts. Although the method is generalisable, we focus on an illustrative example by training an adapter from DNA-sequence embeddings to a single-cell foundation model's token embedding space. As qualitative evaluation, we assess the impact of DNA sequence changes on the model’s learned gene regulatory networks by mutating the transcriptional start site of the transcription factor \textit{GATA4} \textit{in silico}, observing predicted expression changes in its target genes in fetal cardiomyocytes.