ProtRNA: A Protein-derived RNA Language Model by Cross-Modality Transfer Learning

While protein language models such as ESM-2 have demonstrated exceptional performance, RNA language models still face challenges due to limited and less conserved sequence data. To bridge this gap, we introduce ProtRNA, a cross-modality transfer learning framework that adapts ESM-2 for RNA sequence modeling. Leveraging the evolutionary priors encoded in ESM-2, ProtRNA achieves performance comparable to or surpassing baseline RNA language models across multiple downstream tasks, while using only 1/8 of the trainable parameters and 1/6 of the training data required by the primary reference model. These findings underscore the potential of cross-modality transfer learning for biological language modeling.