Benchmarking foundation models for splice site and exon annotation
Abstract
Recent foundation and deep learning models have brought a generational leap in improving the quality of genome annotation, particularly in identifying genes and their structural elements, including exons and splice sites. However, they are trained on reduced datasets that may not capture biological complexity, such as differences between coding versus non-coding, terminal versus internal, constitutive versus alternatively spliced, and transposable element (TE)-derived exons. We evaluate several foundation models for gene and splice site annotation, including the transformer-based SegmentNT, Enformer and Borzoi, coupled with a segmentation head for per-base resolution, and the CNN-based SpliceAI and AlphaGenome, along with a newly developed fine-tuned model, STEP2h, on different classes of gene elements as described above. We found that the performance of all methods is highest for the class of exons found in their training data class and decreases drastically for classes of exons poorly represented. In particular, performance is highest for protein-coding genes, coding exons, and constitutive exons, and decreases drastically by up to 2-4 fold for non-coding internal exons, terminal exons, and exons that undergo alternative splicing. Similarly, performance is impaired on LINE-1 and Alu-derived exons. In contrast, a locally developed CNN model fine-tuned on a specialized TE-exon dataset showed improved performance in this category. Our study highlights the outstanding…
Read the paper
Related studies
- Regulation of nucleolus assembly by non-coding RNA polymerase II transcripts
- For a Healthy Histone Code, a Little SUMO in the Tail Keeps the Acetyl Away
- Discovering a binary CTCF code with a little help from BORIS
- Article Commentary: Stabilizing the Code–-Methods to Preserve RNA Prove Their Worth
- Foreword to the Special Issue for the RINTC (The Implicit Seismic Risk of Code-Conforming Structures) Project
- Commentary: Promoting Health and Well-being in Pediatric Primary Care Settings: Using Health and Behavior Codes at Routine Well-child Visits
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.