An Interpretable Transformer-Based Autoencoder Framework for Identifying Driver Mutation in Colorectal Cancer

Öz

Identification of driving genetic mutations in colorectal cancer is one of the main challenges given the multi-dimensional nature and the difficulty of genomic mutation data. In this study, a deep learning model that is based on the transformer-based Autoencoder was proposed to identify influential genetic mutations linked with colorectal cancer, using data from the cosmic database. The proposed model is dependent on a self-attention mechanism to deal with complex interfaces between genes and long-range associations between mutations. To assess the its efficacy, the model was compared with a classical Autoencoder that is dependent on dense layers as a reference framework. Additionally, a new indicator identified as the Mutation Importance Score (MIS), that integrates attention weights and reconstruction error, with an attempt to rank mutations based on their possible potential biological implication. Experimentation results showed that the Transformer-based model attained a significant reduction in reconstruction error in comparison with the reference model, as the test loss value reduced from 0.154 to 0.107, to reflect an enhancement around 30%. The proposed model indicated a clear enhancement in the ability to find biologically significant genes, with an advanced overlap with the Cancer Gene Census database of (87% vs. 63%), as well as an enhancement in the Precision@20 scale of (0.85 vs. 0.65). These results indicate the model's ability to give exact priority to possible motor mutations. These results indicate that deep learning models based on the self- attention mechanism are strong tools to analyze large-scale genomic mutation data, and may contribute to the growth of computational methods in the field of cancer genomics and accuracy medicine.

Anahtar Kelimeler

• transformer based autoencoder
• colorectal cancer
• self attention
• driving mutation
• deep learning

Destekleyen Kurum

The authors received no specific funding or institutional support for this research.

Etik Beyan

The authors confirm that this research was conducted in accordance with ethical standards. The manuscript is original, has not been published previously, and is not under consideration elsewhere. All authors have approved the final version of the manuscript and declare that there are no conflicts of interest related to this work.

Teşekkür

The authors would like to express their sincere gratitude to Al-Nahrain University for providing support and facilities that contributed to the completion of this research.

Kaynakça

1. [1] Abdul Rahman, H., Ottom, M.A. & Dinov, I.D. (2023). Machine learning based colorectal cancer prediction using global dietary data. BMC Cancer, (23), 144. https://doi.org/10.1186/s12885-023-10587-x
2. [2] Kumar, S., Warrell, J., Li, S., McGillivray, P. D., Meyerson, W., Salichos, L., ... & Gerstein, M. B. (2020). Passenger mutations in more than 2,500 cancer genomes: overall molecular functional impact and consequences. Cell, 180(5), 915-927. 10.1016/j.cell.2020.01.032
3. [3] Zhang, J., & Zhang, S. (2016). The discovery of mutated driver pathways in cancer: models and algorithms. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 15(3), 988-998. https://doi.org/10.1109/tcbb.2016.2640963
4. [4] Alharbi, W. S., & Rashid, M. (2022). A review of deep learning applications in human genomics using next-generation sequencing data. Human Genomics, 16(1). https://doi.org/10.1186/s40246-022-00396-x
5. [5] Nourbakhsh, M., Bruun Degn, K., Brix Saksager, A., Tiberti, M., & Papaleo, E. (2024). Prediction of cancer driver genes and mutations: the potential of integrative computational frameworks. Briefings in Bioinformatics, 25(2). https://doi.org/10.1093/bib/bbad519
6. [6] Zubair, M., Haider Khan, A., Fakhar Bilal, S., & Li, J. (2025). Deep learning approaches for resolving genomic discrepancies in cancer: a systematic review and clinical perspective. Briefings in Bioinformatics, 26(6). https://doi.org/10.1093/bib/bbaf541
7. [7] Sartori, F., Codicè, F., Caranzano, I., Rollo, C., Birolo, G., Fariselli, P., & Pancotti, C. (2025). A comprehensive review of deep learning applications with multi-omics data in cancer research. Genes, 16(6), 648. https://doi.org/10.3390/genes16060648
8. [8] Luo, P., Ding, Y., Lei, X., & Wu, F. (2019). DeepDriver: Predicting cancer driver genes based on somatic mutations using deep convolutional neural networks. Frontiers in Genetics, 10, 13-13. https://doi.org/10.3389/fgene.2019.00013

9. [9] Zeng, Z., Mao, C., Vo, A., Li, X., Nugent, J. O., Khan, S. A., Clare, S. E., & Luo, Y. (2021). Deep learning for cancer type classification and driver gene identification. BMC Bioinformatics, 22(4). https://doi.org/10.1186/s12859-021-04400-4
10. [10] Ji, Y., Zhou, Z., Liu, H., & Davuluri, R. V. (2020). DNABERT: Pre-trained bidirectional encoder representations from transformers model for dna-language in genome. BioRxiv. https://doi.org/10.1101/2020.09.17.301879
11. [11] Wagner, S. J., Reisenbüchler, D., West, N. P., Niehues, J. M., Zhu, J., Foersch, S., Veldhuizen, G. P., Quirke, P., Grabsch, H. I., van den Brandt, P. A., Hutchins, G. G. A., Richman, S. D., Yuan, T., Langer, R., Jenniskens, J. C. A., Offermans, K., Mueller, W., Gray, R., Gruber, S. B., ... Kather, J. N. (2023). Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study. Cancer Cell, 41(9), 1650-1661.e4. https://doi.org/10.1016/j.ccell.2023.08.002
12. [12] Dalla-Torre, H., Gonzalez, L., Mendoza-Revilla, J., Lopez Carranza, N., Grzywaczewski, A. H., Oteri, F., Dallago, C., Trop, E., de Almeida, B. P., Sirelkhatim, H., Richard, G., Skwark, M., Beguir, K., Lopez, M., & Pierrot, T. (2024). Nucleotide Transformer: building and evaluating robust foundation models for human genomics. Nature Methods, 22(2), 287-297. https://doi.org/10.1038/s41592-024-02523-z
13. [13] Heaton, J. (2017). Ian Goodfellow, Yoshua Bengio, and Aaron Courville: Deep learning. Genetic Programming and Evolvable Machines, 19(1-2), 305-307. https://doi.org/10.1007/s10710-017-9314-z
14. [14] Eraslan, G., Avsec, Ž., Gagneur, J., & Theis, F. J. (2019). Deep learning: new computational modelling techniques for genomics. Nature Reviews Genetics, 20(7), 389-403. https://doi.org/10.1038/s41576-019-0122-6
15. [15] G. Tate, J., Bamford, S., C. Jubb, H., Sońdka, Z., M. Beare, D., Bindal, N., Boutselakis, H., G. Cole, C., Creatore, C., Dawson, E., Fish, P., Harsha, B., Hathaway, C., Jupe, S., Yin Kok, C., noble, K., Ponting, L., ramshaw, C., E. Rye, C., ... Andrew Forbes, S. (2018). COSMIC: The catalogue of somatic mutations in cancer. Nucleic Acids Research, 47(D1), D941-D947. https://doi.org/10.1093/nar/gky1015
16. [16] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., N.Gomez, A., Kaiser, L., & Polosukhin, I. (2025). Attention is all you need. Shenzhen Medical Academy of Research and Translation. https://doi.org/10.65215/ysbyhc05
17. [17] Song, H., Dong, T., Wang, W., Yan, X., Geng, C., Bai, S., & Hu, H. (2025). A deep autoencoder compression-based genomic prediction method for whole-genome sequencing data. Biology, 14(11), 1622-1622. https://doi.org/10.3390/biology14111622
18. [18] Xuan, P., Gong, Z., Cui, H., Li, B., & Zhang, T. (2022). Fully connected autoencoder and convolutional neural network with attention-based method for inferring disease-related lncRNAs. Briefings in Bioinformatics, 23(3). https://doi.org/10.1093/bib/bbac089
19. [19] Bishop, C. (2006). Pattern Recognition and Machine Learning (Information Science and Statistics). Springer eBooks. https://openalex.org/W1506806321
20. [20] Li, J., Chen, X., Hovy, E., & Jurafsky, D. (2016). Visualizing and understanding neural models in NLP. K. Knight, A. Nenkova O. Rambow (Eds.) Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 681-691. https://doi.org/10.18653/v1/n16-1082
21. [21] Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504-507. https://doi.org/10.1126/science.1127647