Protein Hücre Altı Lokalizasyonunda Dondurulmuş ESM-2 Gömüleri ile Dikkat Destekli Bi-LSTM Topluluklarının Rekabetçi Performansı

Abstract

Protein hücre altı konumlarının hesaplamalı olarak tahmin edilmesi, büyük protein veri setlerinin analizi için hayati öneme sahiptir. Bu alandaki temel sorunlardan biri, eğitim ve test setlerinde yer alan benzer dizilerin doğruluk tahminlerini yapay olarak yükseltmesidir. Bu çalışma, karmaşık mimariler yerine basit sınıflandırıcılar kullanılarak, yalnızca Protein Dil Modeli (PLM) öznitelikleriyle güçlü tahminler elde edilip edilemeyeceğini araştırmaktadır. Çalışmada, önceden eğitilmiş ESM-2 gömülerini (embeddings) dikkat mekanizması ile güçlendirilmiş bir Bi-LSTM ağı ile birleştiren ve yumuşak oylama (soft voting) kullanan 3-katlı bir topluluk (ensemble) yapısında, sadeleştirilmiş bir derin öğrenme çerçevesi geliştirilmiştir. Titiz bir değerlendirme sağlamak amacıyla, eğitim sürecinde %40 veya daha az benzerliğe sahip ökaryotik diziler kullanılmıştır. Geliştirilen model, test verilerinde %86,81 doğruluk (MCC = 0,825) elde ederek, SVM temel modeline (%64,54, MCC = 0,530) kıyasla %22,27 oranında bir iyileşme sağlamıştır. 2024 yılında yayınlanan 86 yeni protein üzerinde yapılan testlerde sistem %88,37 doğruluk (MCC = 0,827) oranına ulaşmış; DeepLoc 2.1 (%80,23, MCC = 0,714, p=0,007) ve MULocDeep (%77,91, MCC = 0,682, p=0,019) modellerini geride bırakmıştır. Ancak, küçük doğrulama kümesi (N=86) ve Mitokondri (N=5) gibi bazı kategorilerdeki sınırlı veri temsili nedeniyle sonuçların temkinli yorumlanması gerekmektedir. Yöntem, çok konumlu proteinleri hariç tutarak, dört hücresel bölme genelinde yalnızca tek konumlu atamaları ele almaktadır. Dikkat ağırlığı analizi, modelin C-terminal zar bölgeleri ve N-terminal mitokondriyal dizileri dahil olmak üzere biyolojik olarak anlamlı sinyalleri başarıyla tanımladığını göstermekte ve ESM-2 gömülerinin basitleştirilmiş mimarilerle etkili performans sağladığını doğrulamaktadır.

Keywords

• Dikkat mekanizması
• Çift Yönlü LSTM
• Derin öğrenme
• ESM-2 gömüleri
• Protein hücre altı lokalizasyonu

Supporting Institution

Kastamonu Universitesi, Türkiye; Mindanao State University - Main Campus, Marawi City, Philippines.

Ethical Statement

Bu çalışma tamamen hesaplamalı bir araştırma olup, UniProt Bilgi Tabanından (UniProt Knowledgebase) alınan halka açık protein dizi verilerini kullanmaktadır. Araştırmada insan katılımcılar veya hayvan denekler yer almamaktadır. Bu nedenle, etik kurul onayı gerekmemektedir.

Thanks

Yazar, kamuya açık UniProt Bilgi Tabanı'nın sürdürülmesindeki katkılarından dolayı UniProt Konsorsiyumu'na ve ESM-2 protein dil modelini açık kaynak lisansı ile kullanıma sunduğu için Meta AI'ya teşekkürlerini sunar. Tüm hesaplamalı analizler Google Colaboratory platformu üzerinde gerçekleştirilmiştir.

Attention-Enhanced Bi-LSTM Ensembles with Frozen ESM-2 Embeddings Achieve Competitive Performance in Protein Subcellular Localization

Abstract

Predicting protein subcellular locations computationally is crucial for analyzing large protein datasets. A key issue is that similar sequences in training and test sets artificially inflate accuracy estimates. This study investigates whether Protein Language Model (PLM) features alone can achieve strong predictions using simple classifiers instead of complex architectures. We developed a streamlined deep learning framework combining pre-trained ESM-2 embeddings with an attention-enhanced Bi-LSTM network, deployed as a 3-fold ensemble with soft voting. Training used eukaryotic sequences with ≤40% similarity to ensure rigorous evaluation. The model achieved 86.81% accuracy (MCC = 0.825) on test data—a +22.27% improvement over an SVM baseline (64.54%, MCC = 0.530). On 86 newly released 2024 proteins, the system reached 88.37% accuracy (MCC = 0.827), surpassing DeepLoc 2.1 (80.23%, MCC = 0.714, p=0.007) and MULocDeep (77.91%, MCC = 0.682, p=0.019). However, the small validation set (N=86) and limited representation in categories like Mitochondrion (N=5) require cautious interpretation. The method only handles single-location assignments across four compartments, excluding multi-location proteins. Attention weight analysis shows the model identifies biologically relevant signals, including C-terminal membrane regions and N-terminal mitochondrial sequences, confirming that ESM-2 embeddings enable effective performance with simplified architectures.

Keywords

• Attention mechanism
• Bi-LSTM
• Deep learning
• ESM-2 embeddings
• Protein subcellular localization

Supporting Institution

Kastamonu University, Türkiye; Mindanao State University - Main Campus Marawi City, Philippines

Ethical Statement

This study is strictly computational and utilizes publicly available protein sequence data retrieved from the UniProt Knowledgebase. No human participants or animal subjects were involved in this research. Therefore, ethical committee approval was not required.

Thanks

The author acknowledges the UniProt Consortium for maintaining the publicly accessible UniProt Knowledgebase and Meta AI for releasing the ESM-2 protein language model under an open-source license. All computational analyses were conducted using Google Colaboratory.

References

1. References
2. 1. Villanueva, E., et al., System-wide analysis of RNA and protein subcellular localization dynamics. Nat Methods, 2024. 21: p. 60–71. Doi: 10.1038/s41592-023-02101-9
3. 2. Martinez-Val, A., et al., Spatial-proteomics reveals phospho-signaling dynamics at subcellular resolution. Nat Commun, 2021. 12: p. 7113. Doi: 10.1038/s41467-021-27398-y
4. 3. Pasha, T., et al., Karyopherin abnormalities in neurodegenerative proteinopathies. Brain, 2021. 144: p. 2915–2932. Doi: 10.1093/brain/awab201
5. 4. Calabrese, G., C. Molzahn, and T. Mayor, Protein interaction networks in neurodegenerative diseases: From physiological function to aggregation. Journal of Biological Chemistry, 2022. 298: p. 102062. Doi: 10.1016/j.jbc.2022.102062
6. 5. Lavoie, H., J. Gagnon, and M. Therrien, ERK signalling: a master regulator of cell behaviour, life and fate. Nat Rev Mol Cell Biol, 2020. 21: p. 607–632. Doi: 10.1038/s41580-020-0255-7
7. 6. Jiang, Y., et al., Exploring potential therapeutic targets for asthma: a proteome-wide Mendelian randomization analysis. J Transl Med, 2024. 22: p. 978. Doi: 10.1186/s12967-024-05782-8
8. 7. Shen, Y., et al., Critical evaluation of web-based prediction tools for human protein subcellular localization. Brief Bioinform, 2020. 21. Doi: 10.1093/bib/bbz106

9. 8. Xiao, H., et al., A Review for Artificial Intelligence Based Protein Subcellular Localization. Biomolecules, 2024. 14: p. 409. Doi: 10.3390/biom14040409
10. 9. Thumuluri, V., et al., DeepLoc 2.0: multi-label subcellular localization prediction using protein language models. Nucleic Acids Res, 2022. 50. Doi: 10.1093/nar/gkac278
11. 10. Teufel, F., et al., SignalP 6.0 predicts all five types of signal peptides using protein language models. Nat Biotechnol, 2022. 40. Doi: 10.1038/s41587-021-01156-3
12. 11. Lin, Z., et al., Evolutionary-scale prediction of atomic-level protein structure with a language model. Science (1979), 2023. 379. Doi: 10.1126/science.ade2574
13. 12. Ødum, M.T., et al., DeepLoc 2.1: multi-label membrane protein type prediction using protein language models. Nucleic Acids Res, 2024. 52: p. W215–W220. Doi: 10.1093/nar/gkae237
14. 13. Jiang, Y., et al., MULocDeep: A deep-learning framework for protein subcellular and suborganellar localization prediction with residue-level interpretation. Comput Struct Biotechnol J, 2021. 19. Doi: 10.1016/j.csbj.2021.08.027
15. 14. Bateman, A., et al., UniProt: the Universal Protein Knowledgebase in 2025. Nucleic Acids Res, 2025. 53. Doi: 10.1093/nar/gkae1010
16. 15. Bateman, A., et al., UniProt: the Universal Protein Knowledgebase in 2023. Nucleic Acids Res, 2023. 51. Doi: 10.1093/nar/gkac1052
17. 16. Bateman, A., et al., UniProt: the universal protein knowledgebase in 2021. Nucleic Acids Res, 2021. 49. Doi: 10.1093/nar/gkaa1100
18. 17. Li, W. and A. Godzik, Cd-hit: A fast program for clustering and comparing large sets of protein or nucleotide sequences. Bioinformatics, 2006. 22. Doi: 10.1093/bioinformatics/btl158
19. 18. Narang, S., et al., Mixed precision training, in 6th International Conference on Learning Representations, ICLR 2018 – Conference Track Proceedings. 2018.
20. 19. Hochreiter, S., The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowldege-Based Systems, 1998. 6. Doi: 10.1142/S0218488598000094
21. 20. Hochreiter, S. and J. Schmidhuber, Long Short-Term Memory. Neural Comput, 1997. 9. Doi: 10.1162/neco.1997.9.8.1735
22. 21. Schuster, M. and K.K. Paliwal, Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 1997. 45: p. 2673–2681. Doi: 10.1109/78.650093
23. 22. Bahdanau, D., K.H. Cho, and Y. Bengio, Neural machine translation by jointly learning to align and translate, in 3rd International Conference on Learning Representations, ICLR 2015 – Conference Track Proceedings. 2015.
24. 23. Lin, T.Y., et al., Focal Loss for Dense Object Detection. IEEE Trans Pattern Anal Mach Intell, 2020. 42. Doi: 10.1109/TPAMI.2018.2858826
25. 24. Deng, Y., J. Jia, and M. Yi, EDCLoc: a prediction model for mRNA subcellular localization using improved focal loss to address multi-label class imbalance. BMC Genomics, 2024. 25: p. 1252. Doi: 10.1186/s12864-024-11173-6
26. 25. Dietterich, T.G., Ensemble methods in machine learning, in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2000.
27. 26. Kohavi, R., A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection, in IJCAI International Joint Conference on Artificial Intelligence. 1995.
28. 27. Vignesh, U., R. Parvathi, and K. Gokul Ram, Ensemble deep learning model for protein secondary structure prediction using NLP metrics and explainable AI. Results in Engineering, 2024. 24. Doi: 10.1016/j.rineng.2024.103435
29. 28. Yuan, L., Y. Ma, and Y. Liu, Ensemble deep learning models for protein secondary structure prediction using bidirectional temporal convolution and bidirectional long short-term memory. Front Bioeng Biotechnol, 2023. 11. Doi: 10.3389/fbioe.2023.1051268
30. 29. Boob, A.G., et al., Design of diverse, functional mitochondrial targeting sequences across eukaryotic organisms using variational autoencoder. Nature Communications, 2025. 16. Doi: 10.1038/s41467-025-59499-3
31. 30. Zhou, Z., et al., Advances in solubilization and stabilization techniques for structural and functional studies of membrane proteins. PeerJ, 2025. 13.
32. 31. He, L. and X. Liu, The Development and Progress in Machine Learning for Protein Subcellular Localization Prediction. Open Bioinforma J, 2022. 15. Doi: 10.2174/18750362-v15-e2208110
33. 32. Chicco, D. and G. Jurman, The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 2020. 21: p. 6. Doi: 10.1186/s12864-019-6413-7
34. 33. Gavel, Y., L. Nilsson, and G. von Heijne, Mitochondrial targeting sequences why “non-amphiphilic” peptides may still be amphiphilic. FEBS Lett, 1988. 235. Doi: 10.1016/0014-5793(88)81257-2
35. 34. Calvo, S.E., et al., Comparative analysis of mitochondrial N-termini from mouse, human, and yeast. Molecular and Cellular Proteomics, 2017. 16. Doi: 10.1074/mcp.M116.063818
36. 35. Kalinin, I.A., et al., Features of membrane protein sequence direct post-translational insertion. Nature Communications, 2024. 15. Doi: 10.1038/s41467-024-54575-6
37. 36. Sun, S. and M. Mariappan, C-terminal tail length guides insertion and assembly of membrane proteins. Journal of Biological Chemistry, 2020. 295. Doi: 10.1074/jbc.RA120.012992
38. 37. Reed, A.L., et al., Interactions of amyloidogenic proteins with mitochondrial protein import machinery in aging-related neurodegenerative diseases. Front Physiol, 2023. 14. Doi: 10.3389/fphys.2023.1263420
39. 38. Hein, M.Y., et al., Global organelle profiling reveals subcellular localization and remodeling at proteome scale. Cell, 2025. 188: p. 1137–1155.e20. Doi: 10.1016/j.cell.2024.11.028
40. 39. Zhang, X., et al., Prediction of protein subcellular localization in single cells. Nat Methods, 2025. 22: p. 1265–1275. Doi: 10.1038/s41592-025-02696-1
41. 40. Briesemeister, S., J. Rahnenführer, and O. Kohlbacher, Going from where to why-interpretable prediction of protein subcellular localization. Bioinformatics, 2010. 26. Doi: 10.1093/bioinformatics/btq115
42. 41. Wan, B., et al., EGA-Ploc: An Efficient Global-Local Attention Model for Multi-Label Protein Subcellular Localization Prediction on the Immunohistochemistry Images. IEEE J Biomed Health Inform, 2025. 29: p. 9299–9311. Doi: 10.1109/JBHI.2025.3613205
43. 42. Zhang, Q., et al., MpsLDA-ProSVM: Predicting multi-label protein subcellular localization by wMLDAe dimensionality reduction and ProSVM classifier. Chemometrics and Intelligent Laboratory Systems, 2021. 208: p. 104216. Doi: 10.1016/j.chemolab.2020.104216
44. 43. Wang, F. and L. Wei, Multi-scale deep learning for the imbalanced multi-label protein subcellular localization prediction based on immunohistochemistry images. Bioinformatics, 2022. 38: p. 2602–2611. Doi: 10.1093/bioinformatics/btac123
45. 44. Guo, X., et al., Human Protein Subcellular Localization with Integrated Source and Multi-label Ensemble Classifier. Sci Rep, 2016. 6: p. 28087. Doi: 10.1038/srep28087
46. 45. Sahu, S.S., C.D. Loaiza, and R. Kaundal, Plant-mSubP: a computational framework for the prediction of single- and multi-target protein subcellular localization using integrated machine-learning approaches. AoB Plants, 2020. 12. Doi: 10.1093/aobpla/plz068
47. 46. Liu, F., S. Xin, and Y. Liu, ProLoc-IHS: Multi-label protein subcellular localization based on immunohistochemical images and sequence information. Int J Biol Macromol, 2025. 313: p. 144096. Doi: 10.1016/j.ijbiomac.2025.144096
48. 47. Zhang, Q., et al., Accurate prediction of multi-label protein subcellular localization through multi-view feature learning with RBRL classifier. Brief Bioinform, 2021. Doi: 10.1093/bib/bbab012