EXTENSIVE ERROR DERIVATIVE REVIEW OF LSTM MODELS WITH SIGN LANGUAGE INTERPRETATION

Year 2025, Volume: 15 Issue: 9, 2331 - 2351, 01.09.2025

Harapriya Kar P. Viswanathan

Abstract

LSTM models are essential for systems that translate sign language, where the model suffers from error loss when processing data. LSTMs reduce error propagation by continuously calculating gradients, unlike traditional back propagation, which causes exponential error accumulation. This paper investigates error flow in bidirectional, hierarchical, and probabilistic long short-term memory models (LSTMs). While hierarchical LSTMs employ multitask learning to anticipate inputs and outputs, minimizing compounding mistakes reliably, bidirectional LSTMs reduce truncation errors. Model accuracy is increased by optimizing the gradients and parameters. This research offers a thorough evaluation of LSTM models from 2021 to 2024, examining their effectiveness in sign language recognition systems by analyzing both accuracy and loss.

Keywords

RNN , LSTM , Bidirectional LSTM , Bayesian LSTM , Hierarchical LSTM , Parametric.

References

• Reference1 Kun, F., Junqi, J., Runpeng, C., Fei, S., Changshui, Z., (2017), Aligning where to see and what to tell: Image captioning with region-based attention and scene-specific contexts, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), pp. 2321–2334.
• Reference2 Bin, W., Zhijian, O., Zhiqiang, T., (2018), Learning transdimensional random fields with applications to language modeling, IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), pp. 876–890.
• Reference3 Ronald, J. W., David, Z., (1995), Gradient-based learning algorithms for recurrent networks and their computational complexity.
• Reference4 Zhiwen, D., Yuquan, L., Junkang, C., Xiang, Y., Yang, Z., Qing, G.,(2024), Tms-net: A multi-feature multi-stream multi-level information sharing network for skeleton-based sign language recognition, Neurocomputing.., 572(3), pp. 3007-3021.
• Reference5 Xu, Y. Z., Fei, Y., Yan, M. Z., Cheng, L. L., Yoshua, B., (2018), Drawing and recognizing chi- nese characters with recurrent neural network, IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), pp. 849–862.
• Reference6 Kyoungoh, L., Woojae, K., Sanghoon, L., (2023), From human pose similarity metric to 3d human pose estimator Temporal propagating lstm networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2), pp. 1781–1797.
• Reference7 Minhyuk, L., Joonbum B., (2020), Deep learning based real-time recognition of dynamic finger gestures using a data glove, IEEE Access, 8, pp. 219923–219933.
• Reference8 Jun, L., Amir, S., Dong, X., Alex C. K., Gang, W., (2018), Skeleton-based action recognition using spatio-temporal lstm network with trust gates, IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12), pp. 3007–3021.
• Reference9 Sepp, H., Jurgen, S., (1997), Long short-term memory, Neural Computation, 9(8), pp. 1735–1780.
• Reference10 Mostafizer, R., Yutaka, W., (2023), Multilingual program code classification using n-layered bi-lstm model with optimized hyperparameters, IEEE Transactions on Emerging Topics in Computational Intelligence, 8, pp.1452–1468.
• Reference11 Khanh, T.P., Nguyen, K. M., Christian, G., (2022), Probabilistic deep learning methodology for uncer- tainty quantification of remaining useful lifetime of multicomponent systems, Reliability Engineering and System Safety, 222.
• Reference12 Weiwen, P., Zhi, S. Y., Nan, C., (2020), Bayesian deep learning based health prognostics toward prognostics uncertainty, IEEE Transactions on Industrial Electronics, 67(3), pp. 2283–2293.
• Reference13 Yuming, Jie, W., Yan, S., Shude, W., Zuan, F., Jie, G., (2022), Ultra-short-term interval prediction model for photovoltaic power based on bayesian optimization, Institute of Electrical and Electronics Engineers, pp. 1138–1144.
• Reference14 Tadas, B., Chaitanya, A., Louis P. M., (2019), Multimodal machine learning: A survey and taxonomy, IEEE Transactions on Pattern Analysis and Machine Intelligence., 41(2), pp. 423–443.
• Reference15 Vincent, L. G., Nicolas, T., (2023), Deep time series forecasting with shape and temporal criteria, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), pp. 342–355.
• Reference16 Sunghyun, S., Dohee, K., Hyerim, B., (2023), Correlation recurrent units: A novel neural architecture for improving the predictive performance of time-series data, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12), pp. 14266–14283.
• Reference17 Wei, W., Yan, Y., Zhen, C., Jiashi, F., Shuicheng, Y., Nicu, S., (2019), Recurrent face aging with hierarchical autoregressive memory, IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(3), pp. 654–668.
• Reference18 Qianli, M., Sen, L., Garrison, W. C., (2022), Adversarial joint learning recurrent neural network for incomplete time series classification, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4), pp. 1765–1776.
• Reference19 Jinhui, T., Xiangbo, S., Rui, Y., Liyan, Z., (2022), Coherence constrained graph lstm for group activity recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(2), pp. 636–647.
• Reference20 Xianyun, W., Weibang, L., (2023), Time series prediction based on lstm-attention-lstm model, IEEE Access, 11, pp. 48322–48331.
• Reference21 Lianli, G., Xiangpeng, L., Jingkuan, S., Heng-Tao, S., (2019), Hierarchical lstms with adaptive at- tention for visual captioning, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1.
• Reference22 Xiangbo S., Jinhui, T., Guo, J. Q., Wei, L., Jian Y., (2021), Hierarchical long short term concurrent memory for human interaction recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(3), pp. 1110–1118.
• Reference23 Bing, Su. and Ying Wu., (2019), Learning low-dimensional temporal representations with latent align- ments, IEEE Transactions on Pattern Analysis and Machine Intelligence., pp. 1–1.
• Reference24 Mehmet, O. T., Stefano, D. A., Jan, W., Konrad S., (2021), Gating revisited: Deep multi-layer rnns that can be trained, IEEE Transactions on Pattern Analysis and Machine Intelligence., pp. 1–1.
• Reference25 Gers, F.A., Schmidhuber, J., Cummins, F., (1999), Learning to forget: continual prediction with lstm, In 1999 Ninth International Conference on Artificial Neural Networks, 2, pp. 850–855.
• Reference26 Felix, A., Gers., Nicol, N. S., Jurgen, S., (2003), Learning precise timing with lstm recurrent networks, J. Mach. Learn. Res., 3, pp. 115–143.
• Reference27 Dong, Q., William, K. C., (2023), Learning hierarchical variational autoencoders with mutual infor- mation maximization for autoregressive sequence modeling, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2), pp. 1949–1962.
• Reference28 Gilmer, V., Jerome, H. F., Fei, J., Efstathios, D. G., (2022), Representational gradient boosting: Backpropagation in the space of functions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12), pp. 10186–10195.
• Reference29 David, E., Rumelhart, G., Hinton, E., Ronald, J. W., (1986), Learning representations by back- propagating errors, Nature, 323(10), pp. 533–536.
• Reference30 Sepp, H., Jurgen, S., (1997), Long short-term memory, Neural Computation., 9(8), pp. 1735–1780.
• Reference31 Qi, L., Jun, Z., (2015), Revisit long short-term memory: An optimization perspective.
• Reference32 Anahita, G., Nurfadhlina, M. S., Fatimah B. S., (2024), Prediction of course grades in computer science higher education program via a combination of loss functions in lstm model, IEEE Access, 12, pp. 30220–30241.
• Reference33 Wenzhao, Z., Jiwen, L., Jie, Z., (2023), Deep metric learning with adaptively composite dynamic constraints, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–17.
• Reference34 Minhee, K., Kaibo, L., (2021), A bayesian deep learning framework for interval estimation of remaining useful life in complex systems by incorporating general degradation characteristics, IISE Transactions, 53(3), pp. 326–340.
• Reference35 Jun, L., Henghui, D., Amir, S., LingYu, D., Xudong, J., Gang, W., Alex, C. K., (2020), Feature boost- ing network for 3d pose estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence., 42(2), pp. 494–501.
• Reference36 Jie, X., Wei, Z., Fei, W., (2021), A(dp)2sgd: Asynchronous decentralized parallel stochastic gradient descent with differential privacy, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1.
• Reference37 Cuihong X., Jingli, J., Ming, Y., Gang, Y., Yingchun G., Yuehao L., (2024 ), Continuous sign lan- guage recognition based on hierarchical memory sequence network, IET Computer Vision, 18(3)., pp. 247–259.
• Reference38 Lingxiang, Y., Worapan, K., Peng, Z., Qiang, W., Jian, Z., (2023), Improving disentangled represen- tation learning for gait recognition using group supervision, IEEE Transactions on Multimedia, 25, pp. 4187–4198.
• Reference39 Yunbo, W., Haixu, W., Jianjin, Z., Zhifeng, G., Jianmin, W., Philip S. Y., Ming, S., (2023), Long. Predrnn: A recurrent neural network for spatiotemporal predictive learning, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2), pp. 2208–2225.
• Reference40 Sunusi, B., Abdullahi, M., Kosin, C., (2022), American sign language words recognition using spa- tiooral prosodic and angle features: A sequential learning approach, IEEE Access, 10, pp. 15911–15923
• Reference41 Huangyue, Yu., Minjie, C., Yunfei, L., Feng, L., (2023) First and third person video coanalysis by learning spatial-temporal joint attention, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6), pp. 6631–6646.
• Reference42 Oscar, K., Necati, C. C., Hermann, N., Richard, B., (2020), Weakly supervised learning with multi- stream cnn-lstm-hmms to discover sequential parallelism in sign language videos, IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(9), pp. 2306– 2320.
• Reference43 Muneer, A., Ghulam, M., Wadood, A., Mansour, A., Mohammed, A. B., Tareq, S. A., Hassan, M., Mohamed, A. M., (2020), Deep learning-based approach for sign language gesture recognition with efficient hand gesture representation, IEEE Access, 8, pp. 192527–192542.
• Reference44 Natarajan, B., Rajalakshmi, E., Elakkiya, R., Ketan, K., Ajith A., Lubna, A. G., Subramaniyaswamy, V., (2022), Development of an end-to-end deep learning framework for sign language recognition, translation, and video generation, IEEE Access, 10, pp. 104358–104374.
• Reference45 Amimul, I., Abrar, F. E., Lutfun, N., Muhammad, A. Kadir., (2024), Medisign: An attention-based cnn-bilstm approach of classifying word level signs for patient doctor interaction in hearing impaired community, IEEE Access, 12, pp. 33803–33815.
• Reference46 Yao, D., Pan, X., Mingye, W., Xiaohui, H., Zheng, Z., Jiaqi, L., (2022), Full transformer network with masking future for word-level sign language recognition, Neurocomputing, 500(8), pp. 115–123.
• Reference47 Gaspard, H., Jong, W. K., Beakcheol, J., (2022), A multi-headed transformer approach for pre- dicting the patient’s clinical time-series variables from charted vital signs, IEEE Access, 10, pp. 105993–106004.
• Reference48 Lipisha, C., Tejaswini, A., Enjamamul, H., Ifeoma, N., (2023), Signnet ii: A transformer-based two- way sign language translation model, IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 45(11), pp. 12896–12907.
• Reference49 Yan, H., Qi, W., Wei, W., Liang, W., (2018), Image and sentence matching via semantic concepts and order learning, IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(3), pp. 636–650.
• Reference50 Neelma, N., Hasan, S., Sara, A., Osman, H., Muhammad, K. E., (2023), Miparesgcn: a multi- input part attention enhanced residual graph convolutional framework for sign language recognition, Computers and Electrical Engineering, 112(12).
• Reference51 Qinkun, X., Xin, C., Xue, Z., Xing L., (2020), Multi-information spatial temporal lstm fusion contin- uous sign language neural machine translation, IEEE Access, 8, pp. 216718– 216728.
• Reference52 Daniel, S. B., Aveen, D., Ajit, J., Phaneendra, K. Y., OmJee, P., Linga, R. C., (2021), Robust hand gestures recognition using a deep cnn and thermal images, IEEE Sensors Journal, 21(12), pp. 26602–26614.
• Reference53 Huijuan, X., Abir, D., Kate, S., (2019), Two-stream region convolutional 3d network for temporal activity detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(10), pp. 2319–2332.
• Reference54 Hamzah, L., (2022), An efficient two-stream network for isolated sign language recognition using accumulative video motion, IEEE Access, 10, pp. 93785–93798.
• Reference55 Pengfei, Z., Cuiling, L., Junliang, X., Wenjun, Z., Jianru, X., Nanning, Z., (2019), View adaptive neural networks for high performance skeleton-based human action recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8), pp. 1963–1978.
• Reference56 Haocong, R., Siqi, W., Xiping, H., Mingkui, T., Yi, G., Jun, C., Xinwang, L., Bin, H., (2022), A self-supervised gait encoding approach with locality awareness for 3d skeleton based person re-identification, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), pp. 6649–6666.
• Reference57 Muneer, A., Ghulam, M., Wadood, A., Mansour, A., Mohamed, A. B., Mohamed, A. M., (2020), Hand gesture recognition for sign language using 3dcnn, IEEE Access., 8, pp. 79491–79509.
• Reference58 Rajalakshmi, E., Elakkiya, R., Subramaniyaswamy, V., Prikhodko Alexey, L., Grif, M., Maxim, B., Ketan, K., Lubna, A. G., Ajith, A., (2023), Multi- semantic discriminative feature learning for sign gesture recognition using hybrid deep neural architecture, IEEE Access, pp. 2226–2238.
• Reference59 Abu, S. M., Mehedi, H. A., Satoshi, N., Jungpil S., (2024), Sign language recognition using graph and general deep neural network based on large scale dataset. IEEE Access, 12, pp. 34553–34569.
• Reference60 Jungpil, S., Abu, S., Musa, M., Kota, S., Koki, H., Mehedi, H. A., (2023), Dynamic korean sign language recognition using pose estimation based and attention-based neural network, IEEE Access, 11, pp. 143501–143513.
• Reference61 Tamer, S., (2023), Two-stage deep learning solution for continuous arabic sign language recognition using word count prediction and motion images, IEEE Access., 11, pp. 126823– 126833.
• Referenc62 Tianyu, L., Tangfei, T., Yizhe, Z., Min, L., Jieli, Z., (2024), A signer independent sign language recognition method for the single-frequency dataset. Neurocomputing, 582(5).
• Reference63 Sunanda, D., Samir, I., Nieb, H. N., Nazmul, S., Hui, W., (2023), A hybrid approach for bangla sign language recognition using deep transfer learning model with random forest classifier, Expert Systems with Applications, 213(3).
• Reference64 Tafia, H. P., et. al., (2024), Fine-Tuning of Predictive Models CNN-LSTM and CONV-LSTM for Nowcasting PM2.5Level, IEEE Access, 12, pp. 28988-29003.
• Reference65 Yan, H., Qi, W., Wei, W., Liang, W., (2020), Image and Sentence Matching via Semantic Concepts and Order Learning, IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(3), pp. 636-650.
• Reference66 Mucahit, E. Y. M., Suleyman, E., (2022), BabyPose: Real-Time Decoding of Baby’s Non-Verbal Communication Using 2D Video-Based Pose Estimation, IEEE Sensors Journal, 22(14), pp. 13776- 13784
• Reference67 Anis, K., et. al., (2020), A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding, IEEE Transactions on Pattern Analysis and Machine Intelligence., 42 (1), pp. 1-14.
• Reference68 Tresa, J., Bindiya, T. S., (2023), Realization and hardware implementation of gating units for long short-term memory network using hyperbolic sine functions, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 42(12), pp. 5141–5145.