ANALYSIS OF EXISTING DEEP LEARNING MODELS IN NATURAL LANGUAGE PROCESSING TASKS

Authors

DOI:

https://doi.org/10.32782/IT/2024-3-7

Keywords:

deep learning, recurrent neural networks, LSTM, GRU, natural language processing, sentiment analysis

Abstract

Natural language processing (NLP) is one of the most relevant branches of artificial intelligence, covering a wide range of tasks such as emotion analysis, machine translation, speech recognition and others. Purpose of work: The purpose of this study is to comprehensively analyze the performance of deep learning models, including recurrent neural networks (RNN), long-short-term memory (LSTM) networks, and guided recurrent units (GRU), in NLP tasks. Special attention is paid to the effectiveness of these models in emotion analysis tasks. Methodology: The study includes several steps: data collection and pre-processing, implementation and training of RNN, LSTM and GRU models on selected data sets, evaluation of their performance using indicators such as precision, recall and F1-score, and analysis of the resource requirements of the models, especially in conditions of limited computing resources. In addition, the paper provides a comparative analysis of models based on their scalability when working with large volumes of data. Scientific novelty: This study offers a detailed comparative analysis of the performance of RNNs, LSTMs, and GRUs in various NLP tasks, with an emphasis on their ability to process sequential data and account for longterm dependencies. The conducted analysis reveals which of the models is the most effective in specific conditions, depending on the available resources and the specifics of the data. Conclusions: As a result of the study, it was found that GRU showed the highest performance in emotion analysis, outperforming RNN and LSTM in terms of precision, recall and F1-score. LSTM proved to be optimal for working with large volumes of data, demonstrating high efficiency and accuracy. RNN, although it provides fast training on small data sets, is inferior to other models in accuracy, which makes it less suitable for complex NLP tasks. The obtained results contain valuable information for researchers and practitioners who are engaged in the application of deep learning models in NLP tasks.

References

Natural Language Processing – DeepLearning.AI. URL: https://www.deeplearning.ai/resources/naturallanguage-processing/ (дата звернення: 05.09.2024).

Afanasieva I., Golian N., Hnatenko O., Daniiel Y., Onyshchenko K. Data exchange model in the Internet of Things concept. Telecommunications and Radio Engineering. New York, 2019. Vol. 78, № 10. P. 869–878.

Golian, V., Golian, N., Afanasieva, I., Halchenko, K., Onyshchenko, K., Dudar, Z. Study of Methods for Determining Types and Measuring of Agricultural Crops due to Satellite Images. 32nd International Scientific Symposium Metrology and Metrology Assurance, MMA 2022, Sozopol, Bulgaria.

Recurrent Neural Networks (RNN) – EDUCBA. URL: https://www.educba.com/recurrent-neural-networksrnn/(дата звернення: 05.09.2024).

Anish Nama. Understanding LSTM Architecture, Pros and Cons, and Implementation. Medium, 2020. URL: https://medium.com/@anishnama20/understanding-lstm-architecture-pros-and-cons-and-implementation-3e0cca194094 (дата звернення: 03.09.2024).

Anish Nama. Understanding Gated Recurrent Unit (GRU) in Deep Learning. Medium, 2020. URL: https://medium.com/@anishnama20/understanding-gated-recurrent-unit-gru-in-deep-learning-2e54923f3e2 (дата звернення: 02.09.2024).

Pandey P. Emotion dataset. Kaggle. URL: https://www.kaggle.com/datasets/parulpandey/emotiondataset (дата звернення: 05.09.2024).

Afanasieva, I., Golian, N., Golian, V., Khovrat, A., Onyshchenko, K. Application of Neural Networks to Identify of Fake News. Computational Linguistics and Intelligent Systems (COLINS 2023): 7th International Conference, Kharkiv, 20 April – 21 April 2023: CEUR workshop proceedings, No. 3396. P. 346–358. URL: https://ceur-ws.org/Vol-3396/paper28.pdf (дата звернення: 02.09.2024).

Turuta O., Afanasieva I., Golian N., Golian V., Onyshchenko K., Suvorov D. Audio processing methods for speech emotion recognition using machine learning. MoMLet-2024: 6th International Workshop on Modern Machine Learning Technologies, 31 травня – 1 червня 2024 р., Львів-Шацьк, Україна. С. 75–108.

James, G., Witten, D., Hastie, T., Tibshirani, R. An Introduction to Statistical Learning with Applications in R. New York: Springer, 2013. 426 p.

Published

2024-12-06