FEATURES OF SOFTWARE ARCHITECTURE FOR COLLECTING AND ANALYZING STATISTICAL INFORMATION ON THE GLOBAL NETWORK
DOI:
https://doi.org/10.32782/IT/2023-2-13Keywords:
software, software testing, levels of software testing, software requirements: functional requirements, non-functional requirements, requirements, statistical methods.Abstract
To study the structure and characteristics of World Wide Web, specialized software is essential. This paper examines the peculiarities of the such software architecture and pays high attention to important aspects of its performing, such as navigation mechanisms, scanning strategies, and the processing of acquired data. The article also addresses the challenges of efficiency and scalability in similar solutions when handling a large number of web resources. The developed software consists of a crawler and an analytical module. The functional purpose of the analytical module is to conduct cluster and statistical analysis of large volumes of data by means of statistical methods. The architecture of the developed application aligns with the latest trends in software development, considering modern requirements and standards. Using this software with integrated analytical module, statistical and cluster characteristics of various segments of WWW have been researched, including the Ukrainian segments (edu.ua, net.ua), the Polish segment (edu.pl), and the Israeli segment (ac.il).
References
Najork M., Heydon A. High-Performance Web Crawling. In: (eds) Handbook of Massive Data Sets. Massive Computing / J. Abello, P.M. Pardalos, M.G.C. Resende, Boston : Springer, 2002. Vol. 4. P. 25-45.
Najork M. Web Crawler Architecture. Encyclopedia of Database Systems, 2017. P. 1–4.
Говорущенко Т.О., Боднар М.А., Кушнір В.О. Сучасні проблеми формування та аналізу вимог до програмного забезпечення. Вимірювальна та обчислювальна техніка в технологічних процесах. 2019. № 1. С. 45–53.
T. Hovorushchenko, O. Pavlova, M. Bodnar. Development of an Intelligent Agent for Analysis of Nonfunctional Characteristics in Specifications of Software Requirements. Eastern-European Journal of Enterprise Technologies. 2019. Vol. 1. No. 2 (97). P. 6–17.
Paech, Barbara & Dutoit, Allen & Kerkow, Daniel & Knethen, Antje. (2002). Functional requirements, nonfunctional requirements, and architecture should not be separated. A position paper. 2002.
Alashqar A., Elfetouh A., El-Bakry H. Requirement Engineering for Non-Functional Requirements. International Journal of Information and Communication Technology Research. 2015. № 5. P. 21–27.
Кириченко О.Л., Kanovsky I., Остапов С.Е. Програмне забезпечення для дослідження статистичних характеристик глобальної мережі WWW. Системи обробки інформації. 2013. Вип. 3 (110). Т. 2. С. 99–104.
Jaiswal М. Software Architecture and Software Design. International Research Journal of Engineering and Technology. 2019. Vol. 6. P. 2452–2454.
Cook J. Docker for Data Science: Building Scalable and Extensible Data Infrastructure Around the Jupyter Notebook Server. Berkeley : Apress, 2017. 257 p.
Umar M. Comprehensive study of software testing: Categories, levels, techniques, and types. International Journal of Advance Research, Ideas and Innovations in Technology. 2019. № 5. P. 32–40.
