METHOD FOR DETECTING MALICIOUS ACTIVITY IN INFECTED PROGRAMS
DOI:
https://doi.org/10.32782/IT/2024-4-21Keywords:
malware detection, emulation, sandbox, avoid techniques.Abstract
Developers of malwares employ various avoid techniques to evade detection during execution on user host. Key classes of such techniques include anti-emulation and polymorphic techniques, due to their ability to counteract hybrid detection methods commonly used in modern antivirus programs. Therefore, studying avoid techniques, their combinations, and their impact on the host will help in developing detection methods. In this context, it is appropriate to use emulation technologies, sandboxes, and distributing systems. The purpose is to detect malicious activity in infected programs that use evasion techniques by analysing changes in their execution behaviour in modified isolated environments. The methodology involves the use of scientific methods: synthesis, analysis, and comparison. The paper presents an analysis of modern evasion techniques in infected programs. It discusses the execution tool for infected programs and the system architecture for organizing distributed detection of malicious activity. An algorithm to form program execution behaviour in an isolated environment is also presented. The scientific novelty lies in developing a method for detecting, malicious activity in infected programs that use anti-emulation and polymorphic techniques. Special attention is given to the strategy of executing such programs in different environments, which impacts their execution behaviour. The presented research results demonstrate the effectiveness of proposed method within the scope of studied evasion techniques. Conclusions. The proposed method improves the detection of infected programs that use polymorphic transformation techniques to evade detection, particularly when the emulated environment has been identified. The architecture of the presented system provides a set of modified isolated environments for analysing program execution.
References
Razaulla S., Fachkha C., Markarian C., Gawanmeh A., Mansoor W., Fung B. C., Assi C. The age of ransomware: A survey on the evolution, taxonomy, and research directions. IEEE Access. 2023. Vol. 11. P. 40698–40723. DOI: 0.1109/ACCESS.2023.3268535
Savenko B., Lysenko S., Bobrovnikova K., Savenko O., Markowsky G. Detection DNS tunneling botnets. 11th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), 22 September, 2021. P. 64-69. DOI: 10.1109/IDAACS53288.2021.9661022
Savenko O., Sachenko A., Lysenko S., Markowsky G.,Vasylkiv N. Botnet detection approach based on the distributed systems. International Journal of Computing. 2020. Vol. 19. № 2. P. 190–198. DOI: 10.47839/ijc.19.2.1761
Lysenko S., Bobrovnikova K., Kharchenko V., Savenko O. IoT multi-vector cyberattack detection based on machine learning algorithms: traffic features analysis, experiments, and efficiency. Algorithms. 2022. Vol 15. № 7. P. 239. DOI: 10.3390/a15070239
Numminen A. Windows technical hardening against the most prevalent threats: Master Thesis : 2023, 61 p.
Omar M. Static Analysis of Malware. Defending Cyber Systems through Reverse Engineering of Criminal Malware. 2022. P. 9–17. DOI: 10.1007/978-3-031-11626-1_2
Leon, R. S., Kiperberg M., Leon Zabag A. A., Zaidenberg N. J. Hypervisor-assisted dynamic malware analysis. Cybersecurity. 2021. Vol. 4. P. 1–14. DOI: 10.1186/s42400-021-00083-9
Yunus Y. K. B. M., Ngah S. B. Review of hybrid analysis technique for malware detection. The 6thInternational Conference on Software Engineering & Computer Systems, Vol. 769, 25–27 September, 2019, Pahang, Malaysia. P. 012075. DOI: 10.1088/1757-899X/769/1/012075
Faruki P., Bhan R., Jain V., Bhatia S., El Madhoun N., Pamula R. A survey and evaluation of androidbased malware evasion techniques and detection frameworks. Information. 2023. Vol. 14. № 7. P.374. DOI: 10.3390/info14070374
D’Elia D. C., Coppa E., Palmaro F., Cavallaro L. On the dissection of evasive malware. IEEE Transactions on Information Forensics and Security. 2020. Vol. 15. P. 2750–2765. DOI: 10.1109/TIFS.2020.2976559
Apostolopoulos T., Katos V., Choo K. K. R., Patsakis C. Resurrecting anti-virtualization and antidebugging: Unhooking your hooks. Future Generation Computer Systems. 2021. Vol. 116. P. 393–405. DOI: 10.1016/j.future.2020.11.004
Nappa A., Papadopoulos P., Varvello M., Gomez D.A., Tapiador J., Lanzi A. Pow-how: An enduring timing side-channel to evade online malware sandboxes. 26th European Symposium on Research in Computer Security, 4-8 October, 2021, Darmstadt, Germany. P. 86–109. DOI: 10.1007/978-3-030-88418-5_5
Nicheporuk A., Savenko O., Nicheporuk A., Nicheporuk Y. An Android Malware Detection Method Based on CNN Mixed-Data Model. 16th International Conference on ICT in Education, Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer, Vol. 2732, October 06-10, 2020, Kharkiv, Ukraine. P. 198–213.
Gorment N. Z., Selamat A., Krejcar O. Obfuscated malware detection: impacts on detection methods. Asian Conference on Intelligent Information and Database Systems, Vol. 1863, 2023. P. 55–66. DOI: 10.1007/978-3-031-42430-4_5
Gibert D., Fredrikson M., Mateu C., Planes J., Le Q. Enhancing the insertion of NOP instructions to obfuscate malware via deep reinforcement learning. Computers & Security. 2022. Vol. 113. P. 102543. DOI: 10.1016/j.cose.2021.102543
Han S. H., Lee D. Kernel-based real-time file access monitoring structure for detecting malware activity. Electronics. 2022. Vol. 11, № 12, 1871. DOI: 10.3390/electronics11121871
Fedák A., Štulrajter J. Evasion of antivirus with the help of packers. Science & Military Journal. 2022. Vol. 17, № 1. P 14–22. DOI: 10.52651/sam.a.2022.1.14-22
Liu S., Feng P., Wang S., Sun K., Cao J. Enhancing malware analysis sandboxes with emulated user behavior. Computers & Security. 2022. Vol. 115, 102613. DOI: 10.1016/j.cose.2022.102613
Arabo A., Dijoux R., Poulain T., Chevalier G. Detecting ransomware using process behavior analysis. Procedia Computer Science. 2020. Vol. 168. P 289–296. DOI: 10.1016/j.procs.2020.02.249
Van Schaik S., Minkin M., Kwong A., Genkin D., Yarom Y. CacheOut: Leaking data on Intel CPUs via cache evictions. IEEE Symposium on Security and Privacy (SP). 24-27 May, 2021, San Francisco, CA, USA. P. 339–354. DOI: 10.1109/SP40001.2021.00064
Amit N., Tsafrir D., Schuster A., Ayoub A., Shlomo, E. Virtual CPU validation. Proceedings of the 25th Symposium on Operating Systems Principles. 04 October, 2015. P. 311–327. DOI: 10.1145/2815400.2815420
Peng P., Soljanin E., Whiting P. Diversity vs. parallelism in distributed computing with redundancy. IEEE International Symposium on Information Theory (ISIT). 21-26 June, 2020, Los Angeles, CA, USA. P. 257–262. DOI: 10.1109/ISIT44484.2020.9174030
Hamdan S., Ayyash M., Almajali, S. Edge-computing architectures for internet of things applications: A survey. Sensors. 2020. Vol. 20, № 22, P. 6441. DOI: 10.3390/s20226441
Kong X., Wu Y., Wang H., Xia F. Edge computing for internet of everything: A survey. IEEE Internet of Things Journal. 2022. Vol 9, № 23, P. 23472–23485. DOI: 10.1109/JIOT.2022.3200431
Rehida P., Sochor T., Martynyuk V., Tarasova O., Orlenko V. A distributed malware detection model based on sandbox technology. Intelligent Information Technologies & Systems of Information Security (IntelITSIS), Vol. 3373, 22-24 March, 2023, Khmelnytskyi, Ukraine. P. 475–485.
