IMAGE RECOGNITION AND PROCESSING METHODS ACCORDING TO THE MODIFIED YOLOV5-V1
DOI:
https://doi.org/10.32782/IT/2023-1-10Keywords:
image processing, object detection, apple yield, YOLOv5, deep learningAbstract
This study proposes a novel deep learning approach for apple detection and counting. The well-recognized YOLOv5 model was employed as a baseline for its high accuracy and fast processing time and further modified to suit the requirements of the apple detection task in the natural environment. The proposed approach involves two stages: apple detection and apple counting. In the detection stage, the novel YOLOv5-v1 model was trained on a manually crafted dataset of apple images to learn the features that distinguish apples from the background. The model contained new layers for the BottleneckCSP-v4 module, replacing the BottleneckCSP module in the original YOLOv5 network’s backbone architecture. In the counting stage, the SENet module is integrated into the enhanced trunk network to better identify the features of medium and large-sized fruits under various conditions. The initial size of the source network’s binding block was adjusted to prevent the misidentification of small objects in the image background and thus improve counting accuracy. To evaluate the performance of the approach, computational experiments were conducted on a dataset of apple images constructed by the authors. The experimental results on the test dataset demonstrated that the improved model could effectively recognize and count fruits captured by the unmanned aerial vehicle camera with recall, precision, mAP, and F1-score of 92.13%, 84.59%, 87.94%, and 89.02%, respectively. The proposed approach was also compared with several other state-of-the-art methods, like YOLOv5, YOLOv3, and YOLOv4 and EfficientDet-D0, and it was found that our model outperformed the analogs in terms of accuracy and speed. The average recognition speed of our model was 0.015 seconds per 1 frame of the video sequence (66.7 frames/s), which was 2.53, 1.13, and 3.53 times higher than that of the EfficientDet-D0, YOLOv4, and YOLOv3 networks, respectively. The obtained results have several potential applications in the agriculture industry, where they can be used for crop monitoring, yield estimation, and quality control. Further research can also be conducted to incorporate additional features, such as fruit shape, using a larger dataset to train the model.
References
Medvedeva Y., Kucher A., Lipsa J. et al. Human health risk assessment on the consumption of apples growing in urbanized areas: Case of Kharkiv, Ukraine. International Journal of Environmental Research and Public Health. Vol. 18, № 4. P. 1504. DOI:10.3390/ijerph18041504.
Suresh K. M., Mohan S. Selective fruit harvesting: Research, trends and developments towards fruit detection and localization – A review. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science. Vol. 237, № 6. P. 1405–1444. DOI:10.1177/09544062221128443.
Yu L., Xiong J., Fang X. et al. A litchi fruit recognition method in a natural environment using RGB-D images. Biosystems Engineering. Vol. 204. 2021. P. 50–63. DOI:10.1016/j.biosystemseng.2021.01.015.
Wan N. S. R., Muhammad A. H., Megat S. A. M. A. et al. Automated image identification, detection and fruit counting of top-view pineapple crown using machine learning. Alexandria Engineering Journal. Vol. 61, № 2. P. 1265–1276. DOI:10.1016/j.aej.2021.06.053.
Fu L., Duan J., Zou X. et al. Banana detection based on color and texture features in the natural environment. Computers and Electronics in Agriculture. Vol. 167. 2019. P. 105057. DOI:10.1016/j.compag.2019.105057.
Radiuk P., Hrypynska N. A framework for exploring and modeling neural architecture search methods. The 4th International Conference on Computational Linguistics and Intelligent Systems (COLINS-2020) : CEURWorkshop Proceedings. Vol. 2604. (Lviv, 23-24 April 2020). Lviv, 2020. P. 1060–1074. URL: http://ceur-ws.org/Vol-2604/
Li W., Feng X. S., Zha K. et al. Summary of target detection algorithms. Journal of Physics: Conference Series. Vol. 1757, № 1. P. 012003. DOI:10.1088/1742-6596/1757/1/012003.
Tang P., Wang X., Wang A. et al. Weakly supervised region proposal network and object detection. Computer Vision – ECCV 2018 (Cham, 2018). Cham : Springer International Publishing, 2018. P. 370–386. DOI:10.1007/978-3-030-01252-6_22.
Zhao G., Li G., Xu R. et al. Collaborative training between region proposal localization and classification for domain adaptive object detection. Computer Vision – ECCV 2020 (Cham, 2020). Cham : Springer International Publishing, 2020. P. 86–102. DOI:10.1007/978-3-030-58523-5_6.
Radiuk P., Pavlova O., Hrypynska N. An ensemble machine learning approach for Twitter sentiment analysis. The 6th International Conference on Computational Linguistics and Intelligent Systems (CoLInS-2022). Volume I: Main Conference : CEUR-Workshop Proceedings. Vol. 3171. (Gliwice, Poland, 12-13 May 2022). Gliwice, 2022. P. 387–397. URL: http://ceur-ws.org/Vol-3171/ С. 11.
Mai X., Zhang H., Jia X. et al. Faster R-CNN with classifier fusion for automatic detection of small fruits. IEEE Transactions on Automation Science and Engineering. Vol. 17, № 3. P. 1555–1569. DOI:10.1109/TASE.2020.2964289.
Wang S. Research towards YOLO-series algorithms: Comparison and analysis of object detection models for real-time UAV applications. Journal of Physics: Conference Series. Vol. 1948, № 1. P. 012021. DOI :10.1088/1742-6596/1948/1/012021.
Bresilla K., Perulli G. D., Boini A. et al. Single-shot convolution neural networks for real-time fruit detection within the tree. Frontiers in Plant Science. Vol. 10. 2019. P. 610. DOI:10.3389/fpls.2019.00611
Huang Z., Zhang P., Liu R. et al. Immature apple detection method based on improved YOLOv3. ASP Transactions on Internet of Things. Vol. 1, № 1. P. 9–13. DOI:10.52810/TIOT.2021.100028.
Chen W., Zhang J., Guo B. et al. An apple detection method based on Des-YOLO v4 algorithm for harvesting robots in complex environments. Mathematical Problems in Engineering. Vol. 2021. 2021. P. e7351470. DOI:10.1155/2021/7351470.
Behera S. K., Mishra N., Sethy P. K. et al. On-tree detection and counting of apple using color thresholding and CHT. 2018 International Conference on Communication and Signal Processing (ICCSP-2018). Vol. 2018, (2018). IEEE Inc., 2018. P. 0224–0228. DOI:10.1109/ICCSP.2018.8524363.
Колокольчикова І. В. Промислове садівництво Півдня України в рамках забезпечення продовольчої безпеки. Science and Education a New Dimension. VII(24), № 200. P. 7–10. DOI:10.31174/SENDNT2019-200VII24-01.
Wang Y., Qin Y., Cui J. Occlusion robust wheat ear counting algorithm based on deep learning. Frontiers in Plant Science. 2021. Vol. 12. P. 645899. DOI:10.3389/fpls.2021.645899
Tan M., Le Q. V. EfficientNet: Rethinking model scaling for convolutional neural networks. Proceedings of the 36th International Conference on Machine Learning, ICML-2019. Long Beach, California, USA : PMLR. org, 2019. URL: P. 6105–6114. http://proceedings.mlr.press/v97/tan19a.html.
Vinci A., Brigante R., Traini C. et al. Geometrical characterization of hazelnut trees in an intensive orchard by an unmanned aerial vehicle (UAV) for precision agriculture applications. Remote Sensing. Vol. 15, № 2. P. 541. DOI:10.3390/rs15020541.
Zhou D., Hou Q., Chen Y. et al. Rethinking bottleneck structure for efficient mobile network design. Computer Vision – ECCV 2020 (Cham, 2020). Cham : Springer International Publishing, 2020. Vol. 12348. P. 680–697. DOI:10.1007/978-3-030-58580-8_40.
Kim H.-U., Bae T.-S. Deep learning-based GNSS network-based real-time kinematic improvement for autonomous ground vehicle navigation. Journal of Sensors. Vol. 2019. 2019. P. e3737265. DOI:10.1155/2019/3737265.
Dąbrowski P. S., Specht C., Specht M. et al. Integration of multi-source geospatial data from GNSS receivers, terrestrial laser scanners, and unmanned aerial vehicles. Canadian Journal of Remote Sensing. Vol. 47, № 4. P. 621–634. DOI:10.1080/07038992.2021.1922879.
Dempsey P. Reviews consumer technology: The teardown: Apple iPhone pro 13 smartphone. Engineering & Technology. Vol. 16, № 11. P. 68–69. DOI:10.1049/et.2021.1122.
Claid.ai: Generate, enhance and edit images at scale via API. 2023. URL: https://claid.ai/ (дата звернення: 01.04.2023).
Mishra A. Machine learning in the AWS cloud: Add intelligence to applications with Amazon Sagemaker and Amazon Rekognition. Amazon SageMaker. John Wiley & Sons, 2019. P. 353–385. URL: https://onlinelibrary.wiley.com/doi/book/10.1002/9781119556749
