Potato Plant Disease Detection Using UAV-Based RGB Imagery and Depth Maps in Deep Learning

Abstract

Accurate detection of potato plant diseases is essential for ensuring crop health and yield. This thesis investigates deep learning approaches for potato plant disease detection using Uncrewed Aerial Vehicles (UAV)-based RGB imagery, depth maps, and fused modalities. This research presents a deep learning-based framework for classifying three major potato conditions—Blackleg, Potato Virus Y, and healthy plants—based on UAV-derived multimodal data. High-resolution RGB orthomosaics were collected from a potato field located in Lelystad, the Netherlands, where the disease status of individual plant was visually assessed by a crop expert. Depth maps were generated from structure from motion (SfM) point clouds to capture structural traits such as plant height and canopy morphology. Three classification models were developed using the ResNet50 architecture: one trained on RGB imagery, one on depth maps, and one on a fusion of both modalities. Model performance was evaluated through five-fold cross-validation. Results indicate that the Fusion model achieved an improved average balanced accuracy of 0.708, outperforming the Depth model and complementing the RGB model. To enhance interpretability, Class Activation Maps (CAMs) were used to reveal the complementary contributions of geometric features in disease detection. The study further explores how different modalities support disease detection, discusses the challenges of using depth maps for below-canopy symptoms detection, such as those caused by Blackleg, and proposes considerations for data fusion strategies.