Deep learning-based methods for automatic estimation of leaf herbivore damage

Abstract

Quantifying the intensity of leaf herbivory pressure is crucial for understanding the interaction between plants and herbivores in both applied and basic science. Visual estimates and digital analysis have been commonly used to estimate leaf herbivore damage but are time-consuming which limits the amount of data that can be collected and prevents answering big picture questions that require large-scale sampling of herbivory pressure.Here we trained generative adversarial networks (GANs) to predict the intact status of damaged leaves and applied image processing technique to estimate the area and percentage of leaf damage. We first described procedures for collecting leaf images, training GAN models, predicting intact leaves and calculating leaf area, with a Python package provided to enable hands-on application of these procedures. Then, we collected a large leaf dataset to train a universal deep learning model and developed an online app HerbiEstim to allow direct use of pretrained models to estimate herbivory damage of leaves. Lastly, we tested our method using both simulated and real leaf damage data.The procedures provided in our study greatly improved the efficiency of leaf herbivore damage estimation. We showed that the reconstruction of damaged leaf resembled the ground-truth with a similarity of 98.8%. The estimation of leaf herbivore damage showed a high accuracy with an averaged error of 1.6% and had a general applicability to different plant taxa and leaf shapes.Overall, our work demonstrates the feasibility of applying deep learning techniques to quantify leaf herbivory intensity. The use of GANs allows automatic estimation of leaf damage, representing a major advantage of the method. The Python package and online Shiny app with pre-trained models will facilitate the use of our method for the analysis of large datasets of plant-herbivore interactions.

Zihui Wang

Postdoctoral Fellow

I’m interested in understanding the factors that shape plant-microbial associations and predict their distribution and function under global change.