Applying Semantic Segmentation and Structure-from-Motion to Monitor Flora in Cliff Ecosystems

Abstract

Cliff ecosystems show extraordinary biodiversity that is increasingly challenged by outside pressures such as climate change, invasive species, and (other) forms of human interference. This presents researchers and conservationists with promising possibilities to do their work, yet they are presented with big challenges due to the nature of the terrain. Large elevation changes make it difficult to access and monitor plants, and current methods of representing this data fail to properly show the three-dimensionality of the mapped cliffs. Our research proposes a pipeline that combines semantic segmentation and 3D-reconstruction using Structure-from-Motion to create a segmented point cloud from UAV footage. We create two datasets of annotated images taken with a drone in Cabo Espichel and Meia Velha in Portugal. One dataset contains the rare endemic Euphorbia Pedroi, while the other dataset contains Opuntius Ficus-Indica, a species of cactus native to Mexico that is invasive in many areas in the world. We evaluate our pipeline with different semantic segmentation models, different pretraining methods, and different data augmentation configurations to help us understand their role in our pipeline. Evaluation of the resulting segmented point cloud reveals that it is highly effective at mapping our target species in 3D, allowing future cliff monitoring to be done in a much more efficient manner.