International benchmarking of terrestrial laser scanning approaches for forest inventories: Part I: Objective, Datasets, Evaluation Criteria and Methods ((joint project of EuroSDR and ISPRS))

The last two decades have witnessed increasing awareness of the potential for terrestrial laser scanning (TLS) in forest applications in both public and commercial sectors, along with tremendous research efforts and progress. It is time to inspect the achievements of and the remaining barriers to TLS-based forest investigations, so further research and application are clearly orientated in operational uses of TLS. In such context, the international TLS benchmarking project was launched in 2014 by the European Spatial Data Research Organization and coordinated by the Finnish Geospatial Research Institute. The main objectives of this benchmarking study are to evaluate the potential of applying TLS in characterizing sample plots, to clarify the strengths and the weaknesses of TLS as a measure of forest digitization, and to reveal the capability of recent algorithms for tree-attribute extraction. The project was designed to benchmark the TLS algorithms by processing identical TLS datasets for a standardized set of forest attribute criteria and by evaluating the results through a common procedure respecting reliable references. Benchmarking results reflect large variances in estimating accuracies, which were unveiled through the 18 compared algorithms and through the evaluation framework, i.e., forest complexity categories, TLS data acquisition approaches, tree attributes and evaluating procedures. The results also reveal some best available forest attribute estimates at this time achieved by a couple of groups using their algorithms, which hints at the potential of TLS in forest environments with the hardware currently available. Some results are well expected, while some are new, e.g., the variances of estimating accuracies between single-/multi-scan, the principle of the algorithm designs and the possibility of a computer outperforming human operation. This paper focuses on the conceptual schema to promote the understanding of the benchmarking results, the potential of TLS in forest modelling, and the fundamental components, i.e., the selection of sample plots, the collection of TLS data, the acquisition of reference datasets, the definition of evaluation criteria (including three criteria proposed in this benchmarking), and the development of evaluation structures (e.g., the algorithm performances are investigated through combining two or more criteria). The TLS datasets are set to open data for further research purposes. New developments can be linked to the eighteen algorithms reported in this benchmarking. © 2019, European Spatial Data Research (EuroSDR). All rights reserved.