News of 10.03.2021
Three-dimensional view of the world with artificial intelligence
Point cloud of mountains, Image: HZDR/HIF
Scientists from the Helmholtz Institute Freiberg for Resource Technology (HIF) at the Helmholtz-Zentrum Dresden-Rossendorf (HZDR) are currently researching a technology that can be used to better process and analyze multimodal point clouds. The project is called Hyper 3D-AI – Artificial Intelligence for three-dimensional (3D) multimodal point cloud classification. The 3-year project is carried out within the framework of the Helmholtz Imaging Platform, an innovative concept of the Helmholtz Association to systematically support and further develop imaging processes and methods of analysis. The HZDR and its project partner, the Institute for Industrial Information Technology of The Karlsruhe Institute of Technology (KIT), receive around 200,000 Euro for their research from the Helmholtz Association's Initiative and Networking Fund.
Spatial information in point clouds
It is of essential importance for many fields of modern research to master imaging processes and intelligent methods for analyzing image data. This is because, independent of the application field, spatially detailed information is commonly provided in the form of image data. Accordingly, major developments in image processing and interpretation are based on a spatially two-dimensional (2D) data grid with a custom number of informative layers. This approach is sufficient for large-scale geographic data.
However, todays’ most crucial image data applications used for resources, energy, mobility and medicine heavily rely on the precise interpretation of the spatial relationship of objects in all three dimensions. In order to avoid distortions of the spatial properties, 2D image data can be scaled up to 3D point clouds, i.e., each data point is provided with specific coordinates of a 3-dimensional vector space. This approach is not only beneficial for the fusion of image data with 3D information, such as orientation, shape and surface roughness, but can also integrate additional characteristic attributes from sensor data.
AI approach for material characterization in 3D
The currently most advanced approaches to process such point clouds are based on artificial intelligence (AI). However, these approaches still reach their limits when both spatial and high-dimensional point information such as spectral signatures and other compositional characteristics of a point cloud are to be analyzed. The Hyper 3D-AI project is now developing machine learning approaches to fill this gap.
"These approaches comprise both the challenging fusion of multiple sensors and the subsequent classification and segmentation using AI methods. In addition to algorithm design, the testing on representative scenarios from different application fields is an important project component, including the creation of reusable benchmark datasets for the validation and future development of algorithms. For the application scenarios, we are working closely with the industrial partners ZF Friedrichshafen and the HIF spin-off TheiaX. If successful, the project will improve the characterization of objects and surfaces for a wide range of potential applications such as exploration and mining, recycling, autonomous systems, quality assessment, or sorting systems," Dr. Sandra Lorenz explains, describing the goals.
From the project coordinator´s perspective, an enhanced material characterization will directly contribute to render processes more material and energy efficient. Due to its versatility in application, the project outcome could support any process that requires multisensor-based discrimination of objects and materials.
Further information:
Dr. Sandra Lorenz
Helmholtz Institute Freiberg for Resource Technology (HIF) at HZDR
Phone: +49 351 260 4487
