Publications Repository - Helmholtz-Zentrum Dresden-Rossendorf

1 Publication

Unsupervised Data Fusion with Deeper Perspective: A Novel Multi-Sensor Deep Clustering Algorithm

Rafiezadeh Shahi, K.; Ghamisi, P.; Rasti, B.; Scheunders, P.; Gloaguen, R.

The ever-growing developments in technology to capture different types of image data (e.g., hyperspectral imaging and Light Detection and Ranging (LiDAR)-derived digital surface model (DSM)), along with new processing techniques, have led to a rising interest in imaging applications for Earth observation. However, analyzing such datasets in parallel remains a challenging task. In this paper, we propose a multi-sensor deep clustering (MDC) algorithm for the joint processing of multi-source imaging data. The architecture of MDC is inspired by autoencoder (AE)-based networks. The MDC paradigm includes three parallel networks, a spectral network using an autoencoder structure, a spatial network using a convolutional autoencoder structure, and lastly, a fusion network that reconstructs the concatenated image information from the concatenated latent features from the spatial and spectral network. The proposed algorithm combines the reconstruction losses obtained by the aforementioned networks to optimize the parameters (i.e., weights and bias) of all three networks simultaneously. To validate the performance of the proposed algorithm, we use two multi-sensor datasets from different applications (i.e., geological and rural sites) as benchmarks. The experimental results confirm the superiority of our proposed deep clustering algorithm compared to a number of state-of-the-art clustering algorithms. The code will be available at: https://github.com/Kasra2020/MDC.

Keywords: Multi-sensor Data Fusion; Deep Learning; Autoencoder; Convolutional Autoencoder; Remote Sensing

Related publications

  • Open Access Logo IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 15(2021), 284-296
    Online First (2021) DOI: 10.1109/JSTARS.2021.3132856

Permalink: https://www.hzdr.de/publications/Publ-33863
Publ.-Id: 33863