Camera Radar LiDAR Fusion via Point-Cloud Projection for Multi-Task Learning

Abstract

Autonomous vehicles require robust perception systems to operate safely under diverse environmental conditions. While cameras provide high-resolution visual data, their performance significantly degrades in low-light and adverse weather. High Definition (HD) radar offers superior reliability and angular resolution under such conditions but lacks rich visual feature recognition. LiDAR, though expensive, delivers highly precise distance measurements and dense point clouds. Each sensor exhibits unique trade-offs, necessitating the development of advanced sensor fusion algorithms to harness complementary strengths while minimizing computational complexity and maintaining accuracy. In this study, the proposed fusion architecture performs early fusion by combining camera images with perspective-view point cloud data from HD radar and LiDAR sensors. The proposed model is trained to detect vehicles and segment free drivable space simultaneously. Experiments are performed on the RADIal dataset where we demonstrate that fusing point cloud data with camera images in front-view surpasses the performances of the most recent standalone and fusion models.