Learning to Adapt in Robotic Systems: Multi-Sensor Fusion, Domain Shift, and Test-Time Planning

Time: 15:00
Place: EEE  Yorgo Istefanopulos Seminar Room - Kare Blok

 

Abstract: Robotic systems operating in real-world environments must perceive, reason, and act under uncertainty. This talk presents a research path from multi-sensor, multi-task perception toward adaptation-aware planning and prediction. I will begin with LiDAR-based perception for autonomous systems, then discuss how radar and infrared sensing can complement LiDAR in challenging scenes. While LiDAR provides accurate geometric structure, radar can offer robustness under adverse conditions, and infrared sensing may capture information that is less accessible to standard visual modalities. Combining these sensors in a multi-task framework allows shared representations to support related perception problems, such as segmentation, scene understanding, and road or object-level reasoning.

Despite these advantages, sensor fusion alone is not sufficient when deployment environments differ from training conditions. I will therefore discuss our work on unsupervised domain adaptation for perception under domain shift. Such methods can improve generalization, but they also introduce practical constraints, including the need for target-domain data, additional training procedures, and sometimes unstable optimization. This motivates the question of how robotic systems can adapt more directly at deployment time.

 

The second part of the talk focuses on this question from the perspective of planning. Although manipulator planning remains difficult, its search space is often more constrained than open-world perception, making it a useful setting for test-time adaptation. I will describe our recent work on diffusion-based planning, where generative models are used to sample and refine candidate motions during deployment. I will also briefly relate this direction to vision-language-action models, which increasingly use diffusion-based action generation and offer another path toward adaptive robot behavior.

Finally, I will connect these ideas back to autonomous vehicles through ego-velocity-dependent trajectory prediction. Overall, the talk frames adaptation as a common theme across perception, planning, and prediction in robotic systems.