PhD Position Foundation Models Automotive Imaging Radar

Job description

Next generation automotive imaging radars provide increasingly rich 3D information, opening the door to more advanced scene understanding and perception tasks such as 3D object detection and free space estimation. However, current neural networks are typically developed for a specific radar configuration and task, and often generalize poorly to other sensors. At the same time, large scale radar datasets with high quality labels remain scarce.

In this PhD project, you will investigate foundation models for automotive imaging radar. The goal is to learn general radar representations from largely unlabelled data that can generalize across different radar configurations and be efficiently adapted to multiple downstream perception tasks with limited labelled data. An important research direction is the transfer of representations from vision and LiDAR based foundation models to radar, as well as the development of multimodal foundation models incorporating radar.

A further challenge is how different radar representations and sensor configurations can be accommodated within a general foundation model framework, and to what extent a common model can generalize across them. Research directions may also include generative or predictive modeling of dynamic radar scenes.

The project combines methodological machine learning research with experiments on real automotive sensor data. You will have access to research vehicles and advanced radar prototypes for collecting and evaluating real world multimodal sensor data.

While the primary application domain is automotive, the developed methods should preferably be sufficiently general to also apply to other domains, such as autonomous vehicles operating in off-road settings.

You will join the Intelligent Vehicles group within the Department of Cognitive Robotics at TU Delft. The project will be carried out in collaboration with Perciv.

AI, and will also involve NXP Semiconductors as an industrial partner. The research is part of the FIND project, funded by the Dutch Research Council (NWO), which investigates foundation models for high tech industry applications.

Your results are expected to be published at leading international venues in machine learning, computer vision, robotics and radar, such as NeurIPS, CVPR, ICRA, IEEE IV and Radar Conf. For your research, you will have access to extensive computing resources at TU Delft, ranging from personal workstations and shared GPU servers to the Delft AI Cluster and the Delft Blue supercomputer.

Your supervisors will be Prof. Dariu Gavrila and Dr. Julian Kooij.

We are looking for a candidate with:

• An MSc degree in Computer Science, Artificial Intelligence, Robotics, Electrical Engineering, or a closely related field.
• A strong academic record and solid background in machine learning and deep learning.
• Ability to develop, understand, and critically evaluate machine learning research software, preferably using Python and PyTorch.
• An interest in foundation models, self supervised learning, multimodal learning, and 3D perception.
• An affinity for translating methodological research into experiments with real automotive sensor data and vehicle demonstrators.
• The ability to work independently as well as collaborate effectively within a larger research team.
• Good written and spoken English.
• Experience with radar, signal processing, computer vision, autonomous driving, or sensor fusion is advantageous.

Working at TU Delft means contributing to solutions that really make a difference.

For over 180 years, we have been training engineers who make an impact worldwide in companies, government bodies, or as entrepreneurs. Our alumni turn knowledge into concrete solutions for the challenges of today and tomorrow.

These challenges are changing rapidly. That is why we focus on themes such as energy, climate, digitalisation, artificial intelligence (AI), and smart mobility every day. Our education and research are directly aligned with what society needs now and in the future.

At TU Delft, our people make the difference. With their knowledge and curiosity, our staff provide a high-quality education and conduct pioneering research that extends beyond the campus. You will have the opportunity to take the initiative, work with others, and grow as a professional.

Working at TU Delft means join an international community of professionals and students. Together, we create knowledge, innovations,…