An evaluation of model-based methods for control in driverless racing
Supervisor: Dr Sebastian Stein
School: Computing Science
Description:
In the Formula Student - Artificial Intelligence competition (FS-AI, [1]), controllers of autonomous race cars have to navigate unknown tracks delimited by traffic cones to achieve minimum lap time while remaining on the track at all times. This is a challenging problem, as the track and state of the car can only be partially observed via noisy sensors (cameras, LiDAR, IMU and GPS), and control actions have long-term consequences such as entering a bend at excessively high speed.
Problems of sequential decision-making under uncertainty - like this one - are commonly addressed in the framework of Partially Observable Markov Decision Processes (POMDPs). Model-based methods for control in POMDPs have shown to be particularly data-efficient and can incorporate conditional-value-at-risk constraints [2], such as veering off track. These are key properties that popular model-free reinforcement learning methods lack. These advantages over model-free methods comes at the cost of having to learn a probabilistic generative model of the environment either on-the-fly or in a pre-training stage.
To date, the University of Glasgow Racing Team's Driverless section (UGRacing - Driverless) has made significant progress on individual components of the generative model including a stochastic track generator, vehicle model and quantification of perception system uncertainty. This internship will give the selected student the opportunity to integrate these efforts with existing model-based controllers and learn how to rigorously apply the scientific method with two objectives in simulation.
First, to comparatively evaluate control methods (active inference and model-based reinforcement learning with Gaussian Process models) with respect to lap time and violation of track boundary constraints. Outcomes have the potential to be integrated with the existing autonomous driving pipeline of the UGRacing -Driverless team and enter the FS-AI 2023 competiton.
Second, to analyse controller sensitivity with respect to individual components of the generative model, relating reduction of uncertainty with expected lap time reduction and reduction in constraint violations. Results will help identify promising directions for future improvement of the autonomous driving pipeline and could inform the teams's objectives for development in the next season.
References
[1] FS-AI: https://www.imeche.org/events/formula-student/team-information/fs-ai
[2] Cowen-Rivers, A.I., Palenicek, D., Moens, V. et al. SAMBA: safe model-based & active reinforcement learning. Mach Learn 111, 173–203 (2022).