Decentralized Multi-Robot Navigation for Autonomous Surface Vehicles with Distributional Reinforcement Learning

Collision avoidance algorithms for Autonomous Surface Vehicles (ASV) thatfollow the Convention on the International Regulations for PreventingCollisions at Sea (COLREGs) have been proposed in recent years. However, it maybe difficult and unsafe to follow COLREGs in congested waters, where multipleASVs are navigating in the presence of static obstacles and strong currents,due to the complex interactions. To address this problem, we propose adecentralized multi-ASV collision avoidance policy based on DistributionalReinforcement Learning, which considers the interactions among ASVs as well aswith static obstacles and current flows. We evaluate the performance of theproposed Distributional RL based policy against a traditional RL-based policyand two classical methods, Artificial Potential Fields (APF) and ReciprocalVelocity Obstacles (RVO), in simulation experiments, which show that theproposed policy achieves superior performance in navigation safety, whilerequiring minimal travel time and energy. A variant of our framework thatautomatically adapts its risk sensitivity is also demonstrated to improve ASVsafety in highly congested environments.

Further reading#

• Access Paper in arXiv.org