A Study of Deep Q Network, Soft Actor Critic Algorithm in CARLA

Abstract

This project is primarily focused on the advancement of machine learning-based automated driving systems. The majority of our time in this mission will be spent studying reinforcement learning (RL) algorithms. We chose to learn because the special role of reinforcement learning gives it an advantage in the creation of autonomous driving models. We must determine which algorithm to use in our project since RL belongs to several algorithms. We tested several methods during this year, like DQN, SAC, etc. We finally decided to use Proximal Policy Optimization (PPO) to develop the self-driving model after becoming acquainted with RL and conducting research. The platform that we need to build and test the self-driving model in is a simulator named CARLA (Car Learning to Act). CARLA is an open-source simulator built with Unreal Engine 4 by Intel Visual Computing Lab for autonomous driving cars testing. For training, the simulation platform provides free models, such as vehicle models and maps. The client, which is written in Python, will enable the autonomous driving system to communicate with the environment in the CARLA server. CARLA server is capable of simulating real-world elements such as lights, weather, and complex actors.