Deep Reinforcement Learning for Self-Driving Cars — An intro
This article on Deep Reinforcement Learning has been updated on https://www.thinkautonomous.ai/blog/?p=deep-reinforcement-learning-for-self-driving-cars-an-intro
In this email, I explain how Reinforcement Learning is applied to Self-Driving cars. Let’s take a look.
Lately, I have noticed a lot of development platforms for reinforcement learning in self-driving cars.
Voyage Deep Drive is a simulation platform released last month where you can build reinforcement learning algorithms in a realistic simulation.
It looks similar to CARLA.
A simulator is a synthetic environment created to imitate the world. Like a video game. We simulate sensors and can directly work with them, like in a real car.
AWS Deep Racer is a self-driving robot where you can learn and develop reinforcement learning algorithms.
There is a simulation platform but it is primarily a physical robot equipped with cameras.
The purpose of the robot is racing cars and competitions are made by Amazon.
Deep Traffic is a course project launched by MIT where you can try and beat traffic using Deep Reinforcement Learning algorithms and a simple simulator.
If you have never done reinforcement learning before, you can simply watch the course and immediately try the project. My average speed is 70.74 km/h, try to beat me here!
Finally, Wayve video demo is the one I like the most.
You can clearly identify the first and last iterations and visualize how the car learns to steer the wheel to stay in its lane and reach the objective.
All those projects look really fascinating, but what is Reinforcement Learning?
Reinforcement Learning is a learning technique like supervised and unsupervised learning are.
Instead of learning from a labeled dataset (or unlabelled), you learn from mistakes generated by a reward system. We can say the agent learns from experience.
In this technique, an agent (the car) interacts with an environment (the world) to reach an optimal state or objective. To interact, the agent performs an action.
If the action is positive, the reward will be big.
If the action is negative (going on the sidewalk, hurting other cars, …), the reward is negative.
The goal of the agent is to choose the right action that gets the biggest reward.
We have seen a lot of reinforcement learning applied to chess or the game of Go.
In this case, the agent is able to foresee the future actions and states and anticipate which action to take now that maximizes future reward.
In short, we are able to calculate the total reward based on all rewards.
While this email doesn’t aim to teach you reinforcement learning, it’s good to have an overview and understand why a lot of research goes there.
When working with reinforcement learning, you can design an environment and use a reinforcement learning algorithm to optimize the driving policy.
With a lot of learning, you can even work with multiple agents that explore multiple paths at the same time and returns you the optimal one.
In the movie Next, Nicolas Cage is a Multi-Agent Reinforcement Learning algorithm.
How to get started?
- If you are not aware of reinforcement learning yet but know Deep Learning, start with the MIT Course I gave earlier.
- 📜 Also, this Medium series explains things amazingly.
I would say it’s safe to spend a lot of time in these articles and get out with a good understanding. - If you’d like to understand more use cases, Neptute.ai has a great article on 10 Real-Life Applications of Reinforcement Learning which includes Healthcare, NLP, and many more…
- These are really good to get started, then you have to complete projects. You can try Deep Reinforcement Learning Nanodegree by Udacity but I don’t know any review. The good point is you get to do projects and that is essential to every learning process.
Reinforcement Learning is a very complicated topic.
If you don’t know your maths well, it will be hell by week 1.
If you’re a starter in AI, try to do Machine Learning and Deep Learning good and improve your maths first.
If you know AI well, try to do projects and fail a lot.
I have done a lot of courses and still don’t understand the topic well.Yet it’s a fascinating topic and I suggest to at least look from far.
What do you think about RL for autonomous driving?
If you’d like to receive more emails like this one, subscribe here.
These emails will give you access to daily tips on AI & Self-Driving Cars, stories from the inside on autonomous tech companies, promotional offers on my courses and premium invites to online or offline events
It is like a continuous training on autonomous technologies!
Every day, you learn a bit more.
You probably missed today, don’t miss tomorrow’s email!
