Member-only story
Teaching a Neural Network to play the Breakout game
When I was a kid, I loved playing the Arkanoid game on the IBM PC XT machine. Many years later, I became interested in whether the artificial neural network can be trained to play the similar game. Let's figure it out.
Reinforcement Learning
The idea of reinforcement learning (RI) is not new. In the 40s Alan Turing described the machine which has two inputs, the “reward” and the “punishment”. In the 50s this idea evolved into a concept of the Markov decision process (MDP), where the interconnections between states, actions and rewards were formalized. In the 60s Richard Bellman developed the famous Bellman’s equation, which now can be seen in almost every paper about RI. The main idea of Reinforcement Learning is that the Agent has its current State, and at each step, it is trying to perform some Actions. The agent’s task is to take the best action, which will help to maximize the Reward. During the learning process, the Agent is trying different actions and ‘remembering’ the results. In the same way as for humans, after some training, the Agent will be able to get some knowledge about which Action is better to do in different situations. Different ways of creating the Agent are available, for example, a so-called “Q-table” can be used, in which the “state, action, reward” triplets are just stored in a table. It works well if the state can be encoded using a small number of variables, for example, in the CartPole experiment:
In this task, the pole is attached to a cart, which moves along a frictionless track. The cart can move left or right, the goal is to prevent the pendulum from falling. The state is described only by 4 variables (cart position, cart velocity, pole angle, pole velocity), and keeping this data in memory is easy. But if we are talking about video games, we don’t have an internal state. Of course, it is easy to rewrite a Breakout-like game from scratch, then controlling the cart will be an easy task. But the challenge is to train the Agent to play like a human, only by watching the game! In this case, if we have a video game with a 640x480 resolution, the current state of a game frame…
