How To Use Neural Network In Gaming?
Face Recognition, Speech Recognition, Handwriting Recognition, just a few of the tasks from many tasks that we as humans are able to quickly solve. We seem to be able, to perform these tasks effortlessly, that is in some cases impossible for even the most sophisticated computer programs to solve that tasks. Then the question arises is “What is the difference between computer and human?”.
Here, we are not going to take more discussion on it but going to take an introductory look at one aspect of it. A massive parallel network of simple computation units forms by the biological structure of the human brain that has been trained to solve problems quickly. When this network simulated on a computer is known as artificial neural network or neural net for short.
In this blog, we are going to get some introductory part of neural networks and how we used a neural network in gaming. It is an interesting area because gaming technology has always been an attractive area.
Let’s start with the introduction part of the Neural Network…
A neural network is a system of software/hardware patterned like a human’s brain neurons. A Neural network is also known as Artificial Neural Networks (ANN). It is a type of deep learning, a concept of machine learning. Commercial applications of these technologies generally focus on solving complex pattern recognition problems.
It is a way to build a computer program that learns from data. It is based on how we think the human brain works.
A neural network made when a collection of software called “neurons” are created and connected together. It allows them to send messages to each other. After that, the network asked to solve a particular problem, which it attempts to do again and again. The strengthening of connections each time leads to success and diminishing those that lead to failure.
We can also understand the Neural Networks/Artificial Neural Network as-
Neural networks are typically made of layers (Input Layer, Hidden Layer, Output Layer). Layers are made up of a number of interconnected ‘nodes’ that contains an ‘activation function’. Patterns are presented to the network through ‘input layer’. This input layer communicates with one or more ‘hidden layers’. The actual processing is done through a system of weighted ‘connections’. The hidden layers then link to an ‘output layer’ and the output comes as:
Most Artificial Neural Networks contains learning rule that modifies the weights of the connections according to the input patterns that it is presented with. Artificial Neural Network learns by example as do their biological counterparts. A child learns to recognize dogs by learning.
There are different learning rules used in an artificial neural network. Some name s are as follows…
- Hebbian Learning Rule
- Perceptron Learning Rule
- Delta Learning Rule
- Correlation Learning Rule
- Out Star Learning Rule
Some applications where neural network used, are as follows:
- It is used to capture associations or discover regularities within a set of patterns.
- the relationships between variables are vaguely understood.
- It is used where the relationships are difficult to describe adequately with conventional approaches.
- where the volume, number of variables or diversity of the data is very high.
- Now, comes on how neural network used in gaming…
Here, I am going to talk about a small practice of using neural networks that are based on training one to play a Snake Game. This practice is for beginners. It would be good if you know something about machine learning, neural networks, and TensorFlow but there is no problem if you don’t know.
This practice consists of three phases.:
- Phase 1
- Phase 2
- and Phase 3
Let’s start with phase 1…
THE GAME PHASE
Firstly, we need to write a game that will have a 3 pieces snake at the starting, a 20×20 field, a randomly generated apple in each moment of time and API to use with the network.
Here is the code for the game:
The game would be like:
Now let’s start with a neural network.
This phase consist of 4 steps are as follows:
- Input data
- Architecture of neural network
Here, we need to create features to teach our snake and to make our snake smart by giving some knowledge data. Always try to choose most useful features because too many features arise a problem that makes hard for a network to decide which are more important and learning will be longer. It is important to add features through which a network will get enough information to be good but if you are not adding enough features, it makes difficult for a network to get good information.
At the first step, we will focus on the snake that how it survive and will not think about apples. To choose a right direction it should know if there are any hurdle around it. On the basis of these hurdle/obstacle and suggested direction the network will decide that it is a good action or not.
We use an array of 4 numbers for the inputs of our neural network:
- If, there is an obstacle to the left of the snake then(1 — yes, 0 — no)
- There is an obstacle in front of the snake then (1 — yes, 0 — no)
- If, there is an obstacle to the right of the snake then (1 — yes, 0 — no)
- and For Suggested direction (-1 — left, 0 — forward, 1 — right)
With these inputs we want some output that would be 1 or 0:
- 1 – Means we should go in the selected direction.
- 0 -Means we should choose another one.
A neural network needs some data to learn. Data is the key element of the machine learning if you have a huge amount of data then you can achieve great results even if you are not having a good architecture. In this time big companies focus on to get all the information that they can get because big data matters to get good results.
We need to create data randomly by choosing a direction and observe the snake is still alive after a turn.
By playing different numbers of a game we collect data around 4500 which is enough for training to survive.
Architecture of Neural Network
A neural network is always hard to choose. It always depends on the task that you trying to solve. In this, you need to choose the number of layers, number of neurons in layers, and types of neurons. It is better to try a different variety and choose one of the best fits more than others.
So, to achieve our task we need only Input Layer and Output Layer there is no need to use Hidden Layers.
In TensorFlow it will look like (I used TFLearn: Deep learning library featuring a higher-level API for TensorFlow):
Here is the full code:
With each turn, we give three arrays of possible actions to the network. Then network chooses one action with better output. After training the snake with data it chose the easiest way to survive:
This phase consist of 5 steps are as follows:
- New feature
- A new output
- New architecture
- First results
- Let’s make snake great again
In this step, our snake having the knowledge about how to survive. So it is a time to think about apples and teach our snake that how to find an apple by adding new features. We can choose an angle between the movement of the snake and direction to an apple.
It is a good approach to normalize our features. If an apple is to the left of snake the number will be positive(+) and if it is to the right then it will be negative(-). In our case, we need to divide the angle by 180 degrees so the number will be from -1 to 1.
We use an array of 5 numbers for the inputs of our neural network:
- Is there an obstacle to the left of the snake (1 — yes, 0 — no)
- an obstacle in front of the snake (1 — yes, 0 — no)
- Is there an obstacle to the right of the snake (1 — yes, 0 — no)
- A normalized angle between the direction of an apple and the direction of the snake’s movement (from -1 to 1)
- Suggested direction (-1 — left, 0 — forward, 1 — right)
In the previous phase, we focused on that the snake is alive after taking a turn or not but in this phase we focus on that the turn was successful or not rather than observing the snake is alive or not. To perform this task we will calculate a distance between snake’s head and an apple before & after the turn of our snake.
In this way, we get the new output as:
- -1 if the snake didn’t survive
- 0 if the snake survived but the direction is wrong
- 1 if the snake survived and the direction is right
After 6805 initial games I have got:
- 2947 turns with a right direction
- 2800 turns with a wrong direction
- 1058 wrong turns
Now, we need to change its architecture to see that is it enough data to teach our snake or not.
With the previous step, we have lots of complicated data and having more options for the output. So, we need to select a neural network architecture in addition to Hidden layer with 25 neurons along with input layer and an output layer.
After learning the snake represents results in 1000 test games as follows:
- Average number of steps —166.61
- Average number of points — 12.171
It looks like:
We can do it better. Let’s look at some inputs when the snake was wrong:
- [ 1, 1, 1, 0.36420025]
- [ 1, 1, 0, 0.04516724] and
- and [ 1, 0, 1, 0.2912856]
where numbers represent:
- an obstacle to the left
- an obstacle in the front
- an obstacle to the right
- angle to an apple
In the first example, there was nothing to do for the snake with the input data.In the next examples, there was a chance to survive and the snake chose the wrong action. What can we do with it?
Let’s improve snake again
There are different things that can change your network (an architecture, features, learning rate, and the number of input samples). If you can get more input data it’s always worth trying to use it. In our case, it is very easy to generate more games. Let’s try 100000 games.
- Average number of steps — 398.959
- Average number of points — 25.333
When the snake dies all the inputs will look like [1, 1, 1, x]. It means the snake dies only when there is no other way to survive.
A neural network is a wide concept of machine learning and used in different real-world applications to make work easier. One of the most interesting application is gaming. In gaming application neural network used to provide knowledge to the model and make them smart by learning. It helps to make a model like human’s mind that helps to take decisions or we can say right decisions.
Here, I used features for our neural network, an architecture and then got some input data. From collected input data network choose the best result for a given features.
In this practice, you can see that how a snake can be knowledgeable by giving some knowledge or knowledge data. It is a small practice on a neural network we can make more complex model using a neural network.
I think this will give you some basic knowledge about Neural Network or Artificial Neural Network (ANNs) and how it is used in gaming technology.