Neural Networks: Can Androids Recognise Electric Sheep?
By: Damien Clarke
Senior Consultant, Data Exploitation
20th September 2017
In 2010, Lt. Gen. David Deptula, the US Air Force deputy chief of staff for intelligence, was quoted as saying:
“We’re going to find ourselves, in the not too distant future, swimming in sensors and drowning in data.”
Since then, this flood of data is showing no signs of slowing. In fact, it is actually accelerating as greater volumes of data are being generated every day. This is just as true in the civilian context as the military context.
For organisations with access to these large volumes of data, it would be profitable to employ data exploitation techniques to convert the raw data into useful information. This can sometimes be achieved by developing custom data processing techniques for specific situations. However, in many cases, it is better to use machine learning techniques to allow computers to learn from data without being explicitly programmed. At Plextek, we’re passionate about developing and implementing the right data exploitation techniques for the application and are working to ensure that humanity stays afloat (and dry) in Deptula’s prediction.
There is a wide range of potential machine learning techniques to choose from, but one approach is to copy nature and mimic biological brains. This was inspired by the fact that one of the primary purposes of a brain is to process sensory inputs and extract useful information for future exploitation. A biological brain can be produced in software form by modelling a large set of connected neurons. This is an artificial neural network.
How does an artificial neural network work?
The basic building block of a neural network is a single neuron. A neuron transforms a set of one or more input values into a single output by applying a mathematical function to the weighted sum of input values. This output value is then passed to one or more connected neurons to be used as a subsequent input value.
The neural network as a whole can, therefore, be defined by three sets of parameters:
– The weight assigned to each input value for each neuron.
– The function which converts the weighted sum of input values into the output value.
– The pattern of connections between neurons.
A simple example neural network consists of three layers. The first layer contains the input values which represent the data being analysed. This layer is then connected to a hidden layer of neurons. The hidden layer then connects to the third and final layer which contains the output neurons whose values represent the processed data. This design allows a complicated relationship between inputs and outputs.
How is a neural network trained?
Just like biological brains, simply creating a neural network is not sufficient to extract information from raw data. It is also necessary to train the network by exposing it to data for which the desired outputs are already known. This process is used to define the weights assigned to each connection throughout the entire network.
As the size and complexity of the neural network increases, the number of weights that must be defined for optimum performance increases significantly. This training process, therefore, requires a large and representative set of labelled data; otherwise, the neural network may not work successfully on future input data. Also, this training process is computationally challenging and may take significant processing time to perform. GPU acceleration can be used to mitigate this; however, the process may still take days for very large data sets.
Conversely, if large volumes of suitable training data are available, it is possible to create a more complex neural network to improve performance. This can be achieved by increasing the number of hidden layers and therefore the total number of connections within the network. This use of complex neural networks with many layers and connections is called deep learning.
What can neural networks be used for?
With a sufficiently large neural network and suitable training data, it is possible to learn complex non-linear relationships between input and output values. This can reveal insights into data which are not possible when using simple linear mathematical models.
While neural networks are suitable as general purpose problem solvers, they are particularly suited for tasks when an understanding of the underlying relationships in the raw data is neither available nor necessarily required and sufficient data is also available for training.
An important example of this capability is the recognition of objects in images. This is achieved through the use of a neural network which has been trained on a large volume of photos of known objects (e.g. ImageNet). While the training process can take a long time, subsequent object recognition is much faster and can potentially be performed in real time. Due to the large volume of training data and the complexity of the neural network used the resulting object recognition performance is close to human level performance. This can be used to in a military context to recognise different vehicles (e.g. a tank) or in a civilian context to see if computers can distinguish between different animals (Do Androids Dream of Electric Sheep?).
Neural networks are not just limited to processing photos and the same approach can be applied to a wide range of sensor and non-sensor data. The most important requirement is that a suitable volume of labelled training data is available to train the network before it can be used on unknown data.
