Neural Networks: Can Androids Recognise Electric Sheep?

Neural Networks: Can Androids Recognise Electric Sheep?

Damien Clarke - Senior Consultant, Data Exploitation

By: Damien Clarke
Senior Consultant, Data Exploitation

20th September 2017

Home » Connected

In 2010, Lt. Gen. David Deptula, the US Air Force deputy chief of staff for intelligence, was quoted as saying:

“We’re going to find ourselves, in the not too distant future, swimming in sensors and drowning in data.”

Since then, this flood of data is showing no signs of slowing. In fact, it is actually accelerating as greater volumes of data are being generated every day. This is just as true in the civilian context as the military context.

For organisations with access to these large volumes of data, it would be profitable to employ data exploitation techniques to convert the raw data into useful information. This can sometimes be achieved by developing custom data processing techniques for specific situations. However, in many cases, it is better to use machine learning techniques to allow computers to learn from data without being explicitly programmed. At Plextek, we’re passionate about developing and implementing the right data exploitation techniques for the application and are working to ensure that humanity stays afloat (and dry) in Deptula’s prediction.

There is a wide range of potential machine learning techniques to choose from, but one approach is to copy nature and mimic biological brains. This was inspired by the fact that one of the primary purposes of a brain is to process sensory inputs and extract useful information for future exploitation. A biological brain can be produced in software form by modelling a large set of connected neurons. This is an artificial neural network.

How does an artificial neural network work?

The basic building block of a neural network is a single neuron. A neuron transforms a set of one or more input values into a single output by applying a mathematical function to the weighted sum of input values. This output value is then passed to one or more connected neurons to be used as a subsequent input value.

The neural network as a whole can, therefore, be defined by three sets of parameters:

  The weight assigned to each input value for each neuron.

  The function which converts the weighted sum of input values into the output value.

  The pattern of connections between neurons.

A simple example neural network consists of three layers. The first layer contains the input values which represent the data being analysed. This layer is then connected to a hidden layer of neurons. The hidden layer then connects to the third and final layer which contains the output neurons whose values represent the processed data. This design allows a complicated relationship between inputs and outputs.



How is a neural network trained?

Just like biological brains, simply creating a neural network is not sufficient to extract information from raw data. It is also necessary to train the network by exposing it to data for which the desired outputs are already known. This process is used to define the weights assigned to each connection throughout the entire network.

As the size and complexity of the neural network increases, the number of weights that must be defined for optimum performance increases significantly. This training process, therefore, requires a large and representative set of labelled data; otherwise, the neural network may not work successfully on future input data. Also, this training process is computationally challenging and may take significant processing time to perform. GPU acceleration can be used to mitigate this; however, the process may still take days for very large data sets.  

Conversely, if large volumes of suitable training data are available, it is possible to create a more complex neural network to improve performance. This can be achieved by increasing the number of hidden layers and therefore the total number of connections within the network. This use of complex neural networks with many layers and connections is called deep learning.



What can neural networks be used for?

With a sufficiently large neural network and suitable training data, it is possible to learn complex non-linear relationships between input and output values. This can reveal insights into data which are not possible when using simple linear mathematical models.

While neural networks are suitable as general purpose problem solvers, they are particularly suited for tasks when an understanding of the underlying relationships in the raw data is neither available nor necessarily required and sufficient data is also available for training.

An important example of this capability is the recognition of objects in images. This is achieved through the use of a neural network which has been trained on a large volume of photos of known objects (e.g. ImageNet). While the training process can take a long time, subsequent object recognition is much faster and can potentially be performed in real time. Due to the large volume of training data and the complexity of the neural network used the resulting object recognition performance is close to human level performance. This can be used to in a military context to recognise different vehicles (e.g. a tank) or in a civilian context to see if computers can distinguish between different animals (Do Androids Dream of Electric Sheep?).



Neural networks are not just limited to processing photos and the same approach can be applied to a wide range of sensor and non-sensor data. The most important requirement is that a suitable volume of labelled training data is available to train the network before it can be used on unknown data.

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

In 2010, Lt. Gen. David Deptula, the US Air Force deputy chief of staff for intelligence, was quoted as saying:

“We’re going to find ourselves, in the not too distant future, swimming in sensors and drowning in data.”

Since then, this flood of data is showing no signs of slowing. In fact, it is actually accelerating as greater volumes of data are being generated every day. This is just as true in the civilian context as the military context.

For organisations with access to these large volumes of data, it would be profitable to employ data exploitation techniques to convert the raw data into useful information. This can sometimes be achieved by developing custom data processing techniques for specific situations. However, in many cases, it is better to use machine learning techniques to allow computers to learn from data without being explicitly programmed. At Plextek, we’re passionate about developing and implementing the right data exploitation techniques for the application and are working to ensure that humanity stays afloat (and dry) in Deptula’s prediction.

There is a wide range of potential machine learning techniques to choose from, but one approach is to copy nature and mimic biological brains. This was inspired by the fact that one of the primary purposes of a brain is to process sensory inputs and extract useful information for future exploitation. A biological brain can be produced in software form by modelling a large set of connected neurons. This is an artificial neural network.

How does an artificial neural network work?

The basic building block of a neural network is a single neuron. A neuron transforms a set of one or more input values into a single output by applying a mathematical function to the weighted sum of input values. This output value is then passed to one or more connected neurons to be used as a subsequent input value.

The neural network as a whole can, therefore, be defined by three sets of parameters:

  The weight assigned to each input value for each neuron.

  The function which converts the weighted sum of input values into the output value.

  The pattern of connections between neurons.

A simple example neural network consists of three layers. The first layer contains the input values which represent the data being analysed. This layer is then connected to a hidden layer of neurons. The hidden layer then connects to the third and final layer which contains the output neurons whose values represent the processed data. This design allows a complicated relationship between inputs and outputs.



How is a neural network trained?

Just like biological brains, simply creating a neural network is not sufficient to extract information from raw data. It is also necessary to train the network by exposing it to data for which the desired outputs are already known. This process is used to define the weights assigned to each connection throughout the entire network.

As the size and complexity of the neural network increases, the number of weights that must be defined for optimum performance increases significantly. This training process, therefore, requires a large and representative set of labelled data; otherwise, the neural network may not work successfully on future input data. Also, this training process is computationally challenging and may take significant processing time to perform. GPU acceleration can be used to mitigate this; however, the process may still take days for very large data sets.  

Conversely, if large volumes of suitable training data are available, it is possible to create a more complex neural network to improve performance. This can be achieved by increasing the number of hidden layers and therefore the total number of connections within the network. This use of complex neural networks with many layers and connections is called deep learning.



What can neural networks be used for?

With a sufficiently large neural network and suitable training data, it is possible to learn complex non-linear relationships between input and output values. This can reveal insights into data which are not possible when using simple linear mathematical models.

While neural networks are suitable as general purpose problem solvers, they are particularly suited for tasks when an understanding of the underlying relationships in the raw data is neither available nor necessarily required and sufficient data is also available for training.

An important example of this capability is the recognition of objects in images. This is achieved through the use of a neural network which has been trained on a large volume of photos of known objects (e.g. ImageNet). While the training process can take a long time, subsequent object recognition is much faster and can potentially be performed in real time. Due to the large volume of training data and the complexity of the neural network used the resulting object recognition performance is close to human level performance. This can be used to in a military context to recognise different vehicles (e.g. a tank) or in a civilian context to see if computers can distinguish between different animals (Do Androids Dream of Electric Sheep?).



Neural networks are not just limited to processing photos and the same approach can be applied to a wide range of sensor and non-sensor data. The most important requirement is that a suitable volume of labelled training data is available to train the network before it can be used on unknown data.

Save

Save

Save

Save

Save

Save

Save

Save

Further Reading

All 'Things' to be considered in IoT

All ‘Things’ to be Considered in IoT

Richard Emmerson - Senior Consultant, Communications Systems

By: Richard Emmerson
Senior Consultant, Communications Systems

3rd May 2017

Home » Connected

So you have a great idea for an internet connected ‘Thing’. You’ve done the business plan, you’ve raised some investment, or maybe you’re staking your own money. All you have to do now is connect your ‘Thing’ to ‘The Internet of Things (IoT)’ and get the product into the market.

Well, there are a few things you should consider before you jump in.

How will the ‘Thing’ connect?

Surely that’s simple, everyone’s using LoRa (long-range, low-power radio), so I can buy some LoRa modules, connect them to my ‘Thing’ and I’m done.

Well, yes and no.

Range, Data Rate & Power Consumption

With any communications system, there is a direct trade-off between range, data rate and power consumption. LoRa is potentially a great system for IoT. When properly designed, it can achieve long range (typically 2 km in urban areas and line of sight in rural areas) and have a battery life that can last for years. However, data rates are limited to between 0.3 kbps to 50 kbps; with the longest range achieved at the lowest data rate. In the EU 868 MHz band, the duty cycle is also limited to 1%, meaning, at the lowest data rate, only 51 user bytes can be sent every 245 seconds. This is fine for a smoke alarm but unsuitable for a security camera.

For higher data rate applications, a 3G or 4G modem module may be a better choice, provided the power is available. For power limited systems, there is also the Narrowband IoT system, which uses the 4G mobile network with data rates between 20-250 kbps and offers impressive battery life.

What about base stations?

BaseStationLoRa can be used in a peer-to-peer mode (communication between nodes). To connect to the internet though requires some kind of base station. This could be a local base station installed in the home or office, or a wide area base station (LoRa-WAN). You might choose to supply customers with their own low-cost base stations, or take advantage of public networks such as ‘The Things Network’.

An alternative may be to use the ’Sigfox’ system. This has similar performance to LoRa but for a small subscription fee accesses an international network of base stations owned and managed by Sigfox. Unlike Sigfox and cellular systems, LoRa has the advantage that if there is no coverage then you can simply add your own base station.

What about the Antenna?

AntennaBoardThe antenna is a key part of any wireless system and is an area where many developers face problems. In order to work efficiently, antennas need an effective area which is made up of the antenna itself and the circuit board it is connected to. Look carefully at the datasheet for that tiny 868 MHz ‘chip’ antenna and you are likely to see that it requires a PCB of approximately 90 mm length.

However, this poses a problem for small devices operating at 868 MHz, as the antenna is unlikely to be efficient, and that 2km range you expected just reduced to 500 m or less. The antenna may also become de-tuned by the presence of breaks in the PCB ground plane, nearby components and caseworks require a matching network to compensate for these effects. For really small devices, it may be worth considering Bluetooth, which with its higher operating frequency of 2.4 GHz requires a smaller PCB, and, with the release of Bluetooth 5, can be used for local area networks.

So that’s it?

Well, not quite. LoRa and Sigfox use the licence free 868 MHz ISM band in Europe and 915 MHz band in the US. Both of which are prone to interference from other users. There is also the platform, encryption, data ownership, and regulatory approvals to consider.

So you have a great idea for an internet connected ‘Thing’. You’ve done the business plan, you’ve raised some investment, or maybe you’re staking your own money. All you have to do now is connect your ‘Thing’ to ‘The Internet of Things (IoT)’ and get the product into the market.

Well, there are a few things you should consider before you jump in.

How will the ‘Thing’ connect?

Surely that’s simple, everyone’s using LoRa (long-range, low-power radio), so I can buy some LoRa modules, connect them to my ‘Thing’ and I’m done.

Well, yes and no.

Range, Data Rate & Power Consumption

With any communications system, there is a direct trade-off between range, data rate and power consumption. LoRa is potentially a great system for IoT. When properly designed, it can achieve long range (typically 2 km in urban areas and line of sight in rural areas) and have a battery life that can last for years. However, data rates are limited to between 0.3 kbps to 50 kbps; with the longest range achieved at the lowest data rate. In the EU 868 MHz band, the duty cycle is also limited to 1%, meaning, at the lowest data rate, only 51 user bytes can be sent every 245 seconds. This is fine for a smoke alarm but unsuitable for a security camera.

For higher data rate applications, a 3G or 4G modem module may be a better choice, provided the power is available. For power limited systems, there is also the Narrowband IoT system, which uses the 4G mobile network with data rates between 20-250 kbps and offers impressive battery life.

What about base stations?

BaseStationLoRa can be used in a peer-to-peer mode (communication between nodes). To connect to the internet though requires some kind of base station. This could be a local base station installed in the home or office, or a wide area base station (LoRa-WAN). You might choose to supply customers with their own low-cost base stations, or take advantage of public networks such as ‘The Things Network’.

An alternative may be to use the ’Sigfox’ system. This has similar performance to LoRa but for a small subscription fee accesses an international network of base stations owned and managed by Sigfox. Unlike Sigfox and cellular systems, LoRa has the advantage that if there is no coverage then you can simply add your own base station.

What about the Antenna?

AntennaBoardThe antenna is a key part of any wireless system and is an area where many developers face problems. In order to work efficiently, antennas need an effective area which is made up of the antenna itself and the circuit board it is connected to. Look carefully at the datasheet for that tiny 868 MHz ‘chip’ antenna and you are likely to see that it requires a PCB of approximately 90 mm length.

However, this poses a problem for small devices operating at 868 MHz, as the antenna is unlikely to be efficient, and that 2km range you expected just reduced to 500 m or less. The antenna may also become de-tuned by the presence of breaks in the PCB ground plane, nearby components and caseworks require a matching network to compensate for these effects. For really small devices, it may be worth considering Bluetooth, which with its higher operating frequency of 2.4 GHz requires a smaller PCB, and, with the release of Bluetooth 5, can be used for local area networks.

So that’s it?

Well, not quite. LoRa and Sigfox use the licence free 868 MHz ISM band in Europe and 915 MHz band in the US. Both of which are prone to interference from other users. There is also the platform, encryption, data ownership, and regulatory approvals to consider.

Save

Further Reading

Save