Distance Finding in Mobile Autonomous Systems
By: Damien Clarke
Lead Consultant, Data Exploitation
7th November 2018
You might already be familiar with FiveAI, a driverless car startup based in Cambridge, and their recent work making headlines but for those that aren’t, allow me to bring you up to speed. FiveAI’s vision is to bring a shared driverless taxi service to London by next year and they have already started gathering data of London’s streets with their iconic blue branded cars.
A key component in the development of mobile autonomous systems is the ability to produce a 3D map of the local environment which can be used for route planning and collision avoidance (e.g. sense and avoid). There are various sensors which can be used to achieve this and each one has specific advantages and disadvantages.
Stereo Vision
The first approach (and one that many animals, including humans, use) is to combine images from a pair of cameras placed at slightly different positions to enable depth perception. This is achieved by determining the horizontal disparity between the same object in both cameras. Nearby objects produce a large disparity in position between the two cameras whereas far objects have a small disparity.
This technique can also be used with a single camera if it is moving as the video is effectively a series of images taken at different positions. This is known as Structure from Motion and is commonly used with airborne cameras, such as those used on small consumer drones.
The primary advantage of this technique is that cameras are small and inexpensive. At close range, good depth resolution can be achieved and fused with the image content itself to produce a 3D colour image. A large number of cameras with overlapping fields of view can potentially produce a 360° panoramic 3D map of the environment around a vehicle.
The main limitation of this approach is that it will only work when suitable images can be produced and therefore adverse environmental conditions (e.g. dust, fog, rain, etc.) will prevent the production of a 3D map. Operation at night time is potentially possible with illumination or the use of thermal imagers rather than standard cameras. Poor camera dynamic range can also be a problem as bright lights (e.g. headlights or the sun) will cause glare. In addition, the processing required to locate features within both images and match them is complex and adds computational burden when using this technique to produce a 3D map.
Lidar
An alternative optical approach to stereo vision is a scanning laser range finder, also known as lidar. This approach uses a laser to send a pulse towards a surface and a sensor to record how long it takes for the reflection to return. The measurement of the time of flight can then be used to determine the range. To produce a 3D map of a scene, this beam must then be scanned in azimuth and elevation. To reduce the amount of scanning, some lidar sensors use multiple beams at different elevation angles and then only scan in azimuth.
Lidar has very good depth resolution and due to the narrow beam can also produce very good lateral resolution. In general, the technology for emitting and sensing light is entirely solid state, however, at present many lidar system still use the mechanical method to scan the beam across the scene. Fully solid state systems would be small and cheap, though this promise has not yet been fully realised in commercial lidar systems which are often large and expensive.
As simple lidar sensors only record the time for the first reflection to return, a drawback of some lidar systems is that they will only detect the nearest object in a specific direction. This is problematic when the environment is dusty or foggy as the first reflection may not be from a solid object and the resulting 3D map will be degraded. More sophisticated (and costly) systems measure the entire reflection over time which then allows a full range profile to be measured through the obscurant. Direct sunlight can also produce problems as the large level of background illumination can make it difficult to detect weak reflections. Similarly, if a surface has low reflectivity (i.e. it is black) then it may not be detected by the lidar. This can be a problem for autonomous vehicles as black car surfaces will only be detected at a closer range than more reflective vehicles.
Radar
Radar is similar to lidar but uses microwaves rather than light (typically 25 or 77 GHz). Lidar was in fact inspired by radar (e.g. laser radar) and only became possible once lasers were invented. The exact mechanism by which the distance is measured varies slightly between different radar systems; however, the concept is the same. A signal is emitted, the length of time it takes for a reflection to return is measured and this is then converted into a range profile. While panoramic mechanically scanned radars are available, it is more common to use an antenna array and calculate the angle of arrival of a reflection by the difference in signal across the array.
One advantage of radar is the ability to measure speed directly via Doppler shift without complex processing. Therefore, objects moving relative to a mainly static scene are generally easy to detect. Poor environmental conditions (e.g. fog, rain and snow) have little impact on the performance of the radar which provides a useful all-weather capability for autonomous vehicles. Single chip radars with integrated processing capabilities are also available for use as small and inexpensive sensor solutions.
A disadvantage of radar is the limited lateral resolution. While the depth resolution can be good, the angular resolution is significantly lower than for optical sensors. However, this is partially mitigated if an object can be uniquely separated from other objects and clutter by its range or velocity value.
Ultrasonic
The final sensor used for range finding on autonomous vehicles is an ultrasonic sensor which emits high-frequency sounds beyond the range of human hearing. Bats are, of course, well-known users of this approach. Ultrasonic sensors are very similar to lidar sensors; however, as the speed of sound in air is vastly slower than the speed of light it is much easier to measure the time for a reflection to return from a surface.
Ultrasonic sensors work well regardless of light level or environmental conditions and are very small and inexpensive. This makes the technology ideal for ultra-short range collision avoidance sensors on small or slow moving vehicles which can be placed in many locations to provide wide area coverage.
The main disadvantage of ultrasonic sensors is their extremely short range as they can only produce distance measurements for surfaces up to a few metres away. For this reason, it is also uncommon for an ultrasonic sensor to be used to explicitly form a 3D map.
Data Fusion
In practice, to achieve a robust and effective sensor solution for autonomous vehicles it is necessary to combine different sensors and perform sensor fusion. As yet there is no standard sensor suite and research is still ongoing to determine the optimum combination with an acceptable performance across all weather conditions.
Furthermore, as an example, Tesla’s latest models that are claimed to be suitable for autonomous operation have eight cameras (with varying fields of view) and twelve ultrasonic sensors to enable panoramic sensing while a single forward-looking radar measures range and speed of objects up to 160m away.
The combination of cameras with radar is a common sensor choice as it provides good lateral and range resolution under various weather conditions for a relatively low price. It remains to be seen whether or not it is sufficient for safe autonomous operation without the addition of lidar.
