Home Projects Overcoming Training Data Bottlenecks in Thermal Imaging with Transfer Learning

The Challenge

AI object detection models for colour imagery are widely available. These have been trained on large quantities of imagery that was painstakingly manually labelled. Similar performance should be achievable with an AI model trained on thermal imagery, if only such a labelled data set existed. Manually labelling a custom data set is possible but time consuming. For example, producing the well-known Common Objects in Context (COCO) dataset took 22 hours per 1000 segmented objects.

The challenge: develop a method to create large-scale labelled thermal image datasets without the high cost of manual annotation.

The Approach​

Plextek developed an automated approach: by co-locating a colour camera and thermal imager with the same field of view, we collected many synchronised image pairs of an identical scene during the day. An AI object recognition model was then used to provide labelled boxes for the scene content in the colour images. This produced 12,392 labelled boxes across 4639 images. Due to the AI model producing performance comparable to a human under well-lit conditions, these labelled boxes could be transferred to the thermal image and used as a source of reliable ground truth. This only required that the colour image was registered to the thermal image so that the pixel positions were equivalent.

Overcoming Training Data Bottlenecks in Thermal Imaging with Transfer Learning
Images show: Two colour and thermal image pairs with object detection bounding boxes produced from the colour images only. The boxes on the thermal images were then used to retrain the AI model to operate directly on the thermal images.
Two colour and thermal image pairs with object detection bounding boxes produced from the colour images only. The boxes on the thermal images were then used to retrain the AI model to operate directly on the thermal images.

 

The resulting labelled thermal images were then used to train an AI model to detect and classify objects within thermal images. While these images could be used to train an image from scratch, a better approach was to use transfer learning to retrain an existing AI model that works on colour images. This had the advantage of leveraging the hundreds of thousands of colour images used to train the AI model to understand edges and textures within a general image. The much smaller number of IR images was then used to teach the AI model how specific patterns of IR edges and textures corresponds to particular objects of interest (i.e. the labelled boxes).

The Outcome

After retraining the AI model on thermal images, its performance was assessed on imagery collected at night, which wasn’t in the training data set. The presence of streetlights meant that the colour camera could still be used to detect objects within the lit parts of the image, however, there were some false negatives when objects moved within less well-lit regions. There were also misclassifications, such as a bollard being classified as a person.

Left: A colour image at night. Middle: A thermal image with bounding boxes produced from the colour image. Right: A thermal image with bounding boxes produced directly from the thermal image itself using the retrained AI model.
Left: A colour image at night.
Middle: A thermal image with bounding boxes produced from the colour image.
Right: A thermal image with bounding boxes produced directly from the thermal image itself using the retrained AI model.

In contrast, the retrained AI model successfully detected people in thermal images with significantly reduced false alarms. There were also fewer misclassifications as bollards do not look like people in thermal imagery.

This improved performance is not unexpected as thermal cameras should work better than a colour camera at night, even with streetlights. However, the key benefit lay in the fact that the ground truth data used for training the AI model on thermal images was generated automatically without a significant manpower requirement. This addresses an important bottleneck in developing automated AI systems which could even prevent an AI model being developed at all.


Partner with Plextek for AI and Sensor Integration

Exceptional Engineering

Plextek combines deep expertise in multi-sensor systems integration with AI and Transfer Learning. Our end-to-end approach, from constructing combined sensor systems, to training AI models for sensor related tasks and beyond, enables fast accurate development of your AI systems. If you’re facing similar challenges in developing AI models with limited training data, we’d welcome the opportunity to discuss a solution.

Get in touch to discuss your requirements

custom motor control solution 790x634
Field Oriented Control: Precision in Modern Motor Systems

Delivering expertise in field-oriented control for demanding motor control solutions

Radar in Modern Sensing The Power of Sensor Fusion
Radar in Modern Sensing: The Power of Sensor Fusion

Radar Sensor Fusion with Cameras and Lidar: Enhanced Sensing Technology for Autonomous Systems

orbital space debris abstract image
mmWave radar: balancing resolution and practicality in space

mmWave radar offers the ideal balance for space applications, detecting small debris and spacecraft features with minimal power and no moving parts.

How radar works

How Does Radar Work? Understanding Radar Technology and Applications with Richard Jacklin

why radar should be on your radar
Why radar should be on your radar

Space missions demand sensors that work in brutal conditions - and radar works when everything else fails.

Addressing the Escalation in GNSS Spoofing with Data Fusion
Addressing the Escalation in GNSS Spoofing with Data Fusion

Plextek's data fusion technology helps maritime vessels maintain navigation accuracy during increasing GNSS spoofing attacks.

5G/6G and Satellites Global Connectivity Revolution
5G/6G and Satellites: Global Connectivity Revolution

Exploring how the integration of 5G/6G technologies with satellite networks is transforming global connectivity through Non-Terrestrial Networks and Direct-to-Device communication.

Measuring micro-debris in real-time and enabling a safer LEO with next generation space radar
Measuring Micro-Debris in LEO with Next-Gen Space Radar

Detecting micro-debris in real-time is key to safer space operations. Next-gen mmWave radar technology enables high-resolution tracking of even the smallest fragments in LEO, reducing collision risks and enhancing space situational awareness. Discover how this innovation supports a more sustainable orbital future.

Yocto or PetaLinux
Yocto or PetaLinux?

Our comprehensive comparison of PetaLinux and Yocto, focusing on system requirements, build engines, and build outputs for developing embedded systems.

Kevin Cobley discussing 5G and the role of Non-Terrestrial Networks
The Future of Connectivity: 5G, 6G, and Space-Based Networks

As industries push the boundaries of global connectivity, the integration of 5G and drive towards 6G with satellite and space-based networks is unlocking new opportunities. Kevin Cobley, an expert in this evolving field, shares his insights on the challenges and innovations shaping the future of non-terrestrial networks (NTN).

Augmenting UAV Safety with Ubiquitous Radar Technology
Augmenting UAV Safety with Ubiquitous Radar Technology

Enhancing UAV safety with ubiquitous radar tech for detect and avoid capabilities in shared airspace.

Urban Challenges Rapid RF Propagation Modelling
Urban Challenges: Rapid RF Propagation Modelling

Discover the challenges and solutions for accurately modelling RF propagation in urban settings. Explore the innovative neural network model revolutionising urban RF systems.

Related Technical Papers

View All
Understanding the challenges of mobile base-station RU design white paper
Mobile Base-Station Radio Unit Design Challenges

This white paper examines the key technical hurdles facing Radio Unit (RU) designers in this newly competitive landscape.

an image of our technical paper
mmWave Imaging Radar

Camera systems are in widespread use as sensors that provide information about the surrounding environment. But this can struggle with image interpretation in complex scenarios. In contrast, mmWave radar technology offers a more straightforward view of the geometry and motion of objects, making it valuable for applications like autonomous vehicles, where radar aids in mapping surroundings and detecting obstacles. Radar’s ability to provide direct 3D location data and motion detection through Doppler effects is advantageous, though traditionally expensive and bulky. Advances in SiGe device integration are producing more compact and cost-effective radar solutions. Plextek aims to develop mm-wave radar prototypes that balance cost, size, weight, power, and real-time data processing for diverse applications, including autonomous vehicles, human-computer interfaces, transport systems, and building security.

an image of our technical paper
Low Cost Millimeter Wave Radio frequency Sensors

This paper presents a range of novel low-cost millimeter-wave radio-frequency sensors that have been developed using the latest advances in commercially available electronic chip-sets. The recent emergence of low-cost, single chip silicon germanium transceiver modules combined with license exempt usage bands is creating a new area in which sensors can be developed. Three example systems using this technology are discussed, including: gas spectroscopy at stand off distances, non-invasive dielectric material characterization and high performance micro radar.

an image of our technical paper
Ku-Band Metamaterial Flat-Panel Antenna for Satcom

This technical paper by Dr. Rabbani and his team presents research on metamaterial-based, high-gain, flat-panel antennas for Ku-band satellite communications. The study focuses on leveraging the unique electromagnetic properties of metamaterials to enhance the performance of flat-panel antenna designs, aiming for compact structures with high gain and efficiency. The research outlines the design methodology involving multi-layer metasurfaces and leaky-wave antennas to achieve a compact antenna system with a realised gain greater than +20 dBi and an operational bandwidth of 200 MHz. Simulations results confirm the antenna's high efficiency and performance within the specified Ku-band frequency range. Significant findings include the antenna's potential for application in low-cost satellite communication systems and its capabilities for THz spectrum operations through design modifications. The paper provides a detailed technical roadmap of the design process, supported by diagrams, simulation results, and references to prior work in the field. This paper contributes to the advancement of antenna technology and metamaterial applications in satellite communications, offering valuable insights for researchers and professionals in telecommunications.

an image of our technical paper
The Kootwijk VLF Antenna: A Numerical Model

A comprehensive analysis of the historical Kootwijk VLF (Very Low Frequency, which covers 3-30 kHz) antenna, including the development of a numerical model to gain insight into its operation. The Kootwijk VLF antenna played a significant role in long-range communication during the early 20th century. The paper addresses the challenge of accurately modelling this electrically small antenna due to limited historical technical information and its complex design. The main goal is to understand if the antenna’s radiation efficiency might explain why “results were disappointing” for the Kootwijk to Malabar (Indonesia) communications link. Through simulations and comparisons with historical records, the numerical model reveals that the Kootwijk VLF antenna had a low radiation efficiency – about 8.9% – for such a long-distance link. This work discusses additional loss mechanisms in the antenna system that might not have been considered previously, including increased transmission-line losses as a result of impedance mismatch, wires having a lower effective conductivity than copper and inductor quality factors being lower than expected. The study provides insights into key antenna parameters, such as the radiation pattern, the antenna’s quality factor, half-power bandwidth and effective height, as well as the radiated power level and the power lost through dissipation. This research presents the first documented numerical analysis of the Kootwijk VLF antenna and contributes to a better understanding of its historical performance. While the focus has been at VLF, this work can aid future modelling efforts for electrically small antennas at other frequency bands.

an image of our technical paper
The Radiation Resistance of Folded Antennas

This technical paper highlights the ambiguity in the antenna technical literature regarding the radiation resistance of folded antennas, such as the half-wave folded dipole (or quarter-wave folded monopole), electrically small self-resonant folded antennas and multiple-tuned antennas. The feed-point impedance of a folded antenna is increased over that of a single-element antenna but does this increase equate to an increase in the antenna’s radiation resistance or does the radiation resistance remain effectively the same and the increase in feed-point impedance is due to transformer action? Through theoretical analysis and numerical simulations, this study shows that the radiation resistance of a folded antenna is effectively the same as its single-element counterpart. This technical paper serves as an important point of clarification in the field of folded antennas. It also showcases Plextek's expertise in antenna theory and technologies. Practitioners in the antenna design field will find valuable information in this paper, contributing to a deeper understanding of folded antennas.

an image of our technical paper
Chilton Ionosonde Data & HF NVIS Predictions during Solar Cycle 23

This paper presents a comparison of Chilton ionosonde critical frequency measurements against vertical-incidence HF propagation predictions using ASAPS (Advanced Stand Alone Prediction System) and VOACAP (Voice of America Coverage Analysis Program). This analysis covers the time period from 1996 to 2010 (thereby covering solar cycle 23) and was carried out in the context of UK-centric near-vertical incidence skywave (NVIS) frequency predictions. Measured and predicted monthly median frequencies are compared, as are the upper and lower decile frequencies (10% and 90% respectively). The ASAPS basic MUF predictions generally agree with fxI (in lieu of fxF2) measurements, whereas those for VOACAP appear to be conservative for the Chilton ionosonde, particularly around solar maximum. Below ~4 MHz during winter nights around solar minimum, both ASAPS and VOACAP MUF predictions tend towards foF2, which is contrary to their underlying theory and requires further investigation. While VOACAP has greater errors at solar maximum, those for ASAPS increase at low or negative T-index values. Finally, VOACAP errors might be large when T-SSN exceeds ~15.

an image of our technical paper
Antenna GT Degradation with Inefficient Receive Antenna at HF

This paper presents the antenna G/T degradation incurred when communications systems use very inefficient receive antennas. This work is relevant when considering propagation predictions at HF (2-30 MHz), where it is commonly assumed that antennas are efficient/lossless and external noise dominates over internally generated noise at the receiver. Knowledge of the antenna G/T degradation enables correction of potentially optimistic HF predictions. Simple rules of-thumb are provided to identify scenarios when receive signal-to-noise ratios might be degraded.

an image of our technical paper
60 GHz F-Scan SIW Meanderline Antenna for Radar Applications

This paper describes the design and characterization of a frequency-scanning meanderline antenna for operation at 60 GHz. The design incorporates SIW techniques and slot radiating elements. The amplitude profile across the antenna aperture has been weighted to reduce sidelobe levels, which makes the design attractive for radar applications. Measured performance agrees with simulations, and the achieved beam profile and sidelobe levels are better than previously documented frequency-scanning designs at V and W bands.

an image of our technical paper
Midlatitude 5 MHz HF NVIS Links: Predictions vs. Measurements

Signal power measurements for a UK-based network of three beacon transmitters and five receiving stations operating on 5.290 MHz were taken over a 23 month period between May 2009 and March 2011, when solar flux levels were low. The median signal levels have been compared with monthly median signal level predictions generated using VOACAP (Voice of America Coverage Analysis Program) and ASAPS (Advanced Stand Alone Prediction System) HF prediction software with the emphasis on the near-vertical incidence sky wave (NVIS) links. Low RMS differences between measurements and predictions for September, October, November, and also March were observed. However, during the spring and summer months (April to August), greater RMS differences were observed that were not well predicted by VOACAP and ASAPS and are attributed to sporadic E and, possibly, deviative absorption influences. Similarly,the measurements showed greater attenuation than was predicted for December, January, and February, consistent with the anomalously high absorption associated with the “winter anomaly.” The summer RMS differences were generally lower for VOACAP than for ASAPS. Conversely, those for ASAPS were lower during the winter for the NVIS links considered in this analysis at the recent low point of the solar cycle. It remains to be seen whether or not these trends in predicted and measured signal levels on 5.290 MHz continue to be observed through the complete solar cycle.

an image of our technical paper
Electrically small monopoles: Classical vs. Self-Resonant

This paper shows that the Q-factor and VSWR of a monopole are significantly lowered when made resonant by reactive loading (as is used in practice). Comparison with a self-resonant Koch fractal monopole of equal height gives similar values of VSWR and Q-factor. Thus, the various electrically small monopoles (self-resonant and reactively loaded) offer comparable performance when ideal and lossless. However, in selecting the optimum design, conductor losses and mechanical construction at the frequency of interest must be considered. In essence, a trade-off is necessary to obtain an efficient, electrically small antenna suitable for the application in hand.

an image of our technical paper
Ku-Band Low-Sidelobe Waveguide Array

The design of a 16-element waveguide array employing radiating T-junctions that operates in the Ku band is described. Amplitude weighting results in low elevation sidelobe levels, while impedance matching provides a satisfactory VSWR, that are both achieved over a wide bandwidth (15.7-17.2 GHz). Simulation and measurement results, that agree very well, are presented. The design forms part of a 16 x 40 element waveguide array that achieves high gain and narrow beamwidths for use in an electronic-scanning radar system.

Contact Plextek | Employees check their contact emails on a tablet

Got a project in mind?

Let’s talk

If you have got a project to discuss, or even just an idea, let's talk