(59) Interpolation of AMG8833 infrared sensor data: ESP8266 microcontroller combined with several displays

by Floris Wouterlood – The Netherlands – July 5, 2022

Summary
Here we tested an AMG8833 8*8 pixel thermal infrared sensor wired to an ESP8266 Wemos D1 mini microcontroller board. The thermal scanning sensor delivers temperature data at a resolution of 8*8 pixels. Thermal image data can however be processed with as result an interpolated, 24*24 pixel improved image. The interpolation sketches presented here are adaptations of the example ‘thermal_cam_interpolate’ in the Adafruit AMG88xx library. Interpolation was tested with several displays: 1.6 inch diagonal 130*130 pixel TFT display (SSD1283A controller), 1.5 inch diagonal 240*240 pixel TFT with ST7789 controller and finally a 3.2 inch diagonal 320*240 TFT display (ILI9341 controller). The 1.6 inch TFT performed the best in terms of frame refresh rate.

figure 1. left: 8*8 color tile ‘selfie’ = traced after an original image (i.e., the one shown in figure 3) acquired with a AMG8833 thermal sensor. Right: the same dataset interpolated, resulting in 24*24 color tiles.

Introduction
A thermal camera at hand can be handy, for instance to find hot spots and cold bridges in the walls of your home if you want to improve its energy efficiency. Besides that a number of applications exist for thermal devices, among them night vision and heat source detection.
For Arduino hobbyists the AMG8833 thermal sensor offers a great opportunity to enter the sometimes surrealistic world of infrared thermal imaging. An AMG8833 offers a modest 8*8 pixel resolution, it is Arduino compatible while at the same time it is an affordable device.

figure 2. AMG8833 Grid-EYE 8*8 pixel thermal scanning sensor mounted on an Arduino-compatible mini breakout board. The breakout has six pins of which four are essential: VIN, GND, SCL (clock) and SDA (data). Communication is via the I2C protocol.

About the sensor
The Japanese electronics giant, Panasonic, produces the Grid-EYE AMG8833, a low-cost, low resolution 8*8 pixel matrix sensor made to fit home appliances, energy saving devices, automatic door sensors and so forth (figure 2). Temperature range is 0-80 ^oC with an accuracy of 2.5 degrees; vertical and horizontal viewing angles are 60 degrees; detection distance (say, detecting a human being) is approximately 7 meters; frame rate of the sensor is 10 per second. I2C address is 0x69. The breakout device is 3.3V-5V compliant.

Working with the AMG8833
In a previous post (*) I described how to wire an AMG8833 with an ESP8266 Wemos D1 mini microcontroller board hooked up to a 130*130 SPI TFT display with SSD1283A controller. The sensor produces series of 64 temperatures, with each temperature corresponding with one sensor pixel. The microcontroller compares each infrared ‘temperature’ with a so-called color lookup table, picks the corresponding color and sends that color to the display, with ‘cold’ as deep blue and ‘warm’ as hot red. In that setup the data from the thermal sensor is displayed in 8×8 color tiles where each tile is 16 pixels high and 16 pixels wide. Because this is ‘live’ processing, the display refresh rate becomes an interesting issue. The display is at the end of the processing pipe and there is complex calculation involved between the sensor and final display. While an AMG8833 may deliver data transfer speed up to 10 Hz all downstream processing takes its toll. The faster the microprocessor, the more streamlined the instructions and the more efficient the display controller, the higher the display refresh rate.
Data processing in the original example sketch is 1:1, which implies that the 64 pixels of sensor data is translated into 64 screen pixels. Since a color display has dimensions 128*128 pixels or bigger we can scale the 8*8 pixel thermal image up to 8×8 color tiles on screen with each tile made up of 16*16 pixels (figure 1, left; figure 3, left). A color display can now be nearly completely filled with color tiles. The disadvantage of up-scaling’ into color tiles is that the final thermal image looks so ‘squary’, such as the image I took as ‘selfie’: my head consists here of one hot colored tile! (figure 1, left; figure 3, left).

Interpolation
A technique often used in the world of image processing is ‘interpolation’. With an interpolation algorithm the temperature value of every ‘thermal’ pixel is compared with that of the ‘thermal’ pixels surrounding it. New, intermediate pixels with averaged values are introduced. This calculation is done for all pixels. The result is that the image becomes larger in terms of sensor pixels, in the current setup 24*24 pixels. The results in a visual representation of the data much more appreciable than the original thermal 8*8 block image, and without loss of information (figure 1, right; figure 3, right).

figure 3. left: Original 8*8 color tile ‘selfie’ image acquired with a AMG8833 thermal sensor. Right: the same dataset, 24*24 color tiles interpolated image. Images from screen.

The example ‘thermal_cam_interpolate’ in the Adafruit AMG88xx library does a nice job in two dimensional interpolation.
Assume a 128*128 display. With the original 8*8 resolution color tile dimensions are 128/8 = 16 screen pixels and the complete 128*128 display pixels can be used, no screen pixel wasted! With the 24*24 thermal pixel output of the interpolation function the color tile dimensions on the display for each interpolated thermal pixel will be 128/24 = 5 screen pixels. One complete color tiled image on the display is now 24 x5 = 120 pixels high and 120 pixels wide. Eight pixels above the screen image and the same amount of pixels aside remain unused. Considered in this way interpolation produces an image that is slightly smaller but vastly more appreciable than the original ‘squary’ image.

Displays tested
Interpolation requires so much calculation that it considerably slows down the frame rate by which the thermal images are presented on screen. At this point the performance of the display comes into focus. Because we use one microcontroller here (Wemos D1 mini ESP8266) the sensor and the microcontroller are fixed assets. Sketches and displays are variable assets. Sketches were developed for three displays from my inventory:

• 1.6 inch diameter 130*130 pixel SPI TFT with SSD1283A controller
• 1.3 inch diameter 240*240 pixel SPI TFT with ST7789 controller
• 3.2 inch diameter 320*240 pixel SPI TFT with ILI9341 controller

My expectation here was that the larger the display (the more screen pixels) the slower the refresh rate. An important factor with any sketch is also the efficiency with which instructions are compiled, with the assistance of libraries, into the binaries that the microcontroller needs. The more streamlining a library gives to a sketch, the higher the performance of the electronics involved.

Interpolation on a 1.6 inch 130*130 pixel SSD1283A display
A picture of this display is provided in figure 4. This particular display is atypical because it is a transflective display with big screen pixels, much bigger than those of the other displays discussed in this post. The on board controller is a SSD1283A chip. For metric system minded people: screen diagonal is a comfortable 40 mm while the breakout board dimensions are 54×35 mm. The interface has 8 pins: LED, SCL, SDA, A0, RST, CS, VCC and GND. Usually these displays are 3.3V and 5V compatible which makes them useful with all members of the Arduino family. Being of the transflective type implies that they can be used in broad daylight.

figure 4. Wiring diagram and pin connection scheme for an AMG8833 thermal infrared sensor breakout board, a Wemos D1 mini with ESP8266 microcontroller and an 1.6 inch, 130*130 pixel TFT with SSD1283A controller. The 4.7kΩ pull up resistors are necessary in the I2C wiring to the AMG8833 thermal scanner. On the display is one of my interpolated ‘selfies’. The selfie’s image dimensions are 120*120 screen pixels. Color tiles measure 5*5 pixels.

Pin connectivity with the Wemos D1 mini is as follows:

This display is attached to an assembly developed previously as a test bench (**) (figure 5). Having a stable and proven test bench at hand is always very practical. The hardware I2C pins (D1, D2) have on the bench their own female socket header, ready to be used. The AMG8833 thermal sensor was plugged into the bench (D1 = SCL and D2 = SDA in I2C communication) and worked immediately, without problems.

figure 5. Bench used for testing the 130*130 pixel TFT with SSD1283A controller. Left: original ‘selfie’ thermal image in 8*8 color tiles, right: interpolated image, 24*24 color tiles. The translation of thermal pixels into color tiles causes on this screen a smaller interpolated picture than the original. Lower part of the figure: One frame of 64-temperature output generated by the AMG8833 captured from Serial Monitor in a separate experiment, copied into the temperatureData array and used off line to generate the screen images used in all illustrations. This combination of sensor, microprocessor and display scored highest in the screen refresh test.

There exist two libraries that can be used with sketches for the 130*130 SD1283A display. One is LCDWIKI_GUI.h by Jean-Marc Zingg while the other is SD1283A.h created by Bodmer in collaboration with Jean-Marc Zingg. The SD1283A.h library is more user-friendly and supports more classes of instructions than LCDWIKI_GUI.h.

Testing frame refresh rate
Frame refresh rate is expressed in frames per second. The displays used here are capable of high frame refresh speeds, much higher than the 10 Hz provided by the AMG8833. It is therefore best to describe performance in cycles per second, using the appropriate unit, Hertz (cycles per second). A ‘cycle’ here is one iteration of void loop ().
The frame refresh rate was tested by measuring how many milliseconds it takes to go through 100 cycles. The resulting score was then sent to Serial Monitor. A few lines of instruction are built in each sketch for this purpose.

Software comes in flavors. There exist two major Arduino libraries supporting the AMG8833 sensor (Adafruit AMG88xx.h and SparkFun GridEye.h), and there exist two major Arduino libraries as well that support the 1283A display controller: LCDWIKI_GUI.h and SD1283A.h. This makes four possible combinations of libraries.
Streaming AMG8833 data directly to the 130*130 display – no interpolation involved – with as major processing steps comparison with a 256 color lookup table and printing to screen in 8*8 color tiles each consisting of 16*16 pixels) was achieved, with the SD1283A and AMG88xx libraries, at a stunning 29.48 Hz. Adding interpolation improved the thermal image on screen (24*24 color tiles each 5*5 pixels) (using the same libraries as in the direct 8*8 processing). Improvement was achieved at the expense of frame refresh speed (reduced to 5.95 Hz) (see Table 1).

Interpolation on a 1.3 inch 240*240 pixel ST7789 display
This particular display is my little jewel because of its bright colors and sharp image. It has tiny pixels compared with the bigger, 130*130 TFT display. Its controller is instructed via commands that need the ST7789.h library. All major graphical functions are supported by this library that as an extra uses the Adafruit_GFX convention. This combination makes this display a convenient, small, high performance TFT. Wiring to an ESP8266 board has been discussed previously (***). In order to keep pins D1 and D2 free for I2C communication with the AMG8833 sensor the wiring for the display was slightly adapted. Figure 5 shows the current wiring diagram for the 240*240 TFT and the AMG8833. The display has seven pins: GND, VCC, SCL, SDA, RES, DC and BLK. BLK is not connected which leaves six wires. Pin connectivity with the Wemos D1 mini is as follows:

Pins D1 and D2 of the Wemos D1 mini provide hardware I2C communication to the AMG8833 sensor (wherein SCL of the sensor wires to pin D1 of the Wemos D1 mini and SDA to pin D2).
The question here was: how good is this display with this particular controller in real time display of thermal images.

Streaming AMG8833 data directly to display – with as processing steps comparison with a 256 color lookup table and printing to screen in 8*8 color tiles of each 16*16 pixels, was achieved at 3.89 Hz (Adafruit AMG88xx library) or 3.38 Hz (SparkFun GridEye library). Adding interpolation improved the thermal image on screen (24*24 color tiles of each 5*5 pixels) at the expense of frame refresh speed (1.92 Hz; Adafruit AMG88xx library; 1.81 Hz; SparkFun GridEye library) (Table 1).

figure 6. The 240*240 TFT with ST7789 controller in full action, another live ‘selfie’.

Interpolation on a 3.2 inch 320*240 pixel display with ILI9341 controller
The 320*240 display is a real workhorse in my collection because it offers space, pixels and bright colors. They are available in ‘parallel’ configuration with two rows of pins that neatly match the headers of an Arduino Uno (a so-called TFT shield) and they are on the market in a ‘SPI’ configuration, with one row of pins. The display controller is in both versions the popular ILI9341. The parallel versions are faster in terms of refresh speed than the SPI variety, however with a Wemos D1 mini it is the reduced number of pins on the microcontroller board that matter. Here we apply the SPI version, and we use the Adafruit_GFX and Adafruit ILI9341 controller libraries.

figure 7. Wiring scheme and pin connection table for an AMG8833 thermal scanner breakout board, a Wemos D1 mini ESP8266 and a 3.2 inch, 320*240 pixel TFT SPI display.

Figure 7 shows the wiring diagram for the 320*240 ILI9341 controller TFT. This display is equipped with one row with 14 pins of which eight support graphical output: VCC, GND, CS, RST, D/C, SDI, SCK and LED. The remaining 6 pins support the SD card reader and are not used here.

Pin connectivity with the Wemos D1 mini is as follows:

Note the power supply: this particular display needs 5V to work properly and to produce sufficient backlight. It is a power-hungry display.

A bench potentially supporting the three components was available from a previous project (****). In order to accommodate the AMG8833 some additional soldering was necessary. Pin CS of the display is here wired to D8 instead of D2. In the current modification of the bench, D1 and D2 female pin headers are present to accept the SCL and SDA wires from the AMG8833 sensor (figures 6, 7).

figure 8. Together on the bench! Test bench with a 3.2 inch, 320*240 SPI TFT with ILI9341 controller. Wiring as in figure 6. For this particular illustration I wrote a special sketch that makes the ‘raw’ thermal image with 8*8 pixel color tiles appear left on the screen and the interpolated 24*24 color tiles right on the screen. Note that the thermal image measures 128 screen pixels high – 128 screen pixels wide while the interpolated image has dimensions 120*120 screen pixels (arrow). This is due to color tile dimensions.

This configuration was tested with sketches compiled with use of the Adafruit_ILI9341.h display controller library. Performance was good compared with the other displays. Streaming AMG8833 data to display directly as 8*8 thermal data – during processing compared with the 256 color lookup table and printing to screen in color tiles of each 30*30 pixels was achieved at 3.96 Hz (Adafruit AMG88xx.h library) and 4.47 Hz (SparkFun library). Adding interpolation improved the thermal image on screen (24*24 color tiles of each 10*10 pixels) at the expense of frame refresh speed: 2.70 Hz using the Adafruit libraries and 2.26 Hz using the SparkFun GridEye library) (Table 1).

Results and discussion
The refresh rate, or the number of times per second that the sketch completes one cycle in its loop () and prints to screen, is an important measure of performance. The higher the rate the better.
The display refresh frequency is calculated in the sketches as follows. The amount of milliseconds required to run 100 frames is recorded using the millis () function. Then the refresh frequency is calculated and printed to Serial Monitor, the next 100 frame cycles are run, and so forth. Serial Monitor is used as sparsely as possible to minimize loss of time due to port use and communication.
Conditions were: 24*24 resolution interpolated thermal images with color tiles matching maximum screen use. On the internet I found two libraries that contain instructions to read the AMG8833 thermal sensor: one published by Adafruit (Adafruit_AMG88xx.h) and the other published by SparkFun Electronics (SparkFun_GridEYE_Arduino_Library.h). All combinations of both thermal sensor libraries and display controller libraries were evaluated for their refresh rate (Table1).

Table 1. Screen refresh rates obtained with various combinations of display, display controller and thermal sensor controller. Interpolation sketches with 24*24 2d interpolation. Refresh rate = number of completed void loop () runs per second.

In all interpolation sketches in which the Adafruit library was implemented there was consistent better performance, with the best performance with the 130*130 TFT compiled with SSD1283A library instructions: nearly 6 Hz. Amazingly, this display/driver library combination clocked an incredible 29.48 Hz with the 8*8 ‘squary’ sketch. The poorest performing display in the interpolation tests was the 240*240 TFT with 1.81 Hz (sensor library SparkFun).
While both the 240*240 TFT and the 320*240 TFT produced images that on screen had dimensions 240*240 pixels, the 240*240 display was significantly slower than the 320*240 TFT. Therefore ‘the bigger the better’ does not hold here. It is the combination of display controller and software library that make a display performing better or slower.

The sketches
In the course of this project I prepared numerous sketches of which four can be downloaded: one for displays 240*240 and 320*240, and two for the 130*130 display (due to two display controller libraries tested). All three are modifications of the ‘thermal_cam’ example in the Adafruit AMG88xx library. They read a 8*8 thermal data array from the AMG33 sensor, and then interpolate via a 2d interpolation function into a 24*24 interpolated image. For the 130*130 screen this tiny image is translated into 24*24 color tiles with individual dimensions 5*5 pixels, while for the other displays color screen tile size is 10*10 pixels.
All sketches print the performance to serial Monitor, expressed as frames per second, units: Hertz

Sketch #1: ESP8266_AMG8833_SSD1283A.ino
Sketch supporting interpolation with the combination AMG8833 thermal infrared sensor – Wemos D1 mini ESP8266 – 1.6 inch 130*130 TFT with sketch code compiled under the SSD1283A.h and AMG88xx libraries.

Sketch #2: ESP8266_AMG8833_LCDWIKI.ino
Sketch supporting interpolation with the combination AMG8833 thermal infrared sensor – Wemos D1 mini ESP8266 – 1.6 inch 130*130 TFT with sketch code compiled under the LCD_WIKI.h and AMG88xx libraries.

Sketch #3: ESP8266_AMG8833_ST7798.ino
Sketch supporting interpolation with the combination AMG8833 thermal infrared sensor – Wemos D1 mini ESP8266 – 1.3 inch 240*240 TFT with sketch code compiled under the ST7789.h and AMG88xx libraries.

Sketch #4: ESP8266_AMG8833_ILI9341.ino
Sketch supporting interpolation with the combination AMG8833 thermal infrared sensor – Wemos D1 mini ESP8266 – 3.2 inch 320*240 TFT with sketch code compiled under the Adafruit_ILI9341.h and AMG88xx libraries.

Downloadable
Sketches are packed in a ZIP file named ‘ESP8266_AMG8833_thermal imager_interpolated.zip’

References:
(*) 8*8 pixel AMG8833 thermal sensor, ESP8266 microcontroller board and a SSD1283 driven TFT display. Thesolaruniverse.wordpress.com – May 2, 2022.

(**) Test bench with a Wemos D1 mini and a 130*130 TFT display with SSD1283A controller
Thesolaruniverse.wordpress.com – July 9, 2020.

(***) Connecting a 240×240 TFT display with ST7789 controller with a NodeMCU ESP8266 or an Arduino Nano
Thesolaruniverse.wordpress.com – December 24, 2019.

(****) Wiring an ILI9341 SPI TFT display with ESP8266 based microcontroller boards: NodeMCU and Wemos D1 mini
Thesolaruniverse.wordpress.com – May 2, 2021.