Tetsuo Tamura

SOLTEC Corporation an NEC Avio Infrared Technologies, Ltd. Company

12977 Arroyo Street
San Fernando, CA

Abstract

Thermography applications have expanded with lower priced infrared camera availability; however, for those applications requiring high performance/high resolution thermography images such as construction fault diagnosis, image processing to achieve a suitable high resolution display has proven to be a difficult task. NEC Avio Infrared Technologies Co., Ltd. has developed new infrared image processing techniques to improve thermography spatial resolution and temperature resolution for QVGA (320 x 240) display optimization. The methods used and resulting effects shall be presented.

Introduction

During recent years, infrared thermography has been widely used in various fields because of its high performance and low price. Also, various types of products such as portable cameras for predictive / preventive maintenance, optical filter built-in type for special measurement, and fixed mount type for process controls have become available on the market. These are achievements of development of the element technology and products to answer very diversified applications causing an expansion of applicable fields of infrared thermography.

Infrared thermography equipment is used for non-destructive diagnosis in building and facility inspections and further improvement of basic performance such as NETD (Noise Equivalent Temperature Difference) and Spatial Resolution is required for both accurate diagnosis and finding small defects.

We have increased sensitivity of the UFPA (Uncooled Focal Plane Array) and improved NETD by a new development of image processing technology. Further, we developed image processing technology for reducing noise, high spatial resolution and panoramic composition. We hereby report our work with infrared thermography and PC Software using these technologies.

Discussion

Improvement of Basic Performance of Infrared Imagers

Infrared thermography is used for building and facility inspections where the objects being measured are at room temperature, so an infrared range of 8 to 13 micron wavelength is selected. Further, due to the requirement of low cost and extended battery operation, the UFPA is used for the infrared detector element. Figure 1 shows the configuration of the infrared imager used this UFPA. The basic performance of NETD and Spatial Resolution is determined by the UFPA, as well as Infrared Signal Processing and Image Processing. Following is a description of the improvements of each important element.

Improvement of NETD

NETD can be improved by increasing the sensitivity for detected infrared by making the sensitivity of the detector higher and by reducing the noise generated when taking measurements.

Improving the Sensitivity of UFAP

Thermal-type infrared detectors do not require cooling as used in the detection principle that “temperature of detector changes by infrared any energy incoming to the detector”; therefore, in order to make its sensitivity higher, you need to thermally separate (insulate) the light-receiving part of the detector from its external parts, making a change of temperature caused by incidental infrared ray energy at the detector. Thermal separation is made by making the light-receiving part in the micro-bridge structure (figure 2) and using vacuum packaging in order to provide thermal insulation from the environment. Also, a TEC (Thermo Electric Cooler) is used to control the temperature of the detector in order to stabilize the output of the high sensitivity detector.

By improving the process of MEMS (Micro Electro Mechanical System) we:

1) Improved thermal separation of the light-receiving part of the detector from the readout circuit. This required making the foot suspending the light- receiving part in the air thinner.

2) Improved the fill-factor by installing eves on the light-receiving part (ratio of effective light-receiving area of detector to the detector element.

3) Realized high sensitivity (low NETD: Noise Equivalent Temperature Difference) by making ROIC (Read Out IC) high performance / low noise (see figure 2).

4) Further, the NEC Guidance and Electro-Optics Division has achieved NETD of 30mK by reducing the thermal conductance of “feet” which separates heat from read-out circuit of the light-receiving part of the detector element.

Noise Reduction (Digital Image Processing)

As a way to eliminate noise from the infrared image, it is typical that multiple frames of image data are processed with a moving average. Temporary noise is reduced to (frame number) 1/2 and the image quality is therefore improved. However, when the infrared imager or the object being measured is moving, motion blur (ghosting) occurs (figure 3-2).

Noise reduction technology developed by NEC achieved Frame Averaging without ghosting while judging if the measured object is moving or standing still. If moving, there will be no Frame Averaging in the moving area, but will Frame Averaging will still be used for the still object (figure 3-3).
The NETD of the original image (figure 3-1) is 30 mK; the Frame Averaging with deghosting is 20mK. The noise was compressed by 0.7 times without image distortion such as motion blur.

Improvement of Spatial Resolution

The spatial resolution must be high resolution to measure small objects (size of object / measuring distance) or to locate a small defect. On the other hand, if the inspection object is wide and it is necessary to measure the wider angle, the number of image elements on the infrared detector is limited, so spatial resolution gets worse.

FOV (Field of View) and resolution have the following relationship:

FOV x = Spatial Resolution x Number of Image Pixels. For example, if resolution = 1.2 mrad, in an infrared imager of 320 x 240 pixels, the FOV is 22 x 16.5.

This super resolution technology is effective in getting higher resolution while keeping the same FOV. The panoramic technology (that is taking multiple images and connecting and fitting them together) is effective to expand the FOV while maintaining the same resolution. I introduced infrared imager PC software which was developed by collaboration with NEC Information Media Processing laboratory as follows:

High Resolution with Super Resolution Technology

Our super resolution method is making high resolution images by using multiple images. It extracts characteristic points from each frame, relates them, matches the position and overlaps them. The overlap also gives the effect of noise reduction (figs 4-1 & 4-2).

High Resolution by Panoramic Composition

In using an infrared imager and shooting multiple frames (figs 5-1 & 5-2), those multiple frames are recorded and made into a wide angle (panoramic) image while orienting certain angles of continuous frames by using the gyroscope built into the infrared imager. The method here is to combine images from each frame which may be connected smoothly by extracting characteristic points of each frame and relating similar positions in each frame (figure 6).

Examples of Applications for using a High Performance Infrared Imager

Image improved NETD

Super High Resolution Image

Conclusion

An infrared imager can view image temperature, darkness and shadow, etc. that cannot be seen with a visible camera. The price of infrared imagers has quickly lowered in recent years and so often it is used as a replacement for visible cameras. In the near future, it will grow more popular as applications such as fire monitoring, security cameras, and automobile forward-monitoring, etc. continue to be used. The key technologies to drive these advancements are the infrared detector and image processing. Smaller and higher sensitivity detectors will make the lens and the enclosure small and lower cost, and furthermore, as the uniformity of the detector is improved and the correction circuit is simplified. By these improvements, the application of infrared thermography will expand and the market will grow much larger.

References

1) Detection of Terehertz Radiation from Quantum Cascade Laser using Vanadium Oxide Microbolometer Focal Plane Array. Noaki Oda, etc., DPIE Defense + Security Conf. 6940.

2) Super-resolved Video Mosaicing for Documents Based on Extrinsic Camera Parameter Estimation. A. Iketani, T. Sato, S. Ikeda, M. Kanbara, N. Nakajima, and N. Yokoya. Proc. Asian Conference on ComputerVision (ACCV2006), Vol. II, pp. 101-110, January 2006.

NEC Avio Infrared Technologies Company, Ltd.
8-1-5 Nishi-gotanda, Shinagawa-ku, Toyko 141-8535, Japan
Ph: +81-3-5436-1659
email: This email address is being protected from spambots. You need JavaScript enabled to view it.
email: This email address is being protected from spambots. You need JavaScript enabled to view it.
email: This email address is being protected from spambots. You need JavaScript enabled to view it.
email: This email address is being protected from spambots. You need JavaScript enabled to view it.
Web: www.nec-avio.co.jp