Low-Resolution Infrared and High-Resolution Visible Image Fusion Based on U-NET

Date of Award

2022

Degree Name

M.S. in Electrical Engineering and Computer Engineering

Department

Department of Electrical and Computer Engineering

Advisor/Chair

Keigo Hirakawa

Abstract

With current sensor technology, visible wavelength (VIS) images can be acquired at very high resolutions (HR) compared to the infrared (IR) images. Therefore, image fusion techniques aim to augment IR images with the superior spatial resolution of VIS images to overcome the resolution problems in IR imaging. This thesis introduces two ways to integrate IR and VIS images, IR image super-resolution and IR and VIS image fusion. The first application is super-resolution (SR) for IR images. We propose an IR image SR algorithm based on U-Net. By fusing the HR image features of the VIS images, the network can produce an IR SR image successfully and efficiently. Secondly, we also propose a novel framework for combining VIS and IR images, guided by feature extraction techniques such as VGG16. By designing the algorithm to preserve the meaningful VGG16 features from both IR and VIS images, the proposed method achieves excellent performance in the qualitative and quantitative aspects. In addition, we propose joint super-resolution and image fusion between IR and VIS images. Finally, we developed a new HR VIS and LR IR image pair dataset. Since this data collection closely resembles the real-world sensing scenarios, it is a valuable resource for continued exploration of this image processing field.

Keywords

Electrical Engineering, Image Fusion, Super-Resolution, Infrared Image, Visible Wavelength Image, Convolutional Neural Network

Rights Statement

Copyright © 2022, author.

Link to Full Text

Share

COinS