Improved super-resolution methods for division-of-focal-plane systems in complex and constrained imaging applications

Date of Award

2015

Degree Name

Ph.D. in Electrical Engineering

Department

Department of Electrical and Computer Engineering

Advisor/Chair

Advisor: Russell C. Hardie

Abstract

Multi-frame super-resolution (SR) image reconstruction methods seek to overcome sampling limitations in staring cameras, leading to the reduction or elimination of aliasing artifacts in imaging systems. Prior research has predominantly been focused on single band or panchromatic imaging applications. Sampling limitations are further exacerbated for cameras that add functionality by implementing spatial filters across the cameras focal plane array. The types of cameras, generally called division-of-focal-plane (DoFP) cameras, include the Bayer color filter array (CFA) design common to commercial three-color cameras. Spatial filtering is accomplished by repeating a spatial filter pattern across the focal plane array. In CFA cameras, each detector element only collects a single color channel, resulting in a mosaic pattern. The other color channels must be estimated at that sample location. This is accomplished by a process called, demosaicing. Extensive research has been accomplished in demosaicing. However, these processes are typically insufficient in eliminating aliasing artifacts.The research described in this work is developed to overcome sampling limitations in DoFP cameras and reduce or eliminate aliasing in images. Two new methods are developed and are presented. Both methods create SR image estimates from multiple image frames in the presence of random frame-to-frame motion. The first method is an interpolation-restoration SR approach that places all frame samples onto a common non-uniform sampling space and applies an optimal filter based on sample pattern and image statistics. This fast method combines multiple SR steps as a spatially-adaptive weighted sum and is called the color adaptive Wiener filter (AWF) SR approach. By leveraging a newly-developed approach to capture channel cross-correlation, this fast approach rivals and exceeds performance of much more computationally-intensive variational methods.The second method in this work is a new and computationally simpler method to address practical challenges of multi-frame SR, including frame-to-frame affine motion and the presence of local motion objects within the image set, common to many airborne imaging applications. The method is a non-iterative, sequential non-uniform interpolation process that fuses multi-frame data using a new and unique weighted sum. This method is called Fusion of Interpolated Frames (FIF) SR. With its combination of simplicity and robustness, FIF SR moves beyond color AWF SR and other DoFP-based SR approaches in terms of performance and computational efficiency. No other method that we know of has been developed to handle DoFP data in these complex imaging scenarios while operating relatively fast and achieving this level of performance.This work provides the background of the problem and describes detailed mathematical formulations for the two proposed SR methods. Multiple experiments using synthetically generated and real image datasets are provided. The proposed methods are compared against prior research SR approaches with both qualitative and quantitative results detailed. Finally, conclusions and recommendations for future work are discussed.

Keywords

High resolution imaging Mathematical models, Imaging systems Image quality Mathematical models, Electrical Engineering, Optics, super-resolution, image restoration, image processing, demosaicing, multi-frame image enhancement

Rights Statement

Copyright © 2015, author

Link to Full Text

Share

COinS