A Convolutional Neural Network for Motion-Based Multiframe Super-Resolution Using Fusion of Interpolated Frames
Date of Award
12-1-2023
Degree Name
Ph.D. in Electrical Engineering
Department
Department of Electrical and Computer Engineering
Advisor/Chair
Advisor: Russell Hardie
Abstract
We present two novel multiframe image super-resolution (SR) algorithms that employ convolutional neural networks (CNNs) to generate high-resolution (HR) images from mul- tiple low-resolution (LR) input frames. The first algorithm, named Fusion of Interpolated Frames Network (FIFNET), utilizes motion-based multiframe SR by fusing multiple input frames in a single CNN based on Random and Fixed shifts. The second algorithm, called the Exponential weighted Fusion of Interpolated Frames Network (EWF-FIFNET), presents two variations, Externally Exponential Weighted Fusion-FIFNET (EEWF-FIFNET) and Inter- nally Exponential Weighted Fusion-FIFNET (IEWF-FIFNET) based on affine motion. A custom layer called the Exponential Weighted Fusion (EWF) layer is developed to combine input frames using a technique inspired by the fusion interpolation frame SR framework within the IEWF-FIFNET model. The EWF-FIFNET network utilizes a modified Residual Channel Attention Network architecture with residual in residual (RIR) structures. The proposed algorithms are trained and tested using a realistic observation camera model that incorporates optical and sensor degradation. Affine motion is also incorporated to address a challenging degradation problem. The experimental results show that the proposed algorithms outperform the existing state-of-the-art methods using both simulated and real camera data. It is noteworthy that the real data is not artificially downsampled or degraded, making the proposed algorithms a promising solution for practical applications. This research contributes significantly to the field of multiframe image SR, particularly in motion-based and exponentially weighted fusion approaches using CNNs.
Keywords
Multiframe super-resolution, convolutional neural network, fusion of interpolated frames, image restoration, subpixel registration
Rights Statement
Copyright © 2023, author.
Recommended Citation
Elwarfalli, Hamed, "A Convolutional Neural Network for Motion-Based Multiframe Super-Resolution Using Fusion of Interpolated Frames" (2023). Graduate Theses and Dissertations. 7344.
https://ecommons.udayton.edu/graduate_theses/7344
