Integrated coarse to fine and shot break detection approach for fast and efficient registration of aerial image sequences

Date of Award


Degree Name

M.S. in Electrical Engineering


Department of Electrical and Computer Engineering


Advisor: Vijayan K. Asari


Image registration is a task that has been focused on in many fields that deal with object detection and tracking on video sequences. When tracking any object throughout a scene, more often than not, image registration is used to align video frames to help segment moving objects from the background. With that in mind, a new registration method employing a two stage approach is proposed that efficiently registers aerial imagery. The proposed coarse to fine approach uses a combination of two efficient algorithms: Speeded Up Robust Features (SURF) for the generation of an estimated homography and the Efficient Second-Order Minimization (ESM) for fine tuning the homography generated from the coarse SURF method. Experiments are performed on several different aerial image databases, which vary in both size and resolution. The proposed algorithm proves to be effective and accurate when dealing with the changing databases; however, there are times when registration fails, specifically when very large warping parameters occur between two scenes. When registering consecutive image pairs within a sequence of images, accurate registration is needed to support many of the tracking algorithm's downstream processes. When one or several bad frames are present within a sequence of images, it becomes necessary to exclude these frames from use. A shot" is a sequence of frames within a video sequence where an object or objects are tracked consistently. In registration terms, it is a sequence of images that have been registered correctly without disrupting the tracks for targets. A shot break occurs when one frame cannot be linked with a transformation homography to another frame within an image sequence. The goal of a shot break detection algorithm is to exclude bad frames from use and detect when shot breaks occur. This thesis implements several internal and external shot break detection algorithms where bad frames and shot breaks are detected within a sequence of images. Internal checks occur within the registration algorithm before producing the transformation homography, to give a pass/fail on any given frame. External checks are used to compare the level of overlap after the homography has been applied to align one image to another. Shot break detection algorithms are tested on a sequence of images, using the proposed registration algorithm to register frame-to-frame. The proposed techniques show good results for detecting bad frames within a sequence, while maintaining speed and accuracy on the good frames. Research is progressing to increase the speed of registration and detection accuracy of shot break detection algorithms."


Aerial photogrammetry Data processing, Image registration, Photographic errors, Digital video Quality, Electrical engineering; image registration; object tracking; shot break detection; SSIM; aerial image; registration; wide area motion imagery; full motion imagery; computer vision; image sequence registration

Rights Statement

Copyright 2013, author