Presenter(s)

Quinn Graehling, Jonathan Schierl

Download

Download Project (1.2 MB)

Description

Multi-modal data is useful for complex imaging scenarios due to the exclusivity of information found in each modality, but there is a lack of meaningful comparisons of different modalities for object detection. In our work, we propose three contributions: (1) Release of a multi-modal ground-based small object detection dataset, (2) A performance comparison of 2D and 3D modalities using state-of-the-art algorithms on data captured in the same environment, and (3) a multi-modal fusion framework for 2D/3D sensing. The new dataset encompasses various small objects for detection in EO, IR, and LiDAR modalities. The labeled data has comparable resolutions across each modality for better performance analysis. The modality comparison conducted in this work uses advanced deep learning algorithms, such as Mask R-CNN for 2D imaging and PointNet++ for 3D imaging. The comparisons are conducted with similar parameter sizes and the results are analyzed for specific instances where each modality performed the best. To complement the effectiveness of different data modalities, we developed a fusion strategy to combine the region proposals of one modality with the classification strength of a different modality for accurate detection and region segmentation. We investigated the functionality of the You Only Look Once (YOLO) algorithm, which computes partitioned image classification and region proposals in parallel for detection. Our fusion strategy learns the optimum features of different modality combinations for appropriate candidate selection for classification. The effectiveness of the proposed fusion method is being evaluated on the multi-modal dataset for object detection and segmentation and we observe superior performance when compared to single-modality algorithms.

Publication Date

4-22-2020

Project Designation

Graduate Research

Primary Advisor

Theus H. Aspiras, Vijayan K. Asari

Primary Advisor's Department

Electrical and Computer Engineering

Keywords

Stander Symposium project, School of Engineering

United Nations Sustainable Development Goals

Industry, Innovation, and Infrastructure

Multi-modal Data Analysis and Fusion for Robust Object Detection in 2D/3D Sensing

Share

COinS