Presenter(s)

Aqsa Sultana

Comments

3:00-4:15, Kennedy Union Ballroom

Download

Download Project (482 KB)

Description

Recent breakthroughs in Natural Language Processing (NLP) have inspired the use of transformers in computer vision, where they have shown significant promise in tasks like image classification. This study focuses on applying transformers for pixel-wise classification in images. By using pure transformers as encoders, images are divided into patches, and these patches are embedded into tokens, which are fed as input to a Vision Transformer. The self-attention mechanism within transformers allows the model to focus on important patches relative to their neighbors, enabling it to capture long-range dependencies and contextual relationships across the image.However, challenges arise when dealing with the complexity of image datasets, scalability of models, and limitations in capturing long-range contextual information. To address these issues, we turn to the Swin Transformer. The Swin Transformer processes images hierarchically, building feature maps progressively by merging image features at deeper layers. Initially, image patches are grouped into regular windows, which are processed separately using self-attention. In the following layers, the window partitioning is shifted, enabling the model to perform self-attention across the boundaries of these windows, thereby enhancing its ability to capture finer details and long-range dependencies.We evaluate the performance of Swin Transformers on Arctic melt pond data, using high-resolution datasets from the Healy-Oden Trans Arctic Expedition (HOTRAX) and NASA’s Operation IceBridge. Our results demonstrate the effectiveness of the Swin Transformer in localizing melt ponds and achieving precise segmentation in complex Arctic imagery.

Publication Date

4-23-2025

Project Designation

Graduate Research

Primary Advisor

Vijayan K. Asari

Primary Advisor's Department

Electrical and Computer Engineering

Keywords

Stander Symposium, School of Engineering

Localization of Melt Pond Regions in the Arctic Using Transformer Models

Share

COinS