Presenter(s)

    Aqsa Sultana

    Comments

    3:00-4:15, Kennedy Union Ballroom

    Download

    Download Project (482 KB)

    Description

    Recent breakthroughs in Natural Language Processing (NLP) have inspired the use of transformers in computer vision, where they have shown significant promise in tasks like image classification. This study focuses on applying transformers for pixel-wise classification in images. By using pure transformers as encoders, images are divided into patches, and these patches are embedded into tokens, which are fed as input to a Vision Transformer. The self-attention mechanism within transformers allows the model to focus on important patches relative to their neighbors, enabling it to capture long-range dependencies and contextual relationships across the image.However, challenges arise when dealing with the complexity of image datasets, scalability of models, and limitations in capturing long-range contextual information. To address these issues, we turn to the Swin Transformer. The Swin Transformer processes images hierarchically, building feature maps progressively by merging image features at deeper layers. Initially, image patches are grouped into regular windows, which are processed separately using self-attention. In the following layers, the window partitioning is shifted, enabling the model to perform self-attention across the boundaries of these windows, thereby enhancing its ability to capture finer details and long-range dependencies.We evaluate the performance of Swin Transformers on Arctic melt pond data, using high-resolution datasets from the Healy-Oden Trans Arctic Expedition (HOTRAX) and NASA’s Operation IceBridge. Our results demonstrate the effectiveness of the Swin Transformer in localizing melt ponds and achieving precise segmentation in complex Arctic imagery.

    Publication Date

    4-23-2025

    Project Designation

    Graduate Research

    Primary Advisor

    Vijayan K. Asari

    Primary Advisor's Department

    Electrical and Computer Engineering

    Keywords

    Stander Symposium, School of Engineering

    Localization of Melt Pond Regions in the Arctic Using Transformer Models

    Share

    COinS