Hierarchical auto-associative polynomial convolutional neural networks

Date of Award

2017

Degree Name

M.S. in Electrical Engineering

Department

Department of Electrical and Computer Engineering

Advisor/Chair

Advisor: Vijayan K. Asari

Abstract

Convolutional neural networks (CNNs) lack ample methods to improve performance without either adding more input data, modifying existing data, or changing network design. This work seeks to add to the methods available that do not require more data or a trial and error approach to network design. This thesis seeks to demonstrate that a polynomial layer inserted into a CNN, compared to all other factors being equal has great potential to improve classification rates. There are some methods that seek to help fill the gap that this research also investigates an alternative solution. Most other methods in the similar problem space look at ways to improve performance of existing layers, such as modifying the type of pooling or activation functions. Also, methods discussed later, Dropout and DropConnect zero out nodes or connections, respectively, seeking to improve performance. This research focused on adding a new type of layer to typical CNNs, the polynomial layer. This layer adds a local connectivity to each of the perceptrons creating N connections up to the Nth power of the initial value of the perceptron. This is done in either the convolutional portion or the fully connected portion, with the idea that the higher dimensionality allows for better description of the input space. This idea was tested on two datasets, MNIST and CIFAR10, both classification databases with 10 classes. These datasets contain 28₉28 grayscale and 32₉32 RGB images, respectively. It was determined that the polynomial layer universally enabled the tested CNN to perform better on the MNIST data and the convolutional layer polynomials aid CNNs that are trained at a lower learning rate on the CIFAR10 dataset. Looking forward, more CNN designs should be analyzed, along with more learning rates, including ones with a variable rate. Additionally, performing tests on a wider range of datasets would also enable a broader understanding.

Keywords

Neural networks (Computer science), System design Data processing, Polynomials, Electrical Engineering, Convolutional Neural Network, Polynomial, CNN, Classification, MNIST

Rights Statement

Copyright © 2017, author

Link to Full Text

Share

COinS