Hierarchical autoassociative polynomial network for deep learning of complex manifolds
Date of Award
2015
Degree Name
Ph.D. in Electrical Engineering
Department
Department of Electrical and Computer Engineering
Advisor/Chair
Advisor: Vijayan K. Asari
Abstract
Artificial neural networks are an area of research that has been explored extensively. With the formation of these networks, models of biological neural networks can be created mathematically for several different purposes. The neural network architecture being explored here is the nonlinear line attractor (NLA) network, which uses a polynomial weighting scheme instead of a linear weighting scheme for specific tasks. We have conducted research on this architecture and found that it works well to converge towards a specific trained pattern and diverge with untrained patterns. We have also improved the architecture with a Gaussian weighting scheme, which provides a modularity in the architecture and reduces redundancy in the network. Testing on the new weighting scheme improves network on different datasets gave better convergence characteristics, quicker training times, and improved recognition rates. The NLA architecture, however, is not able to reduce the dimensionality, thus a nonlinear dimensionality reduction technique is used. To improve the architecture further, we must be able to decompose the NLA architecture further to alleviate problems in the original structures and allow further improvements. We propose a hierarchical autoassociative polynomial network (HAP Net) which reorders the NLA architecture to include different ways to use polynomial weighting. In each layer, we can have orders of each input connected by a weight set, which can be trained by a backpropagation algorithm. By combining different architectures based on the understanding of MLP, attractor, and modular networks, we create a multi-purpose architecture including all aspects of the previous architecture which is far improved for classification and recognition tasks. Experiments conducted on the standard dataset, MNIST, shows very promising results of the HAP Net framework. Research work is progressing in evaluating performance on HAP Net on various datasets and also incorporating advanced learning strategies, convolutional neural networks, and extreme learning machine to investigate the performance.
Keywords
Neural networks (Computer science), Complex manifolds, Electrical Engineering, Computer Engineering, Polynomial Neural Network, Complex Manifolds, Deep Learning, Nonlinear Weighting, Modular, Classification, MNIST, HAP net
Rights Statement
Copyright © 2015, author
Recommended Citation
Aspiras, Theus H., "Hierarchical autoassociative polynomial network for deep learning of complex manifolds" (2015). Graduate Theses and Dissertations. 1091.
https://ecommons.udayton.edu/graduate_theses/1091
