A State-of-the-Art Survey on Deep Learning Theory and Architectures

Author(s)

Md Zahangir Alom, University of DaytonFollow
Tarek M. Taha, University of DaytonFollow
Christopher Yakopcic, University of DaytonFollow
Stefan Westberg, University of DaytonFollow
Paheding Sidike (0000-0003-4712-9672), Department of Earth and Atmospheric SciencesFollow
Mst Shamima Nasrin, University of DaytonFollow
Mahmudul Hasan, Comcast LabsFollow
Brian C. Van Essen, Lawrence Livermore National LaboratoryFollow
Abdul A. S. Awwal, Lawrence Livermore National LaboratoryFollow
Vijayan K. Asari, University of DaytonFollow

Document Type

Article

Publication Date

3-5-2019

Publication Source

Electronics

Abstract

In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been applied to most traditional application domains, as well as some new areas that present more opportunities. Different methods have been proposed based on different categories of learning, including supervised, semi-supervised, and un-supervised learning. Experimental results show state-of-the-art performance using deep learning when compared to traditional machine learning approaches in the fields of image processing, computer vision, speech recognition, machine translation, art, medical imaging, medical information processing, robotics and control, bioinformatics, natural language processing, cybersecurity, and many others. This survey presents a brief survey on the advances that have occurred in the area of Deep Learning (DL), starting with the Deep Neural Network (DNN). The survey goes on to cover Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). Additionally, we have discussed recent developments, such as advanced variant DL techniques based on these DL approaches. This work considers most of the papers published after 2012 from when the history of deep learning began. Furthermore, DL approaches that have been explored and evaluated in different application domains are also included in this survey. We also included recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys that have been published on DL using neural networks and a survey on Reinforcement Learning (RL). However, those papers have not discussed individual advanced techniques for training large-scale deep learning models and the recently developed method of generative models.

ISBN/ISSN

2079-9292

Document Version

Published Version

Comments

This open-access article is provided for download in compliance with the publisher’s policy on self-archiving. To view the version of record, use the DOI: https://doi.org/10.3390/electronics8030292

Publisher

MDPI

Volume

8

Peer Reviewed

yes

Issue

3

Sponsoring Agency

National Science Foundation (NSF) ; NSF - Directorate for Computer & Information Science & Engineering (CISE) ; NSF - Directorate for Engineering (ENG) ; NSF - Division of Electrical, Communications & Cyber Systems (ECCS)

eCommons Citation

Alom, Md Zahangir; Taha, Tarek M.; Yakopcic, Christopher; Westberg, Stefan; Sidike, Paheding; Nasrin, Mst Shamima; Hasan, Mahmudul; Essen, Brian C. Van; Awwal, Abdul A. S.; and Asari, Vijayan K., "A State-of-the-Art Survey on Deep Learning Theory and Architectures" (2019). Electrical and Computer Engineering Faculty Publications. 457.
https://ecommons.udayton.edu/ece_fac_pub/457