Using Hadoop to cluster data in energy system
Date of Award
M.S. in Computer Science
Department of Computer Science
Advisor: Zhongmei Yao
With the large amount of data generated by various devices, data scientists face big challenges since conditional machine learning algorithms applied on a single computer can no longer be used for processing/analyzing such large data sets. This thesis takes a distributed computing approach built upon Apache Hadoop, which is a distributed data analysis framework running on multiple computers. The main components of this work includes implementation of k-means machine learning algorithms on the Hadoop Map-Reduce framework, processing raw data from real energy systems, classifying the data using k-means algorithms in Hadoop, and improvement on seed selection for k-means algorithms. Finally, this thesis demonstrates the efficiency and effectiveness of our approach using different data sets.
Public utilities Data processing Case studies, Electronic data processing Distributed processing Case studies, Data mining Case studies, Computer algorithms, Computer Science, Hadoop, K-means, energy data, clustering analysis
Copyright 2015, author
Hou, Jun, "Using Hadoop to cluster data in energy system" (2015). Graduate Theses and Dissertations. 1044.