Using Hadoop to cluster data in energy system

Date of Award


Degree Name

M.S. in Computer Science


Department of Computer Science


Advisor: Zhongmei Yao


With the large amount of data generated by various devices, data scientists face big challenges since conditional machine learning algorithms applied on a single computer can no longer be used for processing/analyzing such large data sets. This thesis takes a distributed computing approach built upon Apache Hadoop, which is a distributed data analysis framework running on multiple computers. The main components of this work includes implementation of k-means machine learning algorithms on the Hadoop Map-Reduce framework, processing raw data from real energy systems, classifying the data using k-means algorithms in Hadoop, and improvement on seed selection for k-means algorithms. Finally, this thesis demonstrates the efficiency and effectiveness of our approach using different data sets.


Public utilities Data processing Case studies, Electronic data processing Distributed processing Case studies, Data mining Case studies, Computer algorithms, Computer Science, Hadoop, K-means, energy data, clustering analysis

Rights Statement

Copyright © 2015, author