2012-10-23, 11:53 PM
I need to build two clusters for work. One Windows HPC cluster for our analysis software and one Linux based HDFS cluster for output.
The HPC one is cake and I have done it several times already. The HDFS one is a PITA though it should be simple as well. I am using Cloudera Manager Free Edition on the head node to install all the needed packages and configure the cluster, but I am having tiny issues.
Their docs are good, but stuff is all over the place. Sections refer to other sections that refer to other sections and it is very easy to get lost.
Maybe I need to install it all individually, but when I need so many things installed, the manager is much simpler. I'd need to install hadoop, hdfs, pig, hive, oozie, zookeeper, and then manually edit all the various config files. Not going ot happen if I can help it.
Pulling my hair out right now though
The HPC one is cake and I have done it several times already. The HDFS one is a PITA though it should be simple as well. I am using Cloudera Manager Free Edition on the head node to install all the needed packages and configure the cluster, but I am having tiny issues.
Their docs are good, but stuff is all over the place. Sections refer to other sections that refer to other sections and it is very easy to get lost.
Maybe I need to install it all individually, but when I need so many things installed, the manager is much simpler. I'd need to install hadoop, hdfs, pig, hive, oozie, zookeeper, and then manually edit all the various config files. Not going ot happen if I can help it.
Pulling my hair out right now though