preface1��introduction2��hdfs goals and motivation design daemons reading and writing data the read path the write path managing filesystem metadata namenode high availability namenode federation access and integration command��line tools fuse rest support3��mapreduce the stages of mapreduce introducing hadoop mapreduce daemons when it all goes wrong yarn4��planning a hadoop cluster picking a distribution and version of hadoop apache hadoop cloudera��s distribution including apache hadoop what should i use? hardware selection master hardware selection worker hardware selection cluster sizing blades��sans��and virtualization operating system selection and preparation deployment layout software hostnames��dns��and identmcation users��groups��and privileges kernel tuning vm��swappiness vm��overcommit_memory disk configuration choosing a filesystem mount options network design network usage in hadoop��a review 1 gb versus 10 gb networks typical network topologies 5��installation andconfiguration installing hadoop apache hadoop cdh configuration��an 0verview the hadoop xml configuration files environment variables and shell scripts logging configuration hdfs identification and location optimization and tuning formatting the namenode creating a��tmp directory namenode high availability fencing options basic configuration automatic failover configuration format and bootstrap the namenodes namenode federation mapreduce identification and location optimization and tuning rack topology security6��identity��authentication��and authorization identity kerberos and hadoop kerberos��a refresher kerberos support in hadoop authorization hdfs mapreduce other tools and systems tying it together7��resojjrcemanagement what is resource management? hdfs quotas mapreduce schedulers the fifo scheduler the fair scheduler the capacity scheduler the future8��clustermaintenance managing hadoop processes starting and stopping processes with into scripts starting and stopping processes manually hdfs maintenance tasks adding a datanode decommissioning a datanode checking filesystem integrity with fsck balancing hdfs block data dealing with a failed disk mapreduce maintenance tasks adding a tasktracker decommissioning a tasktracker killing a mapreduce job killing a mapreduce task dealing with a blacklisted tasktracker9��troubleshooting differential diagnosis applied to systems