Some Key concepts used in HDFS

Below are the important link for detailed description of topics covered in Audio:

1. The following documents describe how to install and set up a Hadoop cluster:

• Single Node Setup for first-time users.
• Cluster Setup for large, distributed clusters.

2.DFS admin commands used in HDFS: The bin/hdfs dfsadmin command supports a few HDFS administration related operations

3.Shell Commands : Hadoop includes various shell-like commands that directly interact with HDFS and other file systems that Hadoop supports

4.Secondary NameNode: The secondary NameNode merges the fsimage and the edits log files periodically and keeps edits log size within a limit. It is usually run on a different machine than the primary NameNode since its memory requirements are on the same order as the primary NameNode.

5.CheckPoint Node and Backup Node: The Checkpoint node periodically creates checkpoints of the namespace

Use of a Backup node provides the option of running the NameNode with no persistent storage, delegating all responsibility for persisting the state of the namespace to the Backup node

6. Balancer: HDFS provides a tool for administrators that analyzes block placement and rebalanaces data across the DataNode.

7.Rack awarewness , Safemode, FSCK, fetchdt

Other Documents for reference:

• Hadoop Site: The home page for the Apache Hadoop site.
• Hadoop Wiki: The home page (FrontPage) for the Hadoop Wiki. Unlike the released documentation, which is part of Hadoop source tree, Hadoop Wiki is regularly edited by Hadoop Community.
• FAQ: The FAQ Wiki page.
• Hadoop JavaDoc API.
• Hadoop User Mailing List: user[at]hadoop.apache.org.
• Explore hdfs-default.xml. It includes brief description of most of the configuration variables available.
• HDFS Commands Guide: HDFS commands usage.

Souurce: https://hadoop.apache.org