HADOOP SINGLE NODE INSTALLATION PROCEDURE (UBUNTU)




COMMANDS

1. Disabling IPV6

Step 1: Open the sysctl file using the following command

$ sudo gedit /etc/sysctl.conf

(Add the following lines end of the file)
# disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

Step 2: Test ipv6 is disabled or not.. Run below command

$ cat /proc/sys/net/ipv6/conf/all/disable_ipv6

Reboot the machine in order to make the changes take effect
A return value of 0 means IPv6 is enabled, a value of 1 means disabled.

2. Setting JAVA PATH

Step 1: Following command checks which version of java is currently being used by system.

$ java –version


Step 2: Following command is used to know the java path

$ sudo update-alternatives –config java

Step 3: To set the java path, "export" command is used as following

$ export JAVA_HOME=/usr/lib/jvm/java-7-oracle

Step 4: The Following command conforms the java path is set

$ echo "$JAVA_HOME"

Step 5: Open ~/.bashrc file using gedit and copy the Java Home path in the ~/.bashrc file

3. Setting Hadoop

Step 1: Download Hadoop from its official site
$ wget http://apache.mesi.com.ar/hadoop/common/hadoop-1.2.1/hadoop-0.20.0.tar.gz

Step 2: Unzip and move the Hadoop folder into opt folder

$ tar –xyz hadoop-0.20.0.tar.gz
$ sudo mv /home/aiktc/Download/hadoop /opt/

Step 3: Give the permission to hadoop folder by using the following command

$ sudo chmod 777 –R /opt/hadoop/

Step 4: To set the hadoop path, "export" command is used as following

$ export HADOOP_HOME=/opt/hadoop

Step 5: Open ~/.bashrc file using gedit and copy the Hadoop Home path in the ~/.bashrc file

4. Configuring Hadoop

Step 1: Change directory to hadoop folder

$ cd /opt/hadoop

Step 2: Edit the following hadoop configuration files and make following changes.

$ sudo conf/core-site.xml

Step 3: Edit the following hadoop configuration files and make following changes.

$ sudo conf/hdfs-site.xml

Step 4: Edit the following hadoop configuration files and make following changes.

$ sudo conf/mapred-site.xml

Step 5: Edit the following hadoop configuration files and Set JAVA_HOME path as per your system configuration for java.

$ sudo conf/hadoop-env.sh

Step 6: Format Name Node

$ bin/hadoop namenode –format

5. Start Hadoop Services

Step 1: The following command start all hadoop services.
$ bin/start-all.sh
6. Test and Access Hadoop Services

Step 1: ‘jps” command checks if all services are started well.

$ jps

7. Configuring Hadoop with Eclipse

Step 1: Download Eclipse Europa from its official site

$ wget http://eclipse.org/downloads/packages/release/europa/eclipse-europa-linux-gtx.tar.gz

Step 2: Unzip and give the +x permission to eclipse folder by following command.

$ sudo chmod +x /opt/eclipse/

Step 3: Copy eclipse-plugin from hadoop folder to eclipse folder using following command.

$ sudo cp /opt/hadoop/contrib/eclipse-plugin/hadoop-0.20.2-eclipse-plugin.jar /opt/eclipse/
plugins/


Comments

  1. P S Lokhande6 September 2014 at 01:05

    Well done Brinda...!
    Very informative Blog.

    Keep it up..
    Prof P S Lokhande,
    TPO- AIKTC
    www.aiktc.org
    pslokhande.blogspot.com

    ReplyDelete
  2. Shivaprakash Ranga9 September 2014 at 01:11

    Its very helpful

    ReplyDelete

Post a Comment

Popular posts from this blog

Study of Differenet Network Types and Different Types of Network Cables and Practically Implement the Cross-Wired cable using Clamping Tool.

Image
EXPERIMENT-1.a Aim: Study of following Network Types. • PAN • LAN • MAN • WAN Apparatus (Software): No software or hardware needed. Procedure: Following should be studied to understand this practical. Types of Networks There are several different types of computer networks. Computer networks can be characterized by their size as well as their purpose. The size of a network can be expressed by the geographic area they occupy and the number of computers that are part of the network. Networks can cover anything from a handful of devices within a single room to millions of devices spread across the entire globe. Some of the different networks based on size are: Personal area network, or PAN Local area network, or LAN Metropolitan area network, or MAN Wide area network, or WAN In terms of purpose, many networks can be considered general purpose, which means they are used for everything from sending files to a prin...
Read more

Error Detection and Correction Techniques

Image
Aim: a. Write a program for Error Detection usinng CRC. b. Write a program for Error C orrection using Hamming Code. Apparatus (Software): Eclipse/ Netbeans Procedure: Following should be studied to understand this practical Introduction Environmental interference and physical defects in the communication medium can cause random bit errors during data transmission. Error coding is a method of detecting and correcting these errors to ensure information is transferred intact from its source to its destination. Types of errors These interferences can change the timing and shape of the signal. If the signal is carrying binary encoded data, such changes can alter the meaning of the data. These errors can be divided into two types: Single-bit error and Burst error. Single-bit Error The term single-bit error means that only one bit of given data unit (such as a byte, character, or data unit) is changed from 1 to 0 or from 0 to 1 as ...
Read more

Challenges in e-governance

Over the years, many e-governance initiatives have been undertaken by various State Governments and Central Ministries to make all government services accessible to the common man in his locality, through common service delivery outlets, and ensure efficiency, transparency, and reliability of such services for a common man.  Although there are many improvements and innovations made in e-governance services but yet in today’s global context, e-governance services must respond proactively and it should be more agile.  There are large numbers of barriers that hinder the potential working of e-Governance. It could be different languages, IT illiteracy, population, lack of integrated services, lack of confidence, etc and it has often complained that the government schemes remain confined to papers and even if they are implemented there are no proper sources that can guarantee the proper working on the ground. Many e-government websites and applications particularly those ...
Read more