Cloudera Distribution for Hadoop (CDH)

Opportunities in Big Data Hadoop

2015-06-03T23:43:00.003-07:00

Watch the exclusive recording of Big Data Session conducted By DataFlair

Are you Ready to Migrate your Career in the Latest upcoming Technology Big Data

Visit: http://data-flair.com/course/big-data-and-hadoop-training/

How Big Data is the Biggest Buzz Word

2015-06-03T23:40:00.001-07:00

What is Big Data

Big data is a buzzword, or catch-phrase, used to describe a massive volume of both structured and unstructured data that is so large it is difficult to process using traditional database and software techniques.

What leaders say About Big Data:

Big Data is the new Oil
- Gartner

Hadoop will grow at CAGR of 58%, will reach $50 billion by 2020

- Experfy

Big Data market will be growing 6 times faster than the overall IT market

- IDC

Why learn Big Data?

It is no secret that the data content in the world is growing exponentially. For last two decades, IT communities have been grappling with the issue of managing the glut of data. Google has been at the forefront of this problem and came up with a framework which is now widely known as Big Data & Hadoop. This framework fundamentally changes the traditional approaches – which are no longer coping with the volumes of data being generated today.

This video Tutorial covers Following Topics:

- Why Big Data is biggest Buzz Word
- Basics of Big Data & Hadoop
- Essence of Big Data (volume, velocity, variety, veracity of Big Data)
- Problems with conventional systems like RDBMS and OS file-system
- Introduction of Hadoop & Hadoop ecosystem
- Real time Hadoop use cases
- Future of Hadoop & Careers in Hadoop
- Job Roles in Hadoop like: Hadoop Analyst, Hadoop Developer, Hadoop Admin, Hadoop Architect etc.
- How DataFlair will help you in making your career in Big Data.

Are you Ready to Migrate your Career in the Latest upcoming Technology Big Data

Visit: http://data-flair.com/course/big-data-and-hadoop-training/

Big Data Hadoop Tutorial For Beginners

2015-06-03T23:27:00.000-07:00

Why learn Big Data Hadoop?

We create 2.5 quintillion bytes of data every day. So much that 90% of the data in the world today has been created in the last two years alone (Source: IBM). These extremely large datasets are hard to deal with using legacy systems such as RDBMS as data exceed the storage and processing capacity of database. The legacy systems are becoming obsolete.
According to Gartner: “Big Data is new Oil”. Big Data is all about finding the needle of value in a haystack of Structured, Semi-structured and Un-structured data. Hadoop (the Solution of All Big Data Problems) has become the most important component in the data stack, which enables rapid processing of data at petabyte scale. Hadoop is expected to be at the core of more than half of all analytics software within the next two years.

Watch the exclusive recording of Big Data Live Session

In this tutorial, you will be gaining knowledge on:
- Basics of Big Data & Hadoop
- Introduction to Big Data
- Why Big Data
- Essence of Big Data-volume, velocity, variety, veracity
- Problems with conventional systems like RDBMS and OS file-system
- Introduction of Hadoop
- Introduction of Map-Reduce and HDFS
- Real time Hadoop use cases
- Introduction of Hadoop ecosystem
- Future of Hadoop
- Careers in Hadoop
- Job Roles in Hadoop like: Hadoop Analyst, Hadoop Developer, Hadoop Admin, etc.

Setup Hadoop in distributed mode

2015-05-16T12:43:00.000-07:00

This tutorial explains How to Setup and configure Hadoop on Multiple machines, i.e. Installation of Hadoop in Distributed Mode. In the cluster setup there is one master and 2 slaves will be configured. During the deployment all the pre-requisites will be installed. Hadoop installation is done on Amazon cloud (AWS).
Follow following video tutorial for the installation and configuration of Hadoop 1 in distributed mode (real cluster mode)on Amazon Cloud:

In this video following topics has been covered:
- Installation and configuration of Hadoop 1.x or Cloudera CDH3Ux in Distributed mode (on multiple node cluster).
- Launch 3 instances on AWS (Amazon Cloud), on which we will setup the real cluster. One instance will act as Master and rest all the instances will act as slaves.
- Prerequisites for hadoop Installation.
-- Installation of Java.
-- Setup of password-less ssh.
- Important configurations properties.
- Setup Configuration in core-site.xml, hdfs-site.xml, map-red-site.xml.
- Format name-node.
- Start hadoop services: NameNode, DataNode, secondary-namenode, JobTracker, TaskTracker.
- Setup environment variables for Hadoop,
- Submit Map-Reduce Job.

Big Data Training First Session - Hadoop Training First Class Recording

2015-05-16T04:11:00.001-07:00

This video covers:
- Introduction to Big Data
- What is the need of Big Data
- Essence of Big Data
- 4Vs of Big Data Volume, Velocity, Variety, Veracity
- Problems with conventional systems like RDBMS and OS file-system
- Introduction to Hadoop
- Introduction to HDFS
- Introduction to MapReduce
- Real life Big Data use cases
- Future of Big Data & Hadoop
- Careers in Big Data & Hadoop
- Job Roles in Big Data & Hadoop like: Hadoop Analyst, Hadoop Developer, Hadoop Admin, etc.
- How DataFlair will help you in making your career in Big Data.

About Big Data - Hadoop Training Course:
An online course designed by Hadoop Experts to provide in-depth knowledge and practical skills in the field of Big Data and Hadoop to make you successful Big Data and Hadoop Developer.

Big Data and Hadoop training course is designed to provide knowledge and skills to make you employable in Big Data industry. In-depth knowledge of concepts such as Hadoop Distributed File System, Map-Reduce, Hadoop Cluster- Single and multi node, Hadoop 2.0, Flume, Sqoop, PIG, Hive, Hbase, Zookeeper, Oozie etc. will be covered in the course.

Course Objectives:
After the completion of the ‘Big Data and Hadoop’ Course you should be able to:

1. Master the concepts of Hadoop Distributed File System and MapReduce framework
2. Setup a Hadoop on single and multi node Cluster
3. Understand Data Loading Techniques using Sqoop and Flume
4. Program in MapReduce (Both MRv1 and MRv2)
5. Learn to write Complex MapReduce programs
6. Program in YARN (MRv2)
7. Perform Data Analytics using Pig and Hive
8. Implement HBase, MapReduce Integration, Advanced Usage and Advanced Indexing
9. Have a good understanding of ZooKeeper service
10. New features in Hadoop 2.0 — YARN, HDFS Federation, NameNode High Availability
11. Implement best Practices for Hadoop Development and Debugging
12. Implement a Hadoop Project
13. Work on a Real Life Project on Big Data Analytics and gain Hands on Project Experience

Please contact us for more details:
http://data-flair.com/
http://data-flair.com/course/big-data-and-hadoop/
Email us: info@data-flair.com
Phone: +91-8451097879

Deep Dive into Map-Reduce Data Flow | Big Data Training

2015-05-11T06:29:00.001-07:00

This video covers: Basics of MapReduce, DataFlow in MapReduce, Basics of Input Split, Mapper, Intermediate output, Data Shuffling, Principle behind Shuffling, Reducer, Wordcount in Map-Reduce, etc.

About Big Data - Hadoop Training Course:
An online course designed by Hadoop Experts to provide indepth knowledge and practical skills in the field of Big Data and Hadoop to make you successful Hadoop Developer.

Big Data and Hadoop training course is designed to provide knowledge and skills to become a successful Hadoop Developer. In-depth knowledge of concepts such as Hadoop Distributed File System, Hadoop Cluster- Single and multi node, Hadoop 2.0, Flume, Sqoop, Map-Reduce, PIG, Hive, Hbase, Zookeeper, Oozie etc. will be covered in the course.

Course Objectives:

After the completion of the ‘Big Data and Hadoop’ Course you should be able to:

1. Master the concepts of Hadoop Distributed File System and MapReduce framework
2. Setup a Hadoop on single and multi node Cluster
3. Understand Data Loading Techniques using Sqoop and Flume
4. Program in MapReduce (Both MRv1 and MRv2)
5. Learn to write Complex MapReduce programs
6. Program in YARN (MRv2)
7. Perform Data Analytics using Pig and Hive
8. Implement HBase, MapReduce Integration, Advanced Usage and Advanced Indexing
9. Have a good understanding of ZooKeeper service
10. New features in Hadoop 2.0 — YARN, HDFS Federation, NameNode High Availability
11. Implement best Practices for Hadoop Development and Debugging
12. Implement a Hadoop Project
13. Work on a Real Life Project on Big Data Analytics and gain Hands on Project Experience

Please contact us for more details: info@data-flair.com, +91-8451097879.

Install Cloudera Hadoop with YARN on Ubuntu

2015-04-13T05:40:00.001-07:00

Objective:

This tutorial describes how to install and configure a single-node Hadoop cluster on Ubuntu OS. Single Node Hadoop cluster is also called as Hadoop Pseudo-Distributed Mode. The tutorial is very simple and to the point, so that you can install Hadoop in 10 Min. Once the installation is done you can perform Hadoop Distributed File System (HDFS) and Hadoop Map-Reduce operations.

Recommended Platform:

OS: Linux is supported as a development and production platform. You can use Ubuntu 14.04 or later (you can also use other Linux flavors like: CentOS, Redhat, etc.)
Hadoop: Cloudera Distribution for Apache hadoop CDH5.x (you can use Apache hadoop 2.x)

Prerequisites:

Install Java 7 (Recommended Oracle Java)

Install Python Software Properties

$sudo apt-get install python-software-properties

Add Repository

$sudo add-apt-repository ppa:webupd8team/java

Update the source list

$sudo apt-get update

Install Java

$sudo apt-get install oracle-java7-installer

Configure SSH

Install Open SSH Server-Client

$sudo apt-get install openssh-server openssh-client

Generate Key Pairs

$ssh-keygen -t rsa -P ""

3.2.3 Configure password-less SSH

$cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

Check by SSH to localhost

$ssh localhost

Install Hadoop

Download Hadoop

http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.5.0-cdh5.3.2.tar.gz

Untar Tar ball

$tar xzf hadoop-2.5.0-cdh5.3.2.tar.gz

Note: All the required jars, scripts, configuration files, etc. are available in HADOOP_HOME directory (hadoop-2.5.0-cdh5.3.2)

Read Complete Post >>

Apache Spark - the Second Generation Big Data Technology

2015-01-19T10:13:00.000-08:00

This tutorial covers: Introduction to Spark – Why, what and Features, Understanding Spark Key Terms, RDDs, Spark Ecosystem, Shark, Spark Streaming, MLLib, GraphX, SparkR, Data flow in Spark, Spark Examples with Demo, Fault tolerance in Spark, Spark Use Cases, Spark streaming, Spark Sql, Behavior with Not Enough RAM, Organizations Using Spark

http://data-flair.com/
Email us: info@data-flair.com
Phone: +91-8451097879

About Big Data - Apache Spark Training Course:
An online course designed by Spark Experts to provide in-depth knowledge and practical skills in the field of Big Data and Spark to make you successful Spark Expert.

Please contact us for more details: info@data-flair.com, +91-8451097879.

Introduction to HDFS | Hadoop Distributed File System Tutorial

2015-01-14T05:26:00.002-08:00

http://data-flair.com/course/big-data-and-hadoop/
Email us: info@data-flair.com
Phone: 8451097879

This video covers: Basics of Hadoop Distributed File System HDFS, Introduction to HDFS, HDFS Architecture, HDFS Nodes, NameNode and DataNode Daemons, HDFS Features, Fault tolerance, High Availability, Reliable Data Storage, Replication, Blocks, File Read Operation, File Write Operation, Rack Awareness, Scalability, Real time Hadoop use cases, etc.

About Big Data - Hadoop Training Course:
An online course designed by Hadoop Experts to provide indepth knowledge and practical skills in the field of Big Data and Hadoop to make you successful Hadoop Developer.

Big Data and Hadoop training course is designed to provide knowledge and skills to become a successful Hadoop Developer. In-depth knowledge of concepts such as Hadoop Distributed File System, Hadoop Cluster- Single and multi node, Hadoop 2.0, Flume, Sqoop, Map-Reduce, PIG, Hive, Hbase, Zookeeper, Oozie etc. will be covered in the course.

Course Objectives:
After the completion of the ‘Big Data and Hadoop’ Course you should be able to:
1. Master the concepts of Hadoop Distributed File System and MapReduce framework
2. Setup a Hadoop on single and multi node Cluster
3. Understand Data Loading Techniques using Sqoop and Flume
4. Program in MapReduce (Both MRv1 and MRv2)
5. Learn to write Complex MapReduce programs
6. Program in YARN (MRv2)
7. Perform Data Analytics using Pig and Hive
8. Implement HBase, MapReduce Integration, Advanced Usage and Advanced Indexing
9. Have a good understanding of ZooKeeper service
10. New features in Hadoop 2.0 — YARN, HDFS Federation, NameNode High Availability
11. Implement best Practices for Hadoop Development and Debugging
12. Implement a Hadoop Project
13. Work on a Real Life Project on Big Data Analytics and gain Hands on Project Experience

Please contact us for more details: info@data-flair.com, +91-8451097879.

DataFlair Student Review of Hadoop Training

2015-01-05T00:16:00.001-08:00

http://data-flair.com/
Email us: info@data-flair.com
Phone: 8451097879

This Video Contains: Feedback about DataFlair from our students.

About Big Data - Hadoop Training Course:
An online course designed by Hadoop Experts to provide indepth knowledge and practical skills in the field of Big Data and Hadoop to make you successful Hadoop Developer.

Big Data and Hadoop training course is designed to provide knowledge and skills to become a successful Hadoop Developer. In-depth knowledge of concepts such as Hadoop Distributed File System, Hadoop Cluster- Single and multi node, Hadoop 2.0, Flume, Sqoop, Map-Reduce, PIG, Hive, Hbase, Zookeeper, Oozie etc. will be covered in the course.

Course Objectives:

After the completion of the ‘Big Data and Hadoop’ Course you should be able to:
1. Master the concepts of Hadoop Distributed File System and MapReduce framework
2. Setup a Hadoop on single and multi node Cluster
3. Understand Data Loading Techniques using Sqoop and Flume
4. Program in MapReduce (Both MRv1 and MRv2)
5. Learn to write Complex MapReduce programs
6. Program in YARN (MRv2)
7. Perform Data Analytics using Pig and Hive
8. Implement HBase, MapReduce Integration, Advanced Usage and Advanced Indexing
9. Have a good understanding of ZooKeeper service
10. New features in Hadoop 2.0 — YARN, HDFS Federation, NameNode High Availability
11. Implement best Practices for Hadoop Development and Debugging
12. Implement a Hadoop Project
13. Work on a Real Life Project on Big Data Analytics and gain Hands on Project Experience

Please contact us for more details: info@data-flair.com, 8451097879.

Learn Big Data with real life use-cases

2015-01-04T01:31:00.001-08:00

http://data-flair.com/course/big-data-and-hadoop/

Email us: info@data-flair.com

Phone: +91-8451097879

This video covers: Basics of Big Data & Hadoop, Introduction to Big Data, Why Big Data, essence of Big Data, volume, velocity, variety, veracity of Big Data, Problems with conventional systems like RDBMS and OS file-system, Introduction of Hadoop, Introduction of Map-Reduce, Introduction of HDFS, Real time Hadoop use cases.

About Big Data - Hadoop Training Course:

An online course designed by Hadoop Experts to provide indepth knowledge and practical skills in the field of Big Data and Hadoop to make you successful Hadoop Developer.

Big Data and Hadoop training course is designed to provide knowledge and skills to become a successful Hadoop Developer. In-depth knowledge of concepts such as Hadoop Distributed File System, Hadoop Cluster- Single and multi node, Hadoop 2.0, Flume, Sqoop, Map-Reduce, PIG, Hive, Hbase, Zookeeper, Oozie etc. will be covered in the course.

Course Objectives:

After the completion of the ‘Big Data and Hadoop’ Course you should be able to:

1. Master the concepts of Hadoop Distributed File System and MapReduce framework

2. Setup a Hadoop on single and multi node Cluster

3. Understand Data Loading Techniques using Sqoop and Flume

4. Program in MapReduce (Both MRv1 and MRv2)

5. Learn to write Complex MapReduce programs

6. Program in YARN (MRv2)

7. Perform Data Analytics using Pig and Hive

8. Implement HBase, MapReduce Integration, Advanced Usage and Advanced Indexing

9. Have a good understanding of ZooKeeper service

10. New features in Hadoop 2.0 — YARN, HDFS Federation, NameNode High Availability

11. Implement best Practices for Hadoop Development and Debugging

12. Implement a Hadoop Project

13. Work on a Real Life Project on Big Data Analytics and gain Hands on Project Experience

Please contact us for more details: info@data-flair.com, +91-8451097879.

Hadoop Map Reduce Training Video

2015-01-02T05:43:00.000-08:00

http://data-flair.com/course/big-data-and-hadoop/

Email us: info@data-flair.com

Phone: 8451097879

This video covers: MapReduce Concepts, What is Map, Map Task, Mapper, What is Reduce, Reduce Task, Reducer, Map & Reduce together, MR Job, Task, Split the job into tasks, Map-Reduce flow, Map-Reduce daemons, Job Tracker, Task Tracker, etc.

About Big Data - Hadoop Training Course:

An online course designed by Hadoop Experts to provide indepth knowledge and practical skills in the field of Big Data and Hadoop to make you successful Hadoop Developer.

Big Data and Hadoop training course is designed to provide knowledge and skills to become a successful Hadoop Developer. In-depth knowledge of concepts such as Hadoop Distributed File System, Hadoop Cluster- Single and multi node, Hadoop 2.0, Flume, Sqoop, Map-Reduce, PIG, Hive, Hbase, Zookeeper, Oozie etc. will be covered in the course.

Course Objectives:

After the completion of the ‘Big Data and Hadoop’ Course you should be able to:

1. Master the concepts of Hadoop Distributed File System and MapReduce framework

2. Setup a Hadoop on single and multi node Cluster

3. Understand Data Loading Techniques using Sqoop and Flume

4. Program in MapReduce (Both MRv1 and MRv2)

5. Learn to write Complex MapReduce programs

6. Program in YARN (MRv2)

7. Perform Data Analytics using Pig and Hive

8. Implement HBase, MapReduce Integration, Advanced Usage and Advanced Indexing

9. Have a good understanding of ZooKeeper service

10. New features in Hadoop 2.0 — YARN, HDFS Federation, NameNode High Availability

11. Implement best Practices for Hadoop Development and Debugging

12. Implement a Hadoop Project

13. Work on a Real Life Project on Big Data Analytics and gain Hands on Project Experience.

Please contact us for more details: info@data-flair.com, +91-8451097879.

Big Data and Hadoop Training for Beginners

2014-11-27T02:03:00.001-08:00

http://data-flair.com/course/big-data-and-hadoop/

Email us: info@data-flair.com

Phone: 8451097879

This video covers: Basics of Big Data & Hadoop, Introduction to Big Data, Why Big Data, essence of Big Data, volume, velocity, variety, veracity of Big Data, Problems with conventional systems like RDBMS and OS file-system, Introduction of Hadoop, Introduction of Map-Reduce, Introduction of HDFS, Real time Hadoop use cases, Introduction of Hadoop ecosystem, Future of Hadoop, Careers in Hadoop, Job Roles in Hadoop like: Hadoop Analyst, Hadoop Developer, Hadoop Admin, etc. How Data will help you in making your career in Big Data.

About Big Data - Hadoop Training Course:

An online course designed by Hadoop Experts to provide indepth knowledge and practical skills in the field of Big Data and Hadoop to make you successful Hadoop Developer.

Big Data and Hadoop training course is designed to provide knowledge and skills to become a successful Hadoop Developer. In-depth knowledge of concepts such as Hadoop Distributed File System, Hadoop Cluster- Single and multi node, Hadoop 2.0, Flume, Sqoop, Map-Reduce, PIG, Hive, Hbase, Zookeeper, Oozie etc. will be covered in the course.

Course Objectives:

After the completion of the ‘Big Data and Hadoop’ Course you should be able to:

1. Master the concepts of Hadoop Distributed File System and MapReduce framework

2. Setup a Hadoop on single and multi node Cluster

3. Understand Data Loading Techniques using Sqoop and Flume

4. Program in MapReduce (Both MRv1 and MRv2)

5. Learn to write Complex MapReduce programs

6. Program in YARN (MRv2)

7. Perform Data Analytics using Pig and Hive

8. Implement HBase, MapReduce Integration, Advanced Usage and Advanced Indexing

9. Have a good understanding of ZooKeeper service

10. New features in Hadoop 2.0 — YARN, HDFS Federation, NameNode High Availability

11. Implement best Practices for Hadoop Development and Debugging

12. Implement a Hadoop Project

13. Work on a Real Life Project on Big Data Analytics and gain Hands on Project Experience

Please contact us for more details: info@data-flair.com, 8451097879.

Installing YARN (Hadoop 2.4.0) on Ubuntu

2014-05-14T02:15:00.001-07:00

Steps to install Hadoop 2.x release (Yarn) on single node cluster setup (pseudo-distributed mode)

Hadoop 2.x release involves many changes to Hadoop and MapReduce. The centralized JobTracker service is replaced with a ResourceManager that manages the resources in the cluster and an ApplicationManager that manages the application lifecycle. These architectural changes enable hadoop to scale to much larger clusters. For more details on architectural changes in Hadoop next-gen (a.k.a. Yarn), watch this video or visit this blog.

This post explains on installing Hadoop 2.x a.k.a. Yarn on a single-node cluster.

Prerequisites:

Java 7 installed
Dedicated user for hadoop (not mandatory)
SSH configured

Steps to install Hadoop 2.x:

1. Download tarball

You can download tarball for hadoop 2.x from http://apache.cs.utah.edu/hadoop/common/current2/ (hadoop-2.4.0.tar.gz ).

Extract it to a folder in your home directory say, $HOME/yarn.

$ cd $HOME/yarn (Optional)

$ sudo chown -R hduser:hadoop hadoop-2.4.0

Nixus Technologies - Hadoop Training Demo Video

2014-05-08T06:01:00.000-07:00

Nixus Technologies is the leading providers of Hadoop training. Its an online training program which will be taken through GTM (GoToMeeting). All Trainers are having 16+ years of industry experience and 4+ years of Big Data/Hadoop experience.

Following is the specialty of Big Data/Hadoop Training course:
- 80% training is practical
- The duration of course is 36 - 40 Hrs
- Training involves multiple workshops to explain the practical concepts
- Regular assignments will be given to the candidates
- Study material, PPTs, Project and POC codes, etc. will be given to the candidates
- Course involves 3 Proof Of Concepts
- Course involves a Real Life Project
- Trainer will assist you for interview preparation

Nixus Technologies aims to provide individual attention to all its trainees. Keeping it in mind, it targets to maximum of 10 candidates per batch to enable better learning by all. Training is highly interactive and provides classroom environment.

Nixus Technologies Big Data Introduction

2014-05-06T21:15:00.001-07:00

Running Cloudera in Distributed Mode

2010-11-21T09:09:00.000-08:00

This section contains instructions for Cloudera Distribution for Hadoop (CDH3) installation on ubuntu. This is CDH quickstart tutorial to setup Cloudera Distribution for Hadoop (CDH3) quickly on debian systems. This is shortest tutorial of Cloudera installation, here you will get all the commands and their description required to install Cloudera in Distributed mode(multi node cluster)

Prerequisite: Before starting Cloudera in distributed mode you must setup Cloudera in pseudo distributed mode and you need at least two machines one for master and another for slave(you can create more then one virtual machine(cluster) on a single machine).

Deploy Cloudera (CDH3) on Cluster:

COMMAND	DESCRIPTION
for x in /etc/init.d/hadoop-* ; do sudo $x stop ; done	Before starting Cloudera in distributed mode first stop each cluster
update-alternatives --display hadoop-0.20-conf	To list alternative Hadoop configurations on Your system
cp -r /etc/hadoop-0.20/conf.empty /etc/hadoop-0.20/conf.cluster	Copy the default configuration to your custom directory
update-alternatives --install /etc/hadoop-0.20/conf hadoop-0.20-conf /etc/hadoop-0.20/conf.cluster 50	To activate the new configuration on Your systems
update-alternatives --display hadoop-0.20-conf	To Check the new configuration on Your systems
or update-alternatives --set hadoop-0.20-conf /etc/hadoop-0.20/conf.cluster	To manually set the configuration
vi /etc/hosts	Then type IP-add master(eg: 192.168.0.1 master) IP-add slave(eg: 192.168.0.2 slave)
sudo apt-get install openssh-server openssh-client	install ssh
ssh-keygen -t rsa -P ""	generating rsa key for passwordless ssh
ssh-copy-id -i $HOME/.ssh/id_rsa.pub slave	setting passwordless ssh
Now go to your custom directory (conf.cluster) and change configuration files
vi masters then erase old contents and type master	masters file defines the namenodes of our multi-node cluster
vi slaves then erase old contents and type slave	slaves file lists the hosts, one per line, where the Hadoop slave daemons (datanodes and tasktrackers) will be run.
vi core-site.xml then type: <property> <name>fs.default.name</name> <value>hdfs://master:54310</value> </property>	Edit configuration file core-site.xml
vi mapred-site.xml then type: <property> <name>mapred.job.tracker</name> <value>master:54311</value> </property>	Edit configuration file mapred-site.xml
vi hdfs-site.xml then type: <property> <name>dfs.replication</name> <value>1</value> </property>	Edit configuration file hdfs-site.xml (value=number of slaves)
Now copy /etc/hadoop-0.20/conf.cluster directory to all nodes in your cluster
update-alternatives --install /etc/hadoop-0.20/conf hadoop-0.20-conf /etc/hadoop-0.20/conf.cluster 50	Set alternative rules on all nodes to activate your configuration.
for service in /etc/init.d/hadoop-0.20-; do sudo $service start; done for x in /etc/init.d/hadoop- ; do sudo $x stop ; done	Restart the daemons on all nodes in your cluster using the service scripts so that the new configuration files are read and then stop them
su -s /bin/bash - hdfs -c 'hadoop namenode -format'	Format namenode manually(Before starting namenode)
You must run the commands on the correct server, according to your role definition
/etc/init.d/hadoop-0.20-namenode start /etc/init.d/hadoop-0.20-secondarynamenode start /etc/init.d/hadoop-0.20-jobtracker start	To start the daemons on namenode on master
/etc/init.d/hadoop-0.20-datanode start /etc/init.d/hadoop-0.20-tasktracker start	To start the daemons on datanode on slave
Congratulations Cloudera CDH setup is completed

Running Cloudera in Pseudo Distributed Mode

2010-11-14T06:20:00.000-08:00

COMMAND	DESCRIPTION
$ sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"	If you are using ubuntu 10.04 LTS run this command
sudo apt-get install sun-java6-jdk	Install java
lsb_release –c	Name of the your distribution (let DISTRO)(eg: hardy or jaunty etc.)
vi /etc/apt/sources.list.d/cloudera.list Then type: deb http://archive.cloudera.com/debian DISTRO-cdh3 contrib deb-src http://archive.cloudera.com/debian DISTRO-cdh3 contrib	A repository enables your package manager to install cloudera replace DISTRO with the name of your distribution
sudo apt-get -y install curl	install curl
curl -s http://archive.cloudera.com/debian/archive.key \| sudo apt-key add -	Add a repository key. Add the Cloudera Public GPG Key to your repository
sudo apt-get update	Update APT package index
sudo apt-get -y install hadoop-0.20-conf-pseudo	Install Hadoop in pseudo-distributed mode: A pseudo-distributed Hadoop installation is composed of one node running all five Hadoop daemons: namenode, jobtracker, secondarynamenode, datanode, and tasktracker
for service in /etc/init.d/hadoop-0.20-*; do sudo $service start; done	Start the Cloudera Daemons
dpkg -L hadoop-0.20-conf-pseudo	Viewing the files on Debian systems
jps	It should give output like this: 14799 NameNode 14977 SecondaryNameNode 15183 DataNode 15596 JobTracker 15897 TaskTracker
Congratulations Cloudrea Setup is Completed. Now lets run some examples
hadoop jar /usr/lib/hadoop/hadoop-*-examples.jar pi 10 100	run pi example
hadoop fs -mkdir input hadoop fs -put /etc/hadoop-0.20/conf/.xml input hadoop-0.20 fs -ls input hadoop-0.20 jar /usr/lib/hadoop-0.20/hadoop--examples.jar grep input output 'dfs[a-z.]+'	run grep example
hadoop-0.20 fs -mkdir inputwords hadoop-0.20 fs -put /etc/hadoop-0.20/conf/.xml inputwords hadoop-0.20 fs -ls inputwords hadoop-0.20 jar /usr/lib/hadoop-0.20/hadoop--examples.jar grep inputwords outputwords 'dfs[a-z.]+'	run word count example
http://localhost:50070/	web based interface for name node
http://localhost:50030/	web based interface for Job tracker
for x in /etc/init.d/hadoop-* ; do sudo $x stop ; done	Shutdown CDH2 Hadoop services

Running Cloudera in Standalone Mode

2010-11-14T05:29:00.000-08:00

COMMAND	DESCRIPTION
$ sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"	If you are using ubuntu 10.04 LTS run this command
sudo apt-get install sun-java6-jdk	Install java
lsb_release –c	Name of the your distribution (let DISTRO)(eg: hardy or jaunty or lucid etc.)
vi /etc/apt/sources.list.d/cloudera.list Then type: deb http://archive.cloudera.com/debian DISTRO-cdh3 contrib deb-src http://archive.cloudera.com/debian DISTRO-cdh3 contrib	A repository enables your package manager to install cloudera replace DISTRO with the name of your distribution
sudo apt-get -y install curl	install curl
curl -s http://archive.cloudera.com/debian/archive.key \| sudo apt-key add -	Add a repository key. Add the Cloudera Public GPG Key to your repository
sudo apt-get update	Update APT package index
apt-cache search hadoop	List Hadoop packages on Debian systems
apt-get -y install hadoop-0.20	Install hadoop
dpkg -L hadoop-0.20	List the installed files
man hier	See that the Hadoop package has been configured
Congratulations Cloudrea Setup is Completed. Now lets run some examples
hadoop jar /usr/lib/hadoop-0.20/hadoop-*-examples.jar pi 10 100	Run pi example
cd /tmp mkdir input cp /etc/hadoop/conf/.xml input hadoop jar /usr/lib/hadoop-0.20/hadoop--examples.jar grep input output 'dfs[a-z.]+' cat output/*	Run grep example
cd /tmp mkdir inputwords cp /etc/hadoop/conf/.xml inputwords hadoop jar /usr/lib/hadoop-0.20/hadoop--examples.jar wordcount inputwords outputwords	Run word count example