Skip to main content

Hadoop Introduction

Apache Hadoop is the most important framework for working with Big Data. 

Hadoop biggest strength is scalability. It upgrades from working on a single node to thousands of nodes without any issue in a seamless manner.

The different domains of Big Data means we are able to manage the data’s are  from videos, text medium, transactional data, sensor information, statistical data, social media conversations, search engine queries, e commerce data, financial information, weather data, news updates, forum discussions, executive reports, and so on

Google’s Doug Cutting and his team members developed an Open Source Project namely known as HADOOP which allows you to handle the very large amount of data. 

Hadoop runs the applications on the basis of MapReduce where the data is processed in parallel and accomplish the entire statistical analysis on large amount of data.



It is a framework which is based on java programming. It is intended to work upon from a single server to thousands of machines each offering local computation and storage. It supports the large collection of data set in a distributed computing environment.

The Apache Hadoop software library based framework that gives permissions to distribute huge amount of data sets processing across clusters of computers using easy programming models.

The Apache Hadoop Module

Hadoop Common: Includes the common utilities which supports the other Hadoop modules

HDFS: Hadoop Distributed File System provides unrestricted, high-speed access to the data application.

Hadoop YARN: This technology is basically used for scheduling of job and efficient management of the cluster resource.

 MapReduce: This is a highly efficient methodology for parallel processing of huge volumes of data.

Comments

Post a Comment

Popular posts from this blog

Hadoop Design Principles

Following are the design principles on which  Hadoop  works: System shall manage and heal itself as per the requirement occurred. Fault Tolerant  are automatically and transparently route are managed around failures speculatively execute redundant tasks if certain nodes are detected to be running of slower phase. Performance is scaled based on linearity. Proportional change in terms of capacity with resource been change (Scalability) Compute must be moved to data. Data Locality is termed as lower latency, lower bandwidth. It is based on simple core, modular and extensible (Economical).
Post a Comment
Read more

HDFS Interview Questions And Answers

Compare HDFS & HBase Criteria HDFS HBase Data write process Append method Bulk incremental, random write Data read process Table scan Table scan/random read/small range scan Hive SQL querying Excellent Average What is the use of Hadoop? With Hadoop the user can run applications on the systems that have thousands of nodes spreading through innumerable terabytes. Rapid data processing and transfer among nodes helps uninterrupted operation even when a node fails preventing system failure. What are the operating systems on which Hadoop works? Windows and Linux are the preferred operating system though Hadoop can work on OS x and BSD. What is meant by Big Data? Big Data refers to assortment of huge amount of data which is difficult capturing, storing, processing or reprieving. Traditional database management tools cannot handle them but Hadoop can. Can you indicate Big Data examples? Facebook alone generates more than 500 terabytes of data daily w...
Post a Comment
Read more

Hadoop Interview Questions And Answers

Compare Hadoop & Spark Criteria Hadoop Spark Dedicated storage HDFS None Speed of processing average excellent Libraries Separate tools available Spark Core, SQL, Streaming, MLlib, GraphX Modes Hadoop can be run in? Hadoop can run in three modes: Standalone Mode:  Default mode of Hadoop, it uses local file stystem for input and output operations. This mode is mainly used for debugging purpose, and it does not support the use of HDFS. Further, in this mode, there is no custom configuration required for mapred-site.xml, core-site.xml, hdfs-site.xml files. Much faster when compared to other modes. Pseudo-Distributed Mode (Single Node Cluster):  In this case, you need configuration for all the three files mentioned above. In this case, all daemons are running on one node and thus, both Master and Slave node are the same. Fully Distributed Mode (Multiple Cluster Node):  This is the production phase of Hadoop (what Hadoop is known for) where data is ...
Post a Comment
Read more