Skip to main content

Hadoop Features and Characteristics

Apache Hadoop is the most popular and powerful big data tool, which  provides world’s best reliable storage layer –HDFS(Hadoop Distributed File System), a batch Processing engine namely  
MapReduce and a Resource Management Layer like YARN. Open-source – Apache Hadoop is an open source project. It means its code can be modified according to business requirements.
  • Distributed Processing– The data storage is maintained in a distributed manner in HDFS across the cluster, data is processed in parallel on cluster of nodes.

  • Fault Tolerance– By default the three replicas of each block is stored across the cluster in Hadoop and it’s changed only when required. Hadoop’s fault tolerant can be examined in such cases when any node goes down, the data on that node can be recovered easily from other nodes. Failures of a particular node or task are recovered automatically by the framework.

  • Reliability– Due to replication of data in the cluster, data can be reliable which is stored on the cluster of machine despite machine failures .Even if your machine goes down, and then also your data will be stored reliably.

  • High Availability– Data is  available and accessible even there occurs a hardware failure due to multiple copies of data. If any incidents occurred such as if your machine or few hardware crashes, then data will be accessed from other path.

  • Scalability– Hadoop is highly scalable and in a unique way hardware can be easily added to the nodes. It also provides horizontal scalability which means new nodes can be added on the top without any downtime.

  • Economic– Hadoop is not very expensive as it runs on cluster of commodity hardware. We do not require any specialized machine for it. Hadoop provides huge cost reduction since it is very easy to add more nodes on the top here. So if the requirement increases, then there is an increase of nodes, without any downtime and without any much of pre planning.

  • Easy to use– No need of client to deal with distributed computing, framework takes care of all the things. So it is easy to use.

  • Data Locality– Hadoop works on data locality principle which states that the movement of computation of data instead of data to computation. When client submits his algorithm, then the algorithm is moved to data in the cluster instead of bringing data to the location where algorithm is submitted and then processing it.

Comments

Post a Comment

Popular posts from this blog

Hadoop Design Principles

Following are the design principles on which  Hadoop  works: System shall manage and heal itself as per the requirement occurred. Fault Tolerant  are automatically and transparently route are managed around failures speculatively execute redundant tasks if certain nodes are detected to be running of slower phase. Performance is scaled based on linearity. Proportional change in terms of capacity with resource been change (Scalability) Compute must be moved to data. Data Locality is termed as lower latency, lower bandwidth. It is based on simple core, modular and extensible (Economical).
Post a Comment
Read more

HDFS Interview Questions And Answers

Compare HDFS & HBase Criteria HDFS HBase Data write process Append method Bulk incremental, random write Data read process Table scan Table scan/random read/small range scan Hive SQL querying Excellent Average What is the use of Hadoop? With Hadoop the user can run applications on the systems that have thousands of nodes spreading through innumerable terabytes. Rapid data processing and transfer among nodes helps uninterrupted operation even when a node fails preventing system failure. What are the operating systems on which Hadoop works? Windows and Linux are the preferred operating system though Hadoop can work on OS x and BSD. What is meant by Big Data? Big Data refers to assortment of huge amount of data which is difficult capturing, storing, processing or reprieving. Traditional database management tools cannot handle them but Hadoop can. Can you indicate Big Data examples? Facebook alone generates more than 500 terabytes of data daily w...
Post a Comment
Read more

Hadoop Interview Questions And Answers

Compare Hadoop & Spark Criteria Hadoop Spark Dedicated storage HDFS None Speed of processing average excellent Libraries Separate tools available Spark Core, SQL, Streaming, MLlib, GraphX Modes Hadoop can be run in? Hadoop can run in three modes: Standalone Mode:  Default mode of Hadoop, it uses local file stystem for input and output operations. This mode is mainly used for debugging purpose, and it does not support the use of HDFS. Further, in this mode, there is no custom configuration required for mapred-site.xml, core-site.xml, hdfs-site.xml files. Much faster when compared to other modes. Pseudo-Distributed Mode (Single Node Cluster):  In this case, you need configuration for all the three files mentioned above. In this case, all daemons are running on one node and thus, both Master and Slave node are the same. Fully Distributed Mode (Multiple Cluster Node):  This is the production phase of Hadoop (what Hadoop is known for) where data is ...
Post a Comment
Read more