May 26, 2021 Zookeeper
Zookeeper provides a flexible, coordinated infrastructure for distributed environments. /b10> The ZooKeeper framework supports many of today's best industrial applications. /b11> We'll discuss some of ZooKeeper's most significant applications in this chapter.
The ZooKeeper framework was originally in "Yahoo!" b uilt in . /b10> Well-designed distributed applications need to meet requirements such as data transparency, better performance, robustness, centralized configuration, and coordination. /b11> So they designed the ZooKeeper framework to meet these requirements.
Apache Hadoop is the driving force for the big data industry. /b10> Hadoop relies on ZooKeeper for configuration management and coordination. /b11> Let's take a look at ZooKeeper's role in Hadoop.
Suppose the Hadoop cluster bridges 100 or more commodity servers. T hen, you need to coordinate and name the service. /b11> So it involves computing a large number of nodes, each of which needs to synchronize with each other, know where to access the server, and know how to configure them. /b12> At this point in time, hadoop clusters need cross-node servers. /b13> ZooKeeper provides the ability to synchronize across nodes and ensure that tasks across Hadoop projects are serialized and synchronized.
Multiple ZooKeeper servers support large Hadoop clusters. /b10> Each client machine communicates with one of the ZooKeeper servers to retrieve and update its synchronization information. Some real-time examples are as follows:
The Human Genome Project - The Human Genome Project contains megabytes of data. /b10> The Hadoop MapReduce framework can be used to analyze data sets and find interesting facts about human development.
Healthcare - Hospitals can store, retrieve, and analyze a large number of patient medical records, typically megabytes.
Apache HBase is an open source, distributed NoSQL database for real-time read/write access to large data sets and running on HDFS. /b10> HBase follows the master-to-master architecture, and HBase-master controls all-access machines. /b11> The from the machine is called a zone server.
The installation of HBase distributed applications depends on the Running ZooKeeper cluster. /b10> Apache HBase uses ZooKeeper to help hosts and regional servers track the status of distributed data through centralized configuration management and distributed mutual exclusion mechanisms. Here are some use cases for HBase:
Telecommunications - The telecommunications industry stores billions of mobile call logs (approximately 30TB/month), and real-time access to these call logs is a huge task. HBase can be used to process all records in real time, easily and efficiently.
Social networking - Similar to the telecommunications industry, sites such as Twitter, LinkedIn and Facebook receive large amounts of data through user-created posts. /b10> HBase can be used to find recent trends and other interesting facts.
Apache Solr is a fast, open source search platform written in Java. /b10> It is a fast, fault-020s distributed search engine. /b11> Built on Lucene, it is a high-performance, full-featured text search engine.
Solr uses every feature of ZooKeeper widely, such as configuration management, leader elections, node management, data locking and synchronization.
Solr has two different sections, Index and Search. /b10> Indexes are processes that store data in an appropriate format so that it can be searched later. /b11> Solr uses ZooKeeper to index and search data in multiple nodes. ZooKeeper offers the following features:
Add/delete nodes as needed
Copy data between nodes and then minimize data loss
Share data between multiple nodes and then search from multiple nodes for faster search results
Some of Apache Solr's use cases include e-commerce, job search, and more.