May 26, 2021 Apache Storm
A working Storm cluster should have a Nimbus and one or more supervisors. /b10> Another important node is Apache ZooKeeper, which will be used for coordination between nimbus and supervisors.
Now let's
Take a closer look at Apache Storm's workflow
−
Initially, nimbus waits for Storm Topology to submit to it.
Once the topology is committed, it processes the topology and collects all tasks to be performed and the order in which they will be performed.
Nimbus then distributes the tasks evenly across all available supervisors.
At certain intervals, all supervisors send a heartbeat to nimbus to notify them that they are still running.
When the supervisor terminates and does not send a heartbeat to the heartbeat, nimbus assigns the task to another supervisor.
When the nimbus itself terminates, supervisor will work on the assigned tasks without any problems.
Once all the tasks are complete, the supervisor waits for the new tasks to come in.
At the same time, terminating nimbus will be automatically restarted by the service monitoring tool.
The restarted network will continue from where it stopped. S imilarly, terminating the supervisor can be restarted automatically. ce. and the supervisor can be restarted automatically, and both will continue as before, Storm guarantees that all tasks will be processed at least once.
Once all topologys have been processed, the network manager waits for the new topology to arrive and, similarly, for new tasks.
By default, there are two modes in the Storm cluster:
Local mode - This mode is used for development, testing, and debugging because it is the easiest way to see all topological components working together. ne. n local mode, the storm topology runs on a single JVM on the local machine.
Production Mode - In this mode, we submit the topology to a working Storm cluster, which consists of many processes, usually running on different machines. A s discussed in Storm's workflow, the working cluster runs indefinitely until it is shut down.