How many workers can one supervisor provide?

Abstract

Client often find one problem, “Failed to submit a job, the number of free worker isn’t enough”, So what’t the algorithm of generating worker list.

SourceCode

Please refer to https://github.com/alibaba/jstorm/blob/master/jstorm-core/src/main/java/com/alibaba/jstorm/daemon/supervisor/Heartbeat.java

Algorithm

 Define the amount of workers that can be run on this machine. Each worker is assigned a port to use for communication

 if supervisor.slots.ports is null, 
 the port list will be generated by cpu cores and system memory size 
 for example, 
 there are cpu_num = system_physical_cpu_num/supervisor.slots.port.cpu.weight
 there are mem_num = system_physical_memory_size/(worker.memory.size * supervisor.slots.port.mem.weight) 
 The final port number is min(cpu_num, mem_num)