Improvment: Wont spawn extra queues #1508
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Consider the follow setup:
Currently, in every environment, the MasterSupervisor run all the supervisors queues.
In a multi server deployment, if you spawn horizon on second server with the --environment=production-2, you always get a production-1 supervisors too (default queue in this case) (picture 1).
![image](https://private-user-images.githubusercontent.com/4535817/376367962-808f939b-299f-4292-8fd9-c88e69891bb5.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk3MDI4MTIsIm5iZiI6MTczOTcwMjUxMiwicGF0aCI6Ii80NTM1ODE3LzM3NjM2Nzk2Mi04MDhmOTM5Yi0yOTlmLTQyOTItOGZkOS1jODhlNjk4OTFiYjUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIxNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMTZUMTA0MTUyWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YWIyY2YzZjBlODdjZjQyZjI2NWZkM2VkZDkyZDg3ODBiNGZhYmFhNWVkODFmYTc3ODA2MjM4MTgyNzFjNWM0ZCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.EvmMIkfg7DnuxONVqu05Y1EpRrpY4zBz-g6SMXwVlsY)
With this change, we can selectible choice the supervisors that we want to run on each enviroments, just checking for the maxProcess > 0.
Imagine that you want a horizon instance to only process slow queues, or you have a high power vm/cpu/vps that you just want to run specific queue.
I cant figure out how to dead with this cenario, without this change.
Maybe we can put a global config, to 'dont spawn queues not listed in supervisor'...
What do you think about?