minProcesses and maxProcesses aren't allocated according to queue priority

[sts-ryan-holton]

Horizon Version

5.21.4

Laravel Version

10.34.2

PHP Version

8.1.26

Redis Driver

PhpRedis

Redis Version

7.x

Database Driver & Version

Maria DB

Description

I have several Queue Worker configurations configured in my Horizon config. fast, default and slow jobs. Each, with different queues specified in the queue array.

My slow jobs are my most important, and typically I have a lot of jobs on my uptime queue, thus why this is closer to the start.

I wanted to have a reserve "min processes" of 10 allocated here and a maximum of 55 so it can scale correctly. However, what ended up happening is it give a minimum of 10 to each queue defined in the array:

• 10 for on-demand-runs-now
• 9 for uptime
• 10 for domains
• 10 for blacklists

etc...

And uptime never went above 9. What's going on here? I need the queues to process in order, and for their to be a higher priority for processes to be allocated to uptime.

Steps To Reproduce

My config I tried:

/*
|--------------------------------------------------------------------------
| Queue Worker Configuration
|--------------------------------------------------------------------------
|
| Here you may define the queue worker settings used by your application
| in all environments. These supervisors and settings handle all your
| queued jobs and will be provisioned by Horizon during deployment.
|
*/

'defaults' => [
    'supervisor-fast-jobs' => [
        'connection' => 'redis-short-running',
        'queue' => ['on-demand-runs-now', 'cron', 'listeners', 'redis-short-running'],
        'balance' => 'auto',
        'autoScalingStrategy' => 'time',
        'maxProcesses' => 1,
        'maxTime' => 300,
        'maxJobs' => 1000,
        'memory' => 128,
        'tries' => 1,
        'timeout' => 60,
        'nice' => 0,
    ],
    'supervisor-jobs' => [
        'connection' => 'redis',
        'queue' => ['on-demand-runs-now', 'notifications', 'dns', 'certificates', 'domains-expired', 'default'],
        'balance' => 'auto',
        'autoScalingStrategy' => 'time',
        'maxProcesses' => 1,
        'maxTime' => 300,
        'maxJobs' => 1000,
        'memory' => 128,
        'tries' => 2,
        'timeout' => 70,
        'nice' => 0,
    ],
    'supervisor-slow-jobs' => [
        'connection' => 'redis-long-running',
        'queue' => ['on-demand-runs-now', 'uptime', 'domains', 'blacklists', 'subscriptions', 'redis-long-running'],
        'balance' => 'auto',
        'autoScalingStrategy' => 'time',
        'maxProcesses' => 1,
        'maxTime' => 300,
        'maxJobs' => 1000,
        'memory' => 128,
        'tries' => 1,
        'timeout' => 280,
        'nice' => 0,
    ],
],

'environments' => [
    'production' => [
        'supervisor-fast-jobs' => [
            'minProcesses' => 10,
            'maxProcesses' => 5,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
        ],
        'supervisor-jobs' => [
            'minProcesses' => 10,
            'maxProcesses' => 1,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
        ],
        'supervisor-slow-jobs' => [
            'minProcesses' => 10,
            'maxProcesses' => 55,
            'balanceMaxShift' => 5,
            'balanceCooldown' => 3,
        ],
    ],
    'local' => [
        'supervisor-fast-jobs' => [
            'minProcesses' => 10,
            'maxProcesses' => 5,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
        ],
        'supervisor-jobs' => [
            'minProcesses' => 10,
            'maxProcesses' => 10,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
        ],
        'supervisor-slow-jobs' => [
            'minProcesses' => 10,
            'maxProcesses' => 55,
            'balanceMaxShift' => 5,
            'balanceCooldown' => 3,
        ],
    ]
],

[driesvints]

Hey there,

Can you first please try one of the support channels below? If you can actually identify this as a bug, feel free to open up a new issue with a link to the original one and we'll gladly help you out.

• Laracasts Forums
• Laravel.io Forums
• StackOverflow
• Discord
• Larachat
• IRC

Thanks!