2017 © Pedro Peláez
 

craft-plugin craft-async-queue

A queue handler that moves queue execution to a non-blocking background process

image

ostark/craft-async-queue

A queue handler that moves queue execution to a non-blocking background process

  • Friday, June 15, 2018
  • by ostark
  • Repository
  • 4 Watchers
  • 40 Stars
  • 5,992 Installations
  • PHP
  • 7 Dependents
  • 0 Suggesters
  • 1 Forks
  • 1 Open issues
  • 17 Versions
  • 62 % Grown

The README.md

Async (Background) Queue

Latest Stable Version Total Downloads Monthly Downloads Buy us a tree, (*1)

With Craft's job queue you can run heavy tasks in the background. Unfortunately, this is not entirely true. When runQueueAutomatically => true (default), the job queue is handled by a ajax (FPM) call. With many jobs in the queue and limited PHP-FPM processes this can break your site. This plugin replaces Craft's default queue handler and moves queue execution to a non-blocking background process. The command craft queue/run gets executed right after you push a job to the queue., (*2)

Here you can find the initial discussion I started at craftcms/cms., (*3)

Development happens in my free time, but also during working hours. Thanks fortrabbit.com!, (*4)

Licence 🌳

This package is Treeware. If you use it in production, then we ask that you buy the world a tree to thank me for my work. By contributing to the Treeware forest you’ll be creating employment for local families and restoring wildlife habitats., (*5)

Requirements

  • Craft 3 or 4
  • Permissions to execute a php binary
  • proc_open()
  • PHP >=7.1 (for PHP 7.0 use ostark/craft-async-queue:1.3.*)

Installation

cd your/craft-project
composer require ostark/craft-async-queue
php craft install/plugin async-queue

If you run into Composer version conflicts:, (*6)

composer config platform --unset
composer update
php craft migrate/all
composer require ostark/craft-async-queue
php craft install/plugin async-queue

Configuration (optional)

The plugin uses symfony/process to execute the php binary. Usually the binary is located in /usr/bin/, but other common locations are auto detected as well. With the ENV var PHP_BINARY you can explicitly set the path, e.g. in your .env file like this:, (*7)

PHP_BINARY="/usr/local/Cellar/php71/7.1.0_11/bin/php"

By default 1 background process handles the queue. With the ASYNC_QUEUE_CONCURRENCY ENV var you can modify this behaviour., (*8)

# No concurrency
ASYNC_QUEUE_CONCURRENCY=1

# Or max 5 background processes
ASYNC_QUEUE_CONCURRENCY=5

To disable the plugin in certain environments, like on Windows which is not supported yet, set the DISABLE_ASYNC_QUEUE ENV var., (*9)

DISABLE_ASYNC_QUEUE=1

Note that the plugin will overwrite the runQueueAutomatically setting with Craft CMS. No configurartion required., (*10)

Tests

Beside the test suite you can run from the command line with this shortcut: composer tests, you can perform a test in the Craft CP. Navigate to Utilities > Async Queue Test and hit the Run test button., (*11)

Events

The command that runs in the background is basically php craft queue/run, however we add some linux specific syntax that executes the command in a non-blocking way. By setting useDefaultDecoration to false you prevent this. You have also the ability to modify the command itself., (*12)

// Add handler
\yii\base\Event::on(
     \ostark\AsyncQueue\QueueCommand::class,
     \ostark\AsyncQueue\QueueCommand::EVENT_PREPARE_COMMAND,
     function(\ostark\AsyncQueue\Events\QueueCommandEvent $event) {
         $event->useDefaultDecoration = false;
         $event->commandLine = "BEFORE {$event->commandLine} AFTER";
     }
);

Under the hood: Process list

Empty queue (only php-fpm master is running), (*13)

$ ps auxf | grep php

root      2953  0.0  0.0 399552 13520 ?        Ss   12:27   0:00 php-fpm: master process (/etc/php/fpm.conf)
````

**New job pushed** (php-fpm master + child + /usr/bin/php daemon started)

$ ps auxf | grep php, (*14)

root 2953 0.0 0.0 399552 13520 ? Ss 12:27 0:00 php-fpm: master process (/etc/php/fpm.conf) app 3031 2.2 0.2 718520 45992 ? S 12:31 0:00 _ php-fpm: pool www app 3033 1.2 0.2 280936 32808 ? S 12:31 0:00 /usr/bin/php craft queue/run app 3034 0.0 0.0 4460 784 ? S 12:31 0:00 _ sh -c /usr/bin/php craft queue/exec "1234" "0" "1" app 3035 1.2 0.2 280928 32280 ? S 12:31 0:00 _ /usr/bin/php craft queue/exec 1234 0 1 ```, (*15)

The Versions

15/06 2018

dev-master

9999999-dev

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

The Development Requires

by Oliver Stark

cms queue craftcms craft-plugin craft asyncqueue

15/06 2018

1.3.3

1.3.3.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

The Development Requires

by Oliver Stark

cms queue craftcms craft-plugin craft

04/06 2018

dev-win

dev-win

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

The Development Requires

by Oliver Stark

cms queue craftcms craft-plugin craft

06/04 2018

1.3.2

1.3.2.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

The Development Requires

by Oliver Stark

cms queue craftcms craft-plugin craft

05/04 2018

1.3.1

1.3.1.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

The Development Requires

by Oliver Stark

cms queue craftcms craft-plugin craft

23/02 2018

1.3.0

1.3.0.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms queue craftcms craft-plugin craft

22/02 2018

dev-lock

dev-lock

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms queue craftcms craft-plugin craft

19/02 2018

1.2.0

1.2.0.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms queue craftcms craft-plugin craft

05/12 2017

1.1.5

1.1.5.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms queue craftcms craft-plugin craft

25/11 2017

1.1.4

1.1.4.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms queue craftcms craft-plugin craft

15/11 2017

1.1.3

1.1.3.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms queue craftcms craft-plugin craft

06/11 2017

1.1.2

1.1.2.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms queue craftcms craft-plugin craft

16/10 2017

1.1.1

1.1.1.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms queue craftcms craft-plugin craft

13/10 2017

1.1.0

1.1.0.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms queue craftcms craft-plugin craft

25/08 2017

1.0.3

1.0.3.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms craftcms craft-plugin craft asyncqueue

25/08 2017

1.0.2

1.0.2.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms craftcms craft-plugin craft asyncqueue

24/08 2017

1.0.1

1.0.1.0

A queue handler that moves queue execution to a non-blocking background process

  Sources   Download

MIT

The Requires

 

by Oliver Stark

cms craftcms craft-plugin craft asyncqueue