2017 © Pedro Peláez
 

library swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

image

kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  • Sunday, July 29, 2018
  • by kcloze
  • Repository
  • 29 Watchers
  • 339 Stars
  • 310 Installations
  • PHP
  • 0 Dependents
  • 0 Suggesters
  • 82 Forks
  • 0 Open issues
  • 60 Versions
  • 27 % Grown

The README.md

swoole-jobs

中文介绍

  • Distributed task processing system,similar to gearman,based on swoole
  • High performance / dynamic multi woker process consumption queue to accelerate backend time consuming service
  • There is no need to configure a crontab like gearman worker, swoole-jobs is responsible for managing all worker states
  • Support for pushing queues by HTTP API(swoole http server) , does not depend on php-fpm

1. Explain

  • Slower logic in web, such as statistical /email/ SMS / picture processing, etc.
  • Support redis/rabbitmq/zeromq or any other queue message store.
  • It is more stable and faster than the Yii / laravel framework itself.
  • With yii2/phalcon/yaf/ThinkPHP5 integration example, other frameworks can refer to src/Action code.
  • yii2 demo
  • ThinkPHP5 demo

2. Architecture diagram

Architecture diagram Process model, (*1)

3. Characteristic

  • job scheduling component based on swoole; distributed task processing system similar to gearman;, (*2)

  • redis/rabbitmq/zeromq and any other queue message store (currently only redis/rabbitmq)., (*3)

  • use swoole process to realize multi process management, the number of processes can be configured, and the worker process will automatically pull up after exiting., (*4)

  • the number of cycles of child processes can be configured to prevent memory leakage from business code; the default stop command will wait for the child process to exit smoothly., (*5)

  • support topic features, different job binding different topic;, (*6)

  • each topic starts the corresponding number of sub processes to eliminate the interaction between different topic., (*7)

  • according to the queue backlog, the sub process starts the process dynamically, and the number of the largest sub processes can be configured., (*8)

  • support composer, which can be integrated with any framework;, (*9)

  • log file automatic cutting, default maximum 100M, up to 5 log files, prevent log brush full disk;, (*10)

  • backlog, support for nail robot and other news alerts., (*11)

4. Install

4.1 composer

git clone https://github.com/kcloze/swoole-jobs.git
cd swoole-jobs

composer install

4.2 docker

  • git clone https://github.com/kcloze/swoole-jobs.git
  • cd swoole-jobs and composer install
  • Building a mirror based on the root directory Dockerfile
  • docker build -t swoole-jobs .
  • docker run -it -v ~/data/code/php:/data swoole-jobs /bin/bash
  • After entering the docker container, enter the project directory:
    • php ./bin/swoole-jobs.php start

5. How to running

5.1 example

1.edit config.php

2.start service
php ./bin/swoole-jobs.php start >> log/system.log 2>&1

3.push jobs
php ./tests/testJobsSerialzie.php

4.start api server
php ./bin/swoole-jobs.php http start

5.stop api server
php ./bin/swoole-jobs.php http stop 

5.2 Start parameter description

NAME
      - manage swoole-jobs

SYNOPSIS
      -php bin/swoole-jobs.php app [options]
        -Manage swoole-jobs daemons.

WORKFLOWS

      -help [command]
        -Show this help, or workflow help for command.

      -restart
        -Stop, then start swoole-jobs master and workers.

      -start
        -Start swoole-jobs master and workers.

      -stop
        -Wait all running workers smooth exit, please check swoole-jobs status for a while.

      -exit
        -Kill all running workers and master PIDs.

      -http start 
        -Start swoole http server for apis.

      -http stop
        -Stop swoole http server for api.

5.3 API parameter description

5.3.1 api url

  • http://localhost:9501/pushJobs

5.3.2 api params:

Params Type Demo
jobData json {"topic":"MyJob","jobClass":"\Kcloze\Jobs\Jobs\MyJob","jobMethod":"test2","jobParams":["kcloze",1532857253,"oop"],"jobExtras":[],"serializeFunc":"php"}

6. Service management

There are two ways to start and close the service online:

6.1 The startup script is added to the crontab timing task, which is executed once a minute (swoole-jobs automatically checks if it is executing, avoiding repeated startup).

* * * * * /usr/local/bin/php /***/swoole-jobs.php start >> /***/log/system.log 2>&1

6.2 Using SYSTEMd Management (failure restart, boot up)

more, (*12)

1. According to your own project path, modify: docs/systemd/swoole-jobs.service
2. sudo cp -f systemd/swoole-jobs.service /etc/systemd/system/
3. sudo systemctl --system daemon-reload
4. Service management

start service

sudo systemctl start swoole-jobs.service, (*13)

reload service

sudo systemctl reload swoole-jobs.service, (*14)

stop service

sudo systemctl stop swoole-jobs.service, (*15)

7.System screenshot

htop

demo, (*16)

status

status, (*17)

dingding message

message, (*18)

8. Change log

9. Matters needing attention

  • If you embed your own framework, you can refer to src/Action code to inherit the abstract class Kcloze\Jobs\Action\BaseAction.
  • Various framework services will start slightly different, for specific reference: Code for example/bin projects.
  • swoole >=4 version,need to set php.ini, disable coroutine

10. Pressure measurement

  • Bottleneck: redis/rabbitmq queue storage itself and job execution speed

11. Thanks

12. Contact

qq group:141059677, (*19)

13. Donation

  • If this project really helps you, please click on the top right corner for a star.

The Versions

29/07 2018

dev-master

9999999-dev https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

29/07 2018

v3.0.0

3.0.0.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

13/07 2018

v2.6.4.4

2.6.4.4 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

12/07 2018

v2.6.4.3

2.6.4.3 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

10/07 2018

v2.6.4.2

2.6.4.2 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

09/07 2018

v2.6.4.1

2.6.4.1 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

09/07 2018

v2.6.4

2.6.4.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

05/07 2018

v2.6.3.5

2.6.3.5 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

04/07 2018

v2.6.3.4

2.6.3.4 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

04/07 2018

v2.6.3.3

2.6.3.3 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

04/07 2018

v2.6.3.2

2.6.3.2 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

04/07 2018

v2.6.3.1

2.6.3.1 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

04/07 2018

v2.6.3

2.6.3.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

08/06 2018

v2.6.2

2.6.2.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

The Development Requires

by Avatar kcloze

scheduler job swoole 调度

01/06 2018

v2.6.1

2.6.1.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

01/06 2018

v2.6.0

2.6.0.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

15/05 2018

v2.5.7

2.5.7.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

25/04 2018

v2.5.6

2.5.6.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

12/04 2018

v2.5.5

2.5.5.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

09/02 2018

v2.5.4

2.5.4.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

02/02 2018

v2.5.3.4

2.5.3.4 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

31/01 2018

v2.5.3.3

2.5.3.3 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

30/01 2018

v2.5.3.2

2.5.3.2 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

30/01 2018

v2.5.3.1

2.5.3.1 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

29/01 2018

v2.5.3

2.5.3.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

29/01 2018

v2.5.2

2.5.2.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

19/01 2018

v2.5.1

2.5.1.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

18/01 2018

v2.5.0

2.5.0.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

09/01 2018

v2.4.7

2.4.7.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

06/01 2018

v2.4.6

2.4.6.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

02/01 2018

v2.4.5.2

2.4.5.2 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

02/01 2018

v2.4.5.1

2.4.5.1 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

31/12 2017

v2.4.5

2.4.5.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

29/12 2017

v2.4.4

2.4.4.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

29/12 2017

v2.4.3

2.4.3.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

27/12 2017

v2.4.2

2.4.2.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

27/12 2017

v2.4.1

2.4.1.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

27/12 2017

v2.3.6.x-dev

2.3.6.9999999-dev https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

27/12 2017

v2.4.0

2.4.0.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

26/12 2017

v2.3.6

2.3.6.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

25/12 2017

v2.3.5

2.3.5.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

23/12 2017

v2.3.4

2.3.4.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

19/12 2017

v2.3.3

2.3.3.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

18/12 2017

v2.3.0.x-dev

2.3.0.9999999-dev https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

18/12 2017

v2.3.x-dev

2.3.9999999.9999999-dev https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

15/12 2017

v2.3.2

2.3.2.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

14/12 2017

v2.3.1

2.3.1.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

14/12 2017

v2.3.0

2.3.0.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

 

by Avatar kcloze

scheduler job swoole 调度

11/12 2017

dev-dev

dev-dev https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

11/12 2017

v2.2.1

2.2.1.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件,类似gearman的分布式任务处理系统

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

04/12 2017

v2.1.5

2.1.5.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

01/12 2017

v2.1.4

2.1.4.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

30/11 2017

v2.1.3

2.1.3.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

30/11 2017

v2.1.2

2.1.2.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=7.0
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

29/11 2017

v2.1.1

2.1.1.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=5.4
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

28/11 2017

v2.1.0

2.1.0.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=5.4
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

28/11 2017

v2.0.1

2.0.1.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=5.4
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

28/11 2017

dev-daydaygo-master

dev-daydaygo-master https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=5.4
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

03/11 2016

v1.0.1

1.0.1.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=5.4
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度

31/10 2016

v1.0.0

1.0.0.0 https://github.com/kcloze/swoole-jobs

基于swoole的job调度组件

  Sources   Download

MIT

The Requires

  • php >=5.4
  • ext-swoole >=1.8.9
  • ext-redis *

 

by Avatar kcloze

scheduler job swoole 调度