Skip to content

Latest commit

 

History

History
39 lines (25 loc) · 1.19 KB

README.md

File metadata and controls

39 lines (25 loc) · 1.19 KB

Goal

Demonstrate how to use Celery to implement a pipeline with one component - this example will run a celery pipeline that consists of multiple functions

Files

  • deployment.py: Implementation of a selection of tasks to be run in the Docker container
  • attr.json: Meta data for the experiment/workflow

Prerequisites

  1. Demo 1 & 2 have been run successfully
  2. Background reading on Celery
  3. Read the API document on pipeline and components here

How to run the example

Do the following in the order listed below

Run the orchestration pipeline

python deployment.py

If you wish to interrupt or stop Celery, change directory to the utilities directory and run the provided script.

cd {YOUR DIRECTORY FOR TwinGraph}/examples/utils
python stop_and_delete.py

If your experiment ran successfully, your output should look something similar to thisCelery pipeline

Understanding the Python Code

You will notice that there is only one change between the previous deployments and this - there are additional parameters for the pipeline.