Tuesday, July 19, 2016

Setting up Celery, python3, Celery flower and rabbitmq on mac 10.9.5

The following are the steps required to install celery, flower and rabbitmq on a mac 10.9.5 


Install python3

> brew install python3
> python3 -V # should display the version

Install and run rabbitmq

> brew install rabbitmq
> sudo rabbitmq-server # should run the rabbitmq server

# if the rabbitmq-server returns an error like “not_a_dets_file”, then delete the dets file. This is usually due to incorrect shutdown of the rabbitmq-server

> rm /usr/local/var/lib/rabbitmq/mnesia/rabbit\@localhost/recovery.dets

RabbitMQ can now be accessed on localhost:15672

Install celery 

> brew install celery
> mkdir & cd celery_tasks
> vim tasks.py

from celery import Celery

# first arg is the name to be prepended to the tasks
# backend param is an optional tag to retrieve the status of a backrgound running task
# broker is the broker service that routes the incoming and the outgoing messages
app = Celery('tasks', backend='amqp', broker='amqp://')

# all tasks must be decorated
@app.task(ignore_result=True)
def print_hello():
    print ('hello there')


# a task to generate prime numbers that should take time
@app.task
def gen_prime(x):
    multiples=[]
    results=[]
    for i in range(2,x+1):
        if i not in multiples:
            results.append(i)
            for j in range(i*i,x+1,i):
                multiples.append(j)
    return results

> celery worker -A tasks &

- To run the actual tasks
> python3
> from tasks import print_hello
> from tasks import gen_prime
> print_hello()
> primes=gen_prime(1000)
> print primes
> # to check if the task is complete
> primes.ready()
> # to get the results
> print primes.get()

Install Celery flower 

> pip3 install flower
> flower —port 5555
> # accessing flower on the port should show the list of active tasks

Celery flower can now be accessed on localhost:5555

Interesting links 

https://www.digitalocean.com/community/tutorials/how-to-use-celery-with-rabbitmq-to-queue-tasks-on-an-ubuntu-vps