Queues

Queues are an optional feature that allows directing a set of tasks to specific workers.

Queues are useful when different tasks have different usage patterns, for instance one task being fast and high priority while another task is slow and low-priority. To prevent the slow task from blocking the execution of the fast one, each task can be attached to its own queue:

import time
import logging

from spinach import Engine, MemoryBroker


logging.basicConfig(
    format='%(asctime)s - %(threadName)s %(levelname)s: %(message)s',
    level=logging.DEBUG
)
spin = Engine(MemoryBroker())


@spin.task(name='fast', queue='high-priority')
def fast():
    time.sleep(1)


@spin.task(name='slow', queue='low-priority')
def slow():
    time.sleep(10)


spin.schedule(slow)
spin.schedule(fast)

spin.start_workers(number=1, queue='high-priority', stop_when_queue_empty=True)

The task decorator accepts an optional queue name that binds the task to a specific queue. Likewise, passing a queue name to start_workers restricts workers to executing only tasks of this particular queue.

Note

By default all tasks and all workers use the spinach queue

Note

Namespaces and queues are different concepts. While queues share the same Spinach Engine, namespaces make two Spinach Engines invisible to each other while still using the same broker.