QueuesΒΆ
Queues are an optional feature that allows directing a set of tasks to specific workers.
Queues are useful when different tasks have different usage patterns, for instance one task being fast and high priority while another task is slow and low-priority. To prevent the slow task from blocking the execution of the fast one, each task can be attached to its own queue:
import time
import logging
from spinach import Engine, MemoryBroker
logging.basicConfig(
format='%(asctime)s - %(threadName)s %(levelname)s: %(message)s',
level=logging.DEBUG
)
spin = Engine(MemoryBroker())
@spin.task(name='fast', queue='high-priority')
def fast():
time.sleep(1)
@spin.task(name='slow', queue='low-priority')
def slow():
time.sleep(10)
spin.schedule(slow)
spin.schedule(fast)
spin.start_workers(number=1, queue='high-priority', stop_when_queue_empty=True)
The task decorator accepts an optional queue name that binds the task to a specific queue. Likewise, passing a queue name to start_workers restricts workers to executing only tasks of this particular queue.
Note
By default all tasks and all workers use the spinach
queue
Note
Namespaces and queues are different concepts. While queues share the same Spinach
Engine
, namespaces make two Spinach Engines invisible to each other while still using
the same broker.