Skip to main content

Task queues

A task queue is where the engine puts the work of a durable function: its workflow tasks and its activities. Workers take work from the queues they serve. Every workspace has one managed queue, and you can add self-hosted queues for work that must run on your own machines.

QueueWho runs the work
managedFluidGrids, in its isolated sandbox. This is the default for every run and activity
A self-hosted queue, such as paymentsThe self-hosted workers you start for it

A self-hosted queue runs the workflow code (from the bundle you pushed) and the user activities that a durable function calls with ctx.activity. Node activities (ctx.run) and runs on managed never need a worker of yours.

Creating a queue​

In the app, open Task queues in the sidebar and select New queue. From the CLI:

fluidgrids queues create payments --max-tasks-per-second 20
fluidgrids queues list
fluidgrids queues describe payments # backlog, rates, pollers and workers
fluidgrids queues builds payments # the build ids its workers serve
fluidgrids queues update payments --clear-max-tasks-per-second
fluidgrids queues delete payments
SettingMeaning
NameLower-case letters, digits and dashes, at most 63 characters. managed is reserved
EnvironmentEvery environment, or one of dev, staging and prod (see Environments)
DescriptionOptional
Activity tasks per secondAn optional rate limit for the whole queue, 0.01 to 10,000
Wait for a worker (seconds)How long an activity may wait for a worker to take it, 10 seconds to 7 days. Default 10 minutes

A queue cannot be deleted while open runs or activities use it. Terminate or finish them first.

Sending work to a queue​

A durable function and its activities run on managed unless you name a queue:

import { declareActivity, workflow } from '@fluidgrids/workflows';

const charge = declareActivity<{ amount: number }, { id: string }>('billing.charge', {
taskQueue: 'payments',
});

export const order = workflow('order', { taskQueue: 'payments' }, async (ctx, input: { amount: number }) => {
return ctx.activity(charge, { args: { amount: input.amount }, startToCloseTimeout: '30s' });
});
  • An activity runs on the queue its call names (ctx.activity(charge, { taskQueue })), else the queue its declaration names, else the run's own queue.
  • A run starts on the workflow's queue, or on the queue given at the start: fluidgrids runs start order --task-queue payments. Child runs and continue-as-new can name another queue.
  • A user activity can never run on managed. fluidgrids push and fluidgrids validate refuse a workflow whose user activity would end up there, and the call fails with taskQueue.userActivityOnManaged if it gets that far.

Waiting for a worker​

When no worker serves the queue, the run page shows Waiting for a worker on queue … above the run. The work starts as soon as a worker connects. If no worker takes an activity within the queue's Wait for a worker time, that attempt times out as a schedule-to-start timeout, and the activity's retry policy decides what happens next. An activity's own scheduleToStartTimeout takes precedence over the queue's setting.

Environments​

A queue can be scoped to one environment. A plain name, such as orders, serves runs of every environment. A scoped queue is written orders@staging (or @dev, @prod) and only takes tasks of runs in that environment. A worker started with --environment staging, or a worker token issued for orders@staging, therefore never takes production work of a queue with the same name. See Self-hosted workers.

Permissions​

Reading queues needs the task-queue read permission, which the Task queues entry in the sidebar also checks. Creating, changing and deleting queues and worker tokens need the matching task-queue permissions; see Roles and permissions.

Next steps​