Skip to content

CLI reference

The taskq CLI is the primary operational interface for managing migrations, running workers, probing health, rendering the operational insights, and serving the admin UI. All commands load settings from TASKQ_* environment variables or .env files via dotenvmodel; environment variables take precedence over .env files.

Installation

The taskq command is installed as part of the taskq-py package:

uv add taskq-py

The console-script entry point is taskq.cli:main. See ../getting-started/quick-start.md for initial environment setup.


Global environment variables

These variables are read by all commands via TaskQSettings.load() or WorkerSettings.load().

Variable Default Description
TASKQ_PG_DSN postgresql://taskq:taskq@localhost:5432/taskq Postgres connection string used by migrate and health commands
TASKQ_SCHEMA_NAME taskq Postgres schema for all TaskQ tables

Worker-specific variables (pool sizes, heartbeat timing, cancellation grace periods, etc.) are documented in workers.md.


taskq dev

Starts a worker in development mode with automatic restart on file changes. Useful during local development: it saves manually killing and restarting the worker after every code edit.

taskq dev MODULE:ATTR [OPTIONS]

Arguments:

Argument Description
MODULE:ATTR module:attr reference to the actor registry, same syntax as taskq worker --actors.

Options:

Option Type Default Description
--watch str (repeatable) current working directory Path to watch for changes. Pass the flag multiple times to watch several directories.
--grace-period int 5 Seconds between SIGTERM and SIGKILL when stopping the worker before a restart. 0 means send SIGTERM then kill immediately if still alive.

How it works:

  1. Imports MODULE:ATTR at startup and exits with a clear error if the module is not found or the attribute does not exist.
  2. Spawns taskq worker --actors MODULE:ATTR as a child subprocess. The child inherits the parent's full environment, so TASKQ_* variables set in the shell are picked up normally.
  3. Watches the specified paths with watchfiles.awatch() using DefaultFilter (ignores .git, __pycache__, editor temp files, etc.) and a 400 ms debounce window.
  4. On any file change: sends SIGTERM to the running worker, waits up to --grace-period seconds, then SIGKILL if still alive. Re-validates the import, then spawns a fresh worker. If the re-import fails (e.g. a syntax error) it logs a warning and waits for the next change rather than crashing.
  5. Ctrl-C (SIGINT) stops the worker cleanly and exits with code 0.

Each restart gets a fresh Python interpreter; no importlib.reload() is involved, so module-level state (Pydantic models, actor registries, config objects) is always clean.

Requirements:

taskq dev requires the reload extra:

uv add "taskq-py[reload]"

This installs watchfiles. Without it the command prints an install hint and exits with code 1.

Example: watch a single package directory

taskq dev myapp.actors:registry --watch src/myapp

Example: watch multiple directories

taskq dev myapp.actors:registry --watch src/myapp --watch config

Example: fast restart (no grace period)

taskq dev myapp.actors:registry --grace-period 0

Exit codes:

Code Condition
0 Clean exit via Ctrl-C
1 Bad MODULE:ATTR syntax, import failure, or watchfiles not installed

Note: taskq dev is intended for local development only. Do not run it in production; use taskq worker directly under a process supervisor (systemd, Docker, Kubernetes).


taskq migrate status

Shows which migrations have been applied and which are pending.

taskq migrate status

Connects to TASKQ_PG_DSN, queries {schema}.schema_migrations, and prints one line per discovered migration file.

Example output:

schema: taskq
applied: 1
  [✔] 01.00.00_01_pre_initial.sql

Options:

Option Type Default Description
--pg-credential-provider str \| None None module:attr reference to a PgCredentialProvider. The connection is opened through it instead of the DSN's static password. Overrides TASKQ_PG_CREDENTIAL_PROVIDER. See Managed identities.

A ✔ marker indicates the migration has been applied. An empty marker indicates it is pending. Migrations are discovered from the built-in migration directory in the package.

Without the flag the command uses TASKQ_PG_DSN and TASKQ_SCHEMA_NAME from the environment.


taskq migrate up

Applies pending migrations in order.

taskq migrate up [OPTIONS]

Options:

Option Type Default Description
--phase pre \| post \| None None Restrict to only pre or only post phase migrations. When absent, applies both phases in order.
--target str \| None None Stop after applying this migration version (inclusive). Version format matches the filename prefix, e.g. 01.00.00_01.
--max-steps int \| None None Maximum number of migrations to apply in this invocation.
--ddl-lock-timeout float 30 Seconds a transactional migration waits for a table lock before it fails and rolls back (SET LOCAL lock_timeout; see Upgrading: the migration gave up waiting for a table lock). 0 waits indefinitely, parking every statement on the table behind the queued DDL. Values below one millisecond and non-finite values (nan, inf) are refused: a sub-millisecond bound would truncate to lock_timeout = 0 and wait indefinitely, the opposite of what was asked.
--allow-checksum-drift bool false Proceed past a drifted checksum ledger: an APPLIED migration's ledger checksum differs from the bundled file, which refuses the run by default. Verify the edit is safe first; the drift is still logged as a migration-checksum-drift warning on every run and the ledger keeps the stored checksum. See Upgrading: the checksum ledger.
--pg-credential-provider str \| None None module:attr reference to a PgCredentialProvider. The connection is opened through it instead of the DSN's static password. Overrides TASKQ_PG_CREDENTIAL_PROVIDER. See Managed identities.

The command is idempotent: each migration is recorded in {schema}.schema_migrations and is skipped on subsequent runs. Running taskq migrate up with no options applies all pending migrations.

On failure (exit code 1) the command never prints a traceback. When a migration fails during apply, the command diagnoses itself: it names the failed migration, reports the state the schema was left in: a clean rollback for transactional migrations, or for -- taskq:no-transaction migrations the statements that remain applied plus any INVALID indexes found, and prints the single action to take. When the database connection itself cannot be established, there is nothing to diagnose, so it prints a short report naming the connection error and the same re-run action.

Example: apply all pending:

taskq migrate up

Example output:

applied 2 migration(s):
  01.01.00_01_pre_add_reservation_slots.sql
  01.01.00_02_post_add_reservation_slots.sql

When there is nothing to apply:

no pending migrations

Example: apply only pre-phase migrations up to a target version:

taskq migrate up --phase pre --target 01.01.00_01

taskq migrate disable-hypertables

Converts the TimescaleDB hypertables back to plain vanilla tables — the disable mirror of the migrate up deploy step's conversion. See TimescaleDB: turning it back off for the full mechanics and the crash-safety guarantees.

taskq migrate disable-hypertables

Options:

Option Type Default Description
--pg-credential-provider str \| None None Same as migrate up's.

The gate: flip TASKQ_TIMESCALEDB_HYPERTABLES=false first. With the flag still true the command exits 1 with the remedy (the library-level gate it wraps is a zero-statement no-op, and the CLI refuses to let a mistyped invocation look like a completed disable). On a server without the extension it is a no-op success (nothing to disable).

The command runs under the same migration advisory lock migrate up holds, re-reads the same settings cascade the workers run with, and on failure prints the same self-diagnosing report (exit 1, never a traceback). A crashed run converges on the re-run.

Example output:

disabled hypertables on 3 table(s):
  restored to plain: jobs_archive
  restored to plain: job_attempts_archive
  restored to plain: job_events
  removed retention policy: jobs_archive:365 days
  removed retention policy: job_attempts_archive:365 days
  removed retention policy: job_events:7 days
  removed compression policy: jobs_archive
  removed compression policy: job_attempts_archive

When there is nothing to disable:

no hypertables to disable: the retention tables are already plain

taskq worker

Starts a TaskQ worker process. Blocks until SIGTERM or SIGINT.

taskq worker --actors MODULE:ATTR [OPTIONS]

Options:

| Option | Type | Default | Env var override | Description | |---|---|---|---|---|---| | --actors | str | required | n/a | module:attr reference to the actor registry | | --queues | list[str] | None | TASKQ_QUEUES | Queue names to consume; repeat the flag once per queue name | | --max-concurrency | int | None | TASKQ_MAX_CONCURRENCY | Upper bound on concurrent jobs | | --poll-interval | float | None | TASKQ_POLL_INTERVAL | Producer loop fallback polling cadence (seconds) | | --worker-group | str | None | TASKQ_WORKER_GROUP | Consumer group name for observability spans | | --worker-label | str | None | TASKQ_WORKER_LABEL | Human-readable label stored in the workers table | | --workgroup-instance | str | None | TASKQ_WORKGROUP_INSTANCE | UUIDv7 identifying the workgroup orchestrator that launched this worker | | --health-socket-path | str | None | TASKQ_HEALTH_SOCKET_PATH | Unix socket path for the health server (use unique paths when running multiple workers) | | --force-update-actor-config | bool | False | TASKQ_FORCE_UPDATE_ACTOR_CONFIG | Overwrite metadata-drifted actor-config rows at startup (queue and capacity are never boot-rewritten) | | --until-idle | bool | False | n/a | Run until all subscribed queues are drained, then exit. Exit 0 if all jobs succeeded, 3 if any failed, 4 if idle-max-runtime was exceeded. Incompatible with cron-driven workloads. | | --idle-settle-window | float | None | TASKQ_IDLE_SETTLE_WINDOW | Seconds to wait after queues appear empty before declaring drained. Default 2.0. Only used with --until-idle. | | --idle-poll-interval | float | None | TASKQ_IDLE_POLL_INTERVAL | How often to check queue depth. Default 1.0. Only used with --until-idle. | | --idle-max-runtime | float | None | TASKQ_IDLE_MAX_RUNTIME | Maximum wall-clock seconds before forcing exit (code 4). Only used with --until-idle. | | --pg-credential-provider | str | None | TASKQ_PG_CREDENTIAL_PROVIDER | module:attr reference to a PgCredentialProvider (an instance, a zero-arg factory, or the provider class also work). Every Postgres pool and dedicated connection is then built through it, so SIGHUP / TASKQ_RELOAD_INTERVAL rotate real credentials. See Managed identities. | | --redis-credential-provider | str | None | TASKQ_REDIS_CREDENTIAL_PROVIDER | module:attr reference to a RedisCredentialProvider, in the same shapes as --pg-credential-provider. Requires TASKQ_REDIS_URL. |

All other worker settings are read from environment variables. See workers.md for the full list.

--actors format

The value must be a module:attr string with exactly one colon separator. Both module and attr must be non-empty.

myapp.actors:registry          # module=myapp.actors, attr=registry
myapp.workers.email:handlers   # module=myapp.workers.email, attr=handlers

The module is imported at startup via importlib.import_module. The attribute must resolve to one of:

  • Mapping[str, ActorRef]: keys are actor names, values are ActorRef instances.
  • Iterable[ActorRef]: names are read from ActorRef.name on each element.

Any other type (including a plain list of non-ActorRef objects) prints an error and exits with code 1.

Example: iterable form

# myapp/actors.py
from taskq.actor import actor
from pydantic import BaseModel


class EmailPayload(BaseModel):
    to: str
    subject: str


@actor(queue="email")
async def send_email(payload: EmailPayload) -> None: ...


registry = [send_email]
taskq worker --actors myapp.actors:registry

Example: mapping form

# myapp/actors.py
registry = {"send_email": send_email, "resize_image": resize_image}
taskq worker --actors myapp.actors:registry

--queues flag

--queues is a multi-value flag. Pass it once for each queue name:

taskq worker --actors myapp.actors:registry --queues default --queues priority --queues email

Do not pass a comma-separated string directly to --queues on the command line. Use the environment variable for comma-separated input:

TASKQ_QUEUES=default,priority,email taskq worker --actors myapp.actors:registry

On the command line the flag must be repeated once per queue.

--force-update-actor-config

At startup the worker compares each registered actor's metadata value against the stored row in {schema}.actor_config. If it differs and this flag is absent, the worker refuses to start and prints:

ActorConfigDriftList: ...
Re-run with --force-update-actor-config to overwrite, or set TASKQ_FORCE_UPDATE_ACTOR_CONFIG=true.

max_concurrent, max_pending, result_ttl, and queue never appear in this error: they are operator-owned once a stored row exists (the @actor(...) literal only seeds them the first time) and can never drift into a refusal. A differing queue literal logs actor-config-queue-override at WARNING and the boot adopts the stored assignment (the rolling-deploy window of taskq actor-config move-queue); a differing capacity literal logs actor-config-capacity-override at info. Use taskq actor-config set to change capacity, taskq actor-config move-queue to move an actor's queue; see ActorConfig sync.

Use this flag on the first new pod of a rolling deploy when an actor's metadata has changed. Remove it for subsequent pods; it is not safe to run permanently as it allows silent metadata drift. See workers.md for the full drift protocol.

taskq actor-config

Inspect and tune stored {schema}.actor_config capacity fields without a code change or a worker restart:

taskq actor-config list
taskq actor-config get checkout_charge
taskq actor-config set checkout_charge --max-concurrent 20
taskq actor-config set checkout_charge --max-pending 5000
taskq actor-config set checkout_charge --clear-max-pending

All three capacity fields are live. --max-concurrent and --result-ttl take effect immediately for every worker in the fleet: the dispatch query re-reads max_concurrent every dispatch cycle, and the terminal-write path re-reads result_ttl on every job completion. --max-pending is enforced by every enqueue-side process (your web app, your workers' sub-job enqueues) through a TTL-bounded cache of the table (default 5s staleness), so a change propagates fleet-wide within seconds, with no redeploy and no worker restart.

Clearing a field writes NULL, which means different things per field on purpose: --clear-max-concurrent makes the actor unlimited (the dispatch SQL cannot see the code literal once a row exists), while --clear-max-pending / --clear-result-ttl revert to the @actor(...) literal; clearing undoes your override and restores the code default. For result_ttl the reverted literal is applied from each job's completion timestamp by the completing worker (never re-pinned to the enqueue time), so a job that sat in the queue longer than its TTL still gets its full result lifetime.

To see why a change is (or isn't) taking effect, diff the stored rows against the code literals in your registry:

taskq actor-config diff --actors myapp.actors:registry

Per actor and capacity field this prints the @actor(...) literal, the stored value, and the value the engine currently enforces (effective). It also flags queue mismatches (assignment drift: boot adopts the stored queue, but the cron leader's fires follow it while producers enqueue by their own literal, so the two routing halves disagree until you run taskq actor-config move-queue or deploy the matching literal), metadata mismatches (startup-blocking), actors with no stored row yet, and leftover rows whose actor is no longer registered. Note the no-row case is field-dependent: an actor with no stored row does not dispatch at all (max_concurrent shows effective=0; the dispatch capacity gate reads only actor_config rows until a worker startup seeds it), while max_pending / result_ttl fall back to the code literal.

taskq actor-config set requires the actor to have a stored row already (created by a worker startup that registered it). queue is moved by taskq actor-config move-queue; metadata is structural and only ever changes by redeploying a new @actor(...) registration (plus --force-update-actor-config if a stored row already exists).

taskq actor-config list

Lists every stored actor_config row, one line per actor:

taskq actor-config list
  checkout_charge: max_concurrent=20 max_pending=5000 queue=default result_ttl=None updated_at=2026-01-01 00:00:00+00:00

An empty table prints no actor_config rows. The list is the authoritative view of what can dispatch: rows exist only once a worker startup registered the actor, and the dispatch capacity gate reads only stored rows, so an actor whose row is missing does not appear here and does not dispatch at all. For stored-vs-code drift per field, use taskq actor-config diff instead.

taskq actor-config get

Shows the stored actor_config row for one actor:

taskq actor-config get checkout_charge

The actor is named positionally, and unlike set, get does not need the actor to be in the loaded registry - it reads the row by name.

Exit codes:

Code Condition
0 Row printed
1 No stored actor_config row for that actor

A missing row is a finding, not a cosmetic absence: the dispatch capacity gate reads only stored rows, so jobs for that actor accumulate pending with no error anywhere until a worker startup seeds the row.

taskq actor-config deregister

Deregister an actor: delete its actor_config row with safety checks.

taskq actor-config deregister <ACTOR> [--force] [--purge-queue]
  • <ACTOR>: actor name (positional argument)
  • --force: cancel pending/scheduled jobs, disable enabled cron schedules, and proceed despite non-terminal jobs. Running jobs still block.
  • --purge-queue: also delete the orphaned queues row if no other actor references it.

Exit code 0 on success, 2 on refusal (with error message), 3 on unknown actor.

taskq actor-config move-queue

Move an actor to a different queue in one operator action, the one-step replacement for the old four-write lockstep (code literal + stored row + consumed-queue set + target queues row) whose fail-closed half refused worker boot mid-move:

taskq actor-config move-queue <ACTOR> <NEW_QUEUE>

The actor's pending/scheduled backlog is rewritten onto the target queue as bounded committed batches, then one final transaction locks the stored assignment, carries the old queue's queues row (mode + max_concurrent) to the target when the target has no row of its own, and flips it; old-queue strays drain through the target's consumers, and a crash mid-drain re-runs cleanly (the abort logs actor-queue-move-aborted with the count already committed). Running jobs finish on the workers that claimed them; any that re-pend instead (failure retry, crash reclaim, operator retry) keep their old queue label as an audit trail but are routed at dispatch by the actor's current assignment, so the running-job tail drains through the target queue's consumers; other actors on the old queue are untouched; cron fires follow the moved assignment from the flip on.

Deploy the matching @actor(queue=...) literal before, during, or after the move, in any order. Workers boot on either side of the window (a stale literal logs actor-config-queue-override and adopts the stored assignment). Ensure workers consume the new queue, and keep consuming the old queue until every producer runs the new literal; stale producers keep enqueueing to it.

The command reports how many pending jobs still carry the old queue label. That count is the residual stale producers keep adding to, and it is what tells you when the retired queue can stop being consumed; keep its consumers up until it reaches zero.

Exit code 0 on success, 2 on refusal (invalid queue name, the actor is already on that queue, the assignment changed concurrently, or a drain batch exceeded its statement timeout: batches committed before the abort are kept, so the move is incomplete and safe to re-run), 3 on unknown actor.

taskq queue migrate

The same move under the queue noun, for when you are thinking about queue lifecycle rather than about the actor_config table the assignment is stored in:

taskq queue migrate <ACTOR> --to <QUEUE>

The target queue is named by an explicit --to rather than positionally. Both arguments of a move are plain strings, and two bare positionals are easy to transpose under pressure, with the consequence that the backlog drains onto a queue that was never the target.

Behaviour, reporting and exit codes are identical to taskq actor-config move-queue.

taskq doctor

A read-only health report for the misconfigurations that produce no error anywhere:

taskq doctor --actors myapp.actors:registry

The report's first finding family is the detected storage mode — vanilla, timescale-apache, or timescale-tsl, detected from the connected server (the extension catalog, then the extension's timescaledb.license setting), never read from settings — with that mode's capability consequences in one glance (which retention mechanisms exist, what the sweeps own). The family's red arms: the flag on in an environment whose server detects vanilla gets named as drift, because the next taskq migrate up refuses; and a license downgrade — an apache-licensed server still carrying conversion-era policy jobs from an earlier timescale-license deployment (they fail on every run under the downgraded license, and the sweeps defer the aged end to them, so aged rows strand) gets named as drift with the license-restore remedy. The mode × capability matrix is timescaledb.md's support matrix.

TaskQ refuses boot only on structural stored-config drift, so a whole family of capacity and configuration problems fails silently, so their only symptom is work that quietly does not happen. doctor names them together:

  • a registered actor with no stored row, which never dispatches (the dispatch capacity gate reads only actor_config rows);
  • pending/scheduled jobs whose actor has no stored actor_config row (a renamed or removed actor that old producers or old rows still reference): they wait forever with no error anywhere, and no registry walk can name them, because the registry no longer knows the name. Re-register the actor and seed its row (a worker startup does this), or purge the jobs if the actor was retired;
  • pending/scheduled jobs routed to a queue no live worker serves: the routing queue is the job's own queue, or the actor's stored assignment once the row is re-pended. Start a worker subscribed to the queue or move the actor onto a served one (see runbooks.md: TaskQStrandedJobs);
  • a stale queues row whose queue no actor is assigned to; it is inert now, but silently applied to the next actor moved onto that name;
  • incoherent capacity combinations: a max_pending below max_concurrent (the actor may queue fewer jobs than it may run at once, so its cap is unreachable), and an actor cap above its queue's cap (the queue binds first, so raising the actor cap changes nothing).

The report's environment family sits upstream of all of these: every TASKQ_-prefixed environment variable that matches no settings field is a typo applying its intended setting's default silently (the settings loader reads only the names it defines — see configuration.md for the full variable tables). Each finding names the offender, states the consequence, and offers the closest real setting name as the remedy hint (a typo is nearly always one character of drift). The wrong value was in force before any worker registered, so this family is checked before every stored-row condition in the report.

The two stranded-jobs families are the same computation the leader's stranded-jobs sweep runs every minute, issued here on demand: doctor is the surface an operator reaches for mid-incident, and it cannot wait on a leader tick. The result is per actor, bounded by the distinct-actor count, never by backlog depth.

When a live worker's workers row metadata carries a non-empty event-loop stall tally, doctor reports one finding per attributed actor: which worker recorded it, the actor, the kind counts (blocking_call: a sync call that released the GIL; gil_held: sync work that held it), and the remedy. The tally is the rolling top-20 the worker's lag watchdog attributed (see runbooks.md: Event-loop stall attribution); the worker's own event-loop-stall-attributed warnings name the exact file:line.

A stored max_concurrent=0 is labelled drain mode and a stored NULL is labelled uncapped, so a deliberate drain is distinguishable from an accidental zero and a real "no actor-level cap" from missing data.

Pass --platform-grace-seconds (Kubernetes terminationGracePeriodSeconds, an ACA/ECS stop timeout, compose stop_grace_period, systemd TimeoutStopSec) and doctor compares it against the worker's modelled worst-case shutdown: a platform grace below the worst case gets SIGKILLed mid-teardown, degrading every shutdown to crash reclaim (leases expire, in-flight work re-runs); the report names the shortfall and the fix. The worker itself cannot see the platform's number, so this is the one check that needs the operator to supply it.

The operational-insight findings

Beyond stored config, doctor reads the live fleet through the operational-insights SQL layer — the same statements the insights guide documents, run over the 24h window (a member of that layer's closed window set: 24h spans one full diurnal traffic cycle, the shortest window that cannot mistake a nightly lull for a fleet to shrink, while staying far inside the default 30-day prune retention). Four families, each naming its data and its remedy:

  • STARVED (queue): the queue's utilization — due depth ÷ effective capacity, where effective capacity is sum(max_concurrent) over the actors routed to the queue × live workers — exceeds 2x, or the oldest due job's age exceeds the queue's own p95 wait by 4x (with a 60s floor). Threshold derivations: 2x means a full second claim wave of due work survives after the first drains entirely — one wave is a burst the dispatcher absorbs by design, two sustained waves is a fleet too small for its arrival rate, and the remedy's own granularity is one wave (one worker, or one max_concurrent step). "Sustained" is load-bearing: the utilization arm only fires when the oldest due job has outlived the 60s floor, because a depth snapshot younger than that is a burst in the act of being absorbed — an arrival-rate claim needs a depth that persisted past the claim-in-flight window. The 4x p95 factor survives the burst confound because the p95 is computed over the same window the burst inflates; the 60s floor is 2x the 30s worker-liveness window, so a claim already in flight is never reported. The remedy names the only two levers that add dispatch capacity: start another worker serving the queue, or raise the serving actor's max_concurrent. utilization IS NULL (due work nothing can serve) has two shapes, reported by different lines: no live worker on the queue is the stranded-jobs family's shape; a live worker behind a stored max_concurrent = 0 is the deliberately-stopped drain mode, named by the stored-capacity line itself — neither is reported twice.
  • OVERPROVISIONED (queue): live workers on a queue with zero due depth and fewer terminalisations across the whole window than workers — fewer than one completion per worker (the verdict fetch_overprovisioning computes). The remedy is to consolidate the workers serving the queue in the workgroup config — never anything destructive; the finding says explicitly that nothing is deleted.
  • SLOW DRAIN (queue): the drain estimate (fetch_drain_estimates's eta_seconds — due depth ÷ the window's completions per second) exceeds the 24h observation window itself, with the due depth having persisted past the 60s persistence floor (the imbalance read's oldest-due age). Derivation: the eta is a throughput extrapolation whose only honest input is the traffic the window actually carried, so the window is the longest horizon the rate has evidence for — and it is the operator's own "will this be done by tomorrow?" period; an eta beyond it means the depth exceeds everything the entire window completed. The persistence floor is what keeps the claim honest for an idle-capacity fleet, whose observed rate is demand-limited, not capacity-limited: a depth younger than the claim-in-flight window is a burst the idle workers are absorbing, and its "eta" is fiction (a 51-job burst against an idle 32-slot worker read as ~1.0 days without it). The finding states the eta and the confidence caveat: the estimate rests on the window's realised traffic (has_traffic), assumes the next window looks like the last one, and does not include the armed wave. A window with no traffic renders no estimate at all (eta_seconds is NULL — never zero, which would read as "already drained"), so no finding fires there.
  • CRON LAG (schedule): a schedule whose fan-out outruns its clearance (fetch_cron_ledger) — either runaway_trending (fires > cleared in both the current and the prior window; one window is a burst, two consecutive is the runaway shape) or an outstanding backlog above one catch-up window's slot capacity, defined as the schedule's own demonstrated clearance in its best window (max(cleared_window, cleared_prior), required positive): what the fleet actually cleared, not a theoretical ceiling — and never zero, because a schedule with no demonstrated clearance has nothing to compare against and its in-flight fires are honest work in flight, not a lag (a capacity of zero would read every first in-flight fire as an uncatchable backlog). The best-of-two guards the ledger's right-edge confound (clearance lags fires at the window's edge, so a burst of fresh fires does not read as uncatchable), and the backlog arm requires the schedule to have fired within the two-window horizon — a weekly cron with one long-running fire is work in flight, not a lag. The finding names the schedule id, actor and cron expression, sizes the backlog, and gives the two honest remedies: slow the cron (widen the interval or raise its budget), or add workers for the actor (raise its max_concurrent so more fires clear per wave).

These families read, they never write — the same read-only contract as the stored-config families above. DST's allof double-fire and budget-deferred fires are the ledger's documented confounds (insights.md carries them per metric); read a CRON LAG verdict against the schedule's dst_strategy before acting on it.

It issues no writing statement, so it is safe to run against production mid-incident. It always exits 0: every condition it reports is one a worker keeps running through, and a diagnostic that fails the shell gets wrapped in || true and then ignored. Gate CI on drift with taskq actor-config diff, which exits non-zero by design.

Exit codes

Code Meaning
0 Clean shutdown (SIGTERM received and shutdown completed)
1 Startup failure or runtime error
3 --until-idle: some jobs failed during the drain
4 --until-idle: idle-max-runtime exceeded before drain completed

ActorConfigDriftList is caught by the CLI and produces a clean one-line error message on stderr. Other bootstrap failures (import errors, wrong attribute type, etc.) produce a Python traceback on stderr. Both exit with code 1.


taskq job show

Shows one job's stored row, reading {schema}.jobs first and falling back to {schema}.jobs_archive: a pruned terminal job is shown from the archive, not reported missing.

taskq job show JOB_ID

Arguments:

Argument Type Description
JOB_ID UUID The job's id. A non-UUID value is rejected as a usage error before any database access.

Prints the operator-facing fields of the stored row: id, actor, queue, status, priority, attempt, max_attempts, retry_kind, the four timestamps, and (only when set) error_class/error_message and idempotency_key. payload, result and error_traceback are deliberately not printed by default, keeping a terminal read from dragging arbitrarily large blobs onto the wire; error_traceback and payload are available under the opt-in flags below. A row found in jobs_archive is marked archived: yes; see jobs-clients.md for the archival lifecycle.

Under retry_kind="indefinite" the stored max_attempts ceiling is inert: the retry path never consults it, so the command renders it as (indefinite), the same framing the admin UI uses; a bare number would advertise a budget the job is not enforcing. See retries.md.

Example output:

id: 018f1c7e-5a2b-7c3d-8e4f-9a0b1c2d3e4f
actor: send_email
queue: default
status: succeeded
priority: 0
attempt: 168
max_attempts: (indefinite)
retry_kind: indefinite
created_at: 2026-01-01 00:00:00+00:00
scheduled_at: 2026-01-01 00:00:00+00:00
started_at: 2026-01-01 00:00:01+00:00
finished_at: 2026-01-01 00:00:02+00:00
archived: yes

Exit codes:

Code Condition
0 Row found in jobs or jobs_archive
1 Invalid job id (not a UUID), or no row with that id in either table

Uses TASKQ_PG_DSN and TASKQ_SCHEMA_NAME from the environment.

Options:

Option Type Default Description
--traceback flag off Also print the stored error_traceback. The default output stays blob-free; this flag is the operator asking for the traceback by name.
--payload flag off Also print the stored payload jsonb.

A blob the job does not carry prints as (none), so an absent value is distinguishable from a flag that did nothing. Both blobs print in full and unbounded.


taskq job cancel

Cancels one job, through the same backend mechanism the admin UI's cancel route calls.

taskq job cancel JOB_ID [--reason TEXT]

A pending or scheduled job moves straight to terminal cancelled. A running job gets a cooperative cancel request (cancel_phase=1): the worker running it stops the attempt at its next cancellation checkpoint and writes the terminal state itself, so the command prints cooperative cancel requested and the job's status stays running until that write lands. See cancellation.md.

Options:

Option Type Default Description
--reason str None Recorded on the job's cancel_request event.

Exit codes:

Code Condition
0 Cancel applied (directly, or requested on a running job)
1 Invalid job id (not a UUID), no job with that id, or the job is already in a terminal state

taskq job retry

Re-pends a resting job, the same backend operation the admin UI's retry route calls.

taskq job retry JOB_ID

Every terminal status is a valid source: failed, crashed, cancelled, and also succeeded (a re-run after a bad deploy - the status records what the actor returned, never that its side effects were right) and abandoned. A pending, scheduled, or running job is refused: re-pending a row while an attempt is live can execute the job twice concurrently, and a job that is already queued has nothing to put back.

The attempt counter is not reset: the re-pended job climbs to fresh attempt numbers, and max_attempts rises to at least attempt + 1 so at least one fresh execution is possible.

Exit codes:

Code Condition
0 Job re-pended (prints the read-back status, pending)
1 Invalid job id, no job with that id, the job is not in a terminal state, or it left a terminal state between the pre-check and the write (a concurrent claim won the race; no rows changed)

taskq job cancel-where

Bulk-cancels every job matching the given filters in one set-based operation, through JobsClient.cancel_where's bounded drain.

taskq job cancel-where [--queue NAME] [--status S]... [--actor NAME] [--tag T]... [--older-than DURATION] [--reason TEXT] [--dry-run]

Pending/scheduled matches move straight to terminal cancelled; running matches get cancel_phase=1 (cooperative cancel). The write executes as bounded committed batches, so a mid-operation failure leaves partial progress rather than rolling everything back; re-running continues where it stopped (already-cancelled rows are skipped).

Guardrail: with no filter at all the command refuses to run - a filter with no predicates would cancel the entire table, and the CLI offers no allow_empty_filter bypass. Add at least one of --queue, --status, --actor, --tag, or --older-than.

Options:

Option Type Default Description
--queue str None Only jobs on this queue.
--status list[str] [] Only jobs in this status. Repeatable. Valid: pending, scheduled, running, succeeded, failed, cancelled, crashed, abandoned.
--actor str None Only jobs for this actor name.
--tag list[str] [] Only jobs carrying this tag. Repeatable; matches any of the given tags.
--older-than duration None Only jobs enqueued before now minus this duration. Formats: 45 (seconds), 30m, 2h, 7d, 2w.
--reason str None Recorded on every matched job's cancel_request event.
--dry-run flag off Print the matching count and up to 5 sample ids (newest first) and write nothing.

Example:

# Preview, then apply: the same predicate set, so the preview answers for the write.
taskq job cancel-where --queue email --status pending --older-than 7d --dry-run
taskq job cancel-where --queue email --status pending --older-than 7d --reason "stale backlog"

Exit codes:

Code Condition
0 Bulk cancel applied (or the dry run completed)
1 No filter predicate given, an unknown --status value, an unparseable --older-than duration, or the backend's empty-filter guardrail refused the write

taskq job events

Lists one job's job_events timeline: occurred_at, the event kind (state_change, cancel_request, heartbeat_miss, progress), and the detail jsonb rendered as one bounded line.

taskq job events JOB_ID

A long detail is truncated with the dropped-character count named. The job must exist in jobs or jobs_archive; a job with no events prints no job_events rows.

Exit codes:

Code Condition
0 Timeline printed (possibly empty)
1 Invalid job id, or no job with that id in either table

taskq queues

Read the stored queues rows and tune the two settings they hold: the dispatch ordering mode and the fleet-wide leased-slot concurrency cap. Queues are implicit - enqueueing onto a name creates the work without creating a row - and a queue with no row runs on the defaults: strict_fifo ordering (so fairness_key has no effect) and no concurrency cap. See workers.md: Queue dispatch modes for the modes and rate-limiting.md: Queue-level concurrency cap for the cap's mechanics.

taskq queues depth

Per-queue depth from one grouped pass over {schema}.jobs: pending/scheduled/running/failed counts plus the age of the oldest pending job.

taskq queues depth
queue   pending  scheduled  running  failed  oldest_pending
email         4          0        2       0           3m12s
default       0          0        1       9               -

The oldest-pending age is computed server-side (clock_timestamp()), never by subtracting the CLI process's clock from a database timestamp. failed is included alongside the live statuses because the command runs once on demand; a - oldest-pending age means the queue has no pending rows. Queues are ordered newest activity first. Read-only, like queues list and queues get.

Exit codes:

Code Condition
0 Depth table printed (or no jobs ... nothing to report on an empty table)

taskq queues list

Lists every configured queues row, ordered by name:

taskq queues list
default  mode=strict_fifo  max_concurrent=unlimited
email  mode=round_robin  max_concurrent=8

Queues absent from this list are not missing - queues are implicit and are created by enqueueing onto them. An absent queue runs on the defaults: strict_fifo ordering (so fairness_key has no effect) and no concurrency cap. An empty table prints no configured queues (all queues run on defaults: mode=strict_fifo, max_concurrent=unlimited).

taskq queues get

Shows one queue's stored row:

taskq queues get email
email  mode=round_robin  max_concurrent=8

A queue with no stored row is not an error: the command prints the same runs-on-defaults explanation an absent list row implies - including that fairness_key has NO effect on a strict_fifo queue - and exits 0.

Exit codes:

Code Condition
0 Row printed, or the runs-on-defaults message for a queue with no row

taskq queues set-mode

Sets a queue's dispatch ordering mode, creating the row if needed:

taskq queues set-mode email round_robin

The mode is a positional argument, strict_fifo or round_robin. round_robin is what makes fairness_key do anything: on the default strict_fifo the key is accepted, stored, and ignored. The write is an UPSERT and takes effect on the next dispatch cycle - no worker restart.

Exit codes:

Code Condition
0 Row written and printed
1 Mode is not one of strict_fifo, round_robin (the accepted modes are in --help), or the queue name is not a valid queue name

taskq queues set-max-concurrent

Sets or clears a queue's fleet-wide leased-slot concurrency cap:

taskq queues set-max-concurrent email --max-concurrent 8
taskq queues set-max-concurrent email --clear

Exactly one of --max-concurrent and --clear is required; passing both or neither is refused with exit code 1.

Options:

Option Type Default Description
--max-concurrent int None New per-queue leased-slot cap. Must be >= 1 (a lower value is rejected by the option parser as a usage error).
--clear flag off Remove the cap (unlimited): writes NULL.

There is no 0 state: NULL (via --clear) is uncapped, and an emergency drain to 0 belongs to actor-config set --max-concurrent 0, which is per-actor. Unlike a per-actor change via actor-config set, the write is not live the moment it lands: each running worker read the cap into memory at startup, so every worker keeps enforcing the old value until it restarts, and a cap change lands fleet-wide only after the workers serving the queue have been restarted. taskq queues get shows the database value; it cannot tell you which workers have restarted onto it. See deployment.md: max_concurrent and max_pending for the live-vs-restart asymmetry and a rolling-restart recipe.

Exit codes:

Code Condition
0 Row written and printed
1 Both --max-concurrent and --clear given, neither given, or the queue name is not a valid queue name
2 --max-concurrent below 1 (option range check, before any database access)

taskq insights

Renders the taskq.insights SQL layer as operator tables: how long work waited, whether the fleet is imbalanced, how long a queue needs to drain, and whether a cron schedule is fanning out faster than it clears. The command is the terminal surface for that module — it writes no SQL of its own, every row comes from the module's statements (over BOTH retention tiers, live jobs + jobs_archive).

Read-only like taskq doctor: SELECT-only statements, exit 0 whether or not there is anything to report. Safe to run against production mid-incident.

taskq insights [wait|balance|drain|cron|all]   # default: all

Options: --window selects the trailing window for the windowed reads (wait, drain, cron); one of 1h, 6h, 24h, 7d (the module's INSIGHTS_WINDOWS). On hypertables a window older than the archive retention answers over the surviving chunks — the retention floor is the analytics floor. --actor is a wait-only grouping switch (see below). --queue keeps only that queue's rows on wait, balance, and drain; it is refused on cron, whose ledger is per schedule and has no queue dimension.

taskq insights wait

Per queue, how long terminalised attempts waited before their claim. Wait per attempt is started_at - scheduled_at (the dispatch claim latency); the rows are SEGMENTED clean vs deferred:

queue   segment  count    p50    p95   max
email     clean      4  1m15s  1m55s  2m0s
email  deferred      1   5.0s   5.0s  5.0s
  • count — terminalised attempts in the window.
  • p50 / p95 / max — the wait distribution's percentiles.
  • segment — clean = first-delivery attempts (snooze_count = 0 and rate_limit_blocked_count = 0), the subset a queue-latency SLO is written against. deferred = rows whose scheduled_at a snooze or a rate limit moved forward: reschedules are excluded from the clean percentiles, and a deferred row's wait measures only the FINAL leg (the deferred time is by construction not in the number). started_at - created_at is NOT their wait — it would fold the operator's own snooze choice into the queue's latency.

--actor ACTOR switches the statement to the module's per-(actor, queue) grouping and keeps one actor's rows — the plain per-queue read folds every actor's attempts together.

taskq insights balance

Per queue, is the fleet imbalanced:

queue  depth  live  effective_capacity  utilization  oldest_due               flag
email    122     1                   4        30.50       15m2s  !! over threshold
idle       1     0                   0            -       10m2s     !! no capacity
  • depth — pending rows due now (scheduled_at <= now): the work a worker could claim this instant.
  • live — workers subscribed to the queue and heartbeating in the last 30s.
  • effective_capacity — the routed actors' summed max_concurrent x live workers: the queue's in-flight ceiling under the current fleet.
  • utilization — depth ÷ effective capacity; - renders the NULL case.
  • oldest_due — age of the oldest due job: a large depth with a tiny age is a burst, a small depth with a large age is a strand.
  • flag — over-threshold rows are MARKED: !! over threshold when utilization > 1 (more due work than one wave of capacity can absorb), !! no capacity when nothing can serve the queue (no live worker or no actor capacity — the starvation shape).

taskq insights drain

Per queue, seconds-to-drain: due depth ÷ completions per second over the window.

queue  depth                   eta
email    122                  1d0h
idle       1  no traffic in window

The eta is a THROUGHPUT extrapolation, not a promise: it assumes the next window looks like the last one, that the queue's workers stay up, and that nothing enqueues behind the current depth. The has_traffic caveat is rendered honestly: a queue whose window carried no completions has NO estimate — its eta renders no traffic in window, never 0, which would read as "already drained". Widen --window (up to the archive retention floor) before trusting anything else.

taskq insights cron

Per schedule, the fan-out ledger:

schedule   actor       cron  fires  cleared  outstanding  verdict
ce6f7920  ticker  * * * * *      3        1            2       ok
  • schedule — the schedule id, shortened to 8 characters.
  • fires / cleared — jobs the schedule enqueued in the window (provenance metadata cron_schedule_id) and how many of those are terminalised. Clearance lags fires at the window's right edge by construction; the ratio is a trend, not an instant.
  • outstanding — the schedule's non-terminal backlog right now, windowless (future-armed fires included, by design).
  • verdict — !! runaway trending when fires > cleared in BOTH the current and the prior equal window (two consecutive windows is the runaway shape; one is a burst), ok otherwise.

Two confounds the verdict's reader must hold: a DST allof schedule legitimately doubles a fire in the overlap hour (read the verdict against the schedule's dst_strategy), and a budget-deferred fire enqueues NO row, so a deferral reads as zero fires here — its record is the taskq.cron.budget_deferrals counter, not this ledger.

Exit codes:

Code Condition
0 Tables printed (or a named nothing-to-report line per empty surface)
1 Unknown surface, invalid --window, --actor outside wait, --queue on cron, or an invalid schema name

taskq health live

Probes the worker's liveness endpoint.

taskq health live

Connects to the Unix socket at TASKQ_HEALTH_SOCKET_PATH and issues GET /live. There is no path auto-discovery: an unconfigured worker binds the per-process default /tmp/taskq_health_<pid>.sock (logged at boot as health-server-started's socket_path), so set TASKQ_HEALTH_SOCKET_PATH to probe it — the static fallback /tmp/taskq_health.sock applies only when the variable IS set, and a probe with no socket at the resolved path exits 1 with health socket unreachable (plus a hint naming the per-process default). The health server schedules a loop.call_later(0.01, ...) callback and waits up to 1.0s for it to fire, confirming the event loop is responsive.

The Unix socket is not reachable via Kubernetes httpGet probes. Use an exec probe:

livenessProbe:
  exec:
    command: ["taskq", "health", "live"]
  initialDelaySeconds: 5
  periodSeconds: 10

Exit codes:

Code Condition
0 HTTP 2xx: event loop is responsive ({"status":"ok"})
1 HTTP 5xx: event loop unresponsive or timeout exceeded
1 Socket unreachable (worker not running or wrong socket path)

Example:

taskq health live && echo "worker is alive"

taskq health ready

Probes the worker's readiness endpoint.

taskq health ready

Connects to the Unix socket and issues GET /ready. The health server checks:

  1. shutdown_phase == NONE (worker is not draining or shutting down)
  2. PG ping succeeds within TASKQ_HEALTH_PG_PING_TIMEOUT (default 0.2s)
  3. when the worker runs a per-slot transaction pool, that pool is pinged too (overlapping probes share a single ping)

All conditions must pass for the response to be 200. During any shutdown phase, the response is 503 regardless of the PG ping result.

Response body (200):

{
  "ready": true,
  "redis_configured": false,
  "active_jobs": 4,
  "is_leader": false,
  "shutdown_phase": null
}

Response body (503):

{
  "ready": false,
  "redis_configured": false,
  "active_jobs": 2,
  "is_leader": false,
  "shutdown_phase": 1
}

shutdown_phase is null when NONE; otherwise the integer value (1=DRAINING, 2=CANCELLING, 3=FORCING, 4=RELEASING).

Exit codes:

Code Condition
0 HTTP 200: worker is ready
1 HTTP 503, not ready (shutting down or PG ping failed)
1 Socket unreachable

Example (Kubernetes readiness probe via exec):

readinessProbe:
  exec:
    command: ["taskq", "health", "ready"]
  initialDelaySeconds: 5
  periodSeconds: 10

taskq health metrics

Fetches the worker's Prometheus-format metrics.

taskq health metrics

Connects to the Unix socket and issues GET /metrics. Always returns 200. Prints the response body to stdout.

Example output:

# HELP taskq_active_jobs Currently in-flight jobs on this worker.
# TYPE taskq_active_jobs gauge
taskq_active_jobs 3
# HELP taskq_is_leader 1 if this worker holds the maintenance leader lock.
# TYPE taskq_is_leader gauge
taskq_is_leader 0
# HELP taskq_shutdown_phase Current shutdown phase enum value (0=NONE).
# TYPE taskq_shutdown_phase gauge
taskq_shutdown_phase 0

Exit codes:

Code Condition
0 HTTP 200: metrics returned
1 Socket unreachable or request timed out

taskq ui serve

Starts the admin UI server (FastAPI + uvicorn).

taskq ui serve [OPTIONS]

Options:

Option Type Env var fallback Default Description
--pg-dsn str TASKQ_PG_DSN postgresql://taskq:taskq@localhost:5432/taskq Postgres DSN for the admin UI pool
--schema str TASKQ_SCHEMA_NAME taskq Postgres schema
--redis-url str TASKQ_REDIS_URL None Redis URL for real-time SSE progress
--host str TASKQ_ADMIN_HOST 0.0.0.0 Bind address
--port int TASKQ_ADMIN_PORT 8080 Bind port
--migrate bool TASKQ_MIGRATE_ON_START false Apply pending migrations before starting. Aborts startup if migrations fail.
--pg-credential-provider str None TASKQ_PG_CREDENTIAL_PROVIDER module:attr reference to a PgCredentialProvider for the admin UI's Postgres pool. See Managed identities.
--redis-credential-provider str None TASKQ_REDIS_CREDENTIAL_PROVIDER module:attr reference to a RedisCredentialProvider. Requires TASKQ_REDIS_URL.

Relevant environment variables:

Variable Default Description
TASKQ_ADMIN_MAX_SSE_CONNECTIONS 50 Maximum concurrent SSE connections the admin UI will serve. Controls the connection-limit semaphore.

All options fall back to the corresponding TASKQ_* environment variable when not supplied on the command line. The admin UI requires the fastapi optional dependency group:

uv add "taskq-py[fastapi]"

The server creates a small asyncpg pool (min_size=1, max_size=4) and optionally a Redis client. The admin router is mounted at /admin. See admin-ui.md for the full UI reference.

Example:

taskq ui serve --port 8001
TASKQ_ADMIN_PORT=8001 taskq ui serve

The process blocks until killed. There is no graceful-shutdown option; use a process manager or container lifecycle hook.

The standalone server configures no Backend, so the backend-mediated admin mutations (job cancel, job retry, schedule run-now) answer 503 from taskq ui serve even with TASKQ_ADMIN_ACTIONS_ENABLED=true; use this CLI's taskq job cancel / taskq job retry for those, or embed the admin router with an explicit backend= (see admin-ui.md).


taskq workgroup validate

Validates a workgroup TOML configuration file without starting any workers.

taskq workgroup validate CONFIG

Arguments:

Argument Type Description
CONFIG PATH Path to the workgroup TOML configuration file.

Prints a summary of each worker's configuration. Exits 1 if the config is missing, malformed, contains invalid values (e.g. negative poll interval, misconfigured health check thresholds), or names an actors reference that cannot be resolved: an unimportable module, a module that raises on import, or a missing attribute.

Validation imports the actors module, so it must run where the application is importable: the same interpreter and PYTHONPATH the workgroup would start under. Resolving the reference here is what makes the check worth having: every child imports it the moment it is spawned, so an unresolvable reference crashes each one at import and the supervisor sees only a run of child exits, restarting them on backoff until the burst budget is spent. A CI or lint invocation that runs without the application on sys.path will report a valid config as invalid; run it from the deployment image instead.

Resolving the reference is also what lets validate warn about an actor whose queue no [[workers]] entry consumes. That is a warning, never an exit-1 condition: another workgroup or deployment may consume the queue, and no single supervisor can know the whole fleet.

Example:

taskq workgroup validate workgroup.toml
# config OK, 2 worker(s), actors='billing.actors:registry'
#   api: queues=['default'] poll=0.5s concurrency=8 health=off
#   batch: queues=['email', 'report'] poll=5.0s concurrency=2 health=on

taskq workgroup start

Starts a workgroup supervisor that manages multiple worker subprocesses from a TOML configuration file.

taskq workgroup start CONFIG

Arguments:

Argument Type Description
CONFIG PATH Path to the workgroup TOML configuration file.

The supervisor spawns one taskq worker subprocess per [[workers]] entry, manages their lifecycle, and cleanly shuts them down on SIGTERM/SIGINT. See workgroups.md for the full configuration reference and operational guidance.

Example config:

actors = "myapp.actors:registry"

[[workers]]
name = "api"
queues = ["default"]
max_concurrency = 8

[[workers]]
name = "batch"
queues = ["email", "report"]
poll_interval = 5.0
max_concurrency = 2

Example:

taskq workgroup start workgroup.toml

Exit codes:

Code Condition
0 Clean shutdown after SIGTERM or SIGINT
1 Config file not found, invalid config, or health-check PG pool initialization failure

Exit codes

Code Meaning
0 Success
1 Any failure: bad arguments, import errors, config drift, PG connection failures, health probe negative result
2 Watchdog force-exit (wedged worker), not returned by taskq worker itself; emitted by os._exit(EXIT_WATCHDOG)
3 taskq worker --until-idle: some jobs failed during the drain
4 taskq worker --until-idle: idle-max-runtime exceeded before drain completed

The taskq worker command exits with the code returned by worker_main(). On clean SIGTERM, worker_main() returns 0. With --until-idle, worker_main() returns 0 (all jobs succeeded), 3 (some jobs failed), or 4 (idle-max-runtime exceeded). A third SIGTERM calls sys.exit(1).

The taskq workgroup start command exits 0 on clean shutdown. It exits 1 if the config file is missing, invalid, or if the optional health-check PG pool fails to initialise.