Configuration Reference¶

Overview¶

All TaskQ configuration is provided through TASKQ_* environment variables, loaded via dotenvmodel. TaskQ never reads raw os.environ; use TaskQSettings.load() for all commands, or WorkerSettings.load() for worker processes.

There are two settings classes:

TaskQSettings: base class; applies to every command (worker, migrate, ui serve, health).
WorkerSettings: extends TaskQSettings; additional fields used only by the worker process.

dotenvmodel resolves a cascading chain of .env files at load time:

.env: base defaults, committed to the repo
.env.local: local overrides, never committed
.env.{env}, e.g. .env.production
.env.{env}.local: local env-specific overrides, never committed

{env} comes from the ENV environment variable (default dev): ENV=production loads .env.production and .env.production.local. Later files in the chain take precedence over earlier ones within the file layer. Never commit .env.local or production env files.

Precedence. Process environment variables beat the merged .env cascade, which beats field defaults (dotenvmodel 1.x semantics, adopted by TaskQ). To opt back into the older files-beat-env-vars behaviour, set DOTENV_OVERRIDE=true or call TaskQSettings.load(override=True).

Load-time knobs are process-environment-only. ENV and the DOTENV_* variables (DOTENV_OVERRIDE, DOTENV_READ_DOTFILES, DOTENV_READ_ENVIRON, DOTENV_LOAD_LOCAL, DOTENV_DIR) are read from the process environment, or passed explicitly to load(), before any .env file is read. Setting them inside a .env file has no effect on that load: a value in a file cannot influence which files are selected or how they are applied. The two read knobs are symmetric: read_dotfiles=False skips the .env cascade (fields resolve from the process environment and defaults); read_environ=False skips the process environment (fields resolve from .env files and defaults).

When the resolved env is test (case-insensitive), .env.local and .env.test.local are skipped, so gitignored local overrides cannot decide test outcomes. Restore them with DOTENV_LOAD_LOCAL=true (or load_local=True).

The `0` convention¶

Several fields use 0 as a sentinel: a special meaning, not the quantity zero, and the polarity differs per family. Learn the families once: every sentinel-0 field in this reference follows one of them, and each family's own entry links back here:

Family	Fields	`0` means
Statement cache size	`TASKQ_STATEMENT_CACHE_SIZE`	cache disabled
Statement cache lifetime	`TASKQ_MAX_CACHED_STATEMENT_LIFETIME`	cached indefinitely
Lock-wait budgets	`TASKQ_MAX_PENDING_LOCK_TIMEOUT_MS`, `TASKQ_UNIQUE_FOR_LOCK_TIMEOUT_MS`, `TASKQ_IDEMPOTENCY_LOCK_TIMEOUT_MS`, `TASKQ_TOKEN_BUCKET_LOCK_TIMEOUT_MS`, `TASKQ_SLIDING_WINDOW_LOCK_TIMEOUT_MS`	wait indefinitely server-side (the `lock_timeout` GUC convention). A TaskQ-built pool still applies its per-query `command_timeout`, so an unbounded server wait is still bounded client-side; prefer a large finite value; see Derived Values
Time-based deletion sweeps	`TASKQ_EVENT_RETENTION_PERIOD`, `TASKQ_KEYED_ROW_RECLAIM_PERIOD`	sweep disabled: for a deletion loop the safe misconfiguration is off
Prune/archive retention	`TASKQ_PRUNE_RETENTION_SUCCEEDED`, `TASKQ_PRUNE_RETENTION_FAILED`, `TASKQ_PRUNE_RETENTION_CANCELLED`, `TASKQ_PRUNE_RETENTION_ABANDONED`, `TASKQ_ARCHIVE_RETENTION_PERIOD`	archive/expire immediately at the next sweep: the prune family's polarity is deliberately opposite to the deletion-sweep family's
Health listener	`TASKQ_HEALTH_PORT`	bind an ephemeral port (tests only). Note this field's real "off" is unset, not `0`

The trap this table exists for: the first two rows sit two fields apart in WorkerSettings and mean opposite things, and the lock-budget family's "wait indefinitely" is the opposite of "fail fast". Never generalise 0 from one family to another; check the table.

`.env` File Setup¶

Minimal .env for a real deployment:

# Required for any real deployment
TASKQ_PG_DSN=postgresql://user:pass@localhost:5432/mydb

# Optional: enables real-time admin UI updates
TASKQ_REDIS_URL=redis://localhost:6379/0

# Schema name (default: taskq)
TASKQ_SCHEMA_NAME=taskq

# Suppress unauthenticated-admin warning in dev
TASKQ_ENVIRONMENT=development

.env is the committed base. .env.local overrides it on a developer's machine without affecting others. Setting ENV=production additionally loads .env.production and .env.production.local. TASKQ_ENVIRONMENT has nothing to do with file selection; it is a TaskQ deployment label that gates the unauthenticated-admin warning (dev/development suppress it; any other value triggers it). Never commit .env.local or production env files.

Typo'd `TASKQ_*` variables: `taskq doctor` reports them¶

Unknown TASKQ_* environment variables and .env keys still load noiselessly at settings time: the loader reads only the field names it defines, so TASKQ_MAX_CONCURENCY=4 (missing the second C), a wrong prefix (TASKQ_MAXCONCURRENCY), or a variable that never existed is ignored without a warning, and the field it was meant to set applies its documented default. The loader stays permissive on purpose — rejecting unknown keys would break forward and backward compatibility across rolling deploys (a newer pod's extra variable must not fail an older pod's load) and would fight the dotenv cascade, where keys legitimately migrate between files and the process environment.

The detection lives in taskq doctor, which scans the process environment for TASKQ_-prefixed names that match no field of the settings models (TaskQSettings, WorkerSettings, and the TASKQ_OIDC_* / TASKQ_SAML_* SSO sub-configs). Each unknown name is a finding naming it, stating the consequence ("a typo applies defaults silently"), and suggesting the closest real setting name — TASKQ_MAX_PENDNG_LOCK_TIMEOUT_MS is reported with "did you mean TASKQ_MAX_PENDING_LOCK_TIMEOUT_MS?". Run it as the deploy smoke test's config-drift arm; the deprecated-no-op settings that do still load (e.g. TASKQ_DISPATCH_SCOPE_BY_HOME_QUEUE) are known names to the scan and keep their worker-startup announcement.

TaskQSettings Reference¶

Applies to all commands: worker, migrate, ui serve, health.

Env Var	Type	Default	Description	Used By
`TASKQ_PG_DSN`	`PostgresDsn`	`postgresql://taskq:taskq@localhost:5432/taskq`	Direct (non-PgBouncer) DSN. LISTEN/NOTIFY and advisory locks require a session-mode connection.	all
`TASKQ_PG_IS_POOLED`	`bool`	`false`	Declare that the DSN(s) TaskQ builds pools from route through a transaction-mode pooler (PgBouncer `pool_mode=transaction`, RDS Proxy, Supavisor). asyncpg cannot detect a pooler from the wire protocol, so the operator declares the topology. When `true`, every pool TaskQ builds disables the prepared-statement cache (`statement_cache_size=0`, `max_cached_statement_lifetime=0`, overriding the two knobs below) so a prepared statement can never cross a pooler's server-connection remap, and the worker treats pooler-remap statement errors (SQLSTATE `26000`, `42P05`) as transient instead of loud errors. Applies only to pools TaskQ builds; bring-your-own pools must set the same `create_pool` kwargs themselves.	all
`TASKQ_SCHEMA_NAME`	`str`	`taskq`	Postgres schema for all TaskQ tables. Must match `^[A-Za-z_][A-Za-z0-9_]$`. Read by workers, the CLI, the admin UI and* the `TaskQ` client (an explicit `schema=` argument to `TaskQ(...)` wins; otherwise the client resolves this variable through the same process-env → `.env` cascade the worker reads, validating it alone; an unrelated malformed `TASKQ_*` value cannot break `TaskQ()` construction).	all
`TASKQ_STATEMENT_CACHE_SIZE`	`int`	`512`	Size of the per-connection prepared-statement LRU asyncpg keeps on every pool TaskQ builds. asyncpg's default of 100 thrashes on TaskQ's read paths (`list_jobs` alone renders 384+ filter-combination variants). `0` disables the cache (see The `0` convention). Applies only to pools TaskQ builds; bring-your-own pools must pass the same `create_pool` kwarg themselves. See ops.md: Database performance knobs.	all
`TASKQ_MAX_CACHED_STATEMENT_LIFETIME`	`int` (seconds)	`3600`	How long a prepared statement may stay in asyncpg's per-connection cache on every pool TaskQ builds. asyncpg's default of 300 s re-prepares statements on long-lived workers. `0` caches indefinitely: the opposite polarity of `TASKQ_STATEMENT_CACHE_SIZE` two rows up (see The `0` convention). Same TaskQ-built-pools-only scope as `TASKQ_STATEMENT_CACHE_SIZE`.	all
`TASKQ_REDIS_URL`	`RedisDsn \\| None`	`None`	Optional Redis URL. Required for real-time SSE progress fanout in the admin UI.	worker, ui serve
`TASKQ_PG_CREDENTIAL_PROVIDER`	`str \\| None`	`None`	`module:attr` reference to a `PgCredentialProvider`. Overridden by `--pg-credential-provider` on `taskq worker` / `migrate` / `ui serve`. See Managed identities.	worker, migrate, ui serve
`TASKQ_REDIS_CREDENTIAL_PROVIDER`	`str \\| None`	`None`	`module:attr` reference to a `RedisCredentialProvider`. Requires `TASKQ_REDIS_URL`. Overridden by `--redis-credential-provider`.	worker, ui serve
`TASKQ_RELOAD_INTERVAL`	`float \\| None` (seconds)	`None` (derived from the lease)	Cadence of the credential hot-reload (the same path as SIGHUP): every provider-backed pool and connection is rebuilt on a fresh credential with no external signal required; the rotation path for platforms without SIGHUP (e.g. Windows) and for hands-off scheduled rotation (e.g. ~720 s for AWS IAM's 15-minute tokens). Unset, the cadence is derived from the lease the provider grants when it reports one: a Vault dynamic credential is rebuilt at half its lease TTL (`taskq.auth.ReloadSchedule`); a username-bearing provider that reports no lease warns at startup, and only SIGHUP / `deps.request_reload()` rotate it. Only factory-backed resources are rebuilt; DSN/static credentials are unaffected. Must be > 0.	worker, ui serve
`TASKQ_ENVIRONMENT`	`str \\| None`	`None`	Deployment label; does not select `.env` files (that is `ENV`'s job). Values `dev` or `development` suppress the unauthenticated-admin warning. Any other value triggers it.	all
`TASKQ_ADMIN_ACQUIRE_TIMEOUT`	`float` (seconds)	`5.0`	Bounds every wait an admin UI or progress request makes for a backend resource before its own query runs: a Postgres pool checkout and a Redis read. A pool with every connection wedged, or a black-holed broker, answers the request with `503` (`Retry-After: 2`) after this long instead of hanging it, and every request behind it, until the client gives up; the query itself is bounded by the pool's `command_timeout`. Must be > 0.	ui serve
`TASKQ_ADMIN_MAX_SSE_CONNECTIONS`	`int`	`50`	Maximum concurrent SSE connections the admin UI will serve. Min: 1.	ui serve
`TASKQ_PROGRESS_MAX_SSE_CONNECTIONS`	`int`	`50`	Maximum concurrent per-job progress SSE streams this process will serve. Each stream holds a Redis pubsub subscription and an asyncio task for as long as the client stays connected, so an uncapped endpoint is a resource-exhaustion surface on the app hosting the pipeline. Min: 1.	ui serve
`TASKQ_PROGRESS_REQUIRE_AUTH`	`bool`	`true`	When `true` (the default), `taskq.web.progress.create_router` raises `RuntimeError` if `auth_dependency` is `None` in a non-dev environment, failing closed. Set to `false` to allow unauthenticated per-job progress/state endpoints in non-dev (not recommended, only for deployments that authenticate at the ingress).	ui serve
`TASKQ_ADMIN_WORKER_LIVENESS_SECONDS`	`int`	`30`	How recently a worker must have written `last_seen_at` to count as alive: the admin UI's orphan-queue banner and leader `watchdog_healthy`, and on the worker side the leader's `taskq.queue.live_workers` gauge and the stranded-jobs detector's unserved-queue arm. Measured by Postgres against the statement clock. Must comfortably exceed `TASKQ_HEARTBEAT_INTERVAL`. Min: 1.	ui serve, worker (leader)
`TASKQ_ADMIN_HOST`	`str`	`0.0.0.0`	Bind address for `taskq ui serve`.	ui serve
`TASKQ_ADMIN_PORT`	`int`	`8080`	Bind port for `taskq ui serve`. Range: 1-65535.	ui serve
`TASKQ_ADMIN_URL`	`str`	`http://localhost:8080`	Public base URL of the admin UI as seen from a browser. Used to construct redirect URLs. Override when admin and app run on different hosts or ports.	ui serve
`TASKQ_ADMIN_UI_POLLING_INTERVAL_SECONDS`	`float`	`2.0`	How often the admin UI polls PG when in polling/degraded mode. Min: 0.1.	ui serve
`TASKQ_ADMIN_UI_ALLOW_RATE_LIMIT_RESET`	`bool`	`false`	When `True`, the admin UI shows a reset button on the rate-limits page and serves the `POST /rate-limits/{bucket_name}/reset` endpoint. Default `False` for safety.	ui serve
`TASKQ_ADMIN_UI_REQUIRE_AUTH`	`bool`	`true`	When `true` (the default), `create_router` raises `RuntimeError` if `auth_dependency` is `None` in a non-dev environment, failing closed. Set to `false` to allow an unauthenticated admin UI in non-dev (not recommended; only for air-gapped or localhost-only deployments).	ui serve
`TASKQ_ADMIN_ACTIONS_ENABLED`	`bool`	`false`	When `true`, the admin UI permits destructive actions (run schedule now, retry job, cancel job). Default `false`: prevents on-demand triggering of registered business logic via the admin UI without explicit opt-in. Separate from `auth_dependency`, which controls read access to all admin routes.	ui serve
`TASKQ_ADMIN_UI_FRAME_ANCESTORS`	`str`	`none`	Who may frame admin pages: `none` (the default, nobody) or `self` (the admin UI's own origin, for a host app that embeds the admin UI in its dashboard). Emitted as both `Content-Security-Policy: frame-ancestors ...` and the legacy `X-Frame-Options` (DENY / SAMEORIGIN). Anything else fails at settings load: a typo that silently became "no header" would take the clickjacking defence off in exactly the deployment that believed it had configured it.	ui serve
`TASKQ_ADMIN_UI_SECURE_COOKIES`	`bool`	`true`	Sets the `Secure` flag on the admin UI's CSRF cookie. A configured value, not one inferred from `request.url.scheme`: behind a TLS-terminating edge (Azure Application Gateway, App Service) the app sees plain http, so an inferred flag is silently dropped on a connection the browser reached over HTTPS. Set `false` only for local http dev, where the browser rejects a Secure cookie and the admin UI stops working.	ui serve
`TASKQ_SSO_BACKEND`	`str`	`none`	Selects the SSO backend for the admin UI: `none` (default, unauthenticated/BYO-auth), `oidc` (`taskq[oidc]`), or `saml` (`taskq[saml]`). See sso.md.	ui serve
`TASKQ_HEALTH_TOKEN`	`SecretStr`	`""` (empty)	Bearer token for machine-to-machine access to health/metrics endpoints. When set, health and metrics routes require a matching `Authorization: Bearer <token>` header. Leave empty for unauthenticated cluster-internal access, but see `TASKQ_HEALTH_REQUIRE_TOKEN`, which fails closed on an empty token outside dev. Typed `SecretStr`, so the value is masked in `repr` and logs.	ui serve
`TASKQ_HEALTH_REQUIRE_TOKEN`	`bool`	`true`	When `true` (the default), `taskq ui serve` raises `RuntimeError` if `TASKQ_HEALTH_TOKEN` is empty in a non-dev environment, failing closed. Set to `false` to allow unauthenticated health/metrics endpoints in non-dev (e.g. when relying on network policy instead of a bearer token).	ui serve
`TASKQ_MIGRATE_ON_START`	`bool`	`false`	Apply pending migrations before the process accepts its first request. Aborts startup if migrations fail.	ui serve
`TASKQ_EXAMPLE_HOST`	`str`	`0.0.0.0`	Bind address for the example trigger app. Ignored by worker and admin.	example app
`TASKQ_EXAMPLE_PORT`	`int`	`8000`	Bind port for the example trigger app. Ignored by worker and admin.	example app

See admin-ui.md for admin-specific behaviour driven by these vars.

Lock-wait budgets¶

Five knobs bound how long enqueue and admission statements wait on their advisory locks before giving up. The first three are TaskQSettings fields (the client/enqueue path: a producer process never loads WorkerSettings, so these budgets live on the base class); the last two are WorkerSettings fields (the worker admission path: the Postgres rate-limit acquires, see rate-limiting.md). All five are finite floats in milliseconds, ship the same 5000 ms default, and share one 0 polarity: 0 or less waits indefinitely server-side (the lock_timeout GUC convention - see The 0 convention; prefer a large finite value). Widening a budget past its default re-derives the per-query bound of every pool TaskQ builds, so the server-side refusal still fires before the pool's client-side timer; see Derived Values.

Env Var	Type	Default	Effect	Used By
`TASKQ_MAX_PENDING_LOCK_TIMEOUT_MS`	`float` (milliseconds)	`5000.0`	Bounded wait for the `max_pending` advisory lock on the single-enqueue path (the count-then-insert serialisation per capped actor). Exhaustion raises `MaxPendingLockTimeoutError` - the same typed backpressure as a cap rejection, and the denial consumes retry budget - so widen during an outage that slows lock holders rather than letting the fixed ceiling convert slow holders into refused enqueues.	client (enqueue)
`TASKQ_UNIQUE_FOR_LOCK_TIMEOUT_MS`	`float` (milliseconds)	`5000.0`	Bounded wait for the `unique_for` single-flight advisory lock on the single-enqueue path (the identity preflight-then-insert serialisation). Exhaustion raises `UniqueForLockTimeoutError` with retry-yields-dedup guidance: the correct contention outcome is usually the dedup return, so a `unique_for` caller may want a longer wait than the `max_pending` budget before giving up on the answer.	client (enqueue)
`TASKQ_IDEMPOTENCY_LOCK_TIMEOUT_MS`	`float` (milliseconds)	`5000.0`	Bounded wait for the idempotency token INSERT's speculative-lock conflict - another transaction's uncommitted row with the same `(idempotency_scope, idempotency_key)`; on a transactional consumer the holder is the actor's own open transaction, so this budget bounds the victim. Exhaustion raises `IdempotencyKeyLockTimeoutError`: the dedup answer could not be determined in time; retry the same enqueue, which typically dedupes against the now-visible winner.	client (enqueue)
`TASKQ_TOKEN_BUCKET_LOCK_TIMEOUT_MS`	`float` (milliseconds)	`5000.0`	Bounded wait for the `rate_limit_buckets` row lock on the token-bucket Postgres acquire and refund. With the Postgres fallback enabled a Redis outage funnels every admission through this lock, so the bound is what stops one black-holed holder stalling a bucket's admission. Exhaustion is an admission denial, not a failure: the acquire fails closed and the denial's retry hint is one more budget.	worker (admission)
`TASKQ_SLIDING_WINDOW_LOCK_TIMEOUT_MS`	`float` (milliseconds)	`5000.0`	Bounded wait for the sliding window's admission lock on Postgres: the per-bucket advisory lock on the log style, the `rate_limit_buckets` row lock on the GCRA style. Separate knob from `TASKQ_TOKEN_BUCKET_LOCK_TIMEOUT_MS` because the two limiter shapes hold their locks across different critical sections. Exhaustion fails closed as a denial whose retry hint is one more budget.	worker (admission)

SSO sub-settings (`TASKQ_OIDC_` / `TASKQ_SAML_`)¶

Selecting TASKQ_SSO_BACKEND=oidc or saml activates two dedicated sub-config classes, OIDCSettings (loaded from TASKQ_OIDC_* env vars) and SAMLSettings (loaded from TASKQ_SAML_*). Each field carries a full description in src/taskq/settings.py; the walkthrough, defaults and per-field tables live in sso.md. Summary:

Class	Env prefix	Fields
`OIDCSettings`	`TASKQ_OIDC_`	`ISSUER`, `CLIENT_ID`, `CLIENT_SECRET` (secret), `REDIRECT_URI`, `SESSION_SECRET` (secret), `SESSION_MAX_AGE_SECONDS`, `SCOPE`, `GROUP_CLAIM`, `ALLOWED_GROUPS`
`SAMLSettings`	`TASKQ_SAML_`	`ENTITY_ID`, `ACS_URL`, `IDP_ENTITY_ID`, `IDP_SSO_URL`, `IDP_X509_CERT`, `SP_X509_CERT`, `SP_PRIVATE_KEY` (secret), `SESSION_SECRET` (secret), `SESSION_MAX_AGE_SECONDS`, `GROUP_ATTRIBUTE`, `ALLOWED_GROUPS`, `ALLOW_COOKIELESS_FALLBACK`

Both SESSION_SECRET fields sign session cookies: use at least 32 bytes of random data, and rotate to invalidate all sessions. ALLOWED_GROUPS is a comma-separated group allowlist on either backend.

WorkerSettings Reference¶

Extends TaskQSettings. All fields below apply to the worker process only.

Database Connections¶

Env Var	Type	Default	Description	Constraints
`TASKQ_PG_DSN_DIRECT`	`PostgresDsn \\| None`	falls back to `TASKQ_PG_DSN`	Bypasses PgBouncer. Used by `dispatcher_pool`, `heartbeat_pool`, `notify_conn`, and `leader_conn`.	n/a
`TASKQ_PG_DSN_POOLED`	`PostgresDsn \\| None`	falls back to `TASKQ_PG_DSN`	May route through PgBouncer transaction mode. Used by `worker_pool` only.	n/a

Pool Sizing¶

Env Var	Type	Default	Description	Constraints
`TASKQ_DISPATCHER_POOL_SIZE`	`int`	`4`	Max connections for the dispatcher pool.	Min: 1
`TASKQ_DISPATCHER_COMMAND_TIMEOUT`	`float` (seconds)	`5.0`	Per-query timeout for the dispatcher pool and the TaskQ-built leader connections (election, cron, monitor), and the single deadline wrapped around each period-1 leader-loop iteration (`scheduled_wake`, cron): a stalled PG errors the iteration instead of hanging the loop past its staleness budget. When the watchdog is enabled, load fails unless `timeout + 1.0 < max(1.0 × TASKQ_WATCHDOG_TICK_GRACE_FACTOR, TASKQ_WATCHDOG_STALE_FLOOR)` for the period-1 leader loops (the producer loop is not checked), so a timeout-capped iteration can never false-trip the stale-loop detector. At defaults this caps the value just under `9.0`: a larger value fails at settings load and the worker will not start. Raising it requires raising `TASKQ_WATCHDOG_STALE_FLOOR` too; see Validation Constraints. (Default was 10.0 before 1.x: equal to the floor, which produced exactly that false trip.) For the dispatcher pool this configured value is the floor of the applied bound, not the bound itself; it is re-derived upward from a widened admission lock budget; see Derived Values.	Min: 1.0; must be < 9.0 at default watchdog settings; cross-field, see above
`TASKQ_DISPATCH_OVERSAMPLE`	`int`	`2`	Multiplier for per-actor candidate gathering in the dispatch SQL. Each LATERAL reads `residual × oversample` candidates. Higher values absorb more identity-key collisions and multi-producer contention. Default 2 (tolerates 50% dupe identities). Set 1 when no `identity_key` is used and single-producer. Range: 1-1000.	Min: 1; Max: 1000
`TASKQ_CLAIM_CURSOR_RESET_SECONDS`	`float` (seconds)	`0` (disabled)	OPT-IN. The claim cursor's jitter reset interval (backend/_claim_cursor.py): when enabled, each worker's dispatch keeps a per-queue high-water mark of successfully-claimed job ids (UUIDv7) and bounds the claim query's candidates at it, so the B-tree seek can land on live tuples instead of degenerating through the dead tuples a pinned MVCC horizon leaves behind. The bound is a selection predicate: a pending row whose id falls below it (a skewed producer's clock, a row the round's window never reached) is invisible to the claim query until the reset forgets the cursor - this interval, jittered ±50% per entry, armed at the entry's creating claim, never extended by later claims: the bound on how long any row can be stranded. Round-robin queues are exempt. Disabled by default on measured evidence (the bound's full-path win is plan-dependent on this codebase's multi-surface claim CTE - see the perf-evidence record and the `TaskQClaimLatencyDegraded` runbook); enable it for single-queue, monotone claim streams with fresh statistics.	Min: 0.0 (0 disables the cursor); Max: 86400.0
`TASKQ_HEARTBEAT_POOL_SIZE`	`int`	`4`	Max connections for the heartbeat pool.	Min: 1
`TASKQ_HEARTBEAT_COMMAND_TIMEOUT`	`float` (seconds)	`2.0`	Per-query timeout for the heartbeat pool: deliberately tighter than `TASKQ_DISPATCHER_COMMAND_TIMEOUT`, since a beat slower than the tick cannot keep a lock lease alive. Raise it on a loaded or cross-region Postgres: `TASKQ_MAX_HEARTBEAT_FAILURES` consecutive timeouts self-terminate the worker.	> 0
`TASKQ_MAX_CONCURRENCY`	`int`	`8`	Max concurrent jobs per worker process. `worker_pool` size is derived as `int(max_concurrency * 1.5)`. When a LOOP-scope `asyncpg.Connection` is registered, it also sizes the per-slot transaction pool (`max_concurrency + 1` direct connections, fully warmed at boot); changes require a worker restart and move the direct-connection budget.	Min: 1
`TASKQ_DISPATCH_SCOPE_BY_HOME_QUEUE`	`bool`	`false`	Deprecated no-op, accepted so configurations that set it keep loading: dispatch is assignment-routed now, so the flag has nothing left to apply. The worker logs a deprecated-setting warning at startup when it is set; remove it from the environment.	n/a

For the fleet-level connection budget these pools feed into (per-worker counts, idle floors, the PgBouncer recommendation threshold), see ops.md: Sizing.

Timing and Liveness¶

Env Var	Type	Default	Description	Constraints
`TASKQ_HEARTBEAT_INTERVAL`	`float` (seconds)	`10.0`	Period between heartbeat ticks.	Min: 0.5
`TASKQ_LOCK_LEASE`	`float` (seconds)	`60.0`	Time before an unrenewed job lock is reclaimed by the sweep. Must be >= max(`TASKQ_HEARTBEAT_INTERVAL`, `TASKQ_HEARTBEAT_COMMAND_TIMEOUT`) + (`TASKQ_MAX_HEARTBEAT_FAILURES` + 1) × (`TASKQ_HEARTBEAT_INTERVAL` + `TASKQ_HEARTBEAT_COMMAND_TIMEOUT`), the worst coherent failed-beat cascade (58 at the defaults), and must exceed `TASKQ_WATCHDOG_LOOP_LAG_BUDGET` + `TASKQ_HEARTBEAT_INTERVAL` (a stalled loop dies before its leases expire). The group-crash corner (a crashed TaskGroup's cancel reaching a consumer before the shutdown anchor stamps, so the release park takes the unanchored cleanup-grace bound and the lease cap does not apply) is only exposed when `lock_lease < cleanup_grace_period + heartbeat_interval + 5s` (the unanchored park plus the terminal-write budget); the cascade invariant already excludes that for heartbeat intervals above `(cleanup_grace + 5s) / 3` (~5s at the default cleanup), and the `cancellation + cleanup < lock_lease` invariant squeezes what is left further (the exposed band also needs `cancellation_grace < heartbeat_interval + 5s`), so only a tightly-graced config with a tiny heartbeat can load into the corner, and it is a startup-warning tier concern, not a validation.	Min: 1.0; see Validation Constraints
`TASKQ_LEADER_LEASE`	`float` (seconds)	`40.0`	How long the maintenance leader's lease is trusted without a renewal; another pod takes leadership once it lapses, so this plus `TASKQ_HEARTBEAT_INTERVAL` bounds failover from a leader that went silent. Renewed every heartbeat interval, and never honoured at less than 4 of them: raising the heartbeat interval alone raises the effective lease rather than shortening the renewal margin.	Min: 1.0
`TASKQ_LEADER_WAKE_JITTER`	`float` (seconds)	`0.25`	Upper bound of the random delay a follower waits between hearing a leader's resign broadcast (the leadership wake channel) and re-running its fenced elect, so a waking fleet does not land on the lease row in the same millisecond. `0` disables the damping delay. The broadcast is a hint: a follower that never hears it elects on the next heartbeat tick, which is also the bound for a leader killed before it can broadcast.	Min: 0.0
`TASKQ_MAX_HEARTBEAT_FAILURES`	`int`	`3`	Consecutive heartbeat failures before the worker self-terminates.	Min: 1

Leader Sweep Intervals¶

The leader runs periodic sweep cycles that reclaim expired locks, expire results, clean up stale workers, evict idle keyed refs, and collect metrics. These settings control the cadence of each sub-task within a sweep cycle, and the event-writer knobs bound each sweep's database work per batch.

Env Var	Type	Default	Description	Constraints
`TASKQ_SWEEP_INTERVAL`	`float` (seconds)	`30.0`	Period between leader sweep loop iterations: reclaim expired locks, sweep expired results, clean up stale workers, and evict idle keyed refs. Lower values reduce recovery latency for crashed workers at the cost of more frequent PG queries.	Min: 1.0
`TASKQ_QUEUE_DEPTH_INTERVAL`	`float` (seconds)	`15.0`	Period between queue-depth metrics sampling iterations.	Min: 1.0
`TASKQ_RESERVATION_SLOTS_INTERVAL`	`float` (seconds)	`15.0`	Period between reservation-slot metrics sampling iterations.	Min: 1.0
`TASKQ_STRANDED_JOBS_INTERVAL`	`float` (seconds)	`60.0`	Period between stranded-jobs checks (pending/scheduled jobs whose actor has no `actor_config` row, or whose routing queue has no live worker); each check publishes `taskq.jobs.stranded` and warns.	Min: 1.0
`TASKQ_EVENT_WRITER_BATCH_SIZE`	`int`	`100`	Rows per committed batch for every writer of `job_events` rows (the expired-lock and deadline sweeps, bulk cancel, actor deregistration), and for the scheduled-to-pending promotion batch, which writes no event rows but takes the same per-tick bound. Keeps each batch transaction inside the `reclaim_event_visibility_delay` margin; the server-side `statement_timeout` remains the enforcement if a batch exceeds it. The loop drains the remainder across further batches/calls.	Range: 1-10000
`TASKQ_EVENT_WRITER_STATEMENT_TIMEOUT_MS`	`float` (milliseconds)	`1750.0`	Server-side `statement_timeout` applied to each event-writer batch transaction via `SET LOCAL`. Defaults to 7/8 of the 2 s `reclaim_event_visibility_delay` margin: a batch that cannot finish inside the watermark margin is aborted by the server rather than silently corrupting reclaim-event delivery.	Min: 50.0
`TASKQ_EVENT_WRITER_REDUCED_BATCH_DIVISOR`	`int`	`4`	Divisor for the reduced event-writer batch tier: once a worker's sweeps trip the batch-size breaker, batches shrink to `max(1, event_writer_batch_size / this)`. Only a degradation ceiling; raising it makes the degraded tier closer to the normal one.	Range: 2-1000
`TASKQ_SWEEP_BREAKER_FAILURE_THRESHOLD`	`int`	`3`	Consecutive sweep-batch cancellations (within `TASKQ_SWEEP_BREAKER_WINDOW_SECS`) before the batch-size breaker latches to the reduced tier for the rest of the process lifetime. Any success between failures resets the consecutive count; a latched breaker does not unlatch.	Min: 1
`TASKQ_SWEEP_BREAKER_WINDOW_SECS`	`float` (seconds)	`600.0`	Rolling window the sweep breaker counts consecutive failures within.	Min: 1.0
`TASKQ_SWEEP_DRAIN_BATCHES`	`int`	`8`	Maximum event-writer batches the leader's sweep loop executes per sweep per tick before leaving the remainder to the next tick. Bounded so one iteration cannot monopolise the loop; every batch commits, so a stopped drain keeps its progress.	Range: 1-1000
`TASKQ_EVENT_RETENTION_PERIOD`	`timedelta`	`7d`	The age at which `job_events` rows are deleted regardless of parent-job status. The crash-reclaim outbox slice (`kind='state_change' AND detail->>'reason'='lock_expired'`) is carved out of this window so an unread reclaim event survives it, but the carve-out is bounded: the same sweep deletes it at 100× this period, so a short retention bounds how far behind a lagging `TaskQ.watch_reclaims()` consumer may run before it silently misses events.	Non-negative; `0` disables the sweep (see The `0` convention)
`TASKQ_KEYED_ROW_RECLAIM_PERIOD`	`timedelta`	`1h`	The idle age (each row's own `last_used_at` stamp) at which fleet-reclaimable keyed rows - keyed `reservation_slots` rows and PG-state-backed keyed `rate_limit_buckets` rows - are deleted. Static buckets and redis-backend keyed rows are never deleted by this sweep.	Non-negative; `0` disables the sweep (see The `0` convention)
`TASKQ_EVENT_RETENTION_BATCH_SIZE`	`int`	`10000`	`job_events` rows deleted per leader sweep tick, one committed batch, once `TASKQ_EVENT_RETENTION_PERIOD` has aged them out.	Min: 1
`TASKQ_KEYED_ROW_RECLAIM_BATCH_SIZE`	`int`	`256`	Rows the fleet keyed-row reclaim sweep deletes per committed batch per tick (`reservation_slots` rows and PG-state-backed keyed `rate_limit_buckets` rows, once `TASKQ_KEYED_ROW_RECLAIM_PERIOD` has aged them idle). The constant-size bound keeps one tick's DELETE independent of the dead-worker backlog it is recovering from.	Min: 1

See maintenance-sweeps.md for why these bounds exist: the failure modes they prevent, the derivation of each default, the breaker's latching rationale, and the failure semantics of the bounded bulk operations.

Graceful Shutdown¶

Env Var	Type	Default	Description	Constraints
`TASKQ_TERMINATION_GRACE_PERIOD`	`float` (seconds)	`85.0`	Total budget from SIGTERM to forced exit (the shutdown watchdog counts it down from the first shutdown signal). Must satisfy: `cancellation_grace + cleanup_grace < termination_grace − 5`, and should cover the modelled worst case `cancellation_grace + cleanup_grace + ~42s` bounded-close teardown tail (eight sequential closes, including the conditional per-slot pool and the two credential-provider closes); the default does (`30 + 10 + 42 = 82s`). The ~87s sibling-crash path (nine closes) exceeds the default by 2s on per-slot workers; raise this setting when per-slot workers need a tight crash budget. A value below the worst case still loads but logs `shutdown-budget-exceeds-termination-grace` at startup.	Min: 5.0; see Validation Constraints
`TASKQ_CANCELLATION_GRACE_PERIOD`	`float` (seconds)	`30.0`	Duration of the cooperative cancel phase before force-cancel.	Min: 0.0
`TASKQ_CLEANUP_GRACE_PERIOD`	`float` (seconds)	`10.0`	Force-cancel cleanup grace period.	Min: 0.0
`TASKQ_RECLAIM_EVENT_VISIBILITY_DELAY`	`float` (seconds)	`2.0`	Trailing-watermark margin that `poll_reclaim_events()` / `TaskQ.watch_reclaims()` apply before returning a `job_events` row, so an out-of-commit-order sibling with a lower `event_id` has time to appear first. Correctness assumes every `job_events` writer transaction commits within this margin of its INSERT; raise it if sweeps run under heavy lock contention or against very large batches, lower it if latency matters more and writes are known to be fast. A writer that exceeds the margin can cause a silently missed event.	Min: 0.0

See workers.md for the shutdown sequence these values control.

Watchdog (hang and deadlock detection)¶

Four independent detectors catch a worker that has stopped making progress but is still running: a state that liveness probes miss, because the event loop can be perfectly responsive while every loop that matters has stopped. On a trip the worker logs the detector and a dump of every asyncio task (name, coroutine, await site), then force-exits with code 2 so the supervisor restarts it. The event-loop lag detector is two-tier: past TASKQ_WATCHDOG_LOOP_LAG_WARN_BUDGET it warns (thread dump + metric + a task dump deferred until the loop recovers, no exit); past TASKQ_WATCHDOG_LOOP_LAG_BUDGET it trips.

Env Var	Type	Default	Description	Constraints
`TASKQ_WATCHDOG_ENABLED`	`bool`	`true`	Master switch for the force-exit detectors (shutdown deadline, stale loop ticks, sibling-contract enforcement, event-loop lag). Observability is NOT switched off with it: a sibling returning cleanly still emits the `sibling-returned-unexpectedly` error, and stale loops still flip `/ready`: a zombie worker must never report Ready.	n/a
`TASKQ_WATCHDOG_CHECK_INTERVAL`	`float` (seconds)	`1.0`	Poll cadence for the stale-tick sweep and the loop-lag thread.	Min: > 0
`TASKQ_WATCHDOG_TICK_GRACE_FACTOR`	`float`	`5.0`	Multiplier on a loop's own iteration period before its tick counts as stale.	Min: > 0
`TASKQ_WATCHDOG_STALE_FLOOR`	`float` (seconds)	`10.0`	Lower bound on any staleness budget, so a short interval cannot produce a hair-trigger.	Min: > 0
`TASKQ_WATCHDOG_LOOP_LAG_BUDGET`	`float` (seconds)	`30.0`	How long the event loop may fail to schedule before the lag detector trips. Tier 2 of the lag detector. Must stay inside `TASKQ_LOCK_LEASE` (see the lag-lease invariant below).	Min: > 0; `watchdog_loop_lag_budget + heartbeat_interval < lock_lease`
`TASKQ_WATCHDOG_LOOP_LAG_WARN_BUDGET`	`float` (seconds)	`5.0`	Non-terminal tier 1 of the lag detector: past this much event-loop lag the worker emits a warning, a `faulthandler` thread dump, the `taskq.worker.watchdog_loop_lag_warns_total` metric, and a deferred asyncio task-stack dump that lands once the loop recovers. It never exits; the terminal tier is `TASKQ_WATCHDOG_LOOP_LAG_BUDGET`.	Min: > 0; must be < `watchdog_loop_lag_budget` (a warn budget at or above the terminal budget silently disables tier 1; the trip fires with no prior warning)
`TASKQ_WATCHDOG_LOOP_LAG_STARTUP_GRACE`	`float` (seconds)	`30.0`	Grace before the lag detector arms, covering import-heavy startup and DI bootstrap.	Min: ≥ 0
`TASKQ_WATCHDOG_DUMP_INTERVAL`	`float` (seconds)	`5.0`	Interval between straggler logs (names + await sites of still-alive siblings) once the dump gate opens.	Min: > 0
`TASKQ_WATCHDOG_DUMP_AFTER_FRACTION`	`float`	`0.5`	Fraction of the shutdown deadline that must be consumed before straggler dumps begin. A drain inside the front half of its budget is within expectations and stays quiet; one `shutdown-watchdog-armed` record is always logged when the countdown starts so the window is never blind.	Range: (0, 1), exclusive; at 1.0 the trip would always fire first

The shutdown deadline is not a separate knob: it reuses TASKQ_TERMINATION_GRACE_PERIOD, measured from the first shutdown signal. That is what finally enforces the total budget the termination-budget constraint already validates.

Choosing values¶

Every trip is terminal, so the defaults err heavily towards missing a hang rather than killing a healthy worker; a false trip under a supervisor can become a restart loop. Two rules follow:

Raise budgets, don't lower them, on constrained or heavily oversubscribed hosts. A loaded host with slow scheduling looks exactly like a mildly wedged one.
Keep the terminal lag budget inside the lease: watchdog_loop_lag_budget + heartbeat_interval < lock_lease is validated at load: a stalled loop must die before its leases expire, or the leader sweep reclaims LIVE jobs' locks mid-stall.
Effective staleness budget for a loop is max(period × TASKQ_WATCHDOG_TICK_GRACE_FACTOR, TASKQ_WATCHDOG_STALE_FLOOR), where period is that loop's own interval. With the defaults, a 2 s sweep loop tolerates 10 s of silence and a 30 s loop tolerates 150 s.

Only loops with an unconditional periodic iteration are watched (heartbeat, progress flush, producer, the leader loops). Loops that legitimately park indefinitely (the NOTIFY listener, job consumers waiting on an empty queue, the credential reload coordinator) are deliberately excluded, since a staleness budget would fire on an idle worker. They are covered by the other three detectors.

Set TASKQ_WATCHDOG_ENABLED=false to disable detection entirely. Prefer raising the budgets first: with it off, a wedged worker stays wedged and silent, which is the failure mode this exists to remove. Two things are deliberately NOT switched off with it: the sibling-contract error log (a clean return outside shutdown still records sibling-returned-unexpectedly; enforcement is off, the signal is not), and the stale-loop readiness check (/ready still flips NotReady on a dead loop, so the zombie stops receiving traffic).

While a shutdown counts down, straggler dumps are gated: nothing is logged until TASKQ_WATCHDOG_DUMP_AFTER_FRACTION of the deadline is consumed (one shutdown-watchdog-armed record marks the start), then TASKQ_WATCHDOG_DUMP_INTERVAL cadence right up to the trip. A normal drain stays quiet; a hung one dies fully diagnosed.

The stale-tick detector interacts with TASKQ_DISPATCHER_COMMAND_TIMEOUT: a loop's worst-case tick gap is timeout + period, so the timeout must fit inside max(period × TASKQ_WATCHDOG_TICK_GRACE_FACTOR, TASKQ_WATCHDOG_STALE_FLOOR) for the period-1 leader loops (the producer loop is exempt; its multi-statement dispatch batch does not fit the timeout + period model). This is enforced at load time when the watchdog is enabled, so an over-large timeout prevents the worker from starting at all; at default watchdog settings the ceiling is just under 9.0; see Validation Constraints.

Retry¶

Env Var	Type	Default	Description	Constraints
`TASKQ_MAX_RETRY_BACKOFF`	`timedelta`	`24h`	Global ceiling on per-attempt retry backoff. Caps `RetryPolicy.cap` fleet-wide to prevent misconfigured actors from stranding jobs indefinitely.	n/a
`TASKQ_DEFAULT_START_TO_CLOSE`	`timedelta \\| None`	`None` (unbounded)	Worker-wide fallback per-attempt execution timeout, applied only when a job has no `start_to_close` of its own (neither passed at enqueue time nor declared as an `@actor(start_to_close=...)` default). Gives every actor on the worker a safety-net wall-clock budget per attempt without configuring it individually.	n/a

See retries.md for the full start_to_close vs schedule_to_close precedence chain.

Rate Limiting¶

Env Var	Type	Default	Description	Constraints
`TASKQ_RATE_LIMIT_PG_FALLBACK_ENABLED`	`bool`	`true`	When `false`, Redis errors propagate instead of triggering the Postgres rate-limit fallback.	n/a
`TASKQ_MAX_KEYED_RESERVATIONS`	`int`	`10000`	Guardrail on the number of distinct keyed-reservation entries tracked in memory. When the limit is reached, new keyed reservations raise `ReservationUnavailable`. Tune to your workload's expected key cardinality.	Min: 1
`TASKQ_MAX_KEYED_RATE_LIMITS`	`int`	`10000`	Guardrail on the number of distinct keyed-rate-limit entries tracked in memory. When the limit is reached, new keyed rate limits raise `ReservationUnavailable`. Independent from `TASKQ_MAX_KEYED_RESERVATIONS`. Tune to your workload's expected key cardinality.	Min: 1

See rate-limiting.md for the fallback behaviour.

Health Server¶

Env Var	Type	Default	Description	Constraints
`TASKQ_HEALTH_ENABLED`	`bool`	`true`	Master switch for the worker health server (both transports).	n/a
`TASKQ_HEALTH_SOCKET_PATH`	`str`	`/tmp/taskq_health.sock`	Unix socket path for the health server. A collision (a live peer owns the path; the bind refuses to steal it) warns via `health-server-unavailable` and the boot continues; with `TASKQ_HEALTH_PORT` set the TCP listener still serves probes, with it unset the worker runs with no listener at all. A worker booted with no explicit path (no CLI option, no value in the env/`.env` cascade) binds the per-process default `/tmp/taskq_health_<pid>.sock` instead — this fallback value applies only when the variable IS set, and one container/replica per worker needs no configuration.	n/a
`TASKQ_HEALTH_PORT`	`int \\| None`	unset	TCP port for the HTTP health listener serving `/live` and `/ready`. Unset means no TCP listener at all: setting a port is the opt-in. Required on Azure Container Apps, whose probes support only `httpGet`/`tcpSocket` and cannot reach a Unix socket. If the port cannot be bound the worker logs `health-http-bind-failed` at ERROR and refuses to start (`HealthTcpBindError`; the probes target this port, and a booting worker with dead probes is the silent kind of down; see deployment.md: Health probes): give each replica a unique port. The unix socket's collision behavior is deliberately different (a live peer owns the path; the boot warns and continues), and the two transports bind independently, so a unix collision does not stop the TCP listener from coming up: with a port set, the boot warns (`health-server-unavailable` with `tcp_listener=serving`) and the port keeps answering probes. `0` binds an ephemeral port (tests only); this family's "off" is unset, not `0` (see The `0` convention).	0-65535
`TASKQ_HEALTH_HOST`	`str`	`0.0.0.0`	Bind address for the TCP health listener and the Prometheus scrape listener. Only used when `TASKQ_HEALTH_PORT` or `TASKQ_METRICS_PORT` is set. Defaults to all interfaces because ACA and Kubernetes probe and scrape the replica over the pod network; narrow to `127.0.0.1` when only a local sidecar probes. The scrape listener alone can be pointed elsewhere with `TASKQ_METRICS_HOST`.	n/a
`TASKQ_HEALTH_PG_PING_TIMEOUT`	`float` (seconds)	`0.2`	Timeout for the readiness PG ping: the role-pool ping, and the per-slot transaction pool's ping when that pool exists (overlapping probes share a single ping).	Min: 0.0
`TASKQ_HEALTH_REQUEST_TIMEOUT`	`float` (seconds)	`2.0`	Time a probe gets to send its whole request line and headers before the connection is dropped unanswered. Bounds a drip-feed client that would otherwise hold a connection open by staying just inside a per-line timeout. Keep at or below the shortest probe `timeoutSeconds` you configure.	> 0
`TASKQ_HEALTH_MAX_HEADER_BYTES`	`int`	`16384`	Cap on a probe request's accumulated request line plus headers. Pairs with `TASKQ_HEALTH_REQUEST_TIMEOUT` to bound a peer sending many small lines fast enough to stay inside the deadline.	> 0
`TASKQ_HEALTH_READINESS_CHECK_TIMEOUT`	`float` (seconds)	`5.0`	Time each check registered via `taskq.worker.health.register_readiness_check` gets before it counts as a readiness failure. Matches the ACA default readiness probe `timeoutSeconds`.	> 0
`TASKQ_HEALTH_TASKS_ENABLED`	`bool`	`false`	Expose the `/tasks` asyncio stack-dump endpoint for live debugging of a stuck worker. Off by default; see below.	n/a

TASKQ_HEALTH_ENABLED=false disables both transports; a TASKQ_HEALTH_PORT set alongside it is not honoured, and probes against that port will fail.

See deployment.md for working Azure Container Apps, Kubernetes and App Service probe configurations, and for registering your own readiness checks.

/tasks returns every live task's name, coroutine and await site; never locals or payload values. It is off by default because that still reveals code structure and file paths. Enabling it also tightens the health socket to mode 0600 (owner-only). It is served on the Unix socket only; the TCP listener answers /tasks with 404 even when this setting is on, because none of the socket's filesystem permissions apply there, and it is never mounted on the admin UI surface.

The same dump is available without enabling the endpoint by sending SIGUSR2 to the worker, which writes it to the log. Reach for either when a worker is alive but idle and you need to know what it is waiting on; that question previously required rebuilding the image with instrumentation.

NOTIFY Listener¶

Env Var	Type	Default	Description	Constraints
`TASKQ_POLL_INTERVAL`	`float` (seconds)	`1.0`	Fallback polling cadence when the NOTIFY listener is unavailable.	n/a
`TASKQ_NOTIFY_HEALTH_CHECK_INTERVAL`	`float` (seconds)	`5.0`	How often the NOTIFY health check issues `SELECT 1`. Detection latency before reconnect is at most this interval.	n/a
`TASKQ_NOTIFY_RECONNECT_BACKOFF_INITIAL`	`float` (seconds)	`1.0`	Initial backoff before the first NOTIFY reconnect. Doubles each attempt, capped at 30 s. Sequence: 1, 2, 4, 8, 16, 30.	n/a
`TASKQ_NOTIFY_LISTENER_SETUP_TIMEOUT`	`float` (seconds)	`10.0`	Bounds each `LISTEN` execute and `add_listener` call during NOTIFY listener setup and reconnect: a half-open PG connection that accepts TCP (or completes the reconnect factory handshake) but stalls on the LISTEN execute or registration would otherwise wedge the notify loop forever. On timeout the connection is closed (bounded) and the reconnect retry loop is entered (or the initial setup raises).	Must be > 0
`TASKQ_NOTIFY_ENABLED`	`bool`	`true`	When `true`, the worker uses LISTEN/NOTIFY for near-zero-latency dispatch wakeups with poll interval as fallback. When `false`, the worker uses poll-only dispatch.	n/a
`TASKQ_NOTIFY_POLL_INTERVAL`	`float` (seconds)	`5.0`	Fallback poll cadence when NOTIFY is enabled. Uses `TASKQ_POLL_INTERVAL` when NOTIFY is disabled.	Min: 0.5

Credential Hot-Reload¶

TASKQ_RELOAD_INTERVAL is a TaskQSettings field (see above): it drives the worker and taskq ui serve alike.

Env Var	Type	Default	Description	Constraints
`TASKQ_RELOAD_FACTORY_TIMEOUT`	`float` (seconds)	`30.0`	Bounds each individual factory call: during a credential hot-reload (`reload_credentials`), at bootstrap when the worker opens its per-slot transaction pool (a fully-warmed open means one connection, and on a managed-identity deployment one credential fetch, per consumer slot), and on the notify listener's health-check reconnect (`reconnect_notify_conn`). A hung token endpoint is marked failed for that resource, or logged as a reconnect attempt and retried, instead of wedging the reload coordinator (and all future SIGHUPs), worker boot, or the reconnect loop.	Must be > 0

A hot-reload rebuilds factory-backed resources; it never re-reads settings. TASKQ_MAX_CONCURRENCY changes require a worker restart; the setting sizes the per-slot transaction pool (max_concurrency + 1 connections), and on the per-slot path (a LOOP-scope asyncpg.Connection registered plus max_concurrency > 1) that restart is boot-blocking: the worker opens the pool at startup and fails to boot if it cannot.

Queue Selection¶

Env Var	Type	Default	Description	Constraints
`TASKQ_QUEUES`	`list[str]`	`["default"]`	Queue names this worker consumes. Set as a comma-separated string: `TASKQ_QUEUES=default,priority`. Each name must match `[A-Za-z0-9_][A-Za-z0-9_.-]*`; rejected at load time otherwise.	n/a
`TASKQ_QUEUES_STRICT`	`bool`	`false`	When `true`, the enqueue-time unserved-queue check becomes hard: a submit whose queue is declared in NEITHER the fleet's registered assignments NOR the deploying process's own queue set (`TASKQ_QUEUES`, when it carries one) raises `UnknownQueueError` before any backend write (direct enqueues, every batch arm and finalizer, sub-job fan-out via `ctx.jobs`) instead of storing an orphaned row no worker claims. The asymmetry is the spine: refusals require positive two-source evidence; every ambiguous state — an EMPTY registration snapshot (no worker ever started, the first deploy), a failed snapshot read, env-vs-snapshot disagreement (a new queue mid-deploy), the TTL registration window — resolves to ALLOW + the diagnostic note. Over-rejection is strictly worse than over-permission. The worker-side twin fails the boot when a configured queue (`TASKQ_QUEUES`) has no registered actor routing to it — worker-only, because actors register only there; its inverse (stored assignments no one here serves or consumes — retired-actor drift) is a loud advisory at every worker boot, never a refusal. Safe deploy order for new queues: migrate → workers → clients; a web client whose settings declare no queue set judges on the registered snapshot alone, so workers-first is the operator's protection for a new queue's first submits (the refusal message says exactly this). Fails open on an unavailable registration snapshot (a database blip or migration window is never a submit outage; the check re-arms on the next refresh). Per-submit escape: `allow_unregistered=True` for genuinely dynamic queue names (the diagnostic note still fires on an escaped call). Recommended for production fleets with a fixed queue topology.	n/a
`TASKQ_POOL_MAX_INACTIVE_LIFETIME`	`float` (seconds)	`300.0`	Closes asyncpg connections idle longer than this. Applied to all four pools: dispatcher, heartbeat, worker, and the conditional per-slot transaction pool.	Min: 0.0
`TASKQ_WORKER_LABEL`	`str \\| None`	`None`	Human-readable label for this worker. Stored in `workers.worker_label` for correlation with workgroup supervisors and external monitoring. When omitted, hostname + pid is used.	n/a
`TASKQ_WORKGROUP_INSTANCE`	`str \\| None`	`None`	UUIDv7 identifying the workgroup orchestrator that launched this worker. Stored in `workers.workgroup_instance` for cross-process correlation. Set automatically by the workgroup supervisor.	n/a

Renaming or retiring a queue under strict mode¶

Queue names live in three places — the @actor(queue=...) literal, the worker's TASKQ_QUEUES declaration, and the fleet's stored actor_config assignments — and strict mode's refusals need TWO of those (the declared set and the stored snapshot) to agree a queue is missing before anything refuses. That gives the safe rename procedure; deploy order does the protecting, never luck:

Declare the new queue. Add it to TASKQ_QUEUES on the workers (and on any producer process that carries a queue set). A submit to the new name during this window is allowed through the registration lag: it is in the declared set even though no assignment routes to it yet, and the unserved-queue note fires as the advisory.
Deploy the workers so the actor(s) on the new queue register — sync_actor_config writes their stored assignments, and every client's snapshot picks the queue up within one TTL (5 s default). Strict refusals are now impossible for the new name on every source.
Move the work: redeploy the producers' @actor(queue=...) / per-call queue= onto the new name. For a window both names are declared and routed; strict has nothing to say about a queue both sources know.
Remove the old declaration once the fleet snapshot shows no new writes to the old queue (taskq queues / the admin UI's per-queue pending counts). Do not delete the old actor_config rows while in-flight jobs reference them — rows mid-retry route by their stored assignment.

Retiring a queue entirely follows the same tail: stop serving it, remove the declaration, and let the retirement advisory do the sweeping — every worker boot emits one aggregated retired-queue-assignments WARNING naming stored assignments that no registered actor serves and no declared set consumes (advisory, never a refusal; a sibling worker may still own the queue). When no worker mentions the queue anymore, the stored rows are provably dead and taskq actor-config deletes them.

Genuinely dynamic names (tenant-prefixed, sharded at runtime) are the escape hatch's job, not the rename ritual's: pass allow_unregistered=True at those submit sites — the diagnostic note still fires, so a typo on an escaped call stays visible.

Observability¶

Env Var	Type	Default	Description	Constraints
`TASKQ_OTEL_ENABLED`	`bool`	`true`	When `false`, suppresses all OTel span and metric creation. Operations still succeed.	n/a
`TASKQ_EXCEPTION_REDACTION_ENABLED`	`bool`	`true`	When `false`, Postgres `DETAIL:` lines (which quote caller-supplied row values) are no longer dropped from exception text on spans and logs. Debugging aid only; the worker logs an `exception-redaction-disabled` WARNING at every startup while it is off. URI credential masking is always applied and is unaffected.	n/a
`TASKQ_WORKER_GROUP`	`str`	`default`	Consumer group name emitted as `messaging.consumer.group.name` on spans.	n/a
`TASKQ_LOG_FORMAT`	`str`	`json`	Log renderer. `json` for production; `console` for human-readable dev output. Only these two values are valid.	Must be `json` or `console`
`TASKQ_LOG_LEVEL`	`str`	`INFO`	Root logger level.	n/a
`TASKQ_LOG_EVENTS_LEVEL`	`str`	`info`	The operational-event-stream verbosity knob: which structlog event lines the worker emits. `info` (default) is today's stream byte-identical; `warning` emits only the failure/anomaly stream; `off` emits only WARNING-and-above (for deployments consuming OTel spans or tailing `job_events`); `debug` adds the per-tick internals. The `job_events` ledger is untouched at every level — the knob suppresses the streaming duplicate only. See the semantics table below.	Must be `info`, `warning`, `off`, or `debug` (case-insensitive)
`TASKQ_OTEL_AUTOCONFIGURE`	`bool`	`true`	When `true`, the `taskq worker` CLI installs SDK tracer and meter providers from the standard OTel environment variables (`OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_TRACES_EXPORTER`, `OTEL_METRICS_EXPORTER`, `OTEL_LOGS_EXPORTER`) and from `TASKQ_METRICS_PORT`, through the same configurator `opentelemetry-instrument` uses; only when the `[otel]` extra is installed and no provider is set yet. Set `false` when the embedding application or a vendor distro configures the SDK itself. `OTEL_SDK_DISABLED=true` is honoured either way. See observability.md: setup.	n/a
`TASKQ_EXCEPTION_MESSAGE_MAX_CHARS`	`int`	`2000`	Bound on exception message text on spans and logs, after scrubbing. Matches the admin UI's traceback bound so there is one number for how much error text is kept, not two. Truncation appends the dropped character count, so an operator can see text was cut and raise this. The stack trace is a separate field and is not bounded by this.	Min: 100
`TASKQ_METRICS_PORT`	`int \\| None`	`None`	TCP port for the worker's standalone Prometheus scrape listener, bound on `TASKQ_METRICS_HOST` (falling back to `TASKQ_HEALTH_HOST`). Unset means no listener, so setting a port is the opt-in (the `TASKQ_HEALTH_PORT` shape). Needs the `[prometheus]` extra and `TASKQ_OTEL_AUTOCONFIGURE=true`; the worker then serves every series it records at `http://<host>:<port>/metrics` (see observability.md: Serving the metrics). A listener that cannot be configured or bound raises `OtelExporterConfigurationError` and the worker exits 1 rather than start with its scrape silently dead.	Range: 1-65535
`TASKQ_METRICS_HOST`	`str \\| None`	`None`	Bind address for the Prometheus scrape listener alone, overriding `TASKQ_HEALTH_HOST` for that listener, so the scrape and the probes can sit on different interfaces: the loopback sidecar scraper next to a pod-network prober is the shape that needs this. Unset falls back to `TASKQ_HEALTH_HOST`. Only used when `TASKQ_METRICS_PORT` is set. The scrape endpoint is unauthenticated: SECURITY.md and observability.md document what it exposes; keep a loopback bind or the pod network.	n/a

See observability.md for OTel configuration.

The event-stream verbosity¶

Every state change is durably recorded in the job_events ledger inside the same transaction as the state itself — the DB ledger is the audit trail. The per-job JSON log lines are a streaming duplicate of those rows, and the hot-path audit measured that duplicate stream at ~23% of the worker's on-CPU time. This knob suppresses the streaming duplicate only; job_events receives every event at every level, guaranteed and pinned by test.

Value	Per-job happy-path lines (`state-change`, `cancel_phase_change`)	Failure/anomaly stream (`job-failed`, `heartbeat-tick-failure`, `isolate-self-*`, reclaims, `worker-watchdog-trip`, backpressure refusals, sweep/cron failures)	Per-tick internals (`loop-lag`, `poll-cadence`)
`info` (default)	emitted (byte-identical to pre-knob behavior)	emitted	suppressed
`warning`	suppressed	emitted (anomaly events survive even when logged at INFO)	suppressed
`off`	suppressed	WARNING-and-above emitted; INFO-level anomaly lines suppressed	suppressed
`debug`	emitted	emitted	emitted

Key properties:

off never blinds the operator to failures. WARNING-and-above anomaly lines (job-failed, terminal-write-failed, heartbeat-tick-failure, worker-watchdog-trip) emit at every level. off is short for "off for the event stream", never "off for failures".
The call sites are unconditional. The knob is applied by a structlog processor in the canonical chain (before the serialization cost), not by branching at call sites, so a call site cannot drift from the setting.
Unclassified events fail open. A future event not in the classification table passes at every level — a new event can never be silently suppressed by a stale table.
The ledger is untouchable. With off, job_events still receives every event (asserted end-to-end in the system tier).

With OTel configured, the spans carry the same events (see observability.md): warning/off keeps the traces while cutting the duplicate log stream.

Actor Config¶

Env Var	Type	Default	Description	Constraints
`TASKQ_FORCE_UPDATE_ACTOR_CONFIG`	`bool`	`false`	When `true`, silently overwrites a stored `actor_config` row's `metadata` if it differs from the registered value. When `false`, metadata drift raises `ActorConfigDriftList` and the worker refuses to start. Does not affect `queue` or the capacity fields (`max_concurrent` / `max_pending` / `result_ttl`); those are operator-owned once a row exists and are never overwritten by the registered literal regardless of this flag; move an actor's queue with `taskq actor-config move-queue`, tune capacity with `taskq actor-config set`. Use for one deploy when intentionally adopting a code-side `metadata` change, then unset.	n/a

Cron Scheduler¶

Env Var	Type	Default	Description	Constraints
`TASKQ_CRON_CATCH_UP_WINDOW`	`timedelta`	`1h`	Missed firings within this window are caught up sequentially; older misses are skipped.	Must not be negative
`TASKQ_CRON_AUTO_DISABLE_THRESHOLD`	`int`	`3`	Consecutive failures before a schedule is auto-disabled.	Min: 1
`TASKQ_CRON_TICK_LIMIT`	`int`	`100`	Maximum schedules one cron tick selects, plans and fires. A catch-up burst larger than this drains across successive one-second ticks instead of one oversized transaction; the remainder stays due and untouched until its tick.	Range: 1-10000
`TASKQ_CRON_PAYLOAD_FACTORY_TIMEOUT`	`float` (seconds)	`5.0`	Per-call deadline for a cron schedule's payload factory (both the off-loop call and the coroutine a factory returns). The tick clamps it to stay strictly inside what is left of the leader's whole-tick deadline (`TASKQ_DISPATCHER_COMMAND_TIMEOUT`), so a value at or above that deadline is an upper bound, not the effective one.	Must be > 0 and finite

See cron.md for cron scheduling details.

Until-idle drain mode¶

These settings only apply when the worker is started with --until-idle (run until the queues drain, then exit):

Env Var	Type	Default	Description	Constraints
`TASKQ_IDLE_SETTLE_WINDOW`	`float` (seconds)	`2.0`	Time the drain monitor waits after queues appear empty before declaring drained.	Min: 0.0
`TASKQ_IDLE_POLL_INTERVAL`	`float` (seconds)	`1.0`	How often the drain monitor checks queue depth.	Min: 0.1
`TASKQ_IDLE_MAX_RUNTIME`	`float \\| None` (seconds)	`None`	Maximum wall-clock time for until-idle mode; when exceeded the worker exits with code 4. `None` means no limit.	Must be > 0 when set

Progress Fanout¶

Env Var	Type	Default	Description	Constraints
`TASKQ_PROGRESS_COALESCE_INTERVAL`	`float` (seconds)	`0.5`	Seconds between periodic flush ticks that write coalesced progress state to Postgres. Lower values shrink the window a hot loop's intermediate values can vanish in.	Min: 0.1
`TASKQ_PROGRESS_DATA_MAX_BYTES`	`int`	`16384`	Maximum serialised byte length of the `data` dict in a single progress call. Exceeding this raises `ProgressTooLarge`. Snapshot-style publishers (full-document progress snapshots) must size the cap at or above their payload bound: `data` only grows, so once one snapshot exceeds the cap every later snapshot raises too, and the durable row freezes at the last accepted state until the job reaches a terminal status.	Range: 1024-1048576
`TASKQ_PROGRESS_PUBLISH_GLOBAL`	`bool`	`true`	When `true`, progress updates are published to the global fanout channel (e.g. Redis). When `false`, progress updates are only written to Postgres.	n/a

See progress.md for progress tracking details.

Idempotency¶

Env Var	Type	Default	Description	Constraints
`TASKQ_IDEMPOTENCY_KEY_MAX_BYTES`	`int`	`1024`	Maximum UTF-8 byte length of `idempotency_key`, and of `idempotency_scope`, each. Bytes rather than characters because the real bound is the composite unique index `jobs_idempotency_scope_key_uniq`: a Postgres btree v4 entry cannot exceed 2704 bytes. Raise it when keys are derived from URLs, composite business keys or opaque vendor continuation cursors.	Range: 1-1300

The ceiling (1300) keeps scope + key + index-tuple overhead under the btree limit, so no value of this setting can turn a valid enqueue into a raw index row size ... exceeds btree version 4 maximum error from Postgres.

Job Results¶

Env Var	Type	Default	Description	Constraints
`TASKQ_RESULT_MAX_BYTES`	`int`	`65536`	Maximum serialised byte length of a job's terminal result dict. A larger result raises `ResultTooLarge`, which is non-retryable: the actor already ran, so a re-run returns the same oversized value.	Range: 1024-1048576

The ceiling matches TASKQ_PROGRESS_DATA_MAX_BYTES, so the durable result can be configured as large as the transient progress payload. Raise it with the row size in mind: the result is stored for the job's result_ttl.

Job Retention and Archive¶

The prune sweep (Sweep 5) runs once daily and moves terminal jobs from jobs into jobs_archive after their per-status retention period has elapsed. The archive expiry sweep (Sweep 6) runs once daily and hard-deletes rows from jobs_archive once their archive retention period has expired. Both sweeps are batched, atomic, and advisory-locked.

Prune schedule and batch size¶

Env Var	Type	Default	Description	Constraints
`TASKQ_PRUNE_SCHEDULE_UTC`	`str`	`03:00`	Daily fire time for the prune sweep in `HH:MM` UTC format. Ignored when `TASKQ_PRUNE_CRON_EXPR` is set.	n/a
`TASKQ_PRUNE_CRON_EXPR`	`str \\| None`	`None`	Full 5-field cron expression for the prune sweep. Takes precedence over `TASKQ_PRUNE_SCHEDULE_UTC`.	n/a
`TASKQ_PRUNE_BATCH_SIZE`	`int`	`10000`	Rows processed per CTE batch. The sweep repeats until no rows remain.	Min: 1

Per-status retention¶

These control how long a terminal job stays in the jobs table before being moved to jobs_archive. Shorter values keep the hot jobs table smaller; longer values make recent history available without querying the archive. For these fields 0 archives the status at the next daily sweep: the prune family's zero-means-now polarity, opposite to the deletion-sweep family's (see The 0 convention).

Env Var	Type	Default	Description
`TASKQ_PRUNE_RETENTION_PERIOD`	`timedelta`	`30d`	Reserved as the global fallback for a terminal status without a per-status knob. Currently inert: the four per-status fields below cover every terminal status, so the sweep never falls back here and setting this changes nothing today. Size the per-status fields instead.
`TASKQ_PRUNE_RETENTION_SUCCEEDED`	`timedelta`	`30d`	Retention for `succeeded` jobs: usually the bulk of terminal volume; the first knob to lower when the hot table grows.
`TASKQ_PRUNE_RETENTION_FAILED`	`timedelta`	`90d`	Retention for `failed` jobs: kept hot longer by default because they are the first incident-audit trail.
`TASKQ_PRUNE_RETENTION_CANCELLED`	`timedelta`	`30d`	Retention for `cancelled` jobs.
`TASKQ_PRUNE_RETENTION_ABANDONED`	`timedelta`	`90d`	Retention for `abandoned` and `crashed` jobs.

Per-actor retention overrides can be set in actor_config.metadata as retention_days. It accepts an integer or ASCII digit string from 0 through 999999999 days, the range supported by Python's timedelta; 0 archives terminal jobs on the next prune sweep. Invalid declarations raise immediately. Invalid legacy or manually edited database values are ignored with a warning, so the actor falls back to global retention without stopping the fleet-wide prune. When valid, an actor's jobs are pruned at min(retention_days, global_per_status_retention). This allows short-lived high-volume actors (e.g. ping jobs) to be pruned faster without affecting the global defaults.

Archive retention and expiry schedule¶

Env Var	Type	Default	Description	Constraints
`TASKQ_ARCHIVE_RETENTION_PERIOD`	`timedelta`	`365d`	How long a row stays in `jobs_archive` before the expiry sweep hard-deletes it.	Non-negative; `0` hard-deletes at the next expiry sweep (see The `0` convention)
`TASKQ_ARCHIVE_EXPIRY_SCHEDULE_UTC`	`str`	`04:00`	Daily fire time for the archive expiry sweep in `HH:MM` UTC format. Ignored when `TASKQ_ARCHIVE_EXPIRY_CRON_EXPR` is set.	n/a
`TASKQ_ARCHIVE_EXPIRY_CRON_EXPR`	`str \\| None`	`None`	Full 5-field cron expression for the archive expiry sweep. Takes precedence over `TASKQ_ARCHIVE_EXPIRY_SCHEDULE_UTC`.	n/a
`TASKQ_TIMESCALEDB_HYPERTABLES`	`bool`	`false`	Opt in to TimescaleDB hypertables for `jobs_archive`, `job_attempts_archive` and `job_events`, with retention policies derived from `TASKQ_ARCHIVE_RETENTION_PERIOD` and `TASKQ_EVENT_RETENTION_PERIOD`. The same deploy arms columnstore (compression) policies on the two archive tables — `compress_after` derives from `TASKQ_ARCHIVE_RETENTION_PERIOD` like everything else; there is no separate compression knob, and `job_events` deliberately stays rowstore. With the flag off (the default), setup issues zero new statements and vanilla Postgres is fully supported. With the flag on, `taskq migrate up` requires the server to offer and load the `timescaledb` extension and fails loudly naming this setting when it does not. Flipping the flag off is also step one of the disable path (see timescaledb.md). For when to opt in, see the ops guide's decision section.	n/a

Storage planning. Each job row is approximately 1-4 KB depending on payload and result sizes. With the default retention settings (30 days in jobs, 365 days in jobs_archive) and 100 000 jobs/day, jobs holds roughly 3 M rows and jobs_archive holds roughly 35 M rows. Tune the retention values and monitor table sizes with SELECT pg_size_pretty(pg_total_relation_size('"taskq".jobs_archive')). On the hypertable mode with the columnstore armed, the measured archive storage is ~6× (−84 %) smaller (benchmarks/results/timescale-compression.json); compressed chunks carry one server prerequisite — timescaledb.max_tuples_decompressed_per_dml_transaction must be raised above its 100000 default or the archive-expiry sweep hard-errors on compressed chunks (the deploy step probes and warns loudly; see timescaledb.md).

See ../architecture.md for the prune/archive schema design and the jobs_archive table structure.

Validation Constraints¶

These cross-field constraints are enforced in post_load at startup. Violations raise ValidationError (or MultipleValidationErrors when several fire at once) before the process enters its main loop.

Lock lease vs heartbeat cadence¶

lock_lease >= max(heartbeat_interval, heartbeat_command_timeout) +
              (max_heartbeat_failures + 1) ×
              (heartbeat_interval + heartbeat_command_timeout)

Rationale: the lease must outlive the worst coherent failed-beat cascade to the heartbeat's isolate decision, the (max_heartbeat_failures + 1)-th consecutive failed beat: the last good beat's tail (max(heartbeat_interval, heartbeat_command_timeout), the gap from that renewal point to the next tick's start) plus that many failed-beat cycles, where each failed beat costs at most heartbeat_interval + heartbeat_command_timeout (the pool acquire is bounded by the interval; the tick's command sequence and its bounded teardown share one heartbeat_command_timeout budget). At the defaults the floor is max(10, 2) + 4 × (10 + 2) = 58 against the 60 s lease. Prevents false abandonment under transient PG connectivity issues AND under pool contention, which the older lock_lease >= 4 × heartbeat_interval rule ignored.

Error pattern: lock_lease (...) must cover the worst coherent failed-beat cascade: max(heartbeat_interval, heartbeat_command_timeout) + (max_heartbeat_failures + 1) * (heartbeat_interval + heartbeat_command_timeout) = ...

Termination budget¶

cancellation_grace_period + cleanup_grace_period < termination_grace_period − 5

Rationale: reserves at least 5 seconds for post-shutdown bookkeeping after both cancel phases complete.

Error pattern: cancellation_grace_period + cleanup_grace_period must be < termination_grace_period - 5

Cancellation phases vs lock lease¶

cancellation_grace_period + cleanup_grace_period < lock_lease

Rationale: ensures the worker finishes its shutdown sequence before the job lock expires and the sweep can reclaim the job.

Error pattern: cancellation_grace_period + cleanup_grace_period must be < lock_lease

Lag watchdog vs lock lease (watchdog on)¶

watchdog_loop_lag_budget + heartbeat_interval < lock_lease

Rationale: a stalled event loop must die (the terminal lag watchdog trips at watchdog_loop_lag_budget) before its leases can expire, otherwise the leader sweep reclaims LIVE jobs' locks mid-stall and the worker wakes to find its work reassigned. The heartbeat_interval term is the worst-case age the last beat can carry when the stall starts, so the trip is guaranteed to land inside the lease. Skipped when watchdog_enabled=false, since no terminal lag detector is armed then. Keep the lag budget comfortably inside lock_lease; both knobs must move together.

Error pattern: watchdog_loop_lag_budget ... must be < lock_lease

Dispatcher command timeout vs staleness budget (watchdog on)¶

Checked for the period-1 leader loops (leader.scheduled_wake, leader.cron), which are the loops wrapped in a single asyncio.timeout. With period = 1.0:

dispatcher_command_timeout + 1.0 < max(1.0 × watchdog_tick_grace_factor, watchdog_stale_floor)

The producer loop is deliberately not checked: dispatch_batch is a multi-statement transaction bounded per-statement by the pool's command_timeout, so its worst-case tick gap is k × timeout + period for k statements, a quantity the invariant cannot express at settings-load time.

You cannot raise dispatcher_command_timeout far: the usable ceiling is under 9.0s at defaults

With the default watchdog_tick_grace_factor=5.0 and watchdog_stale_floor=10.0, the budget is max(1.0 × 5.0, 10.0) = 10.0, so the timeout must satisfy timeout + 1.0 < 10.0, i.e. strictly less than 9.0 (default 5.0).

This is enforced at settings load, so an out-of-range value does not degrade gracefully: the worker fails to start and a container restarts into CrashLoopBackOff. Setting TASKQ_DISPATCHER_COMMAND_TIMEOUT=60 as an emergency mitigation for a slow database takes the fleet down rather than making it more patient.

To raise it meaningfully, both knobs must move together: raise the budget first, then the timeout:

# Budget becomes max(1.0 × 5.0, 30.0) = 30.0 → timeout may go up to 28.9
TASKQ_WATCHDOG_STALE_FLOOR=30.0
TASKQ_DISPATCHER_COMMAND_TIMEOUT=20.0

Understand the trade before doing so: the staleness budget is how long a wedged leader loop goes undetected. Raising it to 30s means a genuinely stuck worker is force-exited 30s late, which is the cost of tolerating a slower Postgres. A long dispatcher timeout is rarely the right fix for a slow database; prefer fixing query latency, and see Performance issues.

Rationale: those loops tick once per iteration and sleep one period afterwards, so their worst-case tick gap is timeout + period. A gap that can reach the loop's staleness budget makes detector 2 force-exit a healthy worker in the middle of the PG degradation it should ride out (measured with the old 10.0 default against the 10.0 floor: an 11s gap and a trip at age 10.008s). Skipped when watchdog_enabled=false, since detector 2 is never spawned then. If the budget side is too small for any legal timeout (budget <= period + 1.0), the error is attributed to watchdog_stale_floor instead.

Error pattern: dispatcher_command_timeout ... must be < the loop's staleness budget

Log format¶

log_format in {"json", "console"}

Error pattern: log_format must be one of ['console', 'json'], got <value>

Log events level¶

log_events_level.lower() in {"info", "warning", "off", "debug"}

Case-insensitive (WARNING loads as warning). Error pattern: log_events_level must be one of ['debug', 'info', 'off', 'warning'], got <value>

Derived Values¶

These values are computed from settings rather than set directly.

`worker_pool_size`¶

worker_pool_size = int(max_concurrency * 1.5)

The worker pool is sized at 1.5× TASKQ_MAX_CONCURRENCY to provide burst headroom: jobs that briefly block on I/O can release connections while new ones are dispatched, preventing pool exhaustion at full concurrency.

`resolved_pg_dsn_direct`¶

TASKQ_PG_DSN_DIRECT  →  falls back to TASKQ_PG_DSN when unset

Used by dispatcher_pool, heartbeat_pool, notify_conn, and leader_conn. Always points to a session-mode connection that supports LISTEN/NOTIFY and advisory locks.

`resolved_pg_dsn_pooled`¶

TASKQ_PG_DSN_POOLED  →  falls back to TASKQ_PG_DSN when unset

Used exclusively by worker_pool. May safely route through PgBouncer in transaction mode because the worker pool does not use session-level features.

`dispatcher_pool` command timeout¶

pool command_timeout = max(TASKQ_DISPATCHER_COMMAND_TIMEOUT,
                           widest widened admission lock budget / 0.8)

The admission-path rate-limit acquires (token bucket / sliding window Postgres locks; see rate-limiting.md) run on the dispatcher pool with server-side lock_timeout budgets from TASKQ_TOKEN_BUCKET_LOCK_TIMEOUT_MS / TASKQ_SLIDING_WINDOW_LOCK_TIMEOUT_MS. asyncpg enforces the pool's command_timeout as its own client-side per-statement timer, independent of those budgets, so a budget widened past the 5.0 s default would be silently truncated by the pool's own 5 s timer before the server-side refusal (the typed denial with its retry hint) could ever fire. To prevent that, every TaskQ-built dispatcher pool re-derives its per-query bound exactly the way the client pool re-derives its own from the enqueue lock budgets: the configured value is the floor, and each admission budget configured above its 5000 ms shipped default raises the bound to budget / 0.8 seconds, so the server-side lock_timeout refusal always fires before the pool's client-side timer. At the shipped defaults the applied bound is exactly TASKQ_DISPATCHER_COMMAND_TIMEOUT; no behavior change unless you widen a budget, which is the case that needs it.

Two corollaries:

A caller-supplied dispatcher pool (WorkerConnections.dispatcher_pool / ..._factory) keeps its own command_timeout; if you widen an admission budget there, size the pool's timeout above the budget yourself.
A budget of 0 or less asks the server for an unbounded wait (the lock_timeout GUC convention; the lock-wait budget family's 0 polarity, see The 0 convention). A TaskQ-built pool still applies its per-query bound to that statement; the black-hole guard every other statement relies on is not dropped for it, so if you truly need an unbounded admission wait, supply your own pool without a client-side bound. Prefer a large finite value.

PgBouncer Configuration Pattern¶

When running PgBouncer in front of Postgres, split the DSN by connection type:

# Direct connection: used for LISTEN/NOTIFY, advisory locks, dispatcher, heartbeat
TASKQ_PG_DSN_DIRECT=postgresql://taskq:pass@postgres:5432/taskq

# Pooled connection: can go through PgBouncer transaction mode
TASKQ_PG_DSN_POOLED=postgresql://taskq:pass@pgbouncer:5432/taskq

If neither TASKQ_PG_DSN_DIRECT nor TASKQ_PG_DSN_POOLED is set, both resolve to TASKQ_PG_DSN. In that case TASKQ_PG_DSN must point directly at Postgres (not PgBouncer), because the direct-connection pools require session mode.

The proven transaction-mode topology¶

This split is proven against a live PgBouncer in transaction mode (tests/test_pgbouncer_transaction_mode.py): full worker boot + job completion with the worker pool behind the pooler, claims/heartbeats/LISTEN direct, admin reads concurrent, and the prepared-statement mechanics below red/green-pinned. The essentials of the working pgbouncer.ini:

[databases]
taskq = host=postgres port=5432 dbname=taskq

[pgbouncer]
listen_port = 5432
pool_mode = transaction
auth_type = scram-sha-256     ; match the Postgres backend's pg_hba auth
; The prepared-statement decision — pick ONE:
max_prepared_statements = 0   ; (a) untracked: TaskQ must disable its client cache (set TASKQ_PG_IS_POOLED=true)
; max_prepared_statements = 200 ; (b) tracked (PgBouncer >= 1.21): asyncpg's cache survives remaps, no TaskQ declaration needed
default_pool_size = 20        ; the saturation point: many client conns, few server conns

What TaskQ configures automatically once TASKQ_PG_IS_POOLED=true is set:

Every TaskQ-built pool passes statement_cache_size=0 and max_cached_statement_lifetime=0 (overriding TASKQ_STATEMENT_CACHE_SIZE / TASKQ_MAX_CACHED_STATEMENT_LIFETIME), so a prepared statement can never cross a server-connection remap. This is not optional tuning: with an untracked pooler, two clients alternately executing their same-named cached statement take SQLSTATE 26000 ("prepared statement does not exist") or 42P05 ("already exists") as soon as they share a server connection — the pinned red leg proves it deterministically under saturation.
The worker treats the pooler-remap statement errors (26000, 42P05) as transient instead of loud.
Only pools TaskQ builds are touched; bring-your-own pools (WorkerConnections factories) must pass the same create_pool kwargs themselves.

What the operator must set: the DSN split above, pool_mode=transaction, the auth type matching the backend, and — if not setting TASKQ_PG_IS_POOLED=true — max_prepared_statements high enough for the fleet's statement variety. Either knob works; exactly one is required.

Measured on the pinned topology (single-job enqueue, sequential): the pooler's per-statement overhead is roughly +0.5–1 ms p50 against the direct DSN (~3–4 ms vs ~2.5–3.5 ms on localhost) — the parse/delegate hop, not a per-query multiplexer; at real RTTs the pooler's connection compression dominates.

Starting configurations¶

Every value below is a starting point, not a recommendation: the right numbers come from your actors' payload sizes, durations, and arrival rates. The derivation is ops.md §4: Sizing; the connection columns here are computed with its fleet formula (direct = M × (dispatcher_pool_size + heartbeat_pool_size + 2) + 2, pooled = M × int(max_concurrency × 1.5), steady state, no per-slot transaction pool); recompute before you scale, and keep the rolling-deploy peak (roughly 2× steady, because orchestrators start new pods before draining old ones) inside Postgres max_connections together with your application's own pools.

Profile	Workers	`TASKQ_MAX_CONCURRENCY`	PG connections (steady)	Rolling-deploy peak	Watch
Small (dev, small managed SKU)	2	4	34 (22 direct + 12 pooled)	~68	Fits a 100-connection Postgres beside a small app pool. On a ~50-connection SKU, drop to 1 worker or lower concurrency further; two default workers (44-48 connections) already exceed it; see the small-SKU note in ops.md §4.
Medium	5	8 (default)	112 (52 direct + 60 pooled)	~224	Crosses TaskQ's 80-connection `pgbouncer_recommended` threshold (logged at worker startup): route `TASKQ_PG_DSN_POOLED` through PgBouncer.
Large	10	16	342 (102 direct + 240 pooled)	~684	The worked example in ops.md §4. PgBouncer on the pooled DSN is mandatory in practice; the direct DSN must stay on session mode (LISTEN/NOTIFY, advisory locks).

Everything not listed stays at its shipped default: pool sizes, heartbeat, sweeps, and the retention family are deliberately non-silent defaults (terminal jobs prune to jobs_archive daily; nothing accumulates forever). Per-slot transactional actors (a LOOP-scope asyncpg.Connection registered in DI) add max_concurrency + 1 direct connections per worker; budget that path with ops.md §4, not this table.

Production Example `.env`¶

TASKQ_PG_DSN=postgresql://taskq:secret@postgres.internal:5432/taskq
TASKQ_PG_DSN_DIRECT=postgresql://taskq:secret@postgres.internal:5432/taskq
TASKQ_PG_DSN_POOLED=postgresql://taskq:secret@pgbouncer.internal:5432/taskq
TASKQ_REDIS_URL=redis://redis.internal:6379/0
TASKQ_SCHEMA_NAME=taskq
TASKQ_ENVIRONMENT=production
TASKQ_MAX_CONCURRENCY=16
TASKQ_QUEUES=default,priority
TASKQ_LOG_FORMAT=json
TASKQ_LOG_LEVEL=INFO
TASKQ_OTEL_ENABLED=true
TASKQ_HEALTH_SOCKET_PATH=/run/taskq/health.sock
TASKQ_TERMINATION_GRACE_PERIOD=120
TASKQ_CANCELLATION_GRACE_PERIOD=60
TASKQ_CLEANUP_GRACE_PERIOD=20
TASKQ_LOCK_LEASE=90
TASKQ_HEARTBEAT_INTERVAL=10

These values satisfy all cross-field constraints: - lock_lease (90) >= cascade floor (58): 90 >= 58 ✓ - cancellation_grace (60) + cleanup_grace (20) < termination_grace (120) − 5: 80 < 115 ✓ - cancellation_grace (60) + cleanup_grace (20) < lock_lease (90): 80 < 90 ✓

TASKQ_ENVIRONMENT=production in this example is the deployment label; it gates the unauthenticated-admin warning, not file loading. Set ENV=production to load .env.production and .env.production.local.

Extending Settings¶

Subclass WorkerSettings to add application-specific config alongside TaskQ settings:

from taskq.settings import WorkerSettings
from dotenvmodel import Field


class AppSettings(WorkerSettings):
    stripe_api_key: str = Field(description="Stripe secret key")
    sentry_dsn: str | None = Field(default=None)

Load with AppSettings.load(): it forwards dotenvmodel's full parameter surface (env, override, env_dir, read_dotfiles, read_environ, load_local). All TASKQ_* validation constraints still apply. Additional fields follow the same dotenvmodel env-var resolution and .env cascade. String field defaults interpolate ${VAR} references at load time (an unset reference resolves to "" rather than keeping the literal ${...} text).