Package Overview

TaskQ public API. The taskq package re-exports the primary types used by application code: the @actor decorator, JobsClient / TaskQ, JobHandle, exceptions, RetryPolicy, cron scheduling, and batch helpers.

from taskq import actor, TaskQ, JobHandle, JobFailed, RetryPolicy
from taskq import cron, ScheduleHandle
from taskq.context import JobContext
from taskq.di import ProviderRegistry, Scope

taskq

TaskQ: async-native, Postgres-backed background job library.

Canonical imports:

from taskq import actor, TaskQ, JobHandle, JobFailed, RetryPolicy
from taskq import cron, ScheduleHandle
from taskq.context import JobContext
from taskq.di import ProviderRegistry, Scope

TERMINAL_STATUSES module-attribute

TERMINAL_STATUSES: frozenset[JobStatus] = frozenset(
    {
        "succeeded",
        "failed",
        "cancelled",
        "crashed",
        "abandoned",
    }
)

IdempotencyKey module-attribute

IdempotencyKey = NewType('IdempotencyKey', str)

Distinguishes idempotency keys from identity keys at call sites.

IdentityKey module-attribute

IdentityKey = NewType('IdentityKey', str)

Distinguishes identity keys from idempotency keys at call sites.

JobId module-attribute

JobId = NewType('JobId', UUID)

Opaque job identifier — prevents UUID mixups across the API.

QueueName module-attribute

QueueName = Annotated[
    str, AfterValidator(_validate_queue_name)
]

Validator alias for queue names — accepts plain str literals.

Why Annotated and not NewType: every studied vendor (river, dramatiq, arq, procrastinate) uses raw str + a separate validator for queue names; no nominal type because no other str field at any call site could be confused with queue. Annotated gives runtime validation in Pydantic models without forcing every caller to wrap literals in QueueName("default").

__all__ module-attribute

__all__ = [
    "TERMINAL_STATUSES",
    "AbortBatchAfter",
    "ActorConfigDriftError",
    "ActorConfigDriftList",
    "ActorConfigRow",
    "ActorDeregistrationError",
    "ActorFn",
    "ActorFnWithCtx",
    "ActorHandler",
    "ActorHasActiveJobsError",
    "ActorHasEnabledSchedulesError",
    "ActorNotFoundError",
    "ActorRef",
    "ActorsClient",
    "Backend",
    "BackpressureError",
    "BatchAbortedError",
    "BatchCompletionStatus",
    "BatchCounts",
    "BatchFailurePolicy",
    "BatchFilter",
    "BatchHandle",
    "BatchRow",
    "BatchSummary",
    "BulkCancelResult",
    "CancelPhase",
    "CancelResult",
    "Clock",
    "ConnFactory",
    "CronScheduleSpec",
    "DIError",
    "DependencyCycle",
    "DeregisterResult",
    "DstStrategy",
    "EmptyBatchError",
    "EmptyFilterError",
    "EnqueueItem",
    "ErrorReporter",
    "EventRow",
    "Fail",
    "FakeClock",
    "IdempotencyKey",
    "IdentityKey",
    "IllegalStateTransition",
    "JobContext",
    "JobEvent",
    "JobFailed",
    "JobFilter",
    "JobHandle",
    "JobId",
    "JobPage",
    "JobRetryState",
    "JobRow",
    "JobSortField",
    "JobStatus",
    "JobsClient",
    "MaxPendingExceededError",
    "MissingProvider",
    "NullErrorReporter",
    "OIDCSettings",
    "OnSuccess",
    "PartialBatchError",
    "PayloadValidationError",
    "PgCredential",
    "PgCredentialProvider",
    "PoolFactory",
    "ProgressEvent",
    "ProgressTooLarge",
    "QueueMode",
    "QueueName",
    "RateLimitBackend",
    "RedisCredential",
    "RedisCredentialProvider",
    "RedisFactory",
    "ReservationUnavailable",
    "ResultTooLarge",
    "ResultUnavailable",
    "Retry",
    "RetryAfter",
    "RetryClassifier",
    "RetryClassifierHook",
    "RetryDecision",
    "RetryKind",
    "RetryOverride",
    "RetryPolicy",
    "SAMLSettings",
    "ScheduleHandle",
    "ScheduleRecord",
    "SchemaNotMigratedError",
    "ScopeViolation",
    "ScopedIdempotencyMigrationPendingError",
    "SingletonCollisionError",
    "Snooze",
    "SubEnqueueError",
    "SubJobEnqueuer",
    "SystemClock",
    "TaskQ",
    "TaskQError",
    "TaskQSettings",
    "WorkerConnections",
    "WorkerOwnershipMismatch",
    "WorkerSettings",
    "__version__",
    "actor",
    "apply_batch_terminal_outcome",
    "cron",
    "enrich_pg_dsn",
    "ensure_sslmode_require",
    "make_dedicated_conn_factory",
    "make_pg_pool_factory",
    "make_redis_client_factory",
    "register_cron",
    "validate_actor_payload",
    "wait_for_batch",
]

__version__ module-attribute

__version__ = importlib.metadata.version('taskq-py')

ActorFn

ActorFn = Callable[[P_], Awaitable[R_]]

Actor handler that takes only a payload.

The dispatcher injects nothing beyond the validated payload model. Use this shape for actors that don't need cancellation cooperation, attempt counters, or other context fields.

ActorFnWithCtx

ActorFnWithCtx = Callable[
    [P_, JobContext[P_]], Awaitable[R_]
]

Actor handler that takes a payload and a typed :class:JobContext.

Declare ctx: JobContext[YourPayload] as the second parameter to opt into context injection. The dispatcher constructs the context per attempt, populates it with the validated payload and a fresh :class:asyncio.Event for cooperative cancellation, then passes it to the handler. Handlers that don't declare ctx skip this work.

JobStatus

JobStatus = Literal[
    "pending",
    "scheduled",
    "running",
    "succeeded",
    "failed",
    "cancelled",
    "crashed",
    "abandoned",
]

DstStrategy

DstStrategy = Literal['skip', 'firstof', 'allof']

QueueMode

QueueMode = Literal['strict_fifo', 'round_robin']

RateLimitBackend

RateLimitBackend = Literal['redis', 'postgres', 'memory']

RetryKind

RetryKind = Literal[
    "transient", "indefinite", "non_retryable"
]

Closed set of retry tiers.

Why Literal and not an Enum: serialization round-trips through model_dump(mode="json") produce plain strings without use_enum_values configuration; pyright exhaustive matching works identically for either; no .value access required at call sites.

ConnFactory

ConnFactory = Callable[[], Awaitable[Connection]]

PoolFactory

PoolFactory = Callable[[], Awaitable[Pool]]

RedisFactory

RedisFactory = Callable[[], Awaitable[Redis]]

OnSuccess

OnSuccess = Callable[
    [JobRow, object], Awaitable[None] | None
]

Hook fired when a job succeeds. Receives (job_row, result).

Why object for the result type (not Any or generic R): the hook is dispatched from the consumer loop, which erases the actor's return type to object. This mirrors the non-generic :data:OnRetryExhausted at the same payload-erasure boundary. Hooks that need a typed result re-validate via the actor's result_adapter.

RetryClassifierHook

RetryClassifierHook = Callable[
    [BaseException, int], RetryOverride | None
]

Optional per-actor hook for exception-instance-level retry classification.

non_retryable_exceptions and the built-in :class:PayloadValidationError check classify by exception type alone. Some integrations need finer granularity — a single exception type (e.g. an HTTP client's status-code error) that should retry indefinitely on a 429, fail immediately on a 404, and use a bounded transient budget on a 5xx, or a server-provided Retry-After value that should drive the actual backoff delay. Register one via @actor(retry_classifier=...).

Invoked with (exception, attempt) for every exception that survives the non_retryable_exceptions/PayloadValidationError checks. Return None to fall back to the actor's static RetryPolicy unchanged, or a :class:RetryOverride to refine kind and/or delay for this specific occurrence. Exceptions raised by the hook itself are caught and logged by :func:decide_after_failure; classification falls back to the static policy in that case — a broken hook can never crash the retry pipeline.

RetryDecision

RetryDecision = Retry | Fail

ActorHandler

Bases: Protocol

Most-general actor signature: payload first, then ctx and/or DI deps.

Pyright infers P_ from the first positional parameter (the payload model) and R_ from the awaited return type, regardless of how many additional ctx / DI parameters the handler declares. This lets the :func:actor decorator preserve full payload-and-result inference for FastAPI-style handlers like::

async def my_actor(
    payload: OrderPayload,
    ctx: JobContext[OrderPayload],
    *,
    db: DbSession,
    http: HttpClient,
) -> OrderResult: ...

Without ActorHandler, callers would have to choose between :data:ActorFn (payload only) and :data:ActorFnWithCtx (payload + ctx), neither of which describes the DI case.

__call__ async

__call__(
    payload: P_, /, *args: object, **kwargs: object
) -> R_

Source code in src/taskq/actor.py

async def __call__(self, payload: P_, /, *args: object, **kwargs: object) -> R_: ...

ActorRef

ActorRef(
    *,
    name: str,
    queue: str,
    fn: Callable[..., object],
    is_sync: bool = False,
    wants_ctx: bool,
    dependencies: dict[str, type[object]],
    payload_type: type[P],
    result_adapter: TypeAdapter[R],
    retry: RetryPolicy,
    result_ttl: timedelta | None,
    singleton: bool = False,
    max_concurrent: int | None = None,
    max_pending: int | None = None,
    metadata: dict[str, object] | None = None,
    unique_for: timedelta | None = None,
    unique_states: tuple[JobStatus, ...] = (
        "pending",
        "scheduled",
        "running",
    ),
    start_to_close: timedelta | None = None,
    rate_limits: list[
        str
        | KeyedRateLimitRef
        | TokenBucket
        | SlidingWindow
    ]
    | None = None,
    reservations: list[
        str | KeyedReservationRef | ConcurrencyReservation
    ]
    | None = None,
    non_retryable_exceptions: tuple[
        type[BaseException], ...
    ] = (),
    retry_classifier: RetryClassifierHook | None = None,
    on_retry_exhausted: OnRetryExhausted | None = None,
    on_retry_exhausted_timeout: float = 3.0,
    on_success: OnSuccess | None = None,
    on_success_timeout: float = 3.0,
    priority: int = 0,
)

Typed reference to a registered actor.

Created by the :func:actor decorator. Not callable directly via the queue path — enqueue jobs by passing this ref to :meth:JobsClient.enqueue(ref, payload) <taskq.client.JobsClient.enqueue>. Direct in-process invocation (await my_actor(payload, ...)) is available for tests and simulators.

The two type parameters carry the actor's payload and result types end-to-end:

• payload_type re-validates raw dispatch-time payloads back to P (via :meth:pydantic.BaseModel.model_validate).
• result_adapter round-trips actor returns through the JSONB result column (dump_python(mode="json") on the worker side, validate_python on the client side).

Attributes:

Name	Type	Description
max_concurrent		Fleet-wide concurrency cap for this actor. None means unbounded — matches the actor_config.max_concurrent IS NULL semantics in the dispatch CTE. Allowed values are None or int >= 0. A value of 0 means no jobs may run for this actor (useful for emergency drain scenarios). max_concurrent may transiently exceed configured value by up to (num_active_producers - 1) * max_concurrent per actor under heavy contention (or (num_producers - 1) * limit_n when limit_n < max_concurrent). For strict correctness, use ConcurrencyReservation.
metadata		Arbitrary key-value metadata stored in actor_config.metadata (jsonb NOT NULL). Must be a plain dict[str, object] — mapping proxies and frozendicts are rejected at decoration time to avoid surprises at JSONB serialization time.

Both are stored as instance fields rather than class metadata so pyright can infer P and R from a constructor call without relying on phantom-type tricks.

wants_ctx records whether the handler declared a :class:JobContext parameter; the dispatcher passes ctx only when set. dependencies maps each DI parameter name to its annotated type — the worker's DI pass resolves these at dispatch time and passes them as keyword arguments to the handler. The DI resolver itself is an erasure boundary (see the resolver operates on the registered provider graph at runtime): user-declared annotations on DI parameters are captured here as type[object] because the resolver operates on the registered provider graph at runtime.

Source code in src/taskq/actor.py

def __init__(
    self,
    *,
    name: str,
    queue: str,
    fn: Callable[..., object],
    is_sync: bool = False,
    wants_ctx: bool,
    dependencies: dict[str, type[object]],
    payload_type: type[P],
    result_adapter: TypeAdapter[R],
    retry: RetryPolicy,
    result_ttl: timedelta | None,
    singleton: bool = False,
    max_concurrent: int | None = None,
    max_pending: int | None = None,
    metadata: dict[str, object] | None = None,
    unique_for: timedelta | None = None,
    unique_states: tuple[JobStatus, ...] = ("pending", "scheduled", "running"),
    start_to_close: timedelta | None = None,
    rate_limits: list[str | KeyedRateLimitRef | TokenBucket | SlidingWindow] | None = None,
    reservations: list[str | KeyedReservationRef | ConcurrencyReservation] | None = None,
    non_retryable_exceptions: tuple[type[BaseException], ...] = (),
    retry_classifier: RetryClassifierHook | None = None,
    on_retry_exhausted: OnRetryExhausted | None = None,
    on_retry_exhausted_timeout: float = 3.0,
    on_success: OnSuccess | None = None,
    on_success_timeout: float = 3.0,
    priority: int = 0,
) -> None:
    self.name = name
    # A malformed declared queue strands every job the actor ever
    # enqueues on a queue no worker drains; fail at decoration time
    # (import time in the common case) instead. Same validator the
    # enqueue path runs — see client._args.build_enqueue_args.
    _validate_queue_name(queue)
    self.queue = queue
    self.is_sync = is_sync
    self.wants_ctx = wants_ctx
    self.dependencies = dependencies
    self.payload_type = payload_type
    self.result_adapter = result_adapter
    self.retry = retry
    self.result_ttl = result_ttl
    self.singleton = singleton
    self.max_concurrent = max_concurrent
    self.max_pending = max_pending
    self.metadata = {} if metadata is None else metadata
    self.unique_for = unique_for
    self.unique_states = unique_states
    self.start_to_close = start_to_close
    self.rate_limits = [] if rate_limits is None else rate_limits
    self.reservations = [] if reservations is None else reservations
    self.non_retryable_exceptions = non_retryable_exceptions
    self.retry_classifier = retry_classifier
    self.on_retry_exhausted = on_retry_exhausted
    self.on_retry_exhausted_timeout = on_retry_exhausted_timeout
    self.on_success = on_success
    self.on_success_timeout = on_success_timeout
    self.priority = priority
    # Single storage slot. Call shape varies by handler — the
    # dispatcher (or :meth:`__call__`) routes based on
    # :attr:`wants_ctx`, :attr:`dependencies`, and :attr:`is_sync`.
    self._fn: Callable[..., object] = fn

__slots__ class-attribute instance-attribute

__slots__ = (
    "_fn",
    "dependencies",
    "is_sync",
    "max_concurrent",
    "max_pending",
    "metadata",
    "name",
    "non_retryable_exceptions",
    "on_retry_exhausted",
    "on_retry_exhausted_timeout",
    "on_success",
    "on_success_timeout",
    "payload_type",
    "priority",
    "queue",
    "rate_limits",
    "reservations",
    "result_adapter",
    "result_ttl",
    "retry",
    "retry_classifier",
    "singleton",
    "start_to_close",
    "unique_for",
    "unique_states",
    "wants_ctx",
)

name instance-attribute

name = name

queue instance-attribute

queue = queue

is_sync instance-attribute

is_sync = is_sync

wants_ctx instance-attribute

wants_ctx = wants_ctx

dependencies instance-attribute

dependencies = dependencies

payload_type instance-attribute

payload_type = payload_type

result_adapter instance-attribute

result_adapter = result_adapter

retry instance-attribute

retry = retry

result_ttl instance-attribute

result_ttl = result_ttl

singleton instance-attribute

singleton = singleton

max_concurrent instance-attribute

max_concurrent = max_concurrent

max_pending instance-attribute

max_pending = max_pending

metadata instance-attribute

metadata = {} if metadata is None else metadata

unique_for instance-attribute

unique_for = unique_for

unique_states instance-attribute

unique_states = unique_states

start_to_close instance-attribute

start_to_close = start_to_close

rate_limits instance-attribute

rate_limits = [] if rate_limits is None else rate_limits

reservations instance-attribute

reservations = [] if reservations is None else reservations

non_retryable_exceptions instance-attribute

non_retryable_exceptions = non_retryable_exceptions

retry_classifier instance-attribute

retry_classifier = retry_classifier

on_retry_exhausted instance-attribute

on_retry_exhausted = on_retry_exhausted

on_retry_exhausted_timeout instance-attribute

on_retry_exhausted_timeout = on_retry_exhausted_timeout

on_success instance-attribute

on_success = on_success

on_success_timeout instance-attribute

on_success_timeout = on_success_timeout

priority instance-attribute

priority = priority

fn property

fn: Callable[..., object]

Return the underlying handler.

May be sync (def) or async (async def). Direct invocation through :meth:__call__ is the safer path because it handles both shapes and enforces the declared signature.

__call__ async

__call__(payload: P, /, **deps: object) -> R

__call__(
    payload: P, ctx: JobContext[P], /, **deps: object
) -> R

__call__(
    payload: P,
    ctx: JobContext[P] | None = None,
    /,
    **deps: object,
) -> R

Direct invocation — bypasses enqueue, runs the handler in-process.

Pass a :class:JobContext only when the registered handler declared one (when :attr:wants_ctx is True); calling with a context against a no-ctx handler raises :class:TypeError, and calling without a context against a ctx handler also raises :class:TypeError.

deps mirrors the keyword arguments the worker's dependency-injection pass supplies in production. Tests may pass them explicitly; the call site is responsible for matching the names recorded in :attr:dependencies. Missing dependencies surface as a runtime TypeError from Python's argument binding.

Production callers go through :meth:JobsClient.enqueue.

Source code in src/taskq/actor.py

async def __call__(
    self,
    payload: P,
    ctx: "JobContext[P] | None" = None,
    /,
    **deps: object,
) -> R:
    """Direct invocation — bypasses enqueue, runs the handler in-process.

    Pass a :class:`JobContext` only when the registered handler
    declared one (when :attr:`wants_ctx` is ``True``); calling with
    a context against a no-ctx handler raises :class:`TypeError`,
    and calling without a context against a ctx handler also
    raises :class:`TypeError`.

    ``deps`` mirrors the keyword arguments the worker's
    dependency-injection pass supplies in production. Tests may
    pass them explicitly; the call site is responsible for
    matching the names recorded in :attr:`dependencies`. Missing
    dependencies surface as a runtime ``TypeError`` from Python's
    argument binding.

    Production callers go through :meth:`JobsClient.enqueue`.
    """
    if self.is_sync:
        actor_kwargs: dict[str, object] = {"payload": payload, **deps}
        if self.wants_ctx:
            if ctx is None:
                raise TypeError(
                    f"actor {self.name!r} declares 'ctx: JobContext'; "
                    "supply a context to direct invocation"
                )
            actor_kwargs["ctx"] = ctx
        elif ctx is not None:
            raise TypeError(
                f"actor {self.name!r} does not declare a context parameter; "
                "call it with payload only"
            )
        return await asyncio.to_thread(self._fn, **actor_kwargs)  # type: ignore[return-value]  # Why: asyncio.to_thread erases the return type to Any; the caller type-narrows through ActorRef[R].
    if self.wants_ctx:
        if ctx is None:
            raise TypeError(
                f"actor {self.name!r} declares 'ctx: JobContext'; "
                "supply a context to direct invocation"
            )
        return await self._fn(payload, ctx, **deps)  # type: ignore[return-value]  # Why: _fn is typed Callable[..., object] (sync or async); caller narrows through ActorRef[R].
    if ctx is not None:
        raise TypeError(
            f"actor {self.name!r} does not declare a context parameter; "
            "call it with payload only"
        )
    return await self._fn(payload, **deps)  # type: ignore[return-value]  # Why: _fn is typed Callable[..., object] (sync or async); caller narrows through ActorRef[R].

ActorConfigRow dataclass

ActorConfigRow(
    actor: str,
    max_concurrent: int | None,
    max_pending: int | None,
    queue: str,
    result_ttl: float | None,
    metadata: dict[str, object],
    updated_at: str,
)

Snapshot of one {schema}.actor_config row.

actor instance-attribute

actor: str

max_concurrent instance-attribute

max_concurrent: int | None

max_pending instance-attribute

max_pending: int | None

queue instance-attribute

queue: str

result_ttl instance-attribute

result_ttl: float | None

metadata instance-attribute

metadata: dict[str, object]

updated_at instance-attribute

updated_at: str

DeregisterResult dataclass

DeregisterResult(
    actor: str,
    queue: str,
    actor_config_deleted: bool,
    schedules_disabled: int,
    jobs_cancelled: int,
    terminal_jobs_remaining: int,
    queue_purged: bool,
)

Outcome of a deregister_actor call.

actor_config_deleted is always True — if the row is not found, deregister_actor raises :class:ActorNotFoundError instead of returning a result with False. The field is retained for API contract clarity and consumer assertions.

actor instance-attribute

actor: str

queue instance-attribute

queue: str

actor_config_deleted instance-attribute

actor_config_deleted: bool

schedules_disabled instance-attribute

schedules_disabled: int

jobs_cancelled instance-attribute

jobs_cancelled: int

terminal_jobs_remaining instance-attribute

terminal_jobs_remaining: int

queue_purged instance-attribute

queue_purged: bool

PgCredential dataclass

PgCredential(password: str, username: str | None = None)

A Postgres credential issued by a rotating-credential provider.

password is always required (a token or dynamic password). username, when set, overrides the DSN's userinfo user - needed by providers that issue a fresh username alongside the password (e.g. Vault dynamic DB creds). When None, the DSN's existing user is preserved.

password instance-attribute

password: str

username class-attribute instance-attribute

username: str | None = None

PgCredentialProvider

Bases: Protocol

Provides rotating Postgres credentials on demand.

Implementations fetch a fresh token / dynamic username+password each call. Called by :func:make_pg_pool_factory / :func:make_dedicated_conn_factory once at pool / connection construction (to resolve user= and fail fast), and then again for every physical connection asyncpg opens thereafter - not on each acquire(), which hands back an already-authenticated connection from the pool. Implementations are expected to cache and only hit the issuing service when the cached credential is near expiry.

get_pg_credential async

get_pg_credential() -> PgCredential

Return a fresh :class:PgCredential.

Source code in src/taskq/auth.py

async def get_pg_credential(self) -> PgCredential:
    """Return a fresh :class:`PgCredential`."""
    ...

RedisCredential dataclass

RedisCredential(username: str, password: str)

A Redis credential issued by a rotating-credential provider.

username instance-attribute

username: str

password instance-attribute

password: str

RedisCredentialProvider

Bases: Protocol

Provides rotating Redis credentials on demand.

Implementations fetch a fresh (username, token/password) each call. Called by :func:make_redis_client_factory on every reconnect via the redis-py CredentialProvider adapter.

get_redis_credential async

get_redis_credential() -> RedisCredential

Return a fresh :class:RedisCredential.

Source code in src/taskq/auth.py

async def get_redis_credential(self) -> RedisCredential:
    """Return a fresh :class:`RedisCredential`."""
    ...

Backend

Bases: Protocol

Contract that both PostgresBackend and InMemoryBackend satisfy.

46 async methods plus two sync methods (subscribe_wake and subscribe_cancel_wake) (48 methods total) covering enqueue, dispatch, heartbeat, terminal writes, attempt history, cancel signals, scheduling / sweeps, read, NOTIFY hook, schedule CRUD, and batch operations. Method order grouped for review-grep ergonomics.

Why monomorphic (no Generic[P, R]): the backend is the DB adapter boundary. Payloads are stored as dict[str, object] (the JSONB payload column) regardless of the actor's typed payload model. Generic parameters here would propagate P and R into every method (dispatch_batch, mark_succeeded, etc.) with no safety benefit at the storage layer. The worker consumer reconstructs the typed JobContext[P] at dispatch time using ActorRef.payload_type.model_validate(row.payload).

BACKEND_PROTOCOL_VERSION class-attribute

BACKEND_PROTOCOL_VERSION: int

supports_transactional_simulation class-attribute

supports_transactional_simulation: bool = False

Whether this backend simulates transactional sub-enqueue via a buffer (True) or relies on real database transactions (False).

PostgresBackend returns False — its real PG transaction provides the atomicity guarantee directly: sub-job INSERTs run on the open LOOP-scope connection and are rolled back along with the parent's writes if the actor raises.

InMemoryBackend returns True — it has no real transaction concept, so SubJobEnqueuer buffers EnqueueArgs and flushes on actor success / discards on failure. A third-party Backend implementation that wants transactional simulation in tests can opt in by overriding this to True.

enqueue async

enqueue(args: EnqueueArgs) -> JobRow

Source code in src/taskq/backend/_protocol.py

async def enqueue(self, args: EnqueueArgs) -> JobRow: ...

enqueue_batch async

enqueue_batch(
    args_list: list[EnqueueArgs],
    *,
    connection: Connection | None = None,
) -> list[JobRow]

Insert multiple jobs in a single batched operation.

All items in args_list must be validated before calling this method — the backend does not re-validate payloads. The list must be non-empty and contain at most 1000 items (enforced by the client layer).

Returns one :class:JobRow per item in args_list, in the same order. For idempotency-key collisions the existing row is returned; its id will differ from the requested args.id.

Source code in src/taskq/backend/_protocol.py

async def enqueue_batch(
    self,
    args_list: list[EnqueueArgs],
    *,
    connection: "asyncpg.Connection | None" = None,
) -> list[JobRow]:
    """Insert multiple jobs in a single batched operation.

    All items in *args_list* must be validated before calling this
    method — the backend does not re-validate payloads.  The list
    must be non-empty and contain at most 1000 items (enforced by the
    client layer).

    Returns one :class:`JobRow` per item in *args_list*, in the same
    order.  For idempotency-key collisions the existing row is
    returned; its ``id`` will differ from the requested ``args.id``.
    """
    ...

enqueue_batch_fast async

enqueue_batch_fast(
    args_list: list[EnqueueArgs],
    *,
    connection: Connection | None = None,
) -> int

Insert multiple jobs via the COPY FROM protocol for maximum throughput.

COPY cannot evaluate expressions or handle conflicts, so the write is two statements inside one transaction: a bare COPY of the domain-insensitive columns, then a corrective UPDATE (enqueue_batch_fast_fixup) that stamps/decides the clock-sensitive ones — status, scheduled_at, schedule_to_close, result_expires_at — from the database clock (clock_timestamp()); created_at takes its DDL default (now()). Nothing is observable half-fixed: both statements commit or abort together.

Consequences of the COPY-no-conflicts shape:

• scheduled_at=None means immediate — the fixup's server-side CASE stamps it and decides pending/scheduled (the same single-arbiter contract as :meth:enqueue/:meth:enqueue_batch).
• schedule_to_close_interval/result_ttl are anchored to the server clock at ENQUEUE time by the fixup — a future-scheduled item with a short interval can therefore fail DeadlineExceeded before it is ever dispatched.
• A duplicate idempotency_key — within the batch or already stored — violates the unique index and aborts the ENTIRE batch (all-or-nothing atomicity; nothing is written).

Returns the count of rows written. On success this is exactly len(args_list) — this path never deduplicates, so the count never includes pre-existing rows. The in-memory mirror implements the same contract: duplicates raise asyncpg.UniqueViolationError before any row is written, and the count is the number of items.

This is a performance-focused variant of :meth:enqueue_batch (which DOES deduplicate idempotency-key collisions via ON CONFLICT). Use for bulk import / backfill with 10K+ rows where collision handling is not needed. Max batch size is 50 000 (client-enforced).

Source code in src/taskq/backend/_protocol.py

async def enqueue_batch_fast(
    self,
    args_list: list[EnqueueArgs],
    *,
    connection: "asyncpg.Connection | None" = None,
) -> int:
    """Insert multiple jobs via the COPY FROM protocol for maximum throughput.

    COPY cannot evaluate expressions or handle conflicts, so the write
    is two statements inside one transaction: a bare COPY of the
    domain-insensitive columns, then a corrective UPDATE
    (``enqueue_batch_fast_fixup``) that stamps/decides the
    clock-sensitive ones — ``status``, ``scheduled_at``,
    ``schedule_to_close``, ``result_expires_at`` — from the database
    clock (``clock_timestamp()``); ``created_at`` takes its DDL
    default (``now()``).  Nothing is observable half-fixed: both
    statements commit or abort together.

    Consequences of the COPY-no-conflicts shape:

    - ``scheduled_at=None`` means immediate — the fixup's server-side
      CASE stamps it and decides ``pending``/``scheduled`` (the same
      single-arbiter contract as :meth:`enqueue`/:meth:`enqueue_batch`).
    - ``schedule_to_close_interval``/``result_ttl`` are anchored to the
      server clock at ENQUEUE time by the fixup — a future-scheduled
      item with a short interval can therefore fail DeadlineExceeded
      before it is ever dispatched.
    - A duplicate ``idempotency_key`` — within the batch or already
      stored — violates the unique index and aborts the ENTIRE batch
      (all-or-nothing atomicity; nothing is written).

    Returns the count of rows written.  On success this is exactly
    ``len(args_list)`` — this path never deduplicates, so the count
    never includes pre-existing rows.  The in-memory mirror implements
    the same contract: duplicates raise
    ``asyncpg.UniqueViolationError`` before any row is written, and
    the count is the number of items.

    This is a performance-focused variant of :meth:`enqueue_batch`
    (which DOES deduplicate idempotency-key collisions via ``ON
    CONFLICT``).  Use for bulk import / backfill with 10K+ rows where
    collision handling is not needed.  Max batch size is 50 000
    (client-enforced).
    """
    ...

enqueue_with_conn async

enqueue_with_conn(
    conn: Connection, args: EnqueueArgs
) -> JobRow

Enqueue a job using the supplied connection.

The connection MUST already be in an open transaction managed by the caller — this method does NOT issue BEGIN/COMMIT. The autonomous variant enqueue(args) acquires its own connection and opens a transaction internally.

Source code in src/taskq/backend/_protocol.py

async def enqueue_with_conn(
    self,
    conn: "asyncpg.Connection",
    args: EnqueueArgs,
) -> JobRow:
    """Enqueue a job using the supplied connection.

    The connection MUST already be in an open transaction managed by
    the caller — this method does NOT issue BEGIN/COMMIT. The
    autonomous variant ``enqueue(args)`` acquires its own connection
    and opens a transaction internally.
    """
    ...

dispatch_batch async

dispatch_batch(
    worker_id: UUID,
    queues: list[str],
    limit: int,
    lock_lease: timedelta,
) -> list[JobRow]

Source code in src/taskq/backend/_protocol.py

async def dispatch_batch(
    self,
    worker_id: UUID,
    queues: list[str],
    limit: int,
    lock_lease: timedelta,
) -> list[JobRow]: ...

heartbeat_jobs async

heartbeat_jobs(
    worker_id: UUID, lock_lease: timedelta
) -> int

Source code in src/taskq/backend/_protocol.py

async def heartbeat_jobs(
    self,
    worker_id: UUID,
    lock_lease: timedelta,
) -> int: ...

extend_reservation_leases async

extend_reservation_leases(
    worker_id: UUID, lock_lease: timedelta
) -> int

Source code in src/taskq/backend/_protocol.py

async def extend_reservation_leases(
    self,
    worker_id: UUID,
    lock_lease: timedelta,
) -> int: ...

mark_succeeded async

mark_succeeded(
    job_id: JobId,
    worker_id: UUID,
    result: dict[str, object] | None,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
    fallback_result_ttl: timedelta | None = None,
) -> bool

Mark a job succeeded, computing result_expires_at at completion.

Expiry resolution, first match wins: a non-NULL stored actor_config.result_ttl (operator-owned) applies; otherwise fallback_result_ttl — the worker-side @actor(result_ttl=...) literal, which the terminal-write SQL cannot see — applies; otherwise the row's existing result_expires_at is kept. The computed arms use clock_timestamp() — the wall-clock time the write executes, not the transaction start — so neither a long queue wait nor a long actor runtime can make a job complete already expired and have its result reaped immediately.

Source code in src/taskq/backend/_protocol.py

async def mark_succeeded(
    self,
    job_id: JobId,
    worker_id: UUID,
    result: dict[str, object] | None,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
    fallback_result_ttl: timedelta | None = None,
) -> bool:
    """Mark a job succeeded, computing ``result_expires_at`` at completion.

    Expiry resolution, first match wins: a non-NULL stored
    ``actor_config.result_ttl`` (operator-owned) applies; otherwise
    *fallback_result_ttl* — the worker-side ``@actor(result_ttl=...)``
    literal, which the terminal-write SQL cannot see — applies;
    otherwise the row's existing ``result_expires_at`` is kept. The
    computed arms use ``clock_timestamp()`` — the wall-clock time the
    write executes, not the transaction start — so neither a long
    queue wait nor a long actor runtime can make a job complete
    already expired and have its result reaped immediately.
    """
    ...

mark_succeeded_with_conn async

mark_succeeded_with_conn(
    conn: Connection,
    job_id: JobId,
    worker_id: UUID,
    result: dict[str, object] | None,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
    fallback_result_ttl: timedelta | None = None,
) -> bool

Mark a job succeeded using the supplied connection.

Used by the consumer when a LOOP-scope asyncpg.Connection is available so the success status update commits atomically with the actor's writes and sub-job INSERTs in the same transaction. The connection MUST already be in an open transaction; this method does NOT open or close one. The autonomous variant mark_succeeded(...) acquires its own connection.

fallback_result_ttl follows the same resolution rule as :meth:mark_succeeded.

Source code in src/taskq/backend/_protocol.py

async def mark_succeeded_with_conn(
    self,
    conn: "asyncpg.Connection",
    job_id: JobId,
    worker_id: UUID,
    result: dict[str, object] | None,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
    fallback_result_ttl: timedelta | None = None,
) -> bool:
    """Mark a job succeeded using the supplied connection.

    Used by the consumer when a LOOP-scope ``asyncpg.Connection`` is
    available so the success status update commits atomically with
    the actor's writes and sub-job INSERTs in the same transaction.
    The connection MUST already be in an open transaction; this
    method does NOT open or close one. The autonomous variant
    ``mark_succeeded(...)`` acquires its own connection.

    ``fallback_result_ttl`` follows the same resolution rule as
    :meth:`mark_succeeded`.
    """
    ...

mark_failed_or_retry async

mark_failed_or_retry(
    job_id: JobId,
    worker_id: UUID,
    error_info: ErrorInfo,
    retry_delay: timedelta | None,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
) -> JobRow

Mark a running job failed, or schedule a retry retry_delay later.

retry_delay=None is the terminal-fail arm (status='failed', the original error_info persisted). A non-None delay is applied by the backend's own clock, never the caller's: scheduled_at = now() + delay and the scheduled/pending status derive from the delay alone (zero → immediate). The same statement arbitrates the schedule_to_close deadline server-side — when clock_timestamp() + delay would land past the deadline, the row is failed with error_class='DeadlineExceeded' instead of retried — so app↔DB clock skew can neither void the retry backoff nor kill a job whose deadline has not actually passed.

Source code in src/taskq/backend/_protocol.py

async def mark_failed_or_retry(
    self,
    job_id: JobId,
    worker_id: UUID,
    error_info: ErrorInfo,
    retry_delay: timedelta | None,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
) -> JobRow:
    """Mark a running job failed, or schedule a retry *retry_delay* later.

    ``retry_delay=None`` is the terminal-fail arm (``status='failed'``,
    the original ``error_info`` persisted).  A non-None delay is applied
    by the backend's own clock, never the caller's: ``scheduled_at =
    now() + delay`` and the ``scheduled``/``pending`` status derive from
    the delay alone (zero → immediate).  The same statement arbitrates
    the ``schedule_to_close`` deadline server-side — when
    ``clock_timestamp() + delay`` would land past the deadline, the row
    is failed with ``error_class='DeadlineExceeded'`` instead of
    retried — so app↔DB clock skew can neither void the retry backoff
    nor kill a job whose deadline has not actually passed.
    """
    ...

mark_cancelled async

mark_cancelled(
    job_id: JobId,
    worker_id: UUID,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
) -> bool

Source code in src/taskq/backend/_protocol.py

async def mark_cancelled(
    self,
    job_id: JobId,
    worker_id: UUID,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
) -> bool: ...

write_cancel_escalation async

write_cancel_escalation(
    job_id: JobId, worker_id: UUID, phase: Literal[2]
) -> bool

Source code in src/taskq/backend/_protocol.py

async def write_cancel_escalation(
    self,
    job_id: JobId,
    worker_id: UUID,
    phase: Literal[2],
) -> bool: ...

mark_abandoned async

mark_abandoned(
    job_id: JobId,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
) -> bool

Source code in src/taskq/backend/_protocol.py

async def mark_abandoned(
    self,
    job_id: JobId,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
) -> bool: ...

mark_snoozed async

mark_snoozed(
    job_id: JobId,
    worker_id: UUID,
    delay: timedelta,
    *,
    metadata_update: dict[str, object] | None = None,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
    outcome: AttemptOutcome = "snoozed",
) -> Literal["scheduled", "failed", "noop"]

Source code in src/taskq/backend/_protocol.py

async def mark_snoozed(
    self,
    job_id: JobId,
    worker_id: UUID,
    delay: timedelta,
    *,
    metadata_update: dict[str, object] | None = None,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
    outcome: AttemptOutcome = "snoozed",
) -> Literal["scheduled", "failed", "noop"]: ...

mark_retry_after async

mark_retry_after(
    job_id: JobId,
    worker_id: UUID,
    delay: timedelta,
    *,
    consume_budget: bool = True,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
) -> Literal[
    "scheduled",
    "failed:DeadlineExceeded",
    "failed:MaxAttemptsExceeded",
    "noop",
]

Source code in src/taskq/backend/_protocol.py

async def mark_retry_after(
    self,
    job_id: JobId,
    worker_id: UUID,
    delay: timedelta,
    *,
    consume_budget: bool = True,
    progress_seq: int = 0,
    progress_state: dict[str, object] | None = None,
) -> Literal["scheduled", "failed:DeadlineExceeded", "failed:MaxAttemptsExceeded", "noop"]: ...

write_attempt async

write_attempt(attempt: AttemptRow) -> None

Source code in src/taskq/backend/_protocol.py

async def write_attempt(self, attempt: AttemptRow) -> None: ...

get_attempts async

get_attempts(job_id: JobId) -> list[AttemptRow]

Source code in src/taskq/backend/_protocol.py

async def get_attempts(self, job_id: JobId) -> list[AttemptRow]: ...

get_events async

get_events(job_id: JobId) -> list[EventRow]

Source code in src/taskq/backend/_protocol.py

async def get_events(self, job_id: JobId) -> list[EventRow]: ...

poll_reclaim_events async

poll_reclaim_events(
    after_id: int,
    limit: int = DEFAULT_RECLAIM_POLL_LIMIT,
    *,
    visibility_delay: timedelta | None = None,
) -> list[EventRow]

Return up to limit crash-reclaim events with event_id > after_id, ascending — the durable cursor behind TaskQ.watch_reclaims.

An event can be silently missed if a job_events writer transaction stays open longer than the visibility-delay margin between its INSERT and its COMMIT — ids are allocated at INSERT time but transactions commit out of order, so a late-committing lower-id row can land behind an already-advanced cursor. Rows are therefore held back by a trailing-watermark filter (visibility_delay; backend-configured default when None — see :data:taskq.constants.RECLAIM_EVENT_VISIBILITY_DELAY for the exact assumption and its violation modes, and PostgresBackend.check_reclaim_visibility_delay_risk for the diagnostic that makes a violation operator-visible).

Source code in src/taskq/backend/_protocol.py

async def poll_reclaim_events(
    self,
    after_id: int,
    limit: int = DEFAULT_RECLAIM_POLL_LIMIT,
    *,
    visibility_delay: timedelta | None = None,
) -> list[EventRow]:
    """Return up to *limit* crash-reclaim events with ``event_id >
    after_id``, ascending — the durable cursor behind
    ``TaskQ.watch_reclaims``.

    **An event can be silently missed if a ``job_events`` writer
    transaction stays open longer than the visibility-delay margin
    between its INSERT and its COMMIT** — ids are allocated at INSERT
    time but transactions commit out of order, so a late-committing
    lower-id row can land behind an already-advanced cursor.  Rows
    are therefore held back by a trailing-watermark filter
    (*visibility_delay*; backend-configured default when ``None`` —
    see :data:`taskq.constants.RECLAIM_EVENT_VISIBILITY_DELAY` for
    the exact assumption and its violation modes, and
    ``PostgresBackend.check_reclaim_visibility_delay_risk`` for the
    diagnostic that makes a violation operator-visible).
    """
    ...

write_cancel_request async

write_cancel_request(
    job_id: JobId, reason: str | None
) -> bool

Source code in src/taskq/backend/_protocol.py

async def write_cancel_request(
    self,
    job_id: JobId,
    reason: str | None,
) -> bool: ...

cancel_where async

cancel_where(
    filter: JobFilter, reason: str | None
) -> BulkCancelResult

Cancel all jobs matching filter in a set-based operation.

Pending/scheduled jobs → terminal 'cancelled'. Running jobs → cancel_phase=1 (cooperative cancel + NOTIFY).

The filter's limit, cursor, and order_by fields are ignored — this is a bulk write, not a paginated read.

Guardrail: the client layer (:meth:JobsClient.cancel_where) rejects empty filters (no predicates) with :class:EmptyFilterError. Backend implementations receive a filter that has already been validated. A direct backend call with JobFilter() renders WHERE TRUE and cancels the entire table — callers using the backend directly are responsible for validating the filter.

Returns a :class:BulkCancelResult with counts and affected IDs.

Source code in src/taskq/backend/_protocol.py

async def cancel_where(
    self,
    filter: JobFilter,
    reason: str | None,
) -> BulkCancelResult:
    """Cancel all jobs matching *filter* in a set-based operation.

    Pending/scheduled jobs → terminal 'cancelled'.
    Running jobs → cancel_phase=1 (cooperative cancel + NOTIFY).

    The filter's ``limit``, ``cursor``, and ``order_by`` fields are
    ignored — this is a bulk write, not a paginated read.

    **Guardrail:** the client layer (:meth:`JobsClient.cancel_where`)
    rejects empty filters (no predicates) with
    :class:`EmptyFilterError`. Backend implementations receive a
    filter that has already been validated. A direct backend call
    with ``JobFilter()`` renders ``WHERE TRUE`` and cancels the
    entire table — callers using the backend directly are
    responsible for validating the filter.

    Returns a :class:`BulkCancelResult` with counts and affected IDs.
    """
    ...

poll_cancel_flags async

poll_cancel_flags(worker_id: UUID) -> list[CancelFlag]

Source code in src/taskq/backend/_protocol.py

async def poll_cancel_flags(
    self,
    worker_id: UUID,
) -> list[CancelFlag]: ...

retry_job async

retry_job(job_id: JobId) -> bool

Reset a terminal job (failed/crashed/cancelled) to pending.

Returns True if the job was retried, False if it was not in a retryable state.

Source code in src/taskq/backend/_protocol.py

async def retry_job(self, job_id: JobId) -> bool:
    """Reset a terminal job (failed/crashed/cancelled) to pending.

    Returns ``True`` if the job was retried, ``False`` if it was not
    in a retryable state.
    """
    ...

scheduled_to_pending async

scheduled_to_pending() -> int

Promote scheduled jobs whose scheduled_at has passed.

The backend's own clock is the arbiter (PG evaluates scheduled_at <= clock_timestamp() server-side; InMemory compares against its injected Clock). Returns the count of promoted rows.

Source code in src/taskq/backend/_protocol.py

async def scheduled_to_pending(self) -> int:
    """Promote ``scheduled`` jobs whose ``scheduled_at`` has passed.

    The backend's own clock is the arbiter (PG evaluates
    ``scheduled_at <= clock_timestamp()`` server-side; InMemory
    compares against its injected Clock).  Returns the count of
    promoted rows.
    """
    ...

deadline_sweep async

deadline_sweep() -> int

Fail pending/scheduled jobs whose schedule_to_close has passed.

Transitions to failed with error_class='DeadlineExceeded', arbitrated by the backend's own clock. Returns the count of swept rows.

Source code in src/taskq/backend/_protocol.py

async def deadline_sweep(self) -> int:
    """Fail pending/scheduled jobs whose ``schedule_to_close`` has passed.

    Transitions to ``failed`` with ``error_class='DeadlineExceeded'``,
    arbitrated by the backend's own clock.  Returns the count of swept
    rows.
    """
    ...

reclaim_expired_locks async

reclaim_expired_locks(
    cancel_grace: timedelta, cleanup_grace: timedelta
) -> int

Reclaim running jobs whose lock has expired.

The expiry check is arbitrated by the backend's own clock; the grace parameters only widen the carve-out for jobs with an in-flight cancel request. Returns the count of reclaimed rows.

Source code in src/taskq/backend/_protocol.py

async def reclaim_expired_locks(
    self,
    cancel_grace: timedelta,
    cleanup_grace: timedelta,
) -> int:
    """Reclaim ``running`` jobs whose lock has expired.

    The expiry check is arbitrated by the backend's own clock; the
    grace parameters only widen the carve-out for jobs with an
    in-flight cancel request.  Returns the count of reclaimed rows.
    """
    ...

get async

get(job_id: JobId) -> JobRow | None

Source code in src/taskq/backend/_protocol.py

async def get(self, job_id: JobId) -> JobRow | None: ...

list_jobs async

list_jobs(filters: JobFilter) -> list[JobRow]

List jobs matching filters, returning at most filters.limit rows in keyset-pagination order.

filters.status accepts a single :data:JobStatus or a sequence of statuses; filters.active is a meta-filter for non-terminal (True) or terminal (False) statuses — 'active' here means 'not yet finished', not Celery's 'currently executing'. See :class:JobFilter for details.

Source code in src/taskq/backend/_protocol.py

async def list_jobs(self, filters: JobFilter) -> list[JobRow]:
    """List jobs matching *filters*, returning at most ``filters.limit``
    rows in keyset-pagination order.

    ``filters.status`` accepts a single :data:`JobStatus` or a
    sequence of statuses; ``filters.active`` is a meta-filter for
    non-terminal (``True``) or terminal (``False``) statuses —
    'active' here means 'not yet finished', not Celery's 'currently
    executing'.  See :class:`JobFilter` for details.
    """
    ...

count_pending_jobs async

count_pending_jobs(actors: list[str]) -> dict[str, int]

Return pending+scheduled job counts per actor.

Returns a dict mapping actor name to count. Only actors with at least one pending or scheduled job appear in the result. Actors not in the result have a count of zero. The actors list is used as an IN/ANY filter — pass all distinct actor names from a batch to fetch all counts in one round-trip.

Source code in src/taskq/backend/_protocol.py

async def count_pending_jobs(self, actors: list[str]) -> dict[str, int]:
    """Return pending+scheduled job counts per actor.

    Returns a dict mapping actor name to count.  Only actors with
    at least one pending or scheduled job appear in the result.
    Actors not in the result have a count of zero.  The ``actors``
    list is used as an ``IN``/``ANY`` filter — pass all distinct actor
    names from a batch to fetch all counts in one round-trip.
    """
    ...

count_active_jobs async

count_active_jobs(queues: list[str]) -> int

Count non-terminal jobs (pending, scheduled, running) in the given queues.

Returns the total count across all specified queues. Used by the drain monitor to detect when queues are empty. An empty queues list returns 0.

Source code in src/taskq/backend/_protocol.py

async def count_active_jobs(self, queues: list[str]) -> int:
    """Count non-terminal jobs (pending, scheduled, running) in the given queues.

    Returns the total count across all specified queues. Used by the
    drain monitor to detect when queues are empty. An empty queues
    list returns 0.
    """
    ...

get_actor_max_pending async

get_actor_max_pending() -> dict[str, int | None]

Return the stored actor_config.max_pending for every actor with a row.

Key present with an int value: the stored (operator-owned) limit. Key present with None: a row exists but the column is NULL (a cleared override). Key absent: no stored row. Client-side capacity resolution (:class:taskq.client._capacity.ActorCapacityCache) treats "absent" and "NULL" identically — both fall back to the @actor(...) literal; the distinction is preserved here only so observability callers can tell them apart.

This is the enqueue-path analog of the dispatch CTE's per-cycle actor_config join: one small whole-table read, consumed through a TTL-bounded cache so the hot path pays no per-enqueue query.

Source code in src/taskq/backend/_protocol.py

async def get_actor_max_pending(self) -> dict[str, int | None]:
    """Return the stored ``actor_config.max_pending`` for every actor
    with a row.

    Key present with an ``int`` value: the stored (operator-owned)
    limit. Key present with ``None``: a row exists but the column is
    NULL (a cleared override). Key absent: no stored row. Client-side
    capacity resolution
    (:class:`taskq.client._capacity.ActorCapacityCache`) treats
    "absent" and "NULL" identically — both fall back to the
    ``@actor(...)`` literal; the distinction is preserved here only
    so observability callers can tell them apart.

    This is the enqueue-path analog of the dispatch CTE's per-cycle
    ``actor_config`` join: one small whole-table read, consumed
    through a TTL-bounded cache so the hot path pays no per-enqueue
    query.
    """
    ...

subscribe_wake

subscribe_wake() -> AsyncContextManager[asyncio.Event]

Source code in src/taskq/backend/_protocol.py

def subscribe_wake(self) -> AsyncContextManager[asyncio.Event]: ...

subscribe_cancel_wake

subscribe_cancel_wake() -> AsyncContextManager[
    asyncio.Event
]

Return an async context manager yielding a fresh asyncio.Event that is set whenever a cancel NOTIFY arrives for any job.

The heartbeat loop uses this to interrupt its sleep immediately on cancel, rather than waiting for the next scheduled tick.

Source code in src/taskq/backend/_protocol.py

def subscribe_cancel_wake(self) -> AsyncContextManager[asyncio.Event]:
    """Return an async context manager yielding a fresh ``asyncio.Event``
    that is set whenever a cancel NOTIFY arrives for any job.

    The heartbeat loop uses this to interrupt its sleep immediately on
    cancel, rather than waiting for the next scheduled tick.
    """
    ...

create_schedule async

create_schedule(args: ScheduleCreateArgs) -> ScheduleRecord

Source code in src/taskq/backend/_protocol.py

async def create_schedule(self, args: ScheduleCreateArgs) -> ScheduleRecord: ...

list_schedules async

list_schedules(
    *, actor: str | None = None, enabled: bool | None = None
) -> list[ScheduleRecord]

Source code in src/taskq/backend/_protocol.py

async def list_schedules(
    self,
    *,
    actor: str | None = None,
    enabled: bool | None = None,
) -> list[ScheduleRecord]: ...

update_schedule async

update_schedule(
    schedule_id: UUID, args: ScheduleUpdateArgs
) -> ScheduleRecord

Source code in src/taskq/backend/_protocol.py

async def update_schedule(
    self,
    schedule_id: UUID,
    args: ScheduleUpdateArgs,
) -> ScheduleRecord: ...

delete_schedule async

delete_schedule(schedule_id: UUID) -> None

Source code in src/taskq/backend/_protocol.py

async def delete_schedule(self, schedule_id: UUID) -> None: ...

enqueue_batch_atomic async

enqueue_batch_atomic(
    items: Iterable[EnqueueArgs],
    *,
    batch_id: UUID,
    queue: str,
    batch_row: BatchRow | None,
    finalizer_args: EnqueueArgs | None,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
) -> list[JobRow]

Source code in src/taskq/backend/_protocol.py

async def enqueue_batch_atomic(
    self,
    items: Iterable[EnqueueArgs],
    *,
    batch_id: UUID,
    queue: str,
    batch_row: BatchRow | None,
    finalizer_args: EnqueueArgs | None,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
) -> list[JobRow]: ...

create_batch async

create_batch(
    batch_id: UUID,
    queue: str,
    expected_size: int,
    failure_threshold: int | None,
    finalizer_job_id: UUID | None,
    originating_actor: str | None,
    *,
    connection: Connection | None = None,
) -> None

Source code in src/taskq/backend/_protocol.py

async def create_batch(
    self,
    batch_id: UUID,
    queue: str,
    expected_size: int,
    failure_threshold: int | None,
    finalizer_job_id: UUID | None,
    originating_actor: str | None,
    *,
    connection: "asyncpg.Connection | None" = None,
) -> None: ...

increment_batch_failures async

increment_batch_failures(
    batch_id: UUID, *, connection: Connection | None = None
) -> tuple[int, int | None, int]

Source code in src/taskq/backend/_protocol.py

async def increment_batch_failures(
    self,
    batch_id: UUID,
    *,
    connection: "asyncpg.Connection | None" = None,
) -> tuple[int, int | None, int]: ...

reset_batch_failures async

reset_batch_failures(
    batch_id: UUID, *, connection: Connection | None = None
) -> int

Source code in src/taskq/backend/_protocol.py

async def reset_batch_failures(
    self,
    batch_id: UUID,
    *,
    connection: "asyncpg.Connection | None" = None,
) -> int: ...

abort_batch async

abort_batch(
    batch_id: UUID, *, connection: Connection | None = None
) -> int

Source code in src/taskq/backend/_protocol.py

async def abort_batch(
    self,
    batch_id: UUID,
    *,
    connection: "asyncpg.Connection | None" = None,
) -> int: ...

complete_batch async

complete_batch(
    batch_id: UUID, *, connection: Connection | None = None
) -> None

Source code in src/taskq/backend/_protocol.py

async def complete_batch(
    self,
    batch_id: UUID,
    *,
    connection: "asyncpg.Connection | None" = None,
) -> None: ...

get_batch async

get_batch(batch_id: UUID) -> BatchRow | None

Source code in src/taskq/backend/_protocol.py

async def get_batch(
    self,
    batch_id: UUID,
) -> BatchRow | None: ...

list_batches async

list_batches(
    filter: BatchFilter,
) -> list[tuple[BatchRow, BatchCounts]]

Source code in src/taskq/backend/_protocol.py

async def list_batches(
    self,
    filter: BatchFilter,
) -> list[tuple[BatchRow, BatchCounts]]: ...

count_batch_non_terminal async

count_batch_non_terminal(
    batch_id: UUID, *, connection: Connection | None = None
) -> int

Source code in src/taskq/backend/_protocol.py

async def count_batch_non_terminal(
    self,
    batch_id: UUID,
    *,
    connection: "asyncpg.Connection | None" = None,
) -> int: ...

prune_old_batches async

prune_old_batches(cutoff: datetime) -> int

Source code in src/taskq/backend/_protocol.py

async def prune_old_batches(
    self,
    cutoff: datetime,
) -> int: ...

BatchCounts dataclass

BatchCounts(
    total: int,
    pending: int,
    succeeded: int,
    failed: int,
    cancelled: int,
    crashed: int,
    abandoned: int,
)

Live job-count aggregate for one batch.

Mirrors BatchCompletionStatus fields; defined at the protocol layer so backends do not import the client-side batch module.

total instance-attribute

total: int

pending instance-attribute

pending: int

succeeded instance-attribute

succeeded: int

failed instance-attribute

failed: int

cancelled instance-attribute

cancelled: int

crashed instance-attribute

crashed: int

abandoned instance-attribute

abandoned: int

BatchFilter dataclass

BatchFilter(
    queue: str | None = None,
    active: bool | None = None,
    batch_id: UUID | None = None,
    limit: int = 100,
    cursor: str | None = None,
)

Filter parameters for Backend.list_batches.

Unlike JobFilter, this only carries fields relevant to batch queries: queue, active (status terminality), batch_id, limit, and cursor. Job-oriented fields (status, actor, tags, order_by, identity_key) are intentionally absent — using JobFilter for batch queries would silently ignore those fields, which is a type trap.

cursor is the same mechanism as :attr:JobFilter.cursor: an opaque keyset token encoding the last row of the previous page, which both backends must decode identically (:func:~taskq.backend._cursor.encode_batch_cursor). It encodes (created_at, id) where the job cursor encodes (priority, scheduled_at, id) — one field fewer, same |-delimited shape. created_at alone is not a total order (the column defaults to now(), the transaction timestamp, so one enqueue_batch_atomic stamps every row it writes identically), so id is the tiebreaker; it is UUIDv7 and therefore time-ordered, which lets both columns sort DESC together and makes the seam a single row-wise comparison.

There is no order_by: list_batches has exactly one ordering, so the cursor cannot disagree with it. list_jobs has three, and binds each to its own cursor shape through :class:~taskq.backend._cursor.JobOrdering for the same reason.

queue class-attribute instance-attribute

queue: str | None = None

active class-attribute instance-attribute

active: bool | None = None

batch_id class-attribute instance-attribute

batch_id: UUID | None = None

limit class-attribute instance-attribute

limit: int = 100

cursor class-attribute instance-attribute

cursor: str | None = None

__post_init__

__post_init__() -> None

Source code in src/taskq/backend/_protocol.py

def __post_init__(self) -> None:
    # Why: no upper bound. The limit bounds one page, not the reachable
    # set -- ``cursor`` is what reaches batch 501 -- but an operator
    # asking for a large single page is paying for it themselves, and a
    # cap here would only re-break the callers that predate the cursor.
    if self.limit < 0:
        raise ValueError(f"limit must be >= 0, got {self.limit}")

BatchRow dataclass

BatchRow(
    id: UUID,
    queue: str,
    status: BatchStatus,
    expected_size: int,
    consecutive_failures: int,
    failure_threshold: int | None,
    finalizer_job_id: UUID | None,
    originating_actor: str | None,
    created_at: datetime,
    completed_at: datetime | None,
    metadata: dict[str, object],
)

Read-model of a taskq.batches row.

id instance-attribute

id: UUID

queue instance-attribute

queue: str

status instance-attribute

status: BatchStatus

expected_size instance-attribute

expected_size: int

consecutive_failures instance-attribute

consecutive_failures: int

failure_threshold instance-attribute

failure_threshold: int | None

finalizer_job_id instance-attribute

finalizer_job_id: UUID | None

originating_actor instance-attribute

originating_actor: str | None

created_at instance-attribute

created_at: datetime

completed_at instance-attribute

completed_at: datetime | None

metadata instance-attribute

metadata: dict[str, object]

CancelPhase

Bases: IntEnum

Phases of cooperative-then-forced cancellation.

Why IntEnum and not Literal[0, 1, 2]: the cancel-poll loop performs arithmetic comparisons (db_phase >= 1, active.cancel_phase < 2) that Literal[int] does not narrow correctly under pyright strict. IntEnum subclasses int, so every existing comparison continues to work, while the typed enum carries the OTel attribute semantics (cancel_phase attribute on transition counters) and prevents bare-int values like 99 from slipping past the type checker.

Values NONE, COOPERATIVE, and FORCED are persistable — they map directly to the PG cancel_phase column whose check constraint is BETWEEN 0 AND 2. ABANDON_PENDING is an in-process sentinel only: the cancel-poll loop sets it on _ActiveJob to mark a job as queued for post-transaction abandonment. It is never written to PG. Keeping it on the same enum lets cancel_phase stay strongly typed end-to-end.

NONE class-attribute instance-attribute

NONE = 0

COOPERATIVE class-attribute instance-attribute

COOPERATIVE = 1

FORCED class-attribute instance-attribute

FORCED = 2

ABANDON_PENDING class-attribute instance-attribute

ABANDON_PENDING = 3

EventRow dataclass

EventRow(
    event_id: int,
    job_id: JobId,
    occurred_at: datetime,
    kind: Literal["state_change", "cancel_request"],
    detail: dict[str, object],
)

Read-model of a taskq.job_events row.

Mirrors the job_events table shape: monotonic event_id, the owning job, timestamp, event kind, and a detail payload.

event_id instance-attribute

event_id: int

job_id instance-attribute

job_id: JobId

occurred_at instance-attribute

occurred_at: datetime

kind instance-attribute

kind: Literal['state_change', 'cancel_request']

detail instance-attribute

detail: dict[str, object]

JobFilter dataclass

JobFilter(
    queue: str | None = None,
    status: JobStatus | Sequence[JobStatus] | None = None,
    actor: str | None = None,
    identity_key: IdentityKey | None = None,
    batch_id: UUID | None = None,
    limit: int = 100,
    cursor: str | None = None,
    tags: tuple[str, ...] | None = None,
    order_by: JobSortField | None = None,
    active: bool | None = None,
)

Filter parameters for :meth:Backend.list_jobs and :meth:Backend.cancel_where.

For cancel_where, the limit, cursor, and order_by fields are ignored — a bulk cancel is not paginated. Use :meth:has_predicates to check whether the filter has at least one predicate before passing it to cancel_where.

Heads-up: active=True is not Celery's 'active' — Celery's means 'currently executing' (running only), TaskQ's means 'not yet finished' (pending + scheduled + running). Read the active section below before relying on the name.

cursor is an opaque keyset-pagination token encoding the sort columns of order_by's ordering from the last row of the previous page -- (priority, scheduled_at, id) for the default, (created_at, id) and (finished_at, id) for the others. Encode it with :func:~taskq.backend._cursor.encode_job_cursor, passing the same order_by: a cursor is only meaningful under the ordering that produced it. Both backends must agree on cursor encoding and comparison semantics.

batch_id is a :class:UUID. The PG backend converts it to its canonical string form at the SQL boundary; the in-memory backend compares the UUID directly. Keeping the typed shape here means JobsClient.list(batch_id=UUID(...)) flows without an implicit str(uuid) coercion.

status accepts either a single :data:JobStatus (backwards compatible — e.g. JobFilter(status="pending")) or a sequence of statuses (e.g. JobFilter(status=["pending", "running"])). An empty sequence (status=[]) matches no jobs — it is not treated as 'no filter'. Unknown status values raise :class:ValueError in :meth:__post_init__, so untrusted input fails identically on both backends instead of surfacing as a PG enum-cast error or a silent empty result. The PG backend renders a single status as status = $n and a sequence as status = ANY($n); the in-memory backend performs a membership check in both cases.

active is a meta-filter that selects statuses by terminality. This is not Celery's 'active'. Celery/Flower use 'active' for tasks currently executing on a worker (running only); here it means 'not yet finished' — a superset that also includes work that has not started yet:

• active=True → non-terminal statuses (pending, scheduled, running)
• active=False → terminal statuses (succeeded, failed, cancelled, crashed, abandoned)
• active=None (default) → no status-terminality filter

The non-terminal set is derived from :data:~taskq.backend.statemachine.ACTIVE_STATUSES, which is itself derived from the state machine — adding a new non-terminal state updates this filter automatically.

status and active are mutually exclusive; specifying both raises :class:ValueError in :meth:__post_init__.

Usage examples::

JobsClient.list(JobFilter(status="pending"))
JobsClient.list(JobFilter(status=["pending", "running"]))
JobsClient.list(JobFilter(active=True))

queue class-attribute instance-attribute

queue: str | None = None

status class-attribute instance-attribute

status: JobStatus | Sequence[JobStatus] | None = None

actor class-attribute instance-attribute

actor: str | None = None

identity_key class-attribute instance-attribute

identity_key: IdentityKey | None = None

batch_id class-attribute instance-attribute

batch_id: UUID | None = None

limit class-attribute instance-attribute

limit: int = 100

cursor class-attribute instance-attribute

cursor: str | None = None

tags class-attribute instance-attribute

tags: tuple[str, ...] | None = None

order_by class-attribute instance-attribute

order_by: JobSortField | None = None

active class-attribute instance-attribute

active: bool | None = None

__post_init__

__post_init__() -> None

Source code in src/taskq/backend/_protocol.py

def __post_init__(self) -> None:
    # A negative limit diverges across backends: PG raises
    # "LIMIT must not be negative" while the in-memory slice would
    # silently drop rows.  Reject it here so both fail identically.
    if self.limit < 0:
        raise ValueError(f"limit must be >= 0, got {self.limit}")
    if self.status is not None:
        values = (self.status,) if isinstance(self.status, str) else tuple(self.status)
        unknown = [v for v in values if v not in JOB_STATUS_VALUES]
        if unknown:
            raise ValueError(
                f"unknown job status value(s): {list(dict.fromkeys(unknown))!r}; "
                f"valid statuses are {sorted(JOB_STATUS_VALUES)}"
            )
    if self.status is not None and self.active is not None:
        raise ValueError(
            "status and active are mutually exclusive; "
            "use status for specific status(es) or active for the "
            "terminal/non-terminal meta-filter"
        )
    # A NUL in a text predicate binds as text on PG (queue/actor/
    # identity_key) or text[] (tags) and surfaces as a raw asyncpg
    # CharacterNotInRepertoireError (SQLSTATE 22021) — the same trap
    # EnqueueArgs._check_no_nul_text guards on the write path.
    # Rejecting here makes both backends' list_jobs/cancel_where
    # paths fail identically with a clean ValueError; cancel_where
    # in particular is a write path (the bulk-cancel feature).
    if self.queue is not None:
        check_no_nul_str(self.queue, what="queue")
    if self.actor is not None:
        check_no_nul_str(self.actor, what="actor")
    if self.identity_key is not None:
        check_no_nul_str(self.identity_key, what="identity_key")
    if self.tags is not None:
        for tag in self.tags:
            check_no_nul_str(tag, what="tag")

has_predicates

has_predicates() -> bool

Return True if at least one filter predicate is set.

Used by JobsClient.cancel_where to reject empty filters that would match the entire table. New predicate fields added to JobFilter MUST be added here and in build_filter_conditions — the two are kept in sync manually. Non-predicate fields (limit, cursor, order_by) are excluded by design.

Source code in src/taskq/backend/_protocol.py

def has_predicates(self) -> bool:
    """Return True if at least one filter predicate is set.

    Used by ``JobsClient.cancel_where`` to reject empty filters that
    would match the entire table. New predicate fields added to
    ``JobFilter`` MUST be added here and in
    ``build_filter_conditions`` — the two are kept in sync manually.
    Non-predicate fields (``limit``, ``cursor``, ``order_by``) are
    excluded by design.
    """
    return (
        self.queue is not None
        or self.status is not None
        or self.actor is not None
        or self.identity_key is not None
        or self.batch_id is not None
        or (self.tags is not None and len(self.tags) > 0)
        or self.active is not None
    )

JobPage dataclass

JobPage(jobs: list[JobRow], next_cursor: str | None)

Paged result from :meth:JobsClient.list. Defined at the protocol layer because cursor encoding is a cross-backend contract. next_cursor is None when no more rows exist.

jobs instance-attribute

jobs: list[JobRow]

next_cursor instance-attribute

next_cursor: str | None

JobRow dataclass

JobRow(
    id: JobId,
    actor: str,
    queue: str,
    identity_key: IdentityKey | None,
    fairness_key: str | None,
    payload: dict[str, object],
    payload_schema_ver: int,
    status: JobStatus,
    priority: int,
    attempt: int,
    max_attempts: int,
    retry_kind: RetryKind,
    schedule_to_close: datetime | None,
    start_to_close: timedelta | None,
    heartbeat_timeout: timedelta | None,
    created_at: datetime,
    scheduled_at: datetime,
    started_at: datetime | None,
    finished_at: datetime | None,
    last_heartbeat_at: datetime | None,
    locked_by_worker: UUID | None,
    lock_expires_at: datetime | None,
    cancel_requested_at: datetime | None,
    cancel_phase: CancelPhase,
    error_class: str | None,
    error_message: str | None,
    error_traceback: str | None,
    progress_state: dict[str, object],
    progress_seq: int,
    result: dict[str, object] | None,
    result_size_bytes: int | None,
    result_expires_at: datetime | None,
    idempotency_key: IdempotencyKey | None,
    idempotency_scope: str,
    trace_id: str | None,
    span_id: str | None,
    metadata: dict[str, object],
    tags: tuple[str, ...],
)

Read-model of a taskq.jobs row. Every column the dispatch loop, heartbeat, and terminal writes need appears as a typed field. status uses a Literal union (8 values) matching the job_status enum in 01.00.00_01_pre_initial.sql.

id instance-attribute

id: JobId

actor instance-attribute

actor: str

queue instance-attribute

queue: str

identity_key instance-attribute

identity_key: IdentityKey | None

fairness_key instance-attribute

fairness_key: str | None

payload instance-attribute

payload: dict[str, object]

payload_schema_ver instance-attribute

payload_schema_ver: int

status instance-attribute

status: JobStatus

priority instance-attribute

priority: int

attempt instance-attribute

attempt: int

max_attempts instance-attribute

max_attempts: int

retry_kind instance-attribute

retry_kind: RetryKind

schedule_to_close instance-attribute

schedule_to_close: datetime | None

start_to_close instance-attribute

start_to_close: timedelta | None

heartbeat_timeout instance-attribute

heartbeat_timeout: timedelta | None

created_at instance-attribute

created_at: datetime

scheduled_at instance-attribute

scheduled_at: datetime

started_at instance-attribute

started_at: datetime | None

finished_at instance-attribute

finished_at: datetime | None

last_heartbeat_at instance-attribute

last_heartbeat_at: datetime | None

locked_by_worker instance-attribute

locked_by_worker: UUID | None

lock_expires_at instance-attribute

lock_expires_at: datetime | None

cancel_requested_at instance-attribute

cancel_requested_at: datetime | None

cancel_phase instance-attribute

cancel_phase: CancelPhase

error_class instance-attribute

error_class: str | None

error_message instance-attribute

error_message: str | None

error_traceback instance-attribute

error_traceback: str | None

progress_state instance-attribute

progress_state: dict[str, object]

progress_seq instance-attribute

progress_seq: int

result instance-attribute

result: dict[str, object] | None

result_size_bytes instance-attribute

result_size_bytes: int | None

result_expires_at instance-attribute

result_expires_at: datetime | None

idempotency_key instance-attribute

idempotency_key: IdempotencyKey | None

idempotency_scope instance-attribute

idempotency_scope: str

trace_id instance-attribute

trace_id: str | None

span_id instance-attribute

span_id: str | None

metadata instance-attribute

metadata: dict[str, object]

tags instance-attribute

tags: tuple[str, ...]

JobSortField

Bases: Enum

Sort ordering for :meth:Backend.list_jobs via :attr:JobFilter.order_by.

SCHEDULED_AT_ASC (and the default None) preserve the canonical dispatch-friendly ordering — priority DESC, scheduled_at ASC, id ASC — so existing list_jobs callers see no behaviour change.

CREATED_AT_DESC and FINISHED_AT_DESC serve "latest run by business key" queries: newest-created first and most-recently-finished first (NULLS LAST) respectively.

Every ordering pages with a cursor. Each one orders id with its primary column rather than against it -- created_at DESC, id DESC, not created_at DESC, id ASC -- which is what makes the page seam a single row-wise comparison. id is UUIDv7 and therefore time-ordered, so running it with a timestamp column reorders nothing in practice. The ordering and the cursor shape it pages with are one object, :class:~taskq.backend._cursor.JobOrdering; a cursor encodes the columns of the ordering it was produced under, so cursors are not interchangeable between orderings.

SCHEDULED_AT_ASC class-attribute instance-attribute

SCHEDULED_AT_ASC = 'scheduled_at_asc'

CREATED_AT_DESC class-attribute instance-attribute

CREATED_AT_DESC = 'created_at_desc'

FINISHED_AT_DESC class-attribute instance-attribute

FINISHED_AT_DESC = 'finished_at_desc'

ScheduleRecord

Bases: BaseModel

Read-only snapshot of a cron schedule row from the database.

model_config = ConfigDict(frozen=True) enforces immutability per public API discipline.

model_config class-attribute instance-attribute

model_config = ConfigDict(frozen=True)

id instance-attribute

id: UUID

actor instance-attribute

actor: str

name class-attribute instance-attribute

name: str = ''

cron_expr instance-attribute

cron_expr: str

timezone instance-attribute

timezone: str

dst_strategy class-attribute instance-attribute

dst_strategy: DstStrategy = 'skip'

payload_factory instance-attribute

payload_factory: str | None

identity_key class-attribute instance-attribute

identity_key: IdentityKey | None = None

enabled instance-attribute

enabled: bool

last_fired_at instance-attribute

last_fired_at: datetime | None

last_fire_error instance-attribute

last_fire_error: str | None

consecutive_failures instance-attribute

consecutive_failures: int

next_fire_at instance-attribute

next_fire_at: datetime

metadata instance-attribute

metadata: dict[str, object]

Clock

Bases: Protocol

Time abstraction injected into Backends.

now() returns wall-clock UTC; monotonic() returns a monotonically non-decreasing float suitable for local elapsed-time deltas (e.g. cancel-phase tracking in).

now

now() -> datetime

Source code in src/taskq/backend/clock.py

def now(self) -> datetime: ...

monotonic

monotonic() -> float

Source code in src/taskq/backend/clock.py

def monotonic(self) -> float: ...

SystemClock dataclass

SystemClock()

Production clock delegating to the standard library.

now() calls datetime.now(UTC) (timezone-aware). monotonic() calls time.monotonic().

now

now() -> datetime

Source code in src/taskq/backend/clock.py

def now(self) -> datetime:
    return datetime.now(UTC)

monotonic

monotonic() -> float

Source code in src/taskq/backend/clock.py

def monotonic(self) -> float:
    return time.monotonic()

BatchCompletionStatus

Bases: BaseModel

Aggregated completion counts for a batch of jobs.

pending counts jobs still in flight (pending, scheduled, or running status). is_complete is True when all jobs have reached a terminal status.

total instance-attribute

total: int

pending instance-attribute

pending: int

succeeded instance-attribute

succeeded: int

failed instance-attribute

failed: int

cancelled instance-attribute

cancelled: int

crashed instance-attribute

crashed: int

abandoned instance-attribute

abandoned: int

is_complete property

is_complete: bool

True when no jobs remain in a non-terminal state.

BatchHandle

Bases: BaseModel

Handle to a group of jobs inserted by a single :meth:~taskq.client.JobsClient.enqueue_batch call.

Invariant: job_handles contains one :class:~taskq.client.JobHandle per item in the original list (including idempotency-key collisions that returned existing rows). When a finalizer was enqueued, the finalizer handle is appended as the last entry of job_handles AND set separately as finalizer_handle. size is the number of non-finalizer items (i.e. len(job_handles) - (1 if finalizer_handle is not None else 0)).

:meth:status queries the database for the current completion counts of the batch.

model_config class-attribute instance-attribute

model_config = {'arbitrary_types_allowed': True}

batch_id instance-attribute

batch_id: UUID

job_handles instance-attribute

job_handles: list[JobHandle[BaseModel | None]]

List of :class:~taskq.client.JobHandle instances, one per enqueued item.

size instance-attribute

size: int

finalizer_handle class-attribute instance-attribute

finalizer_handle: JobHandle[BaseModel | None] | None = None

The :class:~taskq.client.JobHandle for the finalizer job, or None when no finalizer was enqueued. When set, the finalizer handle is also appended as the last entry of :attr:job_handles for backward compat.

status async

status(
    db: Connection, *, schema: str = "taskq"
) -> BatchCompletionStatus

Query live completion counts for all jobs in this batch.

Uses a JSONB containment query against the metadata column so the jobs_metadata_gin_idx GIN index is used (@> is supported by jsonb_path_ops). The query groups by status in a single round-trip.

schema must match the schema used when the :class:PostgresBackend was constructed (default "taskq").

Source code in src/taskq/batch.py

async def status(
    self,
    db: "asyncpg.Connection",
    *,
    schema: str = "taskq",
) -> BatchCompletionStatus:
    """Query live completion counts for all jobs in this batch.

    Uses a JSONB containment query against the ``metadata`` column so
    the ``jobs_metadata_gin_idx`` GIN index is used (``@>`` is
    supported by ``jsonb_path_ops``).  The query groups by status in a
    single round-trip.

    ``schema`` must match the schema used when the :class:`PostgresBackend`
    was constructed (default ``"taskq"``).
    """
    if not _IDENT_RE.match(schema):
        raise ValueError(f"invalid schema identifier: {schema!r}")

    containment = dumps_str({"batch_id": str(self.batch_id)})
    records = await db.fetch(
        f"SELECT status, count(*)::int AS cnt "  # noqa: S608  # Why: schema validated against _IDENT_RE immediately above.
        f'FROM "{schema}".jobs '
        "WHERE metadata @> $1::jsonb "
        "GROUP BY status",
        containment,
    )

    counts: dict[str, int] = {}
    for rec in records:
        counts[str(rec["status"])] = int(rec["cnt"])

    pending = counts.get("pending", 0) + counts.get("scheduled", 0) + counts.get("running", 0)
    return BatchCompletionStatus(
        total=sum(counts.values()),
        pending=pending,
        succeeded=counts.get("succeeded", 0),
        failed=counts.get("failed", 0),
        cancelled=counts.get("cancelled", 0),
        crashed=counts.get("crashed", 0),
        abandoned=counts.get("abandoned", 0),
    )

BatchSummary dataclass

BatchSummary(
    batch_id: UUID,
    queue: str,
    status: BatchStatus,
    expected_size: int,
    consecutive_failures: int,
    failure_threshold: int | None,
    finalizer_job_id: UUID | None,
    originating_actor: str | None,
    created_at: datetime,
    completed_at: datetime | None,
    completion: BatchCompletionStatus,
)

One row from the batches table, augmented with live job counts.

batch_id instance-attribute

batch_id: UUID

queue instance-attribute

queue: str

status instance-attribute

status: BatchStatus

expected_size instance-attribute

expected_size: int

consecutive_failures instance-attribute

consecutive_failures: int

failure_threshold instance-attribute

failure_threshold: int | None

finalizer_job_id instance-attribute

finalizer_job_id: UUID | None

originating_actor instance-attribute

originating_actor: str | None

created_at instance-attribute

created_at: datetime

completed_at instance-attribute

completed_at: datetime | None

completion instance-attribute

completion: BatchCompletionStatus

EnqueueItem

Bases: BaseModel

One item in a :meth:~taskq.client.JobsClient.enqueue_batch call.

actor_ref is an :class:~taskq.actor.ActorRef for any payload and result type. payload is the Pydantic model that will be serialized into the job row — it is validated by the actor's payload_type inside :meth:~taskq.client.JobsClient.enqueue_batch before any INSERT.

metadata is merged with the library-injected batch_id key before the row is written; callers MUST NOT set metadata.batch_id manually.

model_config class-attribute instance-attribute

model_config = {
    "arbitrary_types_allowed": True,
    "frozen": True,
}

actor_ref instance-attribute

actor_ref: ActorRef[Any, Any]

payload instance-attribute

payload: BaseModel

scheduled_at class-attribute instance-attribute

scheduled_at: datetime | None = None

priority class-attribute instance-attribute

priority: int | None = None

fairness_key class-attribute instance-attribute

fairness_key: str | None = None

idempotency_key class-attribute instance-attribute

idempotency_key: IdempotencyKey | str | None = None

idempotency_scope class-attribute instance-attribute

idempotency_scope: str | None = None

identity_key class-attribute instance-attribute

identity_key: IdentityKey | None = None

metadata class-attribute instance-attribute

metadata: dict[str, object] = Field(default_factory=dict)

tags class-attribute instance-attribute

tags: list[str] | None = None

start_to_close class-attribute instance-attribute

start_to_close: timedelta | None = None

AbortBatchAfter dataclass

AbortBatchAfter(
    consecutive_failures: int,
    *,
    failure_threshold: int | None = None,
)

Bases: BatchFailurePolicy

Abort the batch after consecutive_failures consecutive failures.

should_abort(n) returns True when n >= consecutive_failures. failure_threshold is set to consecutive_failures in __post_init__ so the client can read it polymorphically via policy.failure_threshold.

Running jobs are NOT cancelled by the abort — only pending and scheduled jobs are cancelled. Running jobs continue to completion. This matches the post-terminal-write hook design: the hook runs after the terminal write, so a job that was dispatched before the abort triggered will run to completion.

consecutive_failures instance-attribute

consecutive_failures: int

__post_init__

__post_init__() -> None

Source code in src/taskq/batch_policy.py

def __post_init__(self) -> None:
    if self.consecutive_failures < 1:
        raise ValueError(f"consecutive_failures must be >= 1, got {self.consecutive_failures}")
    object.__setattr__(self, "failure_threshold", self.consecutive_failures)

should_abort

should_abort(consecutive_failures: int) -> bool

Source code in src/taskq/batch_policy.py

def should_abort(self, consecutive_failures: int) -> bool:
    return consecutive_failures >= self.consecutive_failures

BatchFailurePolicy dataclass

BatchFailurePolicy(*, failure_threshold: int | None = None)

Abstract base class for batch failure policies.

Subclasses implement :meth:should_abort to decide whether a batch should be aborted given the current count of consecutive failures.

failure_threshold is the consecutive-failure count at which the batch should be aborted. None disables abort (the default on the base class). :class:AbortBatchAfter sets this to its consecutive_failures value in __post_init__; custom policies should set it to their computed threshold. The client reads failure_policy.failure_threshold polymorphically — no isinstance check is needed.

failure_threshold class-attribute instance-attribute

failure_threshold: int | None = field(
    default=None, kw_only=True
)

__post_init__

__post_init__() -> None

Source code in src/taskq/batch_policy.py

def __post_init__(self) -> None:
    raise TypeError("BatchFailurePolicy is abstract; use AbortBatchAfter or a custom subclass")

should_abort

should_abort(consecutive_failures: int) -> bool

Return True if the batch should be aborted.

Subclasses must override this method.

Source code in src/taskq/batch_policy.py

def should_abort(self, consecutive_failures: int) -> bool:
    """Return ``True`` if the batch should be aborted.

    Subclasses must override this method.
    """
    raise NotImplementedError

BulkCancelResult

Bases: BaseModel

Structured outcome of a bulk cancellation request.

Returned by JobsClient.cancel_where() so callers can inspect how many jobs were cancelled directly (pending/scheduled → terminal 'cancelled') vs how many had cooperative cancel requested (running → cancel_phase=1).

model_config class-attribute instance-attribute

model_config = ConfigDict(frozen=True)

cancelled_directly instance-attribute

cancelled_directly: int

Count of pending/scheduled jobs moved straight to terminal 'cancelled'.

cancel_requested instance-attribute

cancel_requested: int

Count of running jobs with cancel_phase=1 set (cooperative cancel).

cancelled_ids instance-attribute

cancelled_ids: tuple[UUID, ...]

IDs of jobs cancelled directly (pending/scheduled → cancelled).

cancel_requested_ids instance-attribute

cancel_requested_ids: tuple[UUID, ...]

IDs of running jobs with cancel requested.

total_affected property

total_affected: int

Total jobs affected by the bulk cancel.

CancelResult

Bases: BaseModel

Structured outcome of a cancellation request.

Returned by JobsClient.cancel() so callers can inspect whether the cancellation was initiated and what the status transition was.

model_config class-attribute instance-attribute

model_config = ConfigDict(frozen=True)

job_id instance-attribute

job_id: JobId

previous_status instance-attribute

previous_status: JobStatus

new_status instance-attribute

new_status: JobStatus

cancellation_initiated instance-attribute

cancellation_initiated: bool

JobEvent

Bases: BaseModel

A single event yielded by :meth:TaskQ.stream.

Represents a point-in-time snapshot of a job's observable state. Yielded on every status transition or progress update; the final event always has terminal=True.

The progress_state and progress_seq fields reflect the last values written by the worker. They are None / 0 until the worker emits a progress update.

Serialises cleanly to JSON via model_dump() for SSE or WebSocket fanout — fields are deliberately flat so the caller can forward the event without transformation::

async for event in tq.stream(job_id):
    await websocket.send_json(event.model_dump())

model_config class-attribute instance-attribute

model_config = ConfigDict(frozen=True)

job_id instance-attribute

job_id: JobId

status instance-attribute

status: JobStatus

progress_state instance-attribute

progress_state: dict[str, object]

progress_seq instance-attribute

progress_seq: int

terminal instance-attribute

terminal: bool

JobHandle

JobHandle(
    *,
    row: JobRow,
    result_adapter: TypeAdapter[R],
    was_existing: bool,
    client: JobsClient | None = None,
    backend: Backend | None = None,
    _redis_client: Redis | None = None,
    _settings: TaskQSettings | None = None,
)

Typed handle to a single enqueued job.

Created by :class:JobsClient methods (:meth:~JobsClient.enqueue, :meth:~JobsClient.get) or by :class:SubJobEnqueuer (with backend= only). The type parameter R flows from the actor's declared return type through :class:ActorRef into this handle: JobHandle[OrderResult] for an actor returning OrderResult, JobHandle[None] for fire-and-forget actors.

At least one of client or backend must be supplied. When client is provided, _backend is filled from client.backend. When only backend is provided, the four read-back methods (:meth:status, :meth:refresh, :meth:attempts, :meth:cancel) raise :class:RuntimeError because they require the client's higher-level coordination. :meth:wait always works (it reads through _backend directly).

Why result_adapter: TypeAdapter[R] is a constructor arg: pyright only infers R for a generic class when the type parameter appears in at least one field or method signature. The adapter is that field — without it R would be phantom and inference would silently fall back to Unknown.

Source code in src/taskq/client/_handle.py

def __init__(
    self,
    *,
    row: JobRow,
    result_adapter: TypeAdapter[R],
    was_existing: bool,
    client: "JobsClient | None" = None,
    backend: Backend | None = None,
    _redis_client: "redis_async.Redis | None" = None,
    _settings: "TaskQSettings | None" = None,
) -> None:
    if client is None and backend is None:
        raise ValueError(
            "JobHandle requires at least one of client= or backend= (received neither)"
        )
    self._row = row
    self._result_adapter = result_adapter
    self.was_existing: bool = was_existing
    self._client = client
    self._backend: Backend = backend if backend is not None else client.backend  # pyright: ignore[reportOptionalMemberAccess]  # Why: client is guaranteed non-None when backend is None; the ValueError above ensures at least one is provided
    self._redis_client: "redis_async.Redis | None" = _redis_client  # noqa: UP037  # Why: redis_async is under TYPE_CHECKING; string annotation prevents a runtime import cycle.
    self._handle_settings: "TaskQSettings | None" = _settings  # noqa: UP037  # Why: TaskQSettings is under TYPE_CHECKING; string annotation prevents a runtime import cycle.

was_existing instance-attribute

was_existing: bool = was_existing

job_id property

job_id: JobId

The job's unique id.

actor_name property

actor_name: str

The actor this job targets.

queue property

queue: str

The queue this job was enqueued on.

row property

row: JobRow

The last :class:JobRow this handle observed.

Seeded at construction with the row the creating call fetched, and advanced by every successful row fetch through the handle — :meth:refresh, :meth:status, and :meth:wait's polling loop each record the row they just read. :meth:progress_stream does not advance it — its Redis path fetches no rows, and advancing only on the PG fallback would make the semantics backend-dependent. Reading the property costs no backend round trip: handle = await tq.get(id) followed by handle.row is the single-read pattern for full row state, and a long-lived handle's row stays current as its owner refreshes or waits.

The row is returned by reference (it is frozen and backends hand the handle an isolated row); repeated reads between fetches return the same object.

status async

status() -> JobStatus

Return the current status of this job (live read).

Cheap, non-blocking: a single backend.get and a status projection. No polling. Use this when you want to know the state without waiting for a terminal transition. Advances :attr:row to the fetched row.

Raises:

Type	Description
RuntimeError	this handle was constructed without a :class:JobsClient.

Source code in src/taskq/client/_handle.py

async def status(self) -> JobStatus:
    """Return the current status of this job (live read).

    Cheap, non-blocking: a single ``backend.get`` and a status
    projection. No polling. Use this when you want to know the
    state without waiting for a terminal transition. Advances
    :attr:`row` to the fetched row.

    Raises:
        RuntimeError: this handle was constructed without a
            :class:`JobsClient`.
    """
    if self._client is None:
        raise RuntimeError(
            "JobHandle.status() requires a JobsClient. "
            "This handle was constructed via ctx.jobs.enqueue(); use "
            "the worker's JobsClient to read job state externally."
        )
    row = await self._client.backend.get(self.job_id)
    if row is None:
        raise KeyError(self.job_id)
    self._observe(row)
    return row.status

refresh async

refresh() -> JobRow

Re-read the row from the backend and return the raw :class:JobRow.

Useful for callers that want full row state (timestamps, attempt counts, error metadata) without going through :meth:wait. Does not block on terminal state — returns the current row whatever its status. Advances :attr:row to the fetched row, so after a refresh handle.row and the return value are the same row.

Raises:

Type	Description
RuntimeError	this handle was constructed without a :class:JobsClient.

Source code in src/taskq/client/_handle.py

async def refresh(self) -> JobRow:
    """Re-read the row from the backend and return the raw
    :class:`JobRow`.

    Useful for callers that want full row state (timestamps,
    attempt counts, error metadata) without going through
    :meth:`wait`. Does not block on terminal state — returns the
    current row whatever its status. Advances :attr:`row` to the
    fetched row, so after a refresh ``handle.row`` and the return
    value are the same row.

    Raises:
        RuntimeError: this handle was constructed without a
            :class:`JobsClient`.
    """
    if self._client is None:
        raise RuntimeError(
            "JobHandle.refresh() requires a JobsClient. "
            "This handle was constructed via ctx.jobs.enqueue(); use "
            "the worker's JobsClient to read job state externally."
        )
    row = await self._client.backend.get(self.job_id)
    if row is None:
        raise KeyError(self.job_id)
    self._observe(row)
    return row

attempts async

attempts() -> list[AttemptRow]

Return the attempt rows for this job, ordered by attempt number.

Raises:

Type	Description
RuntimeError	this handle was constructed without a :class:JobsClient.

Source code in src/taskq/client/_handle.py

async def attempts(self) -> list[AttemptRow]:
    """Return the attempt rows for this job, ordered by attempt number.

    Raises:
        RuntimeError: this handle was constructed without a
            :class:`JobsClient`.
    """
    if self._client is None:
        raise RuntimeError(
            "JobHandle.attempts() requires a JobsClient. "
            "This handle was constructed via ctx.jobs.enqueue(); use "
            "the worker's JobsClient to read job state externally."
        )
    return await self._client.backend.get_attempts(self.job_id)

cancel async

cancel(reason: str | None = None) -> CancelResult

Delegate to :meth:JobsClient.cancel.

Raises:

Type	Description
RuntimeError	this handle was constructed without a :class:JobsClient.

Source code in src/taskq/client/_handle.py

async def cancel(self, reason: str | None = None) -> CancelResult:
    """Delegate to :meth:`JobsClient.cancel`.

    Raises:
        RuntimeError: this handle was constructed without a
            :class:`JobsClient`.
    """
    if self._client is None:
        raise RuntimeError(
            "JobHandle.cancel() requires a JobsClient. "
            "This handle was constructed via ctx.jobs.enqueue(); use "
            "the worker's JobsClient to read job state externally."
        )
    return await self._client.cancel(self.job_id, reason)

wait async

wait(*, timeout: float | None = None) -> R

Block until the job reaches a terminal status, then return R.

Returns the actor's return value, validated through :attr:result_adapter. The result type is R exactly — never R | None. Missing or failed results raise. Advances :attr:row to each row the polling loop fetches — on return, the terminal row the result was extracted from.

Raises:

Type	Description
ResultUnavailable	terminal state reached but no result was stored (result TTL expired, actor returned None while R is non-None, etc.).
JobFailed	the job ended in a non-success terminal state (failed / cancelled / crashed / abandoned); the row is attached to the exception for inspection.
TimeoutError	timeout elapsed before any terminal transition was observed.

Source code in src/taskq/client/_handle.py

async def wait(self, *, timeout: float | None = None) -> R:  # noqa: ASYNC109  # Why: timeout is part of the public API contract; asyncio.timeout() context-manager doesn't fit a polling loop
    """Block until the job reaches a terminal status, then return ``R``.

    Returns the actor's return value, validated through
    :attr:`result_adapter`. The result type is ``R`` exactly —
    never ``R | None``. Missing or failed results raise. Advances
    :attr:`row` to each row the polling loop fetches — on return,
    the terminal row the result was extracted from.

    Raises:
        ResultUnavailable: terminal state reached but no result was
            stored (result TTL expired, actor returned ``None``
            while ``R`` is non-``None``, etc.).
        JobFailed: the job ended in a non-success terminal state
            (``failed`` / ``cancelled`` / ``crashed`` / ``abandoned``);
            the row is attached to the exception for inspection.
        TimeoutError: ``timeout`` elapsed before any terminal
            transition was observed.
    """
    deadline: float | None = None
    if timeout is not None:
        deadline = asyncio.get_running_loop().time() + timeout

    while True:
        row = await self._backend.get(self.job_id)
        if row is None:
            raise KeyError(self.job_id)
        self._observe(row)
        if row.status in TERMINAL_STATUSES:
            return self._extract_result(row)

        remaining: float
        if deadline is not None:
            remaining = deadline - asyncio.get_running_loop().time()
            if remaining <= 0:
                raise TimeoutError()
            sleep = min(_WAIT_POLL_INTERVAL, remaining)
        else:
            sleep = _WAIT_POLL_INTERVAL

        await asyncio.sleep(sleep)

progress_stream async

progress_stream() -> AsyncIterator[ProgressEvent]

Stream live progress events for this job.

When Redis is configured, subscribes to the per-job Redis pub/sub channel and yields :class:~taskq.progress.ProgressEvent objects in real time. When Redis is not available, falls back to polling Postgres at 500 ms intervals and synthesising events from row diffs.

Raises :class:NotImplementedError when the in-memory backend is detected — the in-memory backend does not support pub/sub.

Does not advance :attr:row — the Redis path fetches no rows, and advancing only on the PG fallback would make the semantics backend-dependent.

Yields events until a terminal=True event is produced.

Source code in src/taskq/client/_handle.py

async def progress_stream(self) -> AsyncIterator[ProgressEvent]:
    """Stream live progress events for this job.

    When Redis is configured, subscribes to the per-job Redis pub/sub
    channel and yields :class:`~taskq.progress.ProgressEvent` objects in
    real time. When Redis is not available, falls back to polling Postgres
    at 500 ms intervals and synthesising events from row diffs.

    Raises :class:`NotImplementedError` when the in-memory backend is
    detected — the in-memory backend does not support pub/sub.

    Does not advance :attr:`row` — the Redis path fetches no rows,
    and advancing only on the PG fallback would make the semantics
    backend-dependent.

    Yields events until a ``terminal=True`` event is produced.
    """
    from taskq.testing.in_memory import InMemoryBackend  # lazy — test-only dep

    if isinstance(self._backend, InMemoryBackend):
        raise NotImplementedError(
            "progress_stream requires Redis; in-memory backend does not support SSE."
        )

    if self._redis_client is not None and self._handle_settings is not None:
        async for event in self._progress_stream_redis():
            yield event
    else:
        async for event in self._progress_stream_pg():
            yield event

JobsClient

JobsClient(
    backend: Backend,
    *,
    clock: Clock | None = None,
    settings: TaskQSettings | None = None,
    capacity_cache_ttl: float = DEFAULT_CAPACITY_CACHE_TTL,
)

Public API for job operations.

Delegates to the injected :class:Backend and wraps results in typed :class:JobHandle[R] instances. The client owns the payload-serialization step that turns a typed P into the dict[str, object] carried by :class:EnqueueArgs; the backend sees only erased payloads.

Source code in src/taskq/client/_jobs.py

def __init__(
    self,
    backend: Backend,
    *,
    clock: Clock | None = None,
    settings: "TaskQSettings | None" = None,
    capacity_cache_ttl: float = DEFAULT_CAPACITY_CACHE_TTL,
) -> None:
    self._backend = backend
    self._clock = clock if clock is not None else SystemClock()
    self._settings: "TaskQSettings | None" = settings  # noqa: UP037  # Why: TaskQSettings is under TYPE_CHECKING; string annotation avoids runtime import.
    self._redis_client: "redis_async.Redis | None" = None  # type: ignore[type-arg]  # noqa: UP037  # Why: redis_async is under TYPE_CHECKING; string annotation avoids runtime import. type-arg: redis-py stubs expose Redis as an unparameterised generic.
    self._exit_stack: AsyncExitStack = AsyncExitStack()
    self._warned_unique_for: set[str] = set()
    self._capacity_cache = ActorCapacityCache(backend, ttl=capacity_cache_ttl)
    # Why resolved here: every enqueue path in this client validates
    # against one number, and a client built without settings still gets
    # the shipped default rather than a second literal.
    self._idempotency_max_bytes: int = (
        settings.idempotency_key_max_bytes
        if settings is not None
        else MAX_IDEMPOTENCY_KEY_BYTES
    )

backend property

backend: Backend

The underlying :class:Backend this client delegates to.

Exposed so :class:JobHandle can read the backend through the client without accessing the private _backend attribute.

close async

close() -> None

Close the Redis client and release resources via the exit stack.

Source code in src/taskq/client/_jobs.py

async def close(self) -> None:
    """Close the Redis client and release resources via the exit stack."""
    await self._exit_stack.aclose()
    self._redis_client = None

invalidate_actor_capacity_cache

invalidate_actor_capacity_cache() -> None

Drop the cached actor_config.max_pending snapshot.

The next enqueue refreshes from the backend instead of waiting out the TTL. Not needed in normal operation (staleness is bounded by capacity_cache_ttl, default 5s); intended for tests and for tooling that knows it just changed the table and cannot wait out the TTL.

Source code in src/taskq/client/_jobs.py

def invalidate_actor_capacity_cache(self) -> None:
    """Drop the cached ``actor_config.max_pending`` snapshot.

    The next enqueue refreshes from the backend instead of waiting
    out the TTL. Not needed in normal operation (staleness is
    bounded by ``capacity_cache_ttl``, default 5s); intended for
    tests and for tooling that knows it just changed the table and
    cannot wait out the TTL.
    """
    self._capacity_cache.invalidate()

enqueue async

enqueue(
    ref: ActorRef[P, R],
    payload: P,
    *,
    queue: QueueName | None = None,
    scheduled_at: datetime | None = None,
    priority: int | None = None,
    schedule_to_close: datetime | None = None,
    start_to_close: timedelta | None = None,
    heartbeat_timeout: timedelta | None = None,
    identity_key: IdentityKey | None = None,
    fairness_key: str | None = None,
    idempotency_key: IdempotencyKey | None = None,
    idempotency_scope: str | None = None,
    trace_id: str | None = None,
    span_id: str | None = None,
    metadata: dict[str, object] | None = None,
    tags: list[str] | None = None,
) -> JobHandle[R]

Enqueue a job for the given actor and return a typed handle.

The payload is serialized through ref.payload_type so the EnqueueArgs.payload carried over the backend boundary is a plain dict[str, object] ready for the JSONB column. The returned :class:JobHandle[R] carries ref.result_adapter so :meth:JobHandle.wait can validate the stored result back to R.

The metadata.singleton key is reserved by the library for singleton enforcement. When ref.singleton is True the library unconditionally writes metadata.singleton = True, overriding any caller-supplied value. Callers MUST NOT set metadata.singleton manually.

max_pending:

• When the actor's effective max_pending is set, a pre-flight count of pending + scheduled jobs for the actor is compared to the limit. If count >= max_pending, :class:MaxPendingExceededError is raised synchronously — the caller decides whether to retry, fail, or wait; the library does not block on capacity.

• The effective limit is operator-owned: a non-NULL stored actor_config.max_pending (set via taskq actor-config set --max-pending) wins over the @actor(max_pending=...) literal; a cleared or absent stored value falls back to the literal. The client reads the stored value through a TTL-bounded cache (default 5s staleness; see :class:taskq.client._capacity.ActorCapacityCache), so an operator change takes effect fleet-wide within seconds without any redeploy or restart.

• Evaluation order at enqueue: unique_for dedup → singleton pre-flight → max_pending count check → idempotency_key INSERT → job INSERT. A unique_for hit bypasses all remaining checks; a singleton collision fires before max_pending to give the caller the more specific SingletonCollisionError.

• idempotency_key does not bypass max_pending — the idempotency ON CONFLICT fires at step 5, after the max_pending check at step 3. Re-enqueuing with a duplicate idempotency_key when the queue is full raises MaxPendingExceededError, not the deduplicated handle. Only unique_for (step 1) bypasses max_pending.

idempotency_key:

• idempotency_key is unique within its idempotency_scope (composite (idempotency_scope, idempotency_key) uniqueness). The default scope (idempotency_scope=None or "") preserves the prior global-until-prune behavior exactly, so existing callers see zero behavior change. Passing an explicit scope (e.g. a run/batch/epoch id) lets two enqueues with the same business key in different scopes both succeed, decoupling the dedupe horizon from prune_retention_*.

• Key length is bounded at idempotency_key_max_bytes (TASKQ_IDEMPOTENCY_KEY_MAX_BYTES, default 1024 UTF-8 bytes) — the bound is the composite unique index's btree entry size, not a round number. Empty and whitespace-only keys raise :class:ValueError at the client boundary before any backend call. The same bound applies to idempotency_scope; an empty scope ("") is valid and equivalent to None (the default/global scope).

• No time-based (TTL) dedupe window. idempotency_scope decouples the dedupe horizon from prune_retention_* by namespace, not by time — there is no idempotency_ttl or equivalent "dedupe for the next N seconds" parameter. A key within a given scope still dedupes until pruned, exactly like the pre-scope global behavior, just scoped to that namespace. This is a deliberate scope decision, not an oversight: a real sliding-window TTL cannot be expressed as a single static unique index the way scope can — every mature job queue that offers one (Oban, River) either gives up the atomic INSERT ... ON CONFLICT for a check-then-insert lock (weaker concurrency guarantee) or buckets time into the key itself (coarser, edge-artifact-prone semantics). If your use case genuinely needs "dedupe for the next hour, not forever," encode the window into the scope yourself (e.g. a time-bucketed scope string) until/unless a TTL parameter ships as a separate feature.

• Rolling-deploy note: if this schema is mid-upgrade (the 01.00.03_01_pre_idempotency_scope.sql migration applied but 01.00.03_01_post_idempotency_scope_drop_old_index.sql not yet applied), reusing the same idempotency_key under two different idempotency_scope values raises :class:~taskq.exceptions.ScopedIdempotencyMigrationPendingError rather than silently dedupe against the wrong scope's job. The trigger is a key existing under a different scope, in either direction — an unscoped call reusing a key first written under a non-default scope raises it too. Only brand-new keys and same-scope repeats are unaffected. See that exception's docstring and the migration file's header comment for the full rationale.

unique_for:

• unique_for deduplication is best-effort. Concurrent enqueues for the same (actor, identity_key) may both insert; the dispatch CTE's running_identities filter ensures only one runs.

• When either dedup mechanism matches an existing job, JobHandle.was_existing is True. This field replaces the need for callers to inspect the row's created_at to detect a dedup return.

Source code in src/taskq/client/_jobs.py

async def enqueue[P: BaseModel, R: BaseModel | None](
    self,
    ref: ActorRef[P, R],
    payload: P,
    *,
    queue: QueueName | None = None,
    scheduled_at: datetime | None = None,
    priority: int | None = None,
    schedule_to_close: datetime | None = None,
    start_to_close: timedelta | None = None,
    heartbeat_timeout: timedelta | None = None,
    identity_key: IdentityKey | None = None,
    fairness_key: str | None = None,
    idempotency_key: IdempotencyKey | None = None,
    idempotency_scope: str | None = None,
    trace_id: str | None = None,
    span_id: str | None = None,
    metadata: dict[str, object] | None = None,
    tags: list[str] | None = None,
) -> JobHandle[R]:
    """Enqueue a job for the given actor and return a typed handle.

    The payload is serialized through ``ref.payload_type`` so the
    ``EnqueueArgs.payload`` carried over the backend boundary is a
    plain ``dict[str, object]`` ready for the JSONB column. The
    returned :class:`JobHandle[R]` carries ``ref.result_adapter`` so
    :meth:`JobHandle.wait` can validate the stored result back to
    ``R``.

    The ``metadata.singleton`` key is reserved by the library for
    singleton enforcement. When ``ref.singleton`` is ``True`` the
    library unconditionally writes ``metadata.singleton = True``,
    overriding any caller-supplied value. Callers MUST NOT set
    ``metadata.singleton`` manually.

    **max_pending:**

    - When the actor's effective ``max_pending`` is set, a pre-flight
      count of ``pending`` + ``scheduled`` jobs for the actor is
      compared to the limit. If ``count >= max_pending``,
      :class:`MaxPendingExceededError` is raised synchronously — the
      caller decides whether to retry, fail, or wait; the library
      does not block on capacity.

    - The effective limit is **operator-owned**: a non-NULL stored
      ``actor_config.max_pending`` (set via
      ``taskq actor-config set --max-pending``) wins over the
      ``@actor(max_pending=...)`` literal; a cleared or absent
      stored value falls back to the literal. The client reads the
      stored value through a TTL-bounded cache (default 5s
      staleness; see :class:`taskq.client._capacity.ActorCapacityCache`),
      so an operator change takes effect fleet-wide within seconds
      without any redeploy or restart.

    - Evaluation order at enqueue: ``unique_for`` dedup →
      singleton pre-flight → ``max_pending`` count check →
      ``idempotency_key`` INSERT → job INSERT. A ``unique_for`` hit
      bypasses all remaining checks; a singleton collision fires before
      ``max_pending`` to give the caller the more specific
      ``SingletonCollisionError``.

    - ``idempotency_key`` does **not** bypass ``max_pending`` — the
      idempotency ON CONFLICT fires at step 5, after the max_pending
      check at step 3. Re-enqueuing with a duplicate
      ``idempotency_key`` when the queue is full raises
      ``MaxPendingExceededError``, not the deduplicated handle. Only
      ``unique_for`` (step 1) bypasses max_pending.

    **idempotency_key:**

    - ``idempotency_key`` is unique within its ``idempotency_scope``
      (composite ``(idempotency_scope, idempotency_key)`` uniqueness).
      The default scope (``idempotency_scope=None`` or ``""``) preserves
      the prior global-until-prune behavior exactly, so existing callers
      see zero behavior change. Passing an explicit scope (e.g. a
      run/batch/epoch id) lets two enqueues with the same business key
      in different scopes both succeed, decoupling the dedupe horizon
      from ``prune_retention_*``.

    - Key length is bounded at ``idempotency_key_max_bytes``
      (``TASKQ_IDEMPOTENCY_KEY_MAX_BYTES``, default 1024 UTF-8 bytes) —
      the bound is the composite unique index's btree entry size, not a
      round number. Empty and whitespace-only keys raise
      :class:`ValueError` at the client boundary before any backend
      call. The same bound applies to ``idempotency_scope``; an empty
      scope (``""``) is valid and equivalent to ``None`` (the
      default/global scope).

    - **No time-based (TTL) dedupe window.** ``idempotency_scope``
      decouples the dedupe horizon from ``prune_retention_*`` by
      namespace, not by time — there is no ``idempotency_ttl`` or
      equivalent "dedupe for the next N seconds" parameter. A key
      within a given scope still dedupes **until pruned**, exactly
      like the pre-scope global behavior, just scoped to that
      namespace. This is a deliberate scope decision, not an
      oversight: a real sliding-window TTL cannot be expressed as a
      single static unique index the way scope can — every mature
      job queue that offers one (Oban, River) either gives up the
      atomic ``INSERT ... ON CONFLICT`` for a check-then-insert lock
      (weaker concurrency guarantee) or buckets time into the key
      itself (coarser, edge-artifact-prone semantics). If your use
      case genuinely needs "dedupe for the next hour, not forever,"
      encode the window into the scope yourself (e.g. a
      time-bucketed scope string) until/unless a TTL parameter ships
      as a separate feature.

    - Rolling-deploy note: if this schema is mid-upgrade (the
      ``01.00.03_01_pre_idempotency_scope.sql`` migration applied but
      ``01.00.03_01_post_idempotency_scope_drop_old_index.sql`` not
      yet applied), reusing the same ``idempotency_key`` under two
      *different* ``idempotency_scope`` values raises
      :class:`~taskq.exceptions.ScopedIdempotencyMigrationPendingError`
      rather than silently dedupe against the wrong scope's job. The
      trigger is a key existing under a different scope, in *either*
      direction — an unscoped call reusing a key first written under
      a non-default scope raises it too. Only brand-new keys and
      same-scope repeats are unaffected. See that exception's
      docstring and the migration file's header comment for the full
      rationale.

    **unique_for:**

    - ``unique_for`` deduplication is **best-effort**. Concurrent
      enqueues for the same ``(actor, identity_key)`` may both insert;
      the dispatch CTE's ``running_identities`` filter ensures only one
      runs.

    - When either dedup mechanism matches an existing job,
      ``JobHandle.was_existing`` is ``True``. This field replaces the
      need for callers to inspect the row's ``created_at`` to detect a
      dedup return.
    """
    resolved_queue = queue if queue is not None else ref.queue
    identity_key_str = str(identity_key) if identity_key is not None else ""

    with enqueue_span(ref.name, resolved_queue, identity_key=identity_key_str) as (
        span,
        extracted_trace_id,
        extracted_span_id,
    ):
        effective_max_pending = await self._capacity_cache.effective_max_pending(
            ref.name, ref.max_pending
        )
        # An explicit trace_id/span_id overrides the ambient span, per
        # docs/guides/jobs-clients.md: "pass explicitly to override or
        # to propagate an external trace context". Both were previously
        # accepted and then dropped in favour of the extracted values,
        # so cross-service propagation silently produced an unlinked
        # consumer span. SubJobEnqueuer.enqueue has no such override and
        # correctly exposes no parameter for one.
        args = build_enqueue_args(
            ref,
            payload,
            queue=queue,
            scheduled_at=scheduled_at,
            priority=priority,
            fairness_key=fairness_key,
            metadata=metadata,
            identity_key=identity_key,
            idempotency_key=idempotency_key,
            idempotency_scope=idempotency_scope,
            trace_id=trace_id if trace_id is not None else extracted_trace_id,
            span_id=span_id if span_id is not None else extracted_span_id,
            schedule_to_close=schedule_to_close,
            start_to_close=start_to_close,
            heartbeat_timeout=heartbeat_timeout,
            max_pending=effective_max_pending,
            tags=tags,
            idempotency_max_bytes=self._idempotency_max_bytes,
        )
        span.set_attribute("messaging.message.id", str(args.id))
        if ref.unique_for is not None and args.identity_key is None:
            self._maybe_warn_unique_for_no_identity(ref)
        with self._translate_schema_errors():
            row = await self._backend.enqueue(args)

    if row.id == args.id:
        logger.debug(
            "job_enqueued",
            kind="job_enqueued",
            job_id=str(row.id),
            actor=row.actor,
            queue=row.queue,
            idempotency_key=row.idempotency_key,
        )
    return JobHandle(
        client=self,
        row=row,
        result_adapter=ref.result_adapter,
        was_existing=(row.id != args.id),
        _redis_client=self._redis_client,
        _settings=self._settings,
    )

enqueue_batch async

enqueue_batch(
    items: list[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: Connection | None = None,
    failure_policy: BatchFailurePolicy | None = None,
    finalizer: EnqueueItem | None = None,
) -> BatchHandle

Enqueue multiple jobs in a single batched INSERT and return a :class:~taskq.batch.BatchHandle.

All items share a single batch_id UUID written into each job's metadata.batch_id field (as a string). When batch_id is not supplied it is auto-generated as a UUIDv7 via :func:~taskq._ids.new_job_id.

failure_policy:

When failure_policy is set (e.g. :class:~taskq.batch_policy.AbortBatchAfter), a batches row is created with the policy's failure threshold. After each child job reaches a terminal state, the :func:~taskq.batch.apply_batch_terminal_outcome hook inspects the outcome: succeeded resets the consecutive-failure counter; failed increments it and aborts the batch if the threshold is reached. Aborting cancels all pending and scheduled child jobs and sets the batch row to aborted.

finalizer:

When finalizer is set, a finalizer job is enqueued alongside the batch. The finalizer is NOT stamped with batch_id metadata (deadlock prevention — if it were, wait_for_batch would count it as a child and the finalizer would wait for itself). The batch row's finalizer_job_id column records the link, and wait_for_batch automatically excludes that job from counts. The finalizer is dispatched immediately; the in-actor wait_for_batch snooze pattern gates on child-job completion.

Transactional enqueue:

When failure_policy or finalizer is set and connection is None, the entire operation (batch row + all child jobs + finalizer) is inserted in a single transaction via :meth:Backend.enqueue_batch_atomic. If any insert fails, no rows are committed. When a connection is provided, the caller controls the transaction boundary; the batch row and finalizer are created as the last statements on that connection.

Validation rules:

• len(items) == 0 raises :class:ValueError.
• len(items) > MAX_BATCH_SIZE raises :class:ValueError.
• ALL payloads are validated before any INSERT. A single failure raises :class:~taskq.exceptions.PayloadValidationError and leaves no rows inserted.

max_pending:

One aggregated SELECT actor, count(*) … WHERE actor = ANY($1) GROUP BY actor is issued for the entire batch. Per-actor effective limits (operator-owned stored value when set, else the @actor(...) literal — same resolution as :meth:enqueue) are checked before the INSERT; any violation raises :class:~taskq.exceptions.MaxPendingExceededError.

idempotency_key collisions:

Items whose idempotency_key collides with an existing row return the existing :class:~taskq.client.JobHandle (same semantics as single-item :meth:enqueue).

Source code in src/taskq/client/_jobs.py

async def enqueue_batch(
    self,
    items: list[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: "asyncpg.Connection | None" = None,
    failure_policy: BatchFailurePolicy | None = None,
    finalizer: EnqueueItem | None = None,
) -> BatchHandle:
    """Enqueue multiple jobs in a single batched INSERT and return a
    :class:`~taskq.batch.BatchHandle`.

    All ``items`` share a single ``batch_id`` UUID written into each
    job's ``metadata.batch_id`` field (as a string).  When
    ``batch_id`` is not supplied it is auto-generated as a UUIDv7 via
    :func:`~taskq._ids.new_job_id`.

    **failure_policy:**

    When ``failure_policy`` is set (e.g.
    :class:`~taskq.batch_policy.AbortBatchAfter`), a ``batches`` row
    is created with the policy's failure threshold. After each child
    job reaches a terminal state, the
    :func:`~taskq.batch.apply_batch_terminal_outcome` hook inspects
    the outcome: ``succeeded`` resets the consecutive-failure
    counter; ``failed`` increments it and aborts the batch if the
    threshold is reached. Aborting cancels all pending and scheduled
    child jobs and sets the batch row to ``aborted``.

    **finalizer:**

    When ``finalizer`` is set, a finalizer job is enqueued alongside
    the batch. The finalizer is NOT stamped with ``batch_id``
    metadata (deadlock prevention — if it were, ``wait_for_batch``
    would count it as a child and the finalizer would wait for
    itself). The batch row's ``finalizer_job_id`` column records the
    link, and ``wait_for_batch`` automatically excludes that job
    from counts. The finalizer is dispatched immediately; the
    in-actor ``wait_for_batch`` snooze pattern gates on child-job
    completion.

    **Transactional enqueue:**

    When ``failure_policy`` or ``finalizer`` is set and
    ``connection`` is ``None``, the entire operation (batch row +
    all child jobs + finalizer) is inserted in a single transaction
    via :meth:`Backend.enqueue_batch_atomic`. If any insert fails,
    no rows are committed. When a ``connection`` is provided, the
    caller controls the transaction boundary; the batch row and
    finalizer are created as the last statements on that connection.

    **Validation rules:**

    - ``len(items) == 0`` raises :class:`ValueError`.
    - ``len(items) > MAX_BATCH_SIZE`` raises :class:`ValueError`.
    - ALL payloads are validated before any INSERT.  A single failure
      raises :class:`~taskq.exceptions.PayloadValidationError` and
      leaves no rows inserted.

    **max_pending:**

    One aggregated ``SELECT actor, count(*) … WHERE actor = ANY($1)
    GROUP BY actor`` is issued for the entire batch.  Per-actor
    effective limits (operator-owned stored value when set, else the
    ``@actor(...)`` literal — same resolution as :meth:`enqueue`)
    are checked before the INSERT; any violation raises
    :class:`~taskq.exceptions.MaxPendingExceededError`.

    **idempotency_key collisions:**

    Items whose ``idempotency_key`` collides with an existing row
    return the existing :class:`~taskq.client.JobHandle` (same
    semantics as single-item :meth:`enqueue`).
    """
    from taskq._ids import new_job_id

    if len(items) == 0:
        raise ValueError("items must not be empty")
    if len(items) > MAX_BATCH_SIZE:
        raise ValueError(
            f"items must contain at most {MAX_BATCH_SIZE} entries, got {len(items)}"
        )

    # Auto-generate batch_id if not provided (UUIDv7)
    resolved_batch_id = UUID(bytes=new_job_id().bytes) if batch_id is None else batch_id

    # Phase 1: Validate ALL payloads (and idempotency keys) before any I/O
    for i, item in enumerate(items):
        ref = item.actor_ref
        validate_actor_payload(ref.payload_type, item.payload, actor=ref.name)
        validate_idempotency(
            item.idempotency_key,
            item.idempotency_scope,
            self._idempotency_max_bytes,
            where=f" for item {i}",
        )

    # Phase 2: Aggregated max_pending check (one query for the whole batch)
    # Resolve the effective limit per actor (stored value wins over the
    # @actor literal — same resolution as enqueue), then count.
    effective_mp: dict[str, int | None] = {}
    for item in items:
        ref = item.actor_ref
        if ref.name not in effective_mp:
            effective_mp[ref.name] = await self._capacity_cache.effective_max_pending(
                ref.name, ref.max_pending
            )
    actors_with_limit = {name: mp for name, mp in effective_mp.items() if mp is not None}

    if actors_with_limit:
        # One aggregated query for all actors that declare max_pending.
        existing_counts = await self._backend.count_pending_jobs(list(actors_with_limit.keys()))
        for actor_name, limit in actors_with_limit.items():
            batch_count = sum(1 for it in items if it.actor_ref.name == actor_name)
            existing_pending_count = existing_counts.get(actor_name, 0)
            # M1: use > (not >=) so a batch that fills the queue exactly
            # to the limit is admitted, matching single-enqueue semantics
            # where current_count >= max_pending rejects (i.e. +1 > limit).
            if existing_pending_count + batch_count > limit:
                from taskq.exceptions import MaxPendingExceededError

                raise MaxPendingExceededError(
                    actor=actor_name,
                    current_count=existing_pending_count,
                    max_pending=limit,
                )

    # Phase 3: Build per-item EnqueueArgs — carrying the resolved
    # limits so a per-item backend check enforces the same value the
    # aggregated check just admitted.
    args_list = build_batch_args(items, resolved_batch_id, max_pending_by_actor=effective_mp)

    queue = items[0].actor_ref.queue
    has_batch_extras = failure_policy is not None or finalizer is not None

    # Build finalizer EnqueueArgs (without batch_id stamping — deadlock prevention).
    finalizer_args: EnqueueArgs | None = None
    if finalizer is not None:
        finalizer_args = build_enqueue_args(
            finalizer.actor_ref,
            finalizer.payload,
            scheduled_at=finalizer.scheduled_at,
            priority=finalizer.priority,
            fairness_key=finalizer.fairness_key,
            identity_key=finalizer.identity_key,
            idempotency_key=finalizer.idempotency_key,
            idempotency_scope=finalizer.idempotency_scope,
            metadata=dict(finalizer.metadata),
            start_to_close=finalizer.start_to_close,
            tags=finalizer.tags,
            idempotency_max_bytes=self._idempotency_max_bytes,
        )

    # Build BatchRow when failure_policy OR finalizer is set (C3:
    # finalizer-only batches also need a row for list_batches
    # discoverability and finalizer_job_id auto-exclusion in
    # wait_for_batch). When only finalizer is set, failure_threshold=None.
    batch_row: BatchRow | None = None
    if failure_policy is not None or finalizer is not None:
        threshold = failure_policy.failure_threshold if failure_policy is not None else None
        batch_row = BatchRow(
            id=resolved_batch_id,
            queue=queue,
            status="active",
            expected_size=len(items),
            consecutive_failures=0,
            failure_threshold=threshold,
            finalizer_job_id=finalizer_args.id if finalizer_args is not None else None,
            originating_actor=None,
            created_at=self._clock.now(),
            completed_at=None,
            metadata={},
        )

    if has_batch_extras and connection is None:
        # Autonomous atomic path: delegate to backend.enqueue_batch_atomic.
        all_rows = await self._backend.enqueue_batch_atomic(
            args_list,
            batch_id=resolved_batch_id,
            queue=queue,
            batch_row=batch_row,
            finalizer_args=finalizer_args,
        )
        rows = all_rows[: len(items)]
        finalizer_row = all_rows[len(items) :][0] if finalizer is not None else None
    else:
        # Caller-owned transaction or no extras: use regular enqueue_batch.
        # M3: create the batch row BEFORE job inserts so the row exists
        # when the first terminal write triggers the batch hook (on an
        # autocommit conn, jobs can dispatch and fail before the row
        # exists otherwise). Insert the finalizer first so its returned
        # row id is known for finalizer_job_id (M4: idempotency collision
        # may return a different id than finalizer_args.id).
        finalizer_row = None
        if finalizer is not None:
            assert finalizer_args is not None
            finalizer_row = await self._backend.enqueue_with_conn(connection, finalizer_args)  # type: ignore[arg-type]  # Why: guarded by has_batch_extras; when connection is provided it is runtime-compatible
        if batch_row is not None:
            if finalizer_row is not None:
                batch_row = replace(batch_row, finalizer_job_id=finalizer_row.id)
            await self._backend.create_batch(
                batch_row.id,
                batch_row.queue,
                batch_row.expected_size,
                batch_row.failure_threshold,
                batch_row.finalizer_job_id,
                batch_row.originating_actor,
                connection=connection,  # type: ignore[arg-type]  # Why: connection may be None but create_batch handles that
            )
        rows = await self._backend.enqueue_batch(args_list, connection=connection)  # type: ignore[call-arg]  # Why: asyncpg.Connection is compatible with the protocol's connection parameter at runtime

    # Phase 5: Wrap rows in JobHandles
    handles: list[JobHandle[BaseModel | None]] = []
    for i, row in enumerate(rows):
        args = args_list[i]
        handle: JobHandle[BaseModel | None] = JobHandle(
            client=self,
            row=row,
            result_adapter=items[i].actor_ref.result_adapter,
            was_existing=(row.id != args.id),
            _redis_client=self._redis_client,
            _settings=self._settings,
        )
        handles.append(handle)

    # Build finalizer handle (if any) and append to job_handles for backward compat.
    finalizer_handle: JobHandle[BaseModel | None] | None = None
    if finalizer is not None and finalizer_row is not None:
        finalizer_handle = JobHandle(
            client=self,
            row=finalizer_row,
            result_adapter=finalizer.actor_ref.result_adapter,
            was_existing=(finalizer_row.id != finalizer_args.id)
            if finalizer_args is not None
            else False,
            _redis_client=self._redis_client,
            _settings=self._settings,
        )
        handles.append(finalizer_handle)

    logger.debug(
        "batch_enqueued",
        kind="batch_enqueued",
        batch_id=str(resolved_batch_id),
        size=len(items),
    )

    return BatchHandle(
        batch_id=resolved_batch_id,
        job_handles=handles,
        size=len(items),
        finalizer_handle=finalizer_handle,
    )

enqueue_batch_streaming async

enqueue_batch_streaming(
    items: Iterable[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: Connection | None = None,
    failure_policy: BatchFailurePolicy | None = None,
    finalizer: EnqueueItem | None = None,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
) -> BatchHandle

Enqueue jobs from a lazy iterable in chunks, returning a single :class:~taskq.batch.BatchHandle.

Unlike :meth:enqueue_batch, this method accepts an :class:~collections.abc.Iterable (including generators) and inserts in chunks of chunk_size (1-MAX_BATCH_SIZE). All items share the same batch_id. Payloads are validated on the fly as each chunk is built.

When failure_policy or finalizer is set and connection is None, the entire operation is delegated to :meth:Backend.enqueue_batch_atomic for single-transaction atomicity. Otherwise, chunks are inserted via :meth:Backend.enqueue_batch on the caller-owned connection, and the batch row + finalizer are created as the last statements.

Failure-policy counting limitation (caller-connection path): the batch row is created AFTER all chunk inserts — it must carry the final expected_size and create_batch is INSERT, not upsert — so a child job enqueued on that connection that reaches a terminal state BEFORE the row exists is not counted toward failure_policy: increment_batch_failures finds no row and returns (0, None, 0). The atomic (no-connection) path is unaffected — its single transaction makes the batch row and the child jobs visible together.

max_pending: NOT enforced on this path — unlike :meth:enqueue_batch, which runs one aggregated per-actor check before the INSERT, neither the chunked :meth:Backend.enqueue_batch inserts nor the atomic delegation consult max_pending. The caller is responsible for ensuring the stream will not exceed actor limits (the same bulk-import semantics :meth:enqueue_batch_fast discloses).

Source code in src/taskq/client/_jobs.py

async def enqueue_batch_streaming(
    self,
    items: Iterable[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: "asyncpg.Connection | None" = None,
    failure_policy: BatchFailurePolicy | None = None,
    finalizer: EnqueueItem | None = None,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
) -> BatchHandle:
    """Enqueue jobs from a lazy iterable in chunks, returning a single
    :class:`~taskq.batch.BatchHandle`.

    Unlike :meth:`enqueue_batch`, this method accepts an
    :class:`~collections.abc.Iterable` (including generators) and
    inserts in chunks of ``chunk_size`` (1-MAX_BATCH_SIZE).  All items share
    the same ``batch_id``.  Payloads are validated on the fly as
    each chunk is built.

    When ``failure_policy`` or ``finalizer`` is set and
    ``connection`` is ``None``, the entire operation is delegated to
    :meth:`Backend.enqueue_batch_atomic` for single-transaction
    atomicity.  Otherwise, chunks are inserted via
    :meth:`Backend.enqueue_batch` on the caller-owned connection,
    and the batch row + finalizer are created as the last
    statements.

    **Failure-policy counting limitation (caller-connection path):**
    the batch row is created AFTER all chunk inserts — it must carry
    the final ``expected_size`` and ``create_batch`` is INSERT, not
    upsert — so a child job enqueued on that connection that
    reaches a terminal state BEFORE the row exists is not counted
    toward ``failure_policy``: ``increment_batch_failures`` finds
    no row and returns ``(0, None, 0)``. The atomic
    (no-connection) path is unaffected — its single transaction
    makes the batch row and the child jobs visible together.

    **max_pending:** NOT enforced on this path — unlike
    :meth:`enqueue_batch`, which runs one aggregated per-actor
    check before the INSERT, neither the chunked
    :meth:`Backend.enqueue_batch` inserts nor the atomic delegation
    consult ``max_pending``. The caller is responsible for ensuring
    the stream will not exceed actor limits (the same bulk-import
    semantics :meth:`enqueue_batch_fast` discloses).
    """
    if chunk_size < 1 or chunk_size > MAX_BATCH_SIZE:
        raise ValueError(f"chunk_size must be in [1, {MAX_BATCH_SIZE}], got {chunk_size}")

    from taskq._ids import new_job_id

    resolved_batch_id = UUID(bytes=new_job_id().bytes) if batch_id is None else batch_id

    # Peek the iterable — empty raises ValueError.
    it = iter(items)
    try:
        first_item = next(it)
    except StopIteration:
        raise ValueError("items must not be empty") from None

    # Re-chain the first item back into the stream.
    def _chain() -> Iterable[EnqueueItem]:
        yield first_item
        yield from it

    has_batch_extras = failure_policy is not None or finalizer is not None

    # Build finalizer args (without batch_id stamping).
    finalizer_args: EnqueueArgs | None = None
    if finalizer is not None:
        finalizer_args = build_enqueue_args(
            finalizer.actor_ref,
            finalizer.payload,
            scheduled_at=finalizer.scheduled_at,
            priority=finalizer.priority,
            fairness_key=finalizer.fairness_key,
            identity_key=finalizer.identity_key,
            idempotency_key=finalizer.idempotency_key,
            idempotency_scope=finalizer.idempotency_scope,
            metadata=dict(finalizer.metadata),
            start_to_close=finalizer.start_to_close,
            tags=finalizer.tags,
            idempotency_max_bytes=self._idempotency_max_bytes,
        )

    # Build a lazy generator of EnqueueArgs, validating payloads on the fly.
    # H4: collect per-item (actor_ref, args_id) as a side effect so handles
    # can be paired by index after the backend returns rows. This avoids
    # using a single actor's result_adapter for all handles (mixed-actor
    # batches would get wrong deserialization).
    item_meta: list[tuple[ActorRef[Any, Any], JobId]] = []

    def _lazy_args(stream: Iterable[EnqueueItem]) -> Iterable[EnqueueArgs]:
        for idx, item in enumerate(stream):
            ref = item.actor_ref
            try:
                ref.payload_type.model_validate(item.payload)
            except ValidationError as exc:
                errs: list[dict[str, object]] = exc.errors()  # type: ignore[assignment]  # Why: pydantic v2 ErrorDetails is a TypedDict (subtype of dict[str, Any]); assignment to list[dict[str,object]] is safe at runtime but pyright cannot prove covariance
                raise PayloadValidationError(
                    f"Payload validation failed for item {idx} (actor={ref.name!r}): {exc}",
                    actor=ref.name,
                    validation_errors=errs,
                ) from exc
            args = build_enqueue_args(
                ref,
                item.payload,
                scheduled_at=item.scheduled_at,
                priority=item.priority,
                fairness_key=item.fairness_key,
                identity_key=item.identity_key,
                idempotency_key=item.idempotency_key,
                idempotency_scope=item.idempotency_scope,
                metadata=dict(item.metadata),
                start_to_close=item.start_to_close,
                tags=item.tags,
                idempotency_max_bytes=self._idempotency_max_bytes,
            )
            # Stamp batch_id AFTER build_enqueue_args, which strips any
            # caller-supplied batch_id as a security boundary (H5).
            args = replace(
                args,
                metadata={**args.metadata, "batch_id": str(resolved_batch_id)},
            )
            item_meta.append((ref, args.id))
            yield args

    # Determine queue from the first item.
    queue = first_item.actor_ref.queue

    # Build BatchRow when failure_policy OR finalizer is set (C3:
    # finalizer-only batches also need a row for list_batches
    # discoverability and finalizer_job_id auto-exclusion in
    # wait_for_batch). When only finalizer is set, failure_threshold=None.
    # expected_size=0 is a sentinel — the backend computes the real count
    # from the iterable (H6: no materialization).
    batch_row: BatchRow | None = None
    if failure_policy is not None or finalizer is not None:
        threshold = failure_policy.failure_threshold if failure_policy is not None else None
        batch_row = BatchRow(
            id=resolved_batch_id,
            queue=queue,
            status="active",
            expected_size=0,
            consecutive_failures=0,
            failure_threshold=threshold,
            finalizer_job_id=finalizer_args.id if finalizer_args is not None else None,
            originating_actor=None,
            created_at=self._clock.now(),
            completed_at=None,
            metadata={},
        )

    all_handles: list[JobHandle[BaseModel | None]] = []
    total_count = 0

    if has_batch_extras and connection is None:
        # Autonomous atomic path. H6: do NOT materialize the iterable —
        # pass the lazy generator directly to the backend, which consumes
        # it in chunks inside its transaction. expected_size=0 is a
        # sentinel; the backend computes the real count from the items
        # consumed. H4: item_meta is populated as a side effect of the
        # generator being consumed, providing per-item actor_refs and
        # args_ids for handle pairing.
        all_rows = await self._backend.enqueue_batch_atomic(
            _lazy_args(_chain()),
            batch_id=resolved_batch_id,
            queue=queue,
            batch_row=batch_row,
            finalizer_args=finalizer_args,
            chunk_size=chunk_size,
        )
        non_finalizer_count = len(all_rows) - (1 if finalizer is not None else 0)
        for i in range(non_finalizer_count):
            row = all_rows[i]
            ref, args_id = item_meta[i]
            all_handles.append(
                JobHandle(
                    client=self,
                    row=row,
                    result_adapter=ref.result_adapter,
                    was_existing=(row.id != args_id),
                    _redis_client=self._redis_client,
                    _settings=self._settings,
                )
            )
        total_count = non_finalizer_count
        finalizer_row = all_rows[-1] if finalizer is not None else None
    else:
        # Chunked path (caller-owned connection or no extras). The
        # batch row is created AFTER all chunk inserts: create_batch is
        # INSERT (not upsert), so the row must carry the real
        # expected_size, which is only known once the stream is drained.
        # KNOWN LIMITATION: until the row exists, a child job reaching a
        # terminal state finds no batch row — increment_batch_failures
        # returns (0, None, 0) and the failure is NOT counted toward
        # failure_policy (see the docstring disclosure above). Insert
        # the finalizer first so its returned row id is known for
        # finalizer_job_id (M4).
        stream = _chain()
        finalizer_row = None
        if finalizer is not None:
            assert finalizer_args is not None
            finalizer_row = await self._backend.enqueue_with_conn(connection, finalizer_args)  # type: ignore[arg-type]  # Why: guarded by has_batch_extras; when connection is provided it is runtime-compatible

        # Consume chunks, validating payloads with global index (M6).
        global_idx = 0
        while True:
            chunk_items = list(islice(stream, chunk_size))
            if not chunk_items:
                break
            for ci in chunk_items:
                ref = ci.actor_ref
                try:
                    ref.payload_type.model_validate(ci.payload)
                except ValidationError as exc:
                    errs_v: list[dict[str, object]] = exc.errors()  # type: ignore[assignment]  # Why: pydantic v2 ErrorDetails is a TypedDict; safe at runtime
                    raise PayloadValidationError(
                        f"Payload validation failed for item {global_idx} "
                        f"(actor={ref.name!r}): {exc}",
                        actor=ref.name,
                        validation_errors=errs_v,
                    ) from exc
                global_idx += 1
            chunk_args = build_batch_args(chunk_items, resolved_batch_id)
            chunk_rows = await self._backend.enqueue_batch(chunk_args, connection=connection)  # type: ignore[call-arg]  # Why: asyncpg.Connection is compatible with the protocol's connection parameter at runtime
            for i, row in enumerate(chunk_rows):
                all_handles.append(
                    JobHandle(
                        client=self,
                        row=row,
                        result_adapter=chunk_items[i].actor_ref.result_adapter,
                        was_existing=(row.id != chunk_args[i].id),
                        _redis_client=self._redis_client,
                        _settings=self._settings,
                    )
                )
            total_count += len(chunk_items)

        if batch_row is not None:
            if finalizer_row is not None:
                batch_row = replace(
                    batch_row, finalizer_job_id=finalizer_row.id, expected_size=total_count
                )
            else:
                batch_row = replace(batch_row, expected_size=total_count)
            await self._backend.create_batch(
                batch_row.id,
                batch_row.queue,
                batch_row.expected_size,
                batch_row.failure_threshold,
                batch_row.finalizer_job_id,
                batch_row.originating_actor,
                connection=connection,  # type: ignore[arg-type]  # Why: connection may be None but create_batch handles that
            )

    # Build finalizer handle.
    finalizer_handle: JobHandle[BaseModel | None] | None = None
    if finalizer is not None and finalizer_row is not None:
        finalizer_handle = JobHandle(
            client=self,
            row=finalizer_row,
            result_adapter=finalizer.actor_ref.result_adapter,
            was_existing=(finalizer_row.id != finalizer_args.id)
            if finalizer_args is not None
            else False,
            _redis_client=self._redis_client,
            _settings=self._settings,
        )
        all_handles.append(finalizer_handle)

    logger.debug(
        "batch-streaming-enqueued",
        kind="batch-streaming-enqueued",
        batch_id=str(resolved_batch_id),
        size=total_count,
    )

    return BatchHandle(
        batch_id=resolved_batch_id,
        job_handles=all_handles,
        size=total_count,
        finalizer_handle=finalizer_handle,
    )

get_batch async

get_batch(batch_id: UUID) -> BatchRow | None

Fetch a single batch row by ID.

Delegates to :meth:Backend.get_batch. Returns None when the batch does not exist.

Source code in src/taskq/client/_jobs.py

async def get_batch(self, batch_id: UUID) -> BatchRow | None:
    """Fetch a single batch row by ID.

    Delegates to :meth:`Backend.get_batch`. Returns ``None`` when the
    batch does not exist.
    """
    return await self._backend.get_batch(batch_id)

list_batches async

list_batches(filter: BatchFilter) -> list[BatchSummary]

List batches matching filter, returning :class:BatchSummary objects.

Delegates to :meth:Backend.list_batches and maps each (BatchRow, BatchCounts) pair to a :class:BatchSummary with a :class:BatchCompletionStatus derived from the live counts.

Source code in src/taskq/client/_jobs.py

async def list_batches(
    self,
    filter: BatchFilter,
) -> list[BatchSummary]:
    """List batches matching *filter*, returning :class:`BatchSummary` objects.

    Delegates to :meth:`Backend.list_batches` and maps each
    ``(BatchRow, BatchCounts)`` pair to a :class:`BatchSummary`
    with a :class:`BatchCompletionStatus` derived from the live counts.
    """
    from taskq.batch import BatchCompletionStatus

    pairs = await self._backend.list_batches(filter)
    summaries: list[BatchSummary] = []
    for row, counts in pairs:
        completion = BatchCompletionStatus(
            total=counts.total,
            pending=counts.pending,
            succeeded=counts.succeeded,
            failed=counts.failed,
            cancelled=counts.cancelled,
            crashed=counts.crashed,
            abandoned=counts.abandoned,
        )
        summaries.append(
            BatchSummary(
                batch_id=row.id,
                queue=row.queue,
                status=row.status,
                expected_size=row.expected_size,
                consecutive_failures=row.consecutive_failures,
                failure_threshold=row.failure_threshold,
                finalizer_job_id=row.finalizer_job_id,
                originating_actor=row.originating_actor,
                created_at=row.created_at,
                completed_at=row.completed_at,
                completion=completion,
            )
        )
    return summaries

enqueue_batch_fast async

enqueue_batch_fast(
    items: list[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: Connection | None = None,
) -> int

Enqueue jobs via COPY FROM protocol for maximum throughput.

WARNING — bulk-import semantics, not general-purpose enqueue: this method does NOT enforce max_pending, does NOT detect or reject idempotency-key collisions (a duplicate key aborts the whole batch instead of being treated as "already enqueued"), and returns a bare row count, not per-job handles — there is no way to await, cancel, or otherwise reference an individual job from the return value. Use :meth:enqueue_batch unless you specifically need COPY-level throughput for a one-shot bulk import/backfill and have already accounted for these gaps.

Returns the count of inserted rows — no :class:~taskq.batch.BatchHandle, no per-job :class:~taskq.client.JobHandle instances.

Validation rules:

• len(items) == 0 raises :class:ValueError.
• len(items) > 50_000 raises :class:ValueError.
• ALL payloads are validated before any INSERT — a single failure raises :class:~taskq.exceptions.PayloadValidationError.

Tradeoffs vs enqueue_batch:

• No idempotency-key collision handling. A duplicate key aborts the entire batch with asyncpg.UniqueViolationError. Callers must pre-deduplicate. One carve-out: during the 01.00.03 pre→post migration window, a key reused across different scopes raises :class:~taskq.exceptions.ScopedIdempotencyMigrationPendingError instead, matching the other enqueue paths.
• No max_pending check. The caller is responsible for ensuring the batch won't exceed actor limits.
• No JobHandle instances. Only the inserted row count is returned. Use batch_id to query rows post-insert.
• All-or-nothing atomicity. No partial success — the entire COPY fails on any constraint violation.

Use for bulk import / backfill with 1K-50K rows where throughput matters more than idempotency guarantees.

Source code in src/taskq/client/_jobs.py

async def enqueue_batch_fast(
    self,
    items: list[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: "asyncpg.Connection | None" = None,
) -> int:
    """Enqueue jobs via COPY FROM protocol for maximum throughput.

    **WARNING — bulk-import semantics, not general-purpose enqueue:**
    this method does NOT enforce ``max_pending``, does NOT detect or
    reject idempotency-key collisions (a duplicate key aborts the
    whole batch instead of being treated as "already enqueued"), and
    returns a bare row **count**, not per-job handles — there is no
    way to await, cancel, or otherwise reference an individual job
    from the return value. Use :meth:`enqueue_batch` unless you
    specifically need COPY-level throughput for a one-shot bulk
    import/backfill and have already accounted for these gaps.

    Returns the count of inserted rows — no :class:`~taskq.batch.BatchHandle`,
    no per-job :class:`~taskq.client.JobHandle` instances.

    **Validation rules:**

    - ``len(items) == 0`` raises :class:`ValueError`.
    - ``len(items) > 50_000`` raises :class:`ValueError`.
    - ALL payloads are validated before any INSERT — a single failure
      raises :class:`~taskq.exceptions.PayloadValidationError`.

    **Tradeoffs vs enqueue_batch:**

    - **No idempotency-key collision handling.** A duplicate key
      aborts the entire batch with ``asyncpg.UniqueViolationError``.
      Callers must pre-deduplicate. One carve-out: during the
      ``01.00.03`` pre→post migration window, a key reused across
      *different* scopes raises
      :class:`~taskq.exceptions.ScopedIdempotencyMigrationPendingError`
      instead, matching the other enqueue paths.
    - **No max_pending check.** The caller is responsible for
      ensuring the batch won't exceed actor limits.
    - **No JobHandle instances.** Only the inserted row count is
      returned.  Use ``batch_id`` to query rows post-insert.
    - **All-or-nothing atomicity.** No partial success — the entire
      COPY fails on any constraint violation.

    Use for bulk import / backfill with 1K-50K rows where throughput
    matters more than idempotency guarantees.
    """
    from taskq._ids import new_job_id

    if len(items) == 0:
        raise ValueError("items must not be empty")
    if len(items) > 50_000:
        raise ValueError(f"items must contain at most 50 000 entries, got {len(items)}")

    # Auto-generate batch_id if not provided (UUIDv7)
    resolved_batch_id = UUID(bytes=new_job_id().bytes) if batch_id is None else batch_id

    # Phase 1: Validate ALL payloads before any I/O
    for item in items:
        ref = item.actor_ref
        validate_actor_payload(ref.payload_type, item.payload, actor=ref.name)

    # Phase 2: Build per-item EnqueueArgs
    args_list = build_batch_args(items, resolved_batch_id)

    # Phase 3: COPY FROM via backend
    count = await self._backend.enqueue_batch_fast(args_list, connection=connection)

    logger.debug(
        "batch_fast_enqueued",
        kind="batch_fast_enqueued",
        batch_id=str(resolved_batch_id),
        count=count,
    )

    return count

get async

get(
    job_id: JobId,
    *,
    result_adapter: TypeAdapter[R] | None = None,
) -> JobHandle[R] | None

Look up a job by id.

Returns None when the job does not exist; otherwise wraps the row in a :class:JobHandle[R]. The caller may supply result_adapter because lookups by id do not carry actor identity — typical sources are my_actor.result_adapter (when reuniting with an actor) or TypeAdapter(type(None)) (when only row metadata is needed). When result_adapter is None it defaults to TypeAdapter(type(None)), which is suitable for status-only lookups.

Source code in src/taskq/client/_jobs.py

async def get[R: BaseModel | None](
    self,
    job_id: JobId,
    *,
    result_adapter: TypeAdapter[R] | None = None,
) -> JobHandle[R] | None:
    """Look up a job by id.

    Returns ``None`` when the job does not exist; otherwise wraps
    the row in a :class:`JobHandle[R]`. The caller may supply
    ``result_adapter`` because lookups by id do not carry actor
    identity — typical sources are
    ``my_actor.result_adapter`` (when reuniting with an actor) or
    ``TypeAdapter(type(None))`` (when only row metadata is needed).
    When *result_adapter* is ``None`` it defaults to
    ``TypeAdapter(type(None))``, which is suitable for status-only
    lookups.
    """
    adapter: TypeAdapter[R] = (
        result_adapter if result_adapter is not None else TypeAdapter(type(None))
    )  # type: ignore[assignment]  # Why: TypeAdapter(type(None)) returns TypeAdapter[None], which does not narrow to TypeAdapter[R] under pyright; runtime behaviour is correct because None is assignable to the R bound
    with self._translate_schema_errors():
        row = await self._backend.get(job_id)
    if row is None:
        return None
    return JobHandle(
        client=self,
        row=row,
        result_adapter=adapter,
        was_existing=False,
        _redis_client=self._redis_client,
        _settings=self._settings,
    )

get_row async

get_row(job_id: JobId) -> JobRow | None

Look up a job by id and return the raw :class:JobRow.

Mirrors :meth:get's contract — one backend.get, None when the job does not exist — without the handle machinery or result adapter. For callers that never need a :class:JobHandle, this is the direct form; for the fresh-read case that does want a handle, prefer get plus the handle's row property (still a single round trip).

Source code in src/taskq/client/_jobs.py

async def get_row(self, job_id: JobId) -> JobRow | None:
    """Look up a job by id and return the raw :class:`JobRow`.

    Mirrors :meth:`get`'s contract — one ``backend.get``, ``None``
    when the job does not exist — without the handle machinery or
    result adapter. For callers that never need a
    :class:`JobHandle`, this is the direct form; for the fresh-read
    case that does want a handle, prefer ``get`` plus the handle's
    ``row`` property (still a single round trip).
    """
    with self._translate_schema_errors():
        return await self._backend.get(job_id)

list async

list(filter: JobFilter) -> JobPage

List jobs matching filter, returning a :class:JobPage.

filter.status accepts a single :data:JobStatus or a sequence of statuses (e.g. JobFilter(status=["pending", "running"])).

filter.active is a meta-filter — not Celery's 'active': active=True selects non-terminal statuses (pending, scheduled, running — 'not yet finished', not 'currently executing') and active=False selects terminal ones. See :class:JobFilter for full semantics.

next_cursor is returned for every ordering, encoded from the columns that ordering actually sorts by, and is only None on the last page.

Source code in src/taskq/client/_jobs.py

async def list(self, filter: JobFilter) -> JobPage:
    """List jobs matching *filter*, returning a :class:`JobPage`.

    ``filter.status`` accepts a single :data:`JobStatus` or a
    sequence of statuses (e.g. ``JobFilter(status=["pending",
    "running"])``).

    ``filter.active`` is a meta-filter — **not Celery's 'active'**:
    ``active=True`` selects *non-terminal* statuses (pending,
    scheduled, running — 'not yet finished', not 'currently
    executing') and ``active=False`` selects terminal ones.  See
    :class:`JobFilter` for full semantics.

    ``next_cursor`` is returned for every ordering, encoded from the
    columns that ordering actually sorts by, and is only ``None`` on
    the last page.
    """
    with self._translate_schema_errors():
        rows = await self._backend.list_jobs(filter)
    next_cursor: str | None = None
    if rows and len(rows) == filter.limit:
        next_cursor = encode_job_cursor(rows[-1], filter.order_by)
    return JobPage(jobs=rows, next_cursor=next_cursor)

cancel async

cancel(
    job_id: JobId, reason: str | None = None
) -> CancelResult

Request cancellation of a job and return a :class:CancelResult.

Reads the row first via :meth:Backend.get. If the job does not exist, raises :class:KeyError — matching Python's stdlib idiom for "asked for an entry by id; it isn't there".

Then calls :meth:Backend.write_cancel_request and reads the row again to capture the new status. The previous_status reflects the row at the first read, not atomically at write-time (TOCTOU per ).

: increments taskq.cancellation.requested exactly once per call, regardless of cancellation_initiated outcome.

Source code in src/taskq/client/_jobs.py

async def cancel(
    self,
    job_id: JobId,
    reason: str | None = None,
) -> CancelResult:
    """Request cancellation of a job and return a :class:`CancelResult`.

    Reads the row first via :meth:`Backend.get`. If the job does not
    exist, raises :class:`KeyError` — matching Python's stdlib
    idiom for "asked for an entry by id; it isn't there".

    Then calls :meth:`Backend.write_cancel_request` and reads the
    row again to capture the new status. The ``previous_status``
    reflects the row at the first read, not atomically at
    write-time (TOCTOU per  ).

    : increments ``taskq.cancellation.requested`` exactly once
    per call, regardless of ``cancellation_initiated`` outcome.
    """
    from taskq.obs import record_cancel_requested

    record_cancel_requested()

    with self._translate_schema_errors():
        row = await self._backend.get(job_id)
    if row is None:
        raise KeyError(job_id)

    previous_status = row.status
    with self._translate_schema_errors():
        initiated = await self._backend.write_cancel_request(job_id, reason)
        new_row = await self._backend.get(job_id)
    if new_row is None:
        msg = (
            f"job {job_id} disappeared after write_cancel_request; "
            "the row existed a moment ago and a write was issued against it"
        )
        raise RuntimeError(msg)
    new_status = new_row.status

    result = CancelResult(
        job_id=job_id,
        previous_status=previous_status,
        new_status=new_status,
        cancellation_initiated=initiated,
    )
    logger.debug(
        "cancel_requested",
        kind="cancel_requested",
        job_id=str(job_id),
        previous_status=previous_status,
        cancellation_initiated=initiated,
    )
    return result

cancel_where async

cancel_where(
    filter: JobFilter,
    reason: str | None = None,
    *,
    allow_empty_filter: bool = False,
) -> BulkCancelResult

Cancel all jobs matching filter in a single set-based operation.

Pending/scheduled jobs are moved straight to terminal 'cancelled' (no running actor to cooperate with). Running jobs get cancel_phase=1 set (cooperative cancel) — the worker's heartbeat-driven cancel controller observes the phase change and sets the in-process cancel_event.

Guardrail: a filter with no predicates (no queue, status, actor, identity_key, batch_id, tags, or active) is rejected with :class:EmptyFilterError unless allow_empty_filter=True is passed.

Filter fields used: queue, status, actor, identity_key, batch_id, tags, active. The limit, cursor, and order_by fields are ignored.

Returns a :class:BulkCancelResult with counts and affected IDs.

Source code in src/taskq/client/_jobs.py

async def cancel_where(
    self,
    filter: JobFilter,
    reason: str | None = None,
    *,
    allow_empty_filter: bool = False,
) -> BulkCancelResult:
    """Cancel all jobs matching *filter* in a single set-based operation.

    Pending/scheduled jobs are moved straight to terminal 'cancelled'
    (no running actor to cooperate with). Running jobs get
    ``cancel_phase=1`` set (cooperative cancel) — the worker's
    heartbeat-driven cancel controller observes the phase change and
    sets the in-process ``cancel_event``.

    **Guardrail:** a filter with no predicates (no queue, status,
    actor, identity_key, batch_id, tags, or active) is rejected with
    :class:`EmptyFilterError` unless ``allow_empty_filter=True`` is
    passed.

    **Filter fields used:** ``queue``, ``status``, ``actor``,
    ``identity_key``, ``batch_id``, ``tags``, ``active``. The
    ``limit``, ``cursor``, and ``order_by`` fields are ignored.

    Returns a :class:`BulkCancelResult` with counts and affected IDs.
    """
    from taskq.obs import record_cancel_requested

    if not allow_empty_filter and not filter.has_predicates():
        raise EmptyFilterError()

    record_cancel_requested()

    with self._translate_schema_errors():
        return await self._backend.cancel_where(filter, reason)

create_schedule async

create_schedule(
    actor: str | ActorRef[P, R],
    cron_expr: str,
    *,
    timezone: str = "UTC",
    dst_strategy: DstStrategy = "skip",
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    name: str = "",
    identity_key: IdentityKey | None = None,
    enabled: bool = True,
) -> ScheduleHandle

Create a cron schedule. Raises :class:ValueError if both payload_factory and static_payload are provided, or if cron_expr is invalid.

The (actor, name) UNIQUE constraint means each (actor, name) pair may have at most one schedule; a second create_schedule for the same pair raises asyncpg.UniqueViolationError (PG) or :class:ValueError (in-memory). Pass distinct name values to run several cron schedules per actor (e.g. a per-property sync).

When identity_key is set, the cron loop propagates it to cron-fired jobs so they dedup against on-demand jobs for the same business key.

Does NOT validate actor existence at creation time — any string actor name is accepted (validation is deferred to fire time).

The first next_fire_at is seeded from the clock that arbitrates its due-check: on a Postgres-backed client the PG server clock is read first (one-row SELECT clock_timestamp() via the backend's pool, mirroring the worker bootstrap), so app↔DB clock skew cannot shift the fire chain; on a pool-less (in-memory) client the seed comes from the client's injected Clock. This matters permanently: the cron loop's normal path recomputes every subsequent fire from the STORED fire time (only a miss beyond cron_catch_up_window re-anchors on the server clock), so the seed — not any per-tick correction — fixes the chain's phase for the schedule's life.

Parameters:

Name	Type	Description	Default
dst_strategy	DstStrategy	How to handle DST gaps and overlaps. skip (default) advances past gaps, uses the first occurrence in overlaps. firstof explicitly selects the earlier wall-clock time in overlaps. allof fires at both occurrences in overlaps.	'skip'

Source code in src/taskq/client/_jobs.py

async def create_schedule[P: BaseModel, R: BaseModel | None](
    self,
    actor: str | ActorRef[P, R],
    cron_expr: str,
    *,
    timezone: str = "UTC",
    dst_strategy: DstStrategy = "skip",
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    name: str = "",
    identity_key: IdentityKey | None = None,
    enabled: bool = True,
) -> "ScheduleHandle":
    """Create a cron schedule.  Raises :class:`ValueError` if both
    *payload_factory* and *static_payload* are provided, or if
    *cron_expr* is invalid.

    The ``(actor, name)`` UNIQUE constraint means each ``(actor, name)``
    pair may have at most one schedule; a second ``create_schedule`` for
    the same pair raises ``asyncpg.UniqueViolationError`` (PG) or
    :class:`ValueError` (in-memory).  Pass distinct *name* values to run
    several cron schedules per actor (e.g. a per-property sync).

    When *identity_key* is set, the cron loop propagates it to cron-fired
    jobs so they dedup against on-demand jobs for the same business key.

    Does NOT validate actor existence at creation time — any string
    actor name is accepted (validation is deferred to fire time).

    The first ``next_fire_at`` is seeded from the clock that
    arbitrates its due-check: on a Postgres-backed client the PG
    server clock is read first (one-row ``SELECT clock_timestamp()``
    via the backend's pool, mirroring the worker bootstrap), so
    app↔DB clock skew cannot shift the fire chain; on a pool-less
    (in-memory) client the seed comes from the client's injected
    Clock. This matters permanently: the cron loop's normal path
    recomputes every subsequent fire from the STORED fire time
    (only a miss beyond ``cron_catch_up_window`` re-anchors on the
    server clock), so the seed — not any per-tick correction —
    fixes the chain's phase for the schedule's life.

    Args:
        dst_strategy: How to handle DST gaps and overlaps.
            ``skip`` (default) advances past gaps, uses the first
            occurrence in overlaps. ``firstof`` explicitly selects
            the earlier wall-clock time in overlaps. ``allof`` fires at
            both occurrences in overlaps.
    """
    from taskq.cron import (
        ScheduleHandle,
        compute_next_fire_after,
    )

    if not croniter.is_valid(cron_expr):
        raise ValueError(f"Invalid cron expression: {cron_expr!r}")
    if payload_factory is not None and static_payload is not None:
        raise ValueError(
            "payload_factory and static_payload are mutually exclusive; "
            "provide one or the other, not both"
        )
    actor_name = actor.name if isinstance(actor, ActorRef) else actor
    # Why: actor is stored as a name string in the DB; payload type is not preserved at the cron-schedule level
    del actor

    metadata: dict[str, object] = {}
    if static_payload is not None:
        metadata["static_payload"] = static_payload

    now = await self._schedule_seed_now()
    next_fire = compute_next_fire_after(cron_expr, timezone, now, dst_strategy=dst_strategy)[0]

    args = ScheduleCreateArgs(
        actor=actor_name,
        cron_expr=cron_expr,
        timezone=timezone,
        next_fire_at=next_fire,
        dst_strategy=dst_strategy,
        payload_factory=payload_factory,
        enabled=enabled,
        name=name,
        identity_key=identity_key,
        metadata=metadata,
    )
    record = await self._backend.create_schedule(args)
    return ScheduleHandle(
        schedule_id=record.id,
        actor=record.actor,
        cron_expr=record.cron_expr,
        timezone=record.timezone,
        dst_strategy=record.dst_strategy,
        enabled=record.enabled,
        next_fire_at=record.next_fire_at,
        name=record.name,
        identity_key=record.identity_key,
        _backend=self._backend,
    )

list_schedules async

list_schedules(
    *, actor: str | None = None, enabled: bool | None = None
) -> list[ScheduleRecord]

List cron schedules, optionally filtered by actor or enabled status.

Source code in src/taskq/client/_jobs.py

async def list_schedules(
    self,
    *,
    actor: str | None = None,
    enabled: bool | None = None,
) -> "list[ScheduleRecord]":
    """List cron schedules, optionally filtered by actor or enabled status."""
    return await self._backend.list_schedules(actor=actor, enabled=enabled)

update_schedule async

update_schedule(
    schedule_id: UUID,
    *,
    cron_expr: str | None = None,
    enabled: bool | None = None,
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    clear_payload_factory: bool = False,
) -> ScheduleRecord

Update a cron schedule. Setting enabled=True clears last_fire_error and resets consecutive_failures to 0.

Raises :class:ValueError if both payload_factory and static_payload are provided, or if cron_expr is invalid.

To explicitly clear payload_factory (set the column to NULL), pass clear_payload_factory=True — None for payload_factory means "don't change this field."

When cron_expr changes, the recomputed next_fire_at is seeded from the same clock as create_schedule (the PG server clock on Postgres-backed clients; the client's injected Clock in-memory) — the stored chain keeps its server-anchored phase.

Source code in src/taskq/client/_jobs.py

async def update_schedule(
    self,
    schedule_id: UUID,
    *,
    cron_expr: str | None = None,
    enabled: bool | None = None,
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    clear_payload_factory: bool = False,
) -> "ScheduleRecord":
    """Update a cron schedule.  Setting ``enabled=True`` clears
    ``last_fire_error`` and resets ``consecutive_failures`` to 0.

    Raises :class:`ValueError` if both *payload_factory* and
    *static_payload* are provided, or if *cron_expr* is invalid.

    To explicitly clear ``payload_factory`` (set the column to NULL),
    pass ``clear_payload_factory=True`` — ``None`` for payload_factory
    means "don't change this field."

    When *cron_expr* changes, the recomputed ``next_fire_at`` is
    seeded from the same clock as ``create_schedule`` (the PG
    server clock on Postgres-backed clients; the client's injected
    Clock in-memory) — the stored chain keeps its server-anchored
    phase.
    """
    from taskq.cron import compute_next_fire_after

    if cron_expr is not None and not croniter.is_valid(cron_expr):
        raise ValueError(f"Invalid cron expression: {cron_expr!r}")
    if payload_factory is not None and static_payload is not None:
        raise ValueError(
            "payload_factory and static_payload are mutually exclusive; "
            "provide one or the other, not both"
        )

    next_fire_at: datetime | None = None
    if cron_expr is not None:
        now = await self._schedule_seed_now()
        records = await self._backend.list_schedules(actor=None, enabled=None)
        existing = next((r for r in records if r.id == schedule_id), None)
        tz = existing.timezone if existing is not None else "UTC"
        next_fire_at = compute_next_fire_after(cron_expr, tz, now)[0]

    metadata: dict[str, object] | None = None
    if static_payload is not None:
        metadata = {"static_payload": static_payload}

    args = ScheduleUpdateArgs(
        cron_expr=cron_expr,
        next_fire_at=next_fire_at,
        enabled=enabled,
        payload_factory=payload_factory,
        clear_payload_factory=clear_payload_factory,
        metadata=metadata,
    )
    return await self._backend.update_schedule(schedule_id, args)

delete_schedule async

delete_schedule(schedule_id: UUID) -> None

Delete a cron schedule by ID. Idempotent — no error if missing.

Source code in src/taskq/client/_jobs.py

async def delete_schedule(self, schedule_id: UUID) -> None:
    """Delete a cron schedule by ID.  Idempotent — no error if missing."""
    await self._backend.delete_schedule(schedule_id)

TaskQ

TaskQ(
    *,
    dsn: str | None = None,
    pool: Pool | None = None,
    pool_factory: PoolFactory | None = None,
    pg_provider: PgCredentialProvider | None = None,
    schema: str = "taskq",
    min_pool_size: int = 1,
    max_pool_size: int = 5,
    redis_url: str | None = None,
    redis_client: Any | None = None,
    pg_conn_factory: ConnFactory | None = None,
    listen_conn: Connection | None = None,
    poll_timeout: float = 30.0,
    reclaim_event_visibility_delay: timedelta | None = None,
)

Postgres-backed TaskQ client.

Manages a connection pool and exposes job operations directly. Supports both the async context manager pattern and explicit open() / close() for frameworks like FastAPI that manage their own lifecycle.

Parameters

dsn: Postgres DSN string. Mutually exclusive with pool. pool: An already-open asyncpg.Pool. The caller retains ownership; close() will not close it. pool_factory: A zero-arg async factory returning an asyncpg.Pool - the same :data:~taskq.connections.PoolFactory the worker takes via :class:~taskq.connections.WorkerConnections. TaskQ invokes it at open() and owns the result (close() closes it), and :meth:reload_credentials re-invokes it to rotate the pool in place. This is the client-side equivalent of a <role>_pool_factory: pair it with :func:taskq.auth.make_pg_pool_factory for a rotating-credential deployment. Mutually exclusive with dsn and pool. pg_provider: A :class:~taskq.auth.PgCredentialProvider. Sugar for pool_factory=make_pg_pool_factory(dsn, pg_provider, min_size=min_pool_size, max_size=max_pool_size) - it is exactly that call and nothing more, so the two paths share one mechanism. Requires dsn; use pool_factory directly when you need the factory's other hooks (init, server_settings, command_timeout). schema: TaskQ schema name. Defaults to "taskq". min_pool_size: Minimum pool connections. Only used when dsn is provided. max_pool_size: Maximum pool connections. Only used when dsn is provided. redis_url: Redis URL string. Mutually exclusive with redis_client. The library creates and owns the Redis client; close() will close it. redis_client: An already-open redis.asyncio.Redis client. The caller retains ownership; close() will not close it. Mutually exclusive with redis_url. pg_conn_factory: A zero-arg async factory returning an asyncpg.Connection for the LISTEN/NOTIFY transport used by :meth:stream. Mutually exclusive with listen_conn. Takes precedence over dsn when set. Use this when you have no DSN (e.g. AAD-managed-identity auth) but still want streaming. TaskQ owns and closes the connection produced by the factory per stream() call. listen_conn: A pre-constructed asyncpg.Connection for the LISTEN transport. Caller-owned; TaskQ does not close it. Mutually exclusive with pg_conn_factory. Takes precedence over dsn when set. Use this to share a dedicated LISTEN conn across callers. poll_timeout: Maximum seconds to wait between transport wakeups before re-fetching job state. Defaults to 30.0.

Source code in src/taskq/client/_taskq.py

def __init__(
    self,
    *,
    dsn: str | None = None,
    pool: "asyncpg.Pool | None" = None,
    pool_factory: "PoolFactory | None" = None,
    pg_provider: "PgCredentialProvider | None" = None,
    schema: str = "taskq",
    min_pool_size: int = 1,
    max_pool_size: int = 5,
    redis_url: str | None = None,
    redis_client: Any | None = None,
    pg_conn_factory: "ConnFactory | None" = None,
    listen_conn: "asyncpg.Connection | None" = None,
    poll_timeout: float = 30.0,
    reclaim_event_visibility_delay: timedelta | None = None,
) -> None:
    if pg_provider is not None:
        if dsn is None:
            raise ValueError(
                "TaskQ 'pg_provider' requires 'dsn' — the provider issues a credential, "
                "not a host. Pass 'pool_factory' instead when there is no DSN."
            )
        if pool_factory is not None:
            raise ValueError("TaskQ accepts 'pg_provider' or 'pool_factory', not both")
        # Why here and not in open(): the factory is the single mechanism
        # (taskq.auth.make_pg_pool_factory, the same builder
        # build_worker_connections uses); pg_provider is sugar that
        # collapses into it, so everything downstream sees one code path.
        from taskq.auth import make_pg_pool_factory

        pool_factory = make_pg_pool_factory(
            dsn, pg_provider, min_size=min_pool_size, max_size=max_pool_size
        )
        dsn = None

    sources = [
        name
        for name, v in (("dsn", dsn), ("pool", pool), ("pool_factory", pool_factory))
        if v is not None
    ]
    if not sources:
        raise ValueError("TaskQ requires one of 'dsn', 'pool' or 'pool_factory'")
    if len(sources) > 1:
        raise ValueError(f"TaskQ accepts one of {sources!r}, not both")
    if redis_url is not None and redis_client is not None:
        raise ValueError("TaskQ accepts 'redis_url' or 'redis_client', not both")
    if redis_url is not None and not redis_url.strip():
        # Why: dotenvmodel coerces "" to None for the optional field, so
        # the os.getenv(..., "") anti-pattern would pass the is-not-None
        # guard and silently disable Redis instead of failing at startup.
        raise ValueError("TaskQ 'redis_url' must be a non-empty URL or None")
    if pg_conn_factory is not None and listen_conn is not None:
        raise ValueError("TaskQ accepts 'pg_conn_factory' or 'listen_conn', not both")

    self._dsn = dsn
    self._pool: "asyncpg.Pool | None" = pool  # noqa: UP037  # Why: asyncpg imported under TYPE_CHECKING; quotes required for runtime resolution.
    self._schema = schema
    self._min_pool_size = min_pool_size
    self._max_pool_size = max_pool_size
    self._redis_url = redis_url
    self._redis_client: "redis_async.Redis | None" = redis_client  # type: ignore[type-arg]  # noqa: UP037  # Why: erasure boundary — redis_async is under TYPE_CHECKING; string annotation avoids runtime import. type-arg: redis-py stubs expose Redis as an unparameterised generic. The caller-supplied client is stored here and forwarded to JobsClient without entering it on the exit stack.
    self._pg_conn_factory = pg_conn_factory
    self._listen_conn = listen_conn
    self._poll_timeout = poll_timeout
    # Passed through to the constructed PostgresBackend's poll_reclaim_events
    # default — see RECLAIM_EVENT_VISIBILITY_DELAY. Must match whatever
    # margin the worker fleet's sweep-adjacent backend uses, since the
    # margin's correctness depends on writer transaction duration, not
    # reader preference.
    self._reclaim_event_visibility_delay = reclaim_event_visibility_delay
    self._pool_factory = pool_factory
    # A pool TaskQ built (from a DSN or a factory) is TaskQ's to close; a
    # caller-supplied one never is. Same ownership rule as
    # taskq.connections.WorkerConnections.
    self._owns_pool = pool is None
    self._client: JobsClient | None = None
    self._actors_client: ActorsClient | None = None
    # Held so reload_credentials can swap the pool into the live backend
    # without reaching through the JobsClient into PostgresBackend._deps.
    self._deps: _ClientDeps | None = None

actors property

actors: ActorsClient

Actor configuration client — list, get, set capacity, deregister.

Raises RuntimeError if called before open() or outside an async with block.

open async

open() -> None

Open the connection pool and prepare the client.

Called automatically by __aenter__. Safe to call explicitly for frameworks that manage lifecycle outside an async with block. Raises :class:RuntimeError if already open.

Source code in src/taskq/client/_taskq.py

async def open(self) -> None:
    """Open the connection pool and prepare the client.

    Called automatically by ``__aenter__``. Safe to call explicitly
    for frameworks that manage lifecycle outside an ``async with`` block.
    Raises :class:`RuntimeError` if already open.
    """
    if self._client is not None:
        raise RuntimeError("TaskQ is already open")

    # Lazy imports keep asyncpg out of the module-level import graph so
    # taskq.testing can be imported without pulling in asyncpg.
    import asyncpg

    from taskq.backend.clock import SystemClock
    from taskq.backend.postgres import PostgresBackend
    from taskq.settings import TaskQSettings

    if self._pool is None:
        if self._pool_factory is not None:
            self._pool = await self._pool_factory()
        else:
            created = await asyncpg.create_pool(
                dsn=self._dsn,
                min_size=self._min_pool_size,
                max_size=self._max_pool_size,
            )
            assert created is not None  # asyncpg returns None only for record_class paths
            self._pool = created

    pool = self._pool
    assert pool is not None
    deps = _ClientDeps(
        settings=_ClientSettings(schema_name=self._schema),
        worker_pool=pool,
        heartbeat_pool=pool,
    )
    self._deps = deps
    backend = PostgresBackend(
        deps,
        clock=SystemClock(),
        cancellation_grace_period=timedelta(seconds=30),
        cleanup_grace_period=timedelta(seconds=10),
        reclaim_event_visibility_delay=(
            self._reclaim_event_visibility_delay
            if self._reclaim_event_visibility_delay is not None
            else RECLAIM_EVENT_VISIBILITY_DELAY
        ),
    )
    # Route the Redis URL through load_from_dict so it is coerced and
    # validated by the field's declared RedisDsn type (TypeCoercionError
    # on an invalid scheme) instead of being stored as a raw str.
    load_data: dict[str, str] = {"TASKQ_SCHEMA_NAME": self._schema}
    if self._redis_url is not None:
        load_data["TASKQ_REDIS_URL"] = self._redis_url
    settings = TaskQSettings.load_from_dict(load_data)
    self._client = JobsClient(backend, settings=settings)
    self._actors_client = ActorsClient(pool, schema=self._schema)
    if self._redis_client is not None:
        self._client._redis_client = self._redis_client  # pyright: ignore[reportPrivateUsage]  # Why: TaskQ owns the JobsClient lifecycle; assigning the caller-owned redis_client directly bypasses _open_redis so the client is NOT entered on the exit stack — TaskQ.close() must not close a caller-owned client.
    elif self._redis_url is not None:
        await self._client._open_redis(settings)  # pyright: ignore[reportPrivateUsage]  # Why: TaskQ owns the JobsClient lifecycle; _open_redis is the canonical hook for the owner to call after construction.

close async

close() -> None

Close the client and release the pool if owned.

Called automatically by __aexit__. Safe to call explicitly. No-op if already closed.

Source code in src/taskq/client/_taskq.py

async def close(self) -> None:
    """Close the client and release the pool if owned.

    Called automatically by ``__aexit__``. Safe to call explicitly.
    No-op if already closed.
    """
    if self._client is not None:
        await self._client.close()
        self._client = None
        self._actors_client = None
        self._deps = None
    if self._owns_pool and self._pool is not None:
        # Why bounded: an enqueue in flight at close time can stall
        # Pool.close() indefinitely against a dead PG.
        await close_pool_bounded(self._pool, "client", CLOSE_TIMEOUT_SECS)
        self._pool = None

reload_credentials async

reload_credentials() -> None

Rebuild the pool from pool_factory and swap it in, live.

The client-side counterpart of :func:taskq.worker.deps.reload_credentials, and the supported replacement for reaching into _pool / _client: it invokes the factory (which fetches a fresh credential), atomically points every subsystem that holds a pool - the backend deps behind enqueue/get/list/cancel, the :attr:actors client, and the stream() LISTEN fallback - at the new pool, then closes the old one with the same bounded drain used at close().

Call this on a schedule shorter than your credential's lifetime (an Entra access token is typically ~60 min), or on any signal your issuer gives you. It is not needed for the ordinary token refresh that :func:taskq.auth.make_pg_pool_factory already does per physical connection; it is how you drop sessions opened under a revoked credential, and the only way to pick up a changed username, which asyncpg resolves once per pool.

Raises :class:RuntimeError if the client is not open, or if the pool is caller-owned (pool=) - TaskQ must never close a pool it does not own, so the caller rotates that one itself.

If the factory fails, the exception propagates and the current pool is left untouched and serving: a transient token-endpoint outage must not turn into a client outage.

Source code in src/taskq/client/_taskq.py

async def reload_credentials(self) -> None:
    """Rebuild the pool from ``pool_factory`` and swap it in, live.

    The client-side counterpart of
    :func:`taskq.worker.deps.reload_credentials`, and the supported
    replacement for reaching into ``_pool`` / ``_client``: it invokes the
    factory (which fetches a fresh credential), atomically points every
    subsystem that holds a pool - the backend deps behind
    ``enqueue``/``get``/``list``/``cancel``, the :attr:`actors` client, and
    the ``stream()`` LISTEN fallback - at the new pool, then closes the old
    one with the same bounded drain used at ``close()``.

    Call this on a schedule shorter than your credential's lifetime (an
    Entra access token is typically ~60 min), or on any signal your issuer
    gives you. It is not needed for the ordinary token refresh that
    :func:`taskq.auth.make_pg_pool_factory` already does per physical
    connection; it is how you drop sessions opened under a revoked
    credential, and the only way to pick up a **changed username**, which
    asyncpg resolves once per pool.

    Raises :class:`RuntimeError` if the client is not open, or if the pool
    is caller-owned (``pool=``) - TaskQ must never close a pool it does not
    own, so the caller rotates that one itself.

    If the factory fails, the exception propagates and the **current pool
    is left untouched and serving**: a transient token-endpoint outage
    must not turn into a client outage.
    """
    if self._client is None or self._deps is None:
        raise RuntimeError("TaskQ is not open — call open() before reload_credentials()")
    if self._pool_factory is None:
        raise RuntimeError(
            "TaskQ.reload_credentials() requires 'pool_factory' (or 'pg_provider'); "
            "a pool passed as 'pool=' is caller-owned and must be rotated by its owner."
        )

    old_pool = self._pool
    # Built before anything is swapped, so a factory failure leaves the
    # live pool in place — see the docstring.
    new_pool = await self._pool_factory()

    self._pool = new_pool
    self._deps.worker_pool = new_pool
    self._deps.heartbeat_pool = new_pool
    # Rebuilt rather than mutated: ActorsClient takes its pool at
    # construction and is a cheap, stateless facade over it.
    self._actors_client = ActorsClient(new_pool, schema=self._schema)

    if old_pool is not None:
        # Why bounded: an enqueue in flight on the old pool can stall
        # Pool.close() indefinitely against a dead PG.
        await close_pool_bounded(old_pool, "client-reload", CLOSE_TIMEOUT_SECS)

__aenter__ async

__aenter__() -> TaskQ

Source code in src/taskq/client/_taskq.py

async def __aenter__(self) -> "TaskQ":
    await self.open()
    return self

__aexit__ async

__aexit__(*_: object) -> None

Source code in src/taskq/client/_taskq.py

async def __aexit__(self, *_: object) -> None:
    await self.close()

enqueue async

enqueue(
    ref: ActorRef[P, R],
    payload: P,
    *,
    queue: QueueName | None = None,
    scheduled_at: datetime | None = None,
    priority: int | None = None,
    schedule_to_close: datetime | None = None,
    start_to_close: timedelta | None = None,
    heartbeat_timeout: timedelta | None = None,
    identity_key: IdentityKey | None = None,
    fairness_key: str | None = None,
    idempotency_key: IdempotencyKey | None = None,
    idempotency_scope: str | None = None,
    trace_id: str | None = None,
    span_id: str | None = None,
    metadata: dict[str, object] | None = None,
    tags: list[str] | None = None,
) -> JobHandle[R]

Enqueue a job and return a typed handle.

schedule_to_close (absolute datetime) is deprecated — it crosses clock domains (the app clock that produced it vs the database clock that evaluates it). Declare retry.time_budget on the actor instead; the interval form is anchored to the database clock.

Source code in src/taskq/client/_taskq.py

async def enqueue[P: BaseModel, R: BaseModel | None](
    self,
    ref: ActorRef[P, R],
    payload: P,
    *,
    queue: QueueName | None = None,
    scheduled_at: datetime | None = None,
    priority: int | None = None,
    schedule_to_close: datetime | None = None,
    start_to_close: timedelta | None = None,
    heartbeat_timeout: timedelta | None = None,
    identity_key: IdentityKey | None = None,
    fairness_key: str | None = None,
    idempotency_key: IdempotencyKey | None = None,
    idempotency_scope: str | None = None,
    trace_id: str | None = None,
    span_id: str | None = None,
    metadata: dict[str, object] | None = None,
    tags: list[str] | None = None,
) -> JobHandle[R]:
    """Enqueue a job and return a typed handle.

    ``schedule_to_close`` (absolute datetime) is deprecated — it crosses
    clock domains (the app clock that produced it vs the database clock
    that evaluates it).  Declare ``retry.time_budget`` on the actor
    instead; the interval form is anchored to the database clock.
    """
    return await self._require_open().enqueue(
        ref,
        payload,
        queue=queue,
        scheduled_at=scheduled_at,
        priority=priority,
        schedule_to_close=schedule_to_close,
        start_to_close=start_to_close,
        heartbeat_timeout=heartbeat_timeout,
        identity_key=identity_key,
        fairness_key=fairness_key,
        idempotency_key=idempotency_key,
        idempotency_scope=idempotency_scope,
        trace_id=trace_id,
        span_id=span_id,
        metadata=metadata,
        tags=tags,
    )

enqueue_batch async

enqueue_batch(
    items: list[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: Connection | None = None,
    failure_policy: BatchFailurePolicy | None = None,
    finalizer: EnqueueItem | None = None,
) -> BatchHandle

Enqueue multiple jobs in a single batched INSERT.

Delegates to :meth:JobsClient.enqueue_batch; see its docstring for validation rules and idempotency-key collision semantics.

Source code in src/taskq/client/_taskq.py

async def enqueue_batch(
    self,
    items: list[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: "asyncpg.Connection | None" = None,
    failure_policy: BatchFailurePolicy | None = None,
    finalizer: EnqueueItem | None = None,
) -> BatchHandle:
    """Enqueue multiple jobs in a single batched INSERT.

    Delegates to :meth:`JobsClient.enqueue_batch`; see its docstring
    for validation rules and idempotency-key collision semantics.
    """
    return await self._require_open().enqueue_batch(
        items,
        batch_id=batch_id,
        connection=connection,
        failure_policy=failure_policy,
        finalizer=finalizer,
    )

enqueue_batch_streaming async

enqueue_batch_streaming(
    items: Iterable[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: Connection | None = None,
    failure_policy: BatchFailurePolicy | None = None,
    finalizer: EnqueueItem | None = None,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
) -> BatchHandle

Enqueue jobs from a lazy iterable in chunks.

Delegates to :meth:JobsClient.enqueue_batch_streaming; see its docstring for chunk_size validation and streaming semantics.

Source code in src/taskq/client/_taskq.py

async def enqueue_batch_streaming(
    self,
    items: Iterable[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: "asyncpg.Connection | None" = None,
    failure_policy: BatchFailurePolicy | None = None,
    finalizer: EnqueueItem | None = None,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
) -> BatchHandle:
    """Enqueue jobs from a lazy iterable in chunks.

    Delegates to :meth:`JobsClient.enqueue_batch_streaming`; see its
    docstring for chunk_size validation and streaming semantics.
    """
    return await self._require_open().enqueue_batch_streaming(
        items,
        batch_id=batch_id,
        connection=connection,
        failure_policy=failure_policy,
        finalizer=finalizer,
        chunk_size=chunk_size,
    )

get_batch async

get_batch(batch_id: UUID) -> BatchRow | None

Fetch a single batch row by ID.

Delegates to :meth:JobsClient.get_batch. Returns None when the batch does not exist.

Source code in src/taskq/client/_taskq.py

async def get_batch(self, batch_id: UUID) -> BatchRow | None:
    """Fetch a single batch row by ID.

    Delegates to :meth:`JobsClient.get_batch`. Returns ``None`` when
    the batch does not exist.
    """
    return await self._require_open().get_batch(batch_id)

list_batches async

list_batches(filter: BatchFilter) -> list[BatchSummary]

List batches matching filter, returning :class:BatchSummary objects.

Delegates to :meth:JobsClient.list_batches.

Source code in src/taskq/client/_taskq.py

async def list_batches(
    self,
    filter: BatchFilter,
) -> list[BatchSummary]:
    """List batches matching *filter*, returning :class:`BatchSummary` objects.

    Delegates to :meth:`JobsClient.list_batches`.
    """
    return await self._require_open().list_batches(filter)

enqueue_batch_fast async

enqueue_batch_fast(
    items: list[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: Connection | None = None,
) -> int

Enqueue jobs via COPY FROM protocol for maximum throughput.

Delegates to :meth:JobsClient.enqueue_batch_fast; see its docstring for tradeoffs vs the regular :meth:enqueue_batch.

Source code in src/taskq/client/_taskq.py

async def enqueue_batch_fast(
    self,
    items: list[EnqueueItem],
    *,
    batch_id: UUID | None = None,
    connection: "asyncpg.Connection | None" = None,
) -> int:
    """Enqueue jobs via COPY FROM protocol for maximum throughput.

    Delegates to :meth:`JobsClient.enqueue_batch_fast`; see its
    docstring for tradeoffs vs the regular :meth:`enqueue_batch`.
    """
    return await self._require_open().enqueue_batch_fast(
        items,
        batch_id=batch_id,
        connection=connection,
    )

get async

get(
    job_id: JobId,
    *,
    result_adapter: TypeAdapter[R] | None = None,
) -> JobHandle[R] | None

Look up a job by id. Returns None when the job does not exist.

Source code in src/taskq/client/_taskq.py

async def get[R: BaseModel | None](
    self,
    job_id: JobId,
    *,
    result_adapter: TypeAdapter[R] | None = None,
) -> JobHandle[R] | None:
    """Look up a job by id. Returns ``None`` when the job does not exist."""
    return await self._require_open().get(job_id, result_adapter=result_adapter)

get_row async

get_row(job_id: JobId) -> JobRow | None

Look up a job by id and return the raw JobRow — no handle.

Delegates to :meth:JobsClient.get_row: one backend read, None when the job does not exist, mirroring :meth:get's contract without the handle machinery.

Source code in src/taskq/client/_taskq.py

async def get_row(self, job_id: JobId) -> JobRow | None:
    """Look up a job by id and return the raw ``JobRow`` — no handle.

    Delegates to :meth:`JobsClient.get_row`: one backend read,
    ``None`` when the job does not exist, mirroring :meth:`get`'s
    contract without the handle machinery.
    """
    return await self._require_open().get_row(job_id)

list async

list(filter: JobFilter) -> JobPage

List jobs matching filter, returning a :class:JobPage.

Delegates to :meth:JobsClient.list — note filter.active is not Celery's 'active' ('currently executing'); it selects by terminality ('not yet finished'). See :class:JobFilter.

Source code in src/taskq/client/_taskq.py

async def list(self, filter: JobFilter) -> JobPage:
    """List jobs matching *filter*, returning a :class:`JobPage`.

    Delegates to :meth:`JobsClient.list` — note ``filter.active``
    is not Celery's 'active' ('currently executing'); it selects by
    terminality ('not yet finished').  See :class:`JobFilter`.
    """
    return await self._require_open().list(filter)

cancel async

cancel(
    job_id: JobId, reason: str | None = None
) -> CancelResult

Request cancellation of a job. Raises :class:KeyError if not found.

Source code in src/taskq/client/_taskq.py

async def cancel(
    self,
    job_id: JobId,
    reason: str | None = None,
) -> CancelResult:
    """Request cancellation of a job. Raises :class:`KeyError` if not found."""
    return await self._require_open().cancel(job_id, reason)

cancel_where async

cancel_where(
    filter: JobFilter,
    reason: str | None = None,
    *,
    allow_empty_filter: bool = False,
) -> BulkCancelResult

Cancel all jobs matching filter. See :meth:JobsClient.cancel_where.

Source code in src/taskq/client/_taskq.py

async def cancel_where(
    self,
    filter: JobFilter,
    reason: str | None = None,
    *,
    allow_empty_filter: bool = False,
) -> BulkCancelResult:
    """Cancel all jobs matching *filter*. See :meth:`JobsClient.cancel_where`."""
    return await self._require_open().cancel_where(
        filter, reason, allow_empty_filter=allow_empty_filter
    )

create_schedule async

create_schedule(
    actor: str | ActorRef[P, R],
    cron_expr: str,
    *,
    timezone: str = "UTC",
    dst_strategy: DstStrategy = "skip",
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    name: str = "",
    identity_key: IdentityKey | None = None,
    enabled: bool = True,
) -> ScheduleHandle

Create a cron schedule. Delegates to :meth:JobsClient.create_schedule.

dst_strategy controls how DST gaps/overlaps are handled; see :meth:JobsClient.create_schedule for the full semantics.

Source code in src/taskq/client/_taskq.py

async def create_schedule[P: BaseModel, R: BaseModel | None](
    self,
    actor: str | ActorRef[P, R],
    cron_expr: str,
    *,
    timezone: str = "UTC",
    dst_strategy: DstStrategy = "skip",
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    name: str = "",
    identity_key: IdentityKey | None = None,
    enabled: bool = True,
) -> ScheduleHandle:
    """Create a cron schedule.  Delegates to :meth:`JobsClient.create_schedule`.

    ``dst_strategy`` controls how DST gaps/overlaps are handled; see
    :meth:`JobsClient.create_schedule` for the full semantics.
    """
    return await self._require_open().create_schedule(
        actor,
        cron_expr,
        timezone=timezone,
        dst_strategy=dst_strategy,
        payload_factory=payload_factory,
        static_payload=static_payload,
        name=name,
        identity_key=identity_key,
        enabled=enabled,
    )

list_schedules async

list_schedules(
    *, actor: str | None = None, enabled: bool | None = None
) -> list[ScheduleRecord]

List cron schedules. Delegates to :meth:JobsClient.list_schedules.

Source code in src/taskq/client/_taskq.py

async def list_schedules(
    self,
    *,
    actor: str | None = None,
    enabled: bool | None = None,
) -> "list[ScheduleRecord]":
    """List cron schedules.  Delegates to :meth:`JobsClient.list_schedules`."""
    return await self._require_open().list_schedules(actor=actor, enabled=enabled)

update_schedule async

update_schedule(
    schedule_id: UUID,
    *,
    cron_expr: str | None = None,
    enabled: bool | None = None,
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    clear_payload_factory: bool = False,
) -> ScheduleRecord

Update a cron schedule. Delegates to :meth:JobsClient.update_schedule.

Source code in src/taskq/client/_taskq.py

async def update_schedule(
    self,
    schedule_id: UUID,
    *,
    cron_expr: str | None = None,
    enabled: bool | None = None,
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    clear_payload_factory: bool = False,
) -> ScheduleRecord:
    """Update a cron schedule.  Delegates to :meth:`JobsClient.update_schedule`."""
    return await self._require_open().update_schedule(
        schedule_id,
        cron_expr=cron_expr,
        enabled=enabled,
        payload_factory=payload_factory,
        static_payload=static_payload,
        clear_payload_factory=clear_payload_factory,
    )

delete_schedule async

delete_schedule(schedule_id: UUID) -> None

Delete a cron schedule. Delegates to :meth:JobsClient.delete_schedule.

Source code in src/taskq/client/_taskq.py

async def delete_schedule(self, schedule_id: UUID) -> None:
    """Delete a cron schedule.  Delegates to :meth:`JobsClient.delete_schedule`."""
    await self._require_open().delete_schedule(schedule_id)

stream async

stream(job_id: JobId) -> AsyncIterator[JobEvent]

Stream live state changes for a job as :class:JobEvent objects.

    Yields one event per observable state transition (status change or
    progress update), terminating automatically when the job reaches a
    terminal state. The final event always has ``terminal=True``.

    Usage::

        async for event in tq.stream(job_id):
            print(event.status, event.progress_state)
            # loop exits automatically when event.terminal is True

        # Or wire directly into a FastAPI SSE response:
        async def event_generator():
            async for event in tq.stream(job_id):
                yield f"data: {event.model_dump_json()}

"

    Raises
    ------
    RuntimeError
        Called before ``tq.open()`` or outside an ``async with`` block.
    KeyError
        The job does not exist.
    RuntimeError
        PG LISTEN transport requested but ``dsn`` was not provided at
        construction (pool-only mode).

Source code in src/taskq/client/_taskq.py

async def stream(self, job_id: JobId) -> AsyncIterator[JobEvent]:
    """Stream live state changes for a job as :class:`JobEvent` objects.

    Yields one event per observable state transition (status change or
    progress update), terminating automatically when the job reaches a
    terminal state. The final event always has ``terminal=True``.

    Usage::

        async for event in tq.stream(job_id):
            print(event.status, event.progress_state)
            # loop exits automatically when event.terminal is True

        # Or wire directly into a FastAPI SSE response:
        async def event_generator():
            async for event in tq.stream(job_id):
                yield f"data: {event.model_dump_json()}\n\n"

    Raises
    ------
    RuntimeError
        Called before ``tq.open()`` or outside an ``async with`` block.
    KeyError
        The job does not exist.
    RuntimeError
        PG LISTEN transport requested but ``dsn`` was not provided at
        construction (pool-only mode).
    """
    client = self._require_open()
    row = await client.backend.get(job_id)
    if row is None:
        raise KeyError(job_id)

    event = _row_to_event(row)
    yield event
    if event.terminal:
        return

    gen: AsyncGenerator[JobEvent, None] = (
        _stream_redis(
            self._redis_client,
            self._schema,
            job_id,
            client,
            self._poll_timeout,
            last_seq=row.progress_seq,
            last_status=row.status,
        )
        if self._redis_client is not None
        else _stream_pg(
            self._dsn,
            self._schema,
            job_id,
            client,
            self._poll_timeout,
            last_seq=row.progress_seq,
            last_status=row.status,
            pg_conn_factory=self._pg_conn_factory,
            listen_conn=self._listen_conn,
        )
    )
    async with contextlib.aclosing(gen) as agen:
        async for evt in agen:
            yield evt
            if evt.terminal:
                return

watch_reclaims async

watch_reclaims(
    after_id: int = 0, *, poll_timeout: float | None = None
) -> AsyncIterator[EventRow]

Stream fleet-wide crash-reclaim events as :class:EventRow objects.

Delivery guarantee. At-least-once, and gap-free only under a bounded writer-transaction assumption: an event is silently and permanently missed if a job_events writer transaction stays open longer than reclaim_event_visibility_delay (default 2s — see :data:taskq.constants.RECLAIM_EVENT_VISIBILITY_DELAY) between its INSERT and its COMMIT. Ids are allocated at INSERT time but transactions commit out of order, so a late-committing lower-id row can land behind the cursor after the cursor has already advanced past its position; no error is raised anywhere. Sweep and terminal-write transactions are a handful of single-round-trip statements, so 2s is a generous bound under normal operation — but it is an assumption enforced by nothing in the SQL, not a property the query guarantees. So this watcher does not leave detection to chance: on a slow cadence (_VISIBILITY_RISK_CHECK_INTERVAL, 60s) it runs the backend's check_reclaim_visibility_delay_risk diagnostic (when the backend implements it — PostgresBackend does) and logs a loud watch-reclaims-visibility-delay-at-risk warning for every long-open job_events writer it finds. That is a proxy warning, not proof of an actual miss.

Yields job_events rows with kind='state_change' and detail['reason']='lock_expired', ordered by the monotonic event_id cursor ascending. to_state is 'pending' for a retried reclaim, 'crashed' or 'cancelled' (cancel was in-flight when the worker died) for a terminal one.

Cursor and duplicate semantics

The caller persists the last-seen event_id and passes it back as after_id on resumption. Persist the cursor after processing each event: a crash between processing and persisting re-delivers that event on the next run — delivery is at-least-once, so consumers must dedupe on event_id. The cursor is a watermark, not a reference: pruning job_events rows at or below it is always safe, but rows pruned before the consumer reads them are gone for good — only prune rows older than your slowest consumer's cursor. A cursor far behind after a long outage drains at query speed (full batches are re-polled immediately, not one batch per poll_timeout).

Shutdown and backpressure

This is a pull-based async generator: events are fetched only as fast as the consumer iterates, so a slow consumer simply polls slower — no internal buffer grows. To stop, break out of the async for (or cancel the consuming task); generator cleanup removes the LISTEN registration and closes any owned connection.

Transport

On Postgres with a LISTEN transport source (dsn, pg_conn_factory, or listen_conn), the method LISTENs on wake_channel(schema) purely as a low-latency wakeup, but always polls backend.poll_reclaim_events(after_id) as the durable source of truth — NOTIFY is an optimisation, never the only path, and a dropped LISTEN connection degrades to (and recovers from) polling automatically. Without a LISTEN transport or on Redis-configured backends, a plain poll loop against poll_reclaim_events runs on poll_timeout.

Usage::

cursor = await load_cursor()  # your own durable store
async for evt in tq.watch_reclaims(after_id=cursor):
    outstanding -= 1          # fan-out completion tracking
    cursor = evt.event_id
    await save_cursor(cursor)  # persist AFTER processing

Raises

RuntimeError Called before tq.open() or outside an async with block.

Source code in src/taskq/client/_taskq.py

async def watch_reclaims(
    self,
    after_id: int = 0,
    *,
    poll_timeout: float | None = None,
) -> AsyncIterator[EventRow]:
    """Stream fleet-wide crash-reclaim events as :class:`EventRow` objects.

    **Delivery guarantee.** At-least-once, and gap-free *only under a
    bounded writer-transaction assumption*: an event is **silently and
    permanently missed** if a ``job_events`` writer transaction stays
    open longer than ``reclaim_event_visibility_delay`` (default 2s —
    see :data:`taskq.constants.RECLAIM_EVENT_VISIBILITY_DELAY`)
    between its INSERT and its COMMIT.  Ids are allocated at INSERT
    time but transactions commit out of order, so a late-committing
    lower-id row can land *behind* the cursor after the cursor has
    already advanced past its position; no error is raised anywhere.
    Sweep and terminal-write transactions are a handful of
    single-round-trip statements, so 2s is a generous bound under
    normal operation — but it is an assumption enforced by nothing in
    the SQL, not a property the query guarantees.  So this watcher
    does not leave detection to chance: on a slow cadence
    (``_VISIBILITY_RISK_CHECK_INTERVAL``, 60s) it runs the backend's
    ``check_reclaim_visibility_delay_risk`` diagnostic (when the
    backend implements it — PostgresBackend does) and logs a loud
    ``watch-reclaims-visibility-delay-at-risk`` warning for every
    long-open ``job_events`` writer it finds.  That is a proxy
    warning, not proof of an actual miss.

    Yields ``job_events`` rows with ``kind='state_change'`` and
    ``detail['reason']='lock_expired'``, ordered by the monotonic
    ``event_id`` cursor ascending.  ``to_state`` is ``'pending'`` for
    a retried reclaim, ``'crashed'`` or ``'cancelled'`` (cancel was
    in-flight when the worker died) for a terminal one.

    Cursor and duplicate semantics
    ------------------------------
    The caller persists the last-seen ``event_id`` and passes it back
    as *after_id* on resumption.  Persist the cursor **after**
    processing each event: a crash between processing and persisting
    re-delivers that event on the next run — delivery is
    at-least-once, so consumers must dedupe on ``event_id``.  The
    cursor is a watermark, not a reference: pruning ``job_events``
    rows at or below it is always safe, but rows pruned *before* the
    consumer reads them are gone for good — only prune rows older
    than your slowest consumer's cursor.  A cursor far behind after
    a long outage drains at query speed (full batches are re-polled
    immediately, not one batch per *poll_timeout*).

    Shutdown and backpressure
    -------------------------
    This is a pull-based async generator: events are fetched only as
    fast as the consumer iterates, so a slow consumer simply polls
    slower — no internal buffer grows.  To stop, break out of the
    ``async for`` (or cancel the consuming task); generator cleanup
    removes the LISTEN registration and closes any owned connection.

    Transport
    ---------
    On Postgres with a LISTEN transport source (``dsn``,
    ``pg_conn_factory``, or ``listen_conn``), the method LISTENs on
    ``wake_channel(schema)`` purely as a low-latency wakeup, but
    always polls ``backend.poll_reclaim_events(after_id)`` as the
    durable source of truth — NOTIFY is an optimisation, never the
    only path, and a dropped LISTEN connection degrades to (and
    recovers from) polling automatically.  Without a LISTEN
    transport or on Redis-configured backends, a plain poll loop
    against ``poll_reclaim_events`` runs on ``poll_timeout``.

    Usage::

        cursor = await load_cursor()  # your own durable store
        async for evt in tq.watch_reclaims(after_id=cursor):
            outstanding -= 1          # fan-out completion tracking
            cursor = evt.event_id
            await save_cursor(cursor)  # persist AFTER processing

    Raises
    ------
    RuntimeError
        Called before ``tq.open()`` or outside an ``async with`` block.
    """
    client = self._require_open()
    timeout = poll_timeout if poll_timeout is not None else self._poll_timeout

    has_listen_source = (
        self._dsn is not None
        or self._pg_conn_factory is not None
        or self._listen_conn is not None
    )
    if self._redis_client is None and has_listen_source:
        gen: AsyncGenerator[EventRow, None] = _watch_reclaims_pg(
            self._dsn,
            self._schema,
            client,
            timeout,
            after_id=after_id,
            pg_conn_factory=self._pg_conn_factory,
            listen_conn=self._listen_conn,
            visibility_delay=(
                self._reclaim_event_visibility_delay
                if self._reclaim_event_visibility_delay is not None
                else RECLAIM_EVENT_VISIBILITY_DELAY
            ),
        )
    else:
        gen = _watch_reclaims_poll(client, timeout, after_id=after_id)

    async with contextlib.aclosing(gen) as agen:
        async for evt in agen:
            yield evt

ActorsClient

ActorsClient(pool: Pool, *, schema: str = 'taskq')

Pool-wrapping facade for actor configuration operations.

Acquires a connection from the injected pool for each call, delegates to taskq.actor_config_ops, and returns the result. The caller must have opened the pool; this class does not manage its lifecycle.

.. note:: This client is Postgres-only — it delegates to :mod:taskq.actor_config_ops, which executes raw SQL against the actor_config table. The :class:~taskq.backend._protocol.Backend protocol does not include actor config operations, so :class:~taskq.testing.InMemoryBackend does not support ActorsClient.

Parameters

pool: An open asyncpg.Pool. The caller retains ownership. schema: TaskQ schema name. Defaults to "taskq".

Source code in src/taskq/client/_actors.py

def __init__(self, pool: "asyncpg.Pool", *, schema: str = "taskq") -> None:
    self._pool = pool
    self._schema = schema

list async

list() -> list[ActorConfigRow]

List all stored actor_config rows, ordered by actor name.

Source code in src/taskq/client/_actors.py

async def list(self) -> list[ActorConfigRow]:
    """List all stored actor_config rows, ordered by actor name."""
    async with self._pool.acquire() as conn:
        return await list_actor_configs(conn, schema=self._schema)

get async

get(actor: str) -> ActorConfigRow | None

Get one actor_config row, or None if not found.

Source code in src/taskq/client/_actors.py

async def get(self, actor: str) -> ActorConfigRow | None:
    """Get one actor_config row, or ``None`` if not found."""
    async with self._pool.acquire() as conn:
        return await get_actor_config(conn, actor, schema=self._schema)

set_capacity async

set_capacity(
    actor: str,
    *,
    max_concurrent: int | Unset | None = UNSET,
    max_pending: int | Unset | None = UNSET,
    result_ttl: float | Unset | None = UNSET,
) -> ActorConfigRow | None

Update capacity fields on an existing actor_config row.

Source code in src/taskq/client/_actors.py

async def set_capacity(
    self,
    actor: str,
    *,
    max_concurrent: int | Unset | None = UNSET,
    max_pending: int | Unset | None = UNSET,
    result_ttl: float | Unset | None = UNSET,
) -> ActorConfigRow | None:
    """Update capacity fields on an existing actor_config row."""
    async with self._pool.acquire() as conn:
        return await set_actor_config_capacity(
            conn,
            actor,
            max_concurrent=max_concurrent,
            max_pending=max_pending,
            result_ttl=result_ttl,
            schema=self._schema,
        )

deregister async

deregister(
    actor: str,
    *,
    force: bool = False,
    purge_queue: bool = False,
) -> DeregisterResult

Deregister an actor with safety checks.

See :func:taskq.actor_config_ops.deregister_actor for the full semantics.

Source code in src/taskq/client/_actors.py

async def deregister(
    self,
    actor: str,
    *,
    force: bool = False,
    purge_queue: bool = False,
) -> DeregisterResult:
    """Deregister an actor with safety checks.

    See :func:`taskq.actor_config_ops.deregister_actor` for
    the full semantics.
    """
    async with self._pool.acquire() as conn:
        return await deregister_actor(
            conn,
            actor,
            force=force,
            purge_queue=purge_queue,
            schema=self._schema,
        )

SubJobEnqueuer

SubJobEnqueuer(
    loop_scope_resolved: Mapping[type, object] | None,
    worker_pool: Pool | None,
    backend: Backend,
    *,
    clock: Clock | None = None,
    capacity_cache: ActorCapacityCache | None = None,
)

Enqueue sub-jobs from within an actor body.

Uses the LOOP-scope DB connection by default (transactional enqueue). Falls back to the worker pool if no LOOP-scope connection is registered. One instance per loop — survives across dispatches so the per-100-enqueue re-warning fires on the loop-level counter, not per-job.

Source code in src/taskq/client/_enqueuer.py

def __init__(
    self,
    loop_scope_resolved: Mapping[type, object] | None,
    worker_pool: asyncpg.Pool | None,
    backend: Backend,
    *,
    clock: Clock | None = None,
    capacity_cache: ActorCapacityCache | None = None,
) -> None:
    self._loop_scope_resolved = loop_scope_resolved
    self._worker_pool = worker_pool
    self._backend = backend
    self._clock = clock if clock is not None else SystemClock()
    self._capacity_cache = (
        capacity_cache if capacity_cache is not None else ActorCapacityCache(backend)
    )
    self._pending_buffer: list[EnqueueArgs] = []
    self._loop_enqueue_args: list[EnqueueArgs] = []
    self._autonomous_enqueue_count: int = 0

pending_count property

pending_count: int

pending_items property

pending_items: Sequence[EnqueueArgs]

enqueue async

enqueue(
    actor_ref: ActorRef[P, R],
    payload: P,
    *,
    connection: Connection | None = None,
    scheduled_at: datetime | None = None,
    priority: int | None = None,
    fairness_key: str | None = None,
    metadata: dict[str, object] | None = None,
    identity_key: IdentityKey | None = None,
    idempotency_key: IdempotencyKey | str | None = None,
    idempotency_scope: str | None = None,
    unique_for: timedelta | None = None,
    unique_states: tuple[JobStatus, ...] | None = None,
    max_pending: int | None = None,
    _batch_id: str | None = None,
    tags: list[str] | None = None,
    inherit_tags: bool = True,
    schedule_to_close: datetime | None = None,
    start_to_close: timedelta | None = None,
    heartbeat_timeout: timedelta | None = None,
) -> JobHandle[R]

Enqueue a sub-job. max_pending is a per-call limit resolved against the operator-owned stored cap and the @actor(...) literal: against a non-NULL stored actor_config.max_pending the tighter of the two wins (min(stored, per_call) — an explicit caller shedding load is never widened by an operator override, and no code path can raise an operator's fleet cap); with no stored value this parameter wins outright over the literal (historical behavior — actor code may loosen its own declaration).

_batch_id is a library-internal parameter used by :meth:enqueue_batch to stamp batch_id into metadata after :func:build_enqueue_args has stripped any caller-supplied batch_id (H5 security boundary). Callers MUST NOT pass it.

Source code in src/taskq/client/_enqueuer.py

async def enqueue[P: BaseModel, R: BaseModel | None](
    self,
    actor_ref: ActorRef[P, R],
    payload: P,
    *,
    connection: asyncpg.Connection | None = None,
    scheduled_at: datetime | None = None,
    priority: int | None = None,
    fairness_key: str | None = None,
    metadata: dict[str, object] | None = None,
    identity_key: IdentityKey | None = None,
    idempotency_key: IdempotencyKey | str | None = None,
    idempotency_scope: str | None = None,
    unique_for: timedelta | None = None,
    unique_states: tuple[JobStatus, ...] | None = None,
    max_pending: int | None = None,
    _batch_id: str | None = None,
    tags: list[str] | None = None,
    inherit_tags: bool = True,
    schedule_to_close: datetime | None = None,
    start_to_close: timedelta | None = None,
    heartbeat_timeout: timedelta | None = None,
) -> JobHandle[R]:
    """Enqueue a sub-job. ``max_pending`` is a per-call limit resolved
    against the operator-owned stored cap and the ``@actor(...)``
    literal: against a non-NULL *stored* ``actor_config.max_pending``
    the tighter of the two wins (``min(stored, per_call)`` — an
    explicit caller shedding load is never widened by an operator
    override, and no code path can raise an operator's fleet cap);
    with no stored value this parameter wins outright over the
    literal (historical behavior — actor code may loosen its own
    declaration).

    ``_batch_id`` is a library-internal parameter used by
    :meth:`enqueue_batch` to stamp ``batch_id`` into metadata after
    :func:`build_enqueue_args` has stripped any caller-supplied
    ``batch_id`` (H5 security boundary). Callers MUST NOT pass it.
    """
    resolved_queue = actor_ref.queue
    identity_key_str = str(identity_key) if identity_key is not None else ""

    with enqueue_span(actor_ref.name, resolved_queue, identity_key=identity_key_str) as (
        span,
        extracted_trace_id,
        extracted_span_id,
    ):
        effective_max_pending = await self._capacity_cache.effective_max_pending(
            actor_ref.name,
            actor_ref.max_pending,
            per_call=max_pending,
        )
        resolved_tags = self._resolve_tags(tags, inherit_tags)
        args = build_enqueue_args(
            actor_ref,
            payload,
            scheduled_at=scheduled_at,
            priority=priority,
            fairness_key=fairness_key,
            metadata=metadata,
            identity_key=identity_key,
            idempotency_key=idempotency_key,
            idempotency_scope=idempotency_scope,
            trace_id=extracted_trace_id,
            span_id=extracted_span_id,
            tags=resolved_tags,
            schedule_to_close=schedule_to_close,
            start_to_close=start_to_close,
            heartbeat_timeout=heartbeat_timeout,
            unique_for=unique_for,
            unique_states=unique_states,
            max_pending=effective_max_pending,
        )
        if _batch_id is not None:
            # H5: stamp batch_id AFTER build_enqueue_args, which strips
            # any caller-supplied batch_id as a security boundary.
            args = replace(
                args,
                metadata={**args.metadata, "batch_id": _batch_id},
            )
        span.set_attribute("messaging.message.id", str(args.id))
        row = await self._do_enqueue(args, connection)
    return JobHandle(
        row=row,
        result_adapter=actor_ref.result_adapter,
        was_existing=(row.id != args.id),
        backend=self._backend,
        client=None,
    )

enqueue_batch async

enqueue_batch(
    items: Sequence[EnqueueItem[Any, Any]],
    *,
    batch_id: UUID | None = None,
    connection: Connection | None = None,
) -> list[JobHandle[Any]]

Enqueue a batch of sub-jobs sharing a single batch_id.

All items share a single batch_id UUID written into each job's metadata.batch_id field (as a string). When batch_id is not supplied it is auto-generated as a UUIDv7 via :func:~taskq._ids.new_job_id — mirrors :meth:~taskq.client.JobsClient.enqueue_batch. Pass an explicit batch_id to correlate this batch with a caller-constructed identifier (e.g. a finalizer job enqueued separately that needs to reference the same batch).

Raises ValueError when items is empty or exceeds MAX_BATCH_SIZE — the same guardrails :meth:~taskq.client.JobsClient.enqueue_batch applies to the identical operation one layer up. Without the empty check the no-connection fallback loop would iterate zero items and return [] silently. The backend binds every item as 21 parallel array parameters to a single unnest INSERT in one transaction, so an uncapped batch enqueued from inside a job body is unbounded fan-out that bypasses the client-side guardrail.

Source code in src/taskq/client/_enqueuer.py

async def enqueue_batch(
    self,
    items: Sequence[EnqueueItem[Any, Any]],
    *,
    batch_id: UUID | None = None,
    connection: asyncpg.Connection | None = None,
) -> list[JobHandle[Any]]:
    """Enqueue a batch of sub-jobs sharing a single ``batch_id``.

    All ``items`` share a single ``batch_id`` UUID written into each
    job's ``metadata.batch_id`` field (as a string). When ``batch_id``
    is not supplied it is auto-generated as a UUIDv7 via
    :func:`~taskq._ids.new_job_id` — mirrors
    :meth:`~taskq.client.JobsClient.enqueue_batch`. Pass an explicit
    ``batch_id`` to correlate this batch with a caller-constructed
    identifier (e.g. a finalizer job enqueued separately that needs to
    reference the same batch).

    Raises ``ValueError`` when ``items`` is empty or exceeds
    ``MAX_BATCH_SIZE`` — the same guardrails
    :meth:`~taskq.client.JobsClient.enqueue_batch` applies to the
    identical operation one layer up. Without the empty check the
    no-connection fallback loop would iterate zero items and return
    ``[]`` silently. The backend binds every item
    as 21 parallel array parameters to a single ``unnest`` INSERT in one
    transaction, so an uncapped batch enqueued from inside a job body is
    unbounded fan-out that bypasses the client-side guardrail.
    """
    if len(items) == 0:
        raise ValueError("items must not be empty")
    if len(items) > MAX_BATCH_SIZE:
        raise ValueError(
            f"items must contain at most {MAX_BATCH_SIZE} entries, got {len(items)}"
        )

    resolved_batch_id = batch_id if batch_id is not None else UUID(bytes=new_job_id().bytes)

    conn, from_loop_scope = self._resolve_connection(connection)

    if conn is not None:
        effective_mp: dict[str, int | None] = {}
        for item in items:
            ref = item.actor_ref
            if ref.name not in effective_mp:
                effective_mp[ref.name] = await self._capacity_cache.effective_max_pending(
                    ref.name, ref.max_pending
                )
        args_list = build_batch_args(
            items, resolved_batch_id, max_pending_by_actor=effective_mp
        )

        if from_loop_scope and self._backend.supports_transactional_simulation:
            for args in args_list:
                self._pending_buffer.append(args)
            return [
                JobHandle(
                    row=self._synthesize_row(args),
                    result_adapter=item.actor_ref.result_adapter,
                    was_existing=False,
                    backend=self._backend,
                    client=None,
                )
                for args, item in zip(args_list, items, strict=True)
            ]

        rows = await self._backend.enqueue_batch(args_list, connection=conn)  # type: ignore[call-arg]  # Why: asyncpg.Connection is compatible with the protocol's connection parameter at runtime
        if from_loop_scope:
            self._loop_enqueue_args.extend(args_list)
        handles: list[JobHandle[Any]] = []
        for i, row in enumerate(rows):
            args = args_list[i]
            handles.append(
                JobHandle(
                    row=row,
                    result_adapter=items[i].actor_ref.result_adapter,
                    was_existing=(row.id != args.id),
                    backend=self._backend,
                    client=None,
                )
            )
        return handles

    if self._worker_pool is None:
        raise RuntimeError("ctx.jobs is only available inside an actor body")

    handles = []
    failed_items: list[tuple[int, Exception]] = []
    batch_id_str = str(resolved_batch_id)
    for i, item in enumerate(items):
        try:
            handle = await self.enqueue(
                item.actor_ref,
                item.payload,
                scheduled_at=item.scheduled_at,
                priority=item.priority,
                fairness_key=item.fairness_key,
                metadata=dict(item.metadata),
                idempotency_key=item.idempotency_key,
                idempotency_scope=item.idempotency_scope,
                identity_key=item.identity_key,
                _batch_id=batch_id_str,
                tags=list(item.tags) if item.tags else None,
                inherit_tags=False,
                start_to_close=item.start_to_close,
            )
            handles.append(handle)
        except Exception as exc:
            failed_items.append((i, exc))

    if failed_items:
        raise PartialBatchError(
            succeeded_count=len(handles),
            failed_items=failed_items,
            total=len(items),
        )

    return handles

flush_buffer async

flush_buffer() -> None

Flush buffered EnqueueArgs to the backend (in-memory simulation).

Called by the consumer on actor success, AFTER the LOOP-scope transaction has committed. Per-item flush failures are collected and re-raised as :class:~taskq.exceptions.SubEnqueueError after the loop completes so callers can detect lost sub-jobs.

Source code in src/taskq/client/_enqueuer.py

async def flush_buffer(self) -> None:
    """Flush buffered EnqueueArgs to the backend (in-memory simulation).

    Called by the consumer on actor success, AFTER the LOOP-scope
    transaction has committed. Per-item flush failures are collected
    and re-raised as :class:`~taskq.exceptions.SubEnqueueError` after
    the loop completes so callers can detect lost sub-jobs.
    """
    snapshot = self._pending_buffer
    self._pending_buffer = []
    self._loop_enqueue_args.clear()
    failed_items: list[tuple[EnqueueArgs, Exception]] = []
    for args in snapshot:
        try:
            await self._backend.enqueue(args)
        except Exception as exc:
            failed_items.append((args, exc))
            _log.warning(
                "sub_enqueue_flush_error",
                kind="sub_enqueue_flush_error",
                job_id=str(args.id),
                error_class=type(exc).__name__,
                error_message=str(exc),
            )
    if failed_items:
        raise SubEnqueueError(failed_items=failed_items)

discard_buffer

discard_buffer() -> None

Clear the pending buffer without flushing.

Source code in src/taskq/client/_enqueuer.py

def discard_buffer(self) -> None:
    """Clear the pending buffer without flushing."""
    self._pending_buffer.clear()
    self._loop_enqueue_args.clear()

drain_for_re_enqueue

drain_for_re_enqueue() -> list[EnqueueArgs]

Return and clear both loop-scope and pending buffers for re-enqueue.

Source code in src/taskq/client/_enqueuer.py

def drain_for_re_enqueue(self) -> list[EnqueueArgs]:
    """Return and clear both loop-scope and pending buffers for re-enqueue."""
    items = self._loop_enqueue_args + list(self._pending_buffer)
    self._loop_enqueue_args = []
    self._pending_buffer = []
    return items

WorkerConnections dataclass

WorkerConnections(
    dispatcher_pool: Pool | None = None,
    dispatcher_pool_factory: PoolFactory | None = None,
    heartbeat_pool: Pool | None = None,
    heartbeat_pool_factory: PoolFactory | None = None,
    worker_pool: Pool | None = None,
    worker_pool_factory: PoolFactory | None = None,
    notify_conn: Connection | None = None,
    notify_conn_factory: ConnFactory | None = None,
    leader_conn: Connection | None = None,
    leader_conn_factory: ConnFactory | None = None,
    redis_client: Redis | None = None,
    redis_client_factory: RedisFactory | None = None,
)

Per-role connection overrides for the worker.

Each role has a <role> (pre-constructed, caller-owned) and a <role>_factory (zero-arg async factory, TaskQ-owned) slot. Leave both None for DSN-based construction (the default).

Example — AAD-managed-identity worker::

from azure.identity.aio import DefaultAzureCredential
from taskq.aad import EntraIdProvider
from taskq.auth import make_pg_pool_factory
from taskq.connections import WorkerConnections

cred = DefaultAzureCredential()
provider = EntraIdProvider(cred)

connections = WorkerConnections(
    dispatcher_pool_factory=make_pg_pool_factory(
        settings.pg_dsn_direct, provider, max_size=settings.dispatcher_pool_size,
    ),
    heartbeat_pool_factory=make_pg_pool_factory(
        settings.pg_dsn_direct, provider,
        max_size=settings.heartbeat_pool_size,
        command_timeout=settings.heartbeat_command_timeout,
    ),
    worker_pool_factory=make_pg_pool_factory(
        settings.pg_dsn_pooled, provider, max_size=settings.worker_pool_size,
    ),
)

Example — share an app-wide pool (caller-owned)::

connections = WorkerConnections(worker_pool=app_state.pg_pool)

dispatcher_pool class-attribute instance-attribute

dispatcher_pool: Pool | None = None

Dispatcher pool (pg_dsn_direct role). Caller-owned if set.

dispatcher_pool_factory class-attribute instance-attribute

dispatcher_pool_factory: PoolFactory | None = None

Factory for the dispatcher pool. TaskQ-owned.

heartbeat_pool class-attribute instance-attribute

heartbeat_pool: Pool | None = None

Heartbeat pool (pg_dsn_direct, heartbeat_command_timeout). Caller-owned.

heartbeat_pool_factory class-attribute instance-attribute

heartbeat_pool_factory: PoolFactory | None = None

Factory for the heartbeat pool. TaskQ-owned. command_timeout is your responsibility when overriding — set it on create_pool.

worker_pool class-attribute instance-attribute

worker_pool: Pool | None = None

Worker pool (pg_dsn_pooled role). Caller-owned.

worker_pool_factory class-attribute instance-attribute

worker_pool_factory: PoolFactory | None = None

Factory for the worker pool. TaskQ-owned.

notify_conn class-attribute instance-attribute

notify_conn: Connection | None = None

Dedicated LISTEN connection. Caller-owned. TaskQ still issues LISTEN.

notify_conn_factory class-attribute instance-attribute

notify_conn_factory: ConnFactory | None = None

Factory for the LISTEN connection. TaskQ-owned.

leader_conn class-attribute instance-attribute

leader_conn: Connection | None = None

Dedicated advisory-lock connection. Caller-owned.

leader_conn_factory class-attribute instance-attribute

leader_conn_factory: ConnFactory | None = None

Factory for the advisory-lock connection. TaskQ-owned.

redis_client class-attribute instance-attribute

redis_client: Redis | None = None

Redis client for progress fanout / rate limiting. Caller-owned.

redis_client_factory class-attribute instance-attribute

redis_client_factory: RedisFactory | None = None

Factory for the Redis client. TaskQ-owned.

__post_init__

__post_init__() -> None

Reject concrete + factory for the same role (configuration error).

Source code in src/taskq/connections.py

def __post_init__(self) -> None:
    """Reject concrete + factory for the same role (configuration error)."""
    for concrete, factory in (
        ("dispatcher_pool", "dispatcher_pool_factory"),
        ("heartbeat_pool", "heartbeat_pool_factory"),
        ("worker_pool", "worker_pool_factory"),
        ("notify_conn", "notify_conn_factory"),
        ("leader_conn", "leader_conn_factory"),
        ("redis_client", "redis_client_factory"),
    ):
        if getattr(self, concrete) is not None and getattr(self, factory) is not None:
            raise ValueError(
                f"WorkerConnections: provide either {concrete!r} or "
                f"{factory!r}, not both (role would be ambiguous)."
            )

has_any

has_any() -> bool

True if any override (concrete or factory) is set.

Source code in src/taskq/connections.py

def has_any(self) -> bool:
    """True if any override (concrete or factory) is set."""
    return any(
        getattr(self, name) is not None
        for name in (
            "dispatcher_pool",
            "dispatcher_pool_factory",
            "heartbeat_pool",
            "heartbeat_pool_factory",
            "worker_pool",
            "worker_pool_factory",
            "notify_conn",
            "notify_conn_factory",
            "leader_conn",
            "leader_conn_factory",
            "redis_client",
            "redis_client_factory",
        )
    )

JobContext dataclass

JobContext(
    job_id: UUID,
    actor: str,
    queue: str,
    attempt: int,
    worker_id: UUID,
    payload: P,
    jobs: SubJobEnqueuer,
    log: BoundLogger,
    span: Span | None = None,
    cancel_event: Event = asyncio.Event(),
    _abort_requested: Event = threading.Event(),
    _progress_buffers: dict[UUID, _ProgressBuffer]
    | None = None,
    _redis_client: Redis | None = None,
    _worker_settings: WorkerSettings | None = None,
    _pending_publish_tasks: set[Task[None]] | None = None,
)

Per-job execution context handed to worker actors.

The cancel_event field is a plain :class:asyncio.Event — never wrapped in a cancel scope or :class:asyncio.TaskGroup ( PEP 789 mitigation). The consumer constructs a fresh event per attempt; the cancel-poll hook sets it on phase 1; user actor code polls :attr:cancellation_requested or awaits cancel_event.wait().

payload is typed as the actor's payload model P. The worker consumer validates the raw dict[str, object] payload from the JobRow against actor_ref.payload_type before constructing the context, so handlers see a fully-validated Pydantic instance.

jobs provides :class:SubJobEnqueuer for enqueuing sub-jobs from within the actor body. The enqueuer resolves the database connection via LOOP-scope DI → worker-pool fallback.

job_id instance-attribute

job_id: UUID

actor instance-attribute

actor: str

queue instance-attribute

queue: str

attempt instance-attribute

attempt: int

worker_id instance-attribute

worker_id: UUID

payload instance-attribute

payload: P

jobs instance-attribute

jobs: SubJobEnqueuer

log instance-attribute

log: BoundLogger

span class-attribute instance-attribute

span: Span | None = None

cancel_event class-attribute instance-attribute

cancel_event: Event = field(default_factory=asyncio.Event)

cancellation_requested property

cancellation_requested: bool

check_cancelled

check_cancelled() -> None

Source code in src/taskq/context.py

def check_cancelled(self) -> None:
    if self.cancel_event.is_set():
        raise asyncio.CancelledError

should_abort

should_abort() -> bool

Synchronous cancellation check for sync actors (thread-safe).

Sync actors cannot await the async :attr:cancel_event, so they poll this method cooperatively. The cancel controller sets the underlying :class:threading.Event during phase 1.

Returns:

Type	Description
bool	True when cancellation has been requested — the sync
bool	actor should return or raise immediately.

Source code in src/taskq/context.py

def should_abort(self) -> bool:
    """Synchronous cancellation check for sync actors (thread-safe).

    Sync actors cannot ``await`` the async :attr:`cancel_event`, so
    they poll this method cooperatively. The cancel controller sets
    the underlying :class:`threading.Event` during phase 1.

    Returns:
        ``True`` when cancellation has been requested — the sync
        actor should return or raise immediately.
    """
    return self._abort_requested.is_set()

progress async

progress(
    *,
    step: int | None = None,
    percent: float | None = None,
    detail: str | None = None,
    data: dict[str, object] | None = None,
) -> None

Report incremental progress for this job.

Updates the in-memory coalesce buffer synchronously, then schedules a best-effort kind="progress" Redis publish as a background task when a client is connected — this call never blocks on the network. Raises :class:~taskq.exceptions.ProgressTooLarge if the serialised data payload exceeds WorkerSettings.progress_data_max_bytes.

All arguments are optional and merged last-writer-wins into the accumulated pending_state. Intermediate calls between periodic flush ticks are coalesced: only the latest value for each field reaches Postgres. seq is strictly monotone across calls.

The Redis publish is genuinely fire-and-forget: it may complete out of order relative to other in-flight publishes for the same job. Consumers reading the SSE/pub-sub stream already discard any event whose seq is not greater than the last one seen (see :mod:taskq.web.progress), so out-of-order or dropped publishes never corrupt displayed state — the buffer mutation above (and the eventual Postgres flush) is the durable source of truth. Failures publishing to Redis are logged and recorded as a metric, never raised here.

Source code in src/taskq/context.py

async def progress(
    self,
    *,
    step: int | None = None,
    percent: float | None = None,
    detail: str | None = None,
    data: dict[str, object] | None = None,
) -> None:
    """Report incremental progress for this job.

    Updates the in-memory coalesce buffer synchronously, then schedules a
    best-effort ``kind="progress"`` Redis publish as a background task
    when a client is connected — this call never blocks on the network.
    Raises :class:`~taskq.exceptions.ProgressTooLarge` if the serialised
    ``data`` payload exceeds ``WorkerSettings.progress_data_max_bytes``.

    All arguments are optional and merged last-writer-wins into the
    accumulated ``pending_state``. Intermediate calls between periodic
    flush ticks are coalesced: only the latest value for each field
    reaches Postgres. ``seq`` is strictly monotone across calls.

    The Redis publish is genuinely fire-and-forget: it may complete out
    of order relative to other in-flight publishes for the same job.
    Consumers reading the SSE/pub-sub stream already discard any event
    whose ``seq`` is not greater than the last one seen (see
    :mod:`taskq.web.progress`), so out-of-order or dropped publishes
    never corrupt displayed state — the buffer mutation above (and the
    eventual Postgres flush) is the durable source of truth. Failures
    publishing to Redis are logged and recorded as a metric, never
    raised here.
    """
    if data is not None and self._worker_settings is not None:
        serialised_len = len(dumps(data))
        limit = self._worker_settings.progress_data_max_bytes
        if serialised_len > limit:
            raise ProgressTooLarge(limit=limit, actual=serialised_len)

    if self._progress_buffers is None:
        return

    buffer = self._progress_buffers.get(self.job_id)
    if buffer is None:
        return

    buffer.pending_seq_delta += 1
    if step is not None:
        buffer.pending_state["step"] = step
    if percent is not None:
        buffer.pending_state["percent"] = percent
    if detail is not None:
        buffer.pending_state["detail"] = detail
    if data is not None:
        buffer.pending_state["data"] = data
    buffer.dirty = True

    seq = buffer.base_seq + buffer.pending_seq_delta

    if self._redis_client is not None and self._worker_settings is not None:
        coro = _publish_progress_event(
            self._redis_client,
            self._worker_settings,
            self.actor,
            self.job_id,
            step=step,
            percent=percent,
            detail=detail,
            data=data,
            seq=seq,
        )
        if self._pending_publish_tasks is not None:
            task = asyncio.create_task(coro, name=f"taskq-progress-publish-{self.job_id}")
            self._pending_publish_tasks.add(task)
            task.add_done_callback(self._pending_publish_tasks.discard)
        else:
            # No shared task set to hold a reference (e.g. a caller
            # constructing JobContext directly without a WorkerDeps) —
            # fall back to awaiting inline rather than risking the
            # scheduled task being garbage-collected mid-publish.
            await coro

CronScheduleSpec dataclass

CronScheduleSpec(
    actor: str,
    cron_expr: str,
    timezone: str = "UTC",
    dst_strategy: DstStrategy = "skip",
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    name: str = "",
    identity_key: IdentityKey | None = None,
    enabled: bool = True,
)

Immutable specification for a cron schedule row.

Created by the :func:cron decorator or constructed directly for register_cron(). payload_factory and static_payload are mutually exclusive — setting both raises :class:ValueError at construction time (via :func:cron).

dst_strategy controls how DST gaps and overlaps are handled. See :data:DstStrategy for the semantics of each strategy.

actor instance-attribute

actor: str

cron_expr instance-attribute

cron_expr: str

timezone class-attribute instance-attribute

timezone: str = 'UTC'

dst_strategy class-attribute instance-attribute

dst_strategy: DstStrategy = 'skip'

payload_factory class-attribute instance-attribute

payload_factory: str | None = None

static_payload class-attribute instance-attribute

static_payload: dict[str, object] | None = None

name class-attribute instance-attribute

name: str = ''

identity_key class-attribute instance-attribute

identity_key: IdentityKey | None = None

enabled class-attribute instance-attribute

enabled: bool = True

ScheduleHandle dataclass

ScheduleHandle(
    schedule_id: UUID,
    actor: str,
    cron_expr: str,
    timezone: str,
    enabled: bool,
    next_fire_at: datetime,
    _backend: Backend = field(),
    dst_strategy: DstStrategy = "skip",
    name: str = "",
    identity_key: IdentityKey | None = None,
)

Immutable handle for a cron schedule, returned by JobsClient methods.

The handle fields are a point-in-time snapshot of schedule state. Async methods delegate to the Backend injected at construction time (not part of the public __init__ signature) via ScheduleUpdateArgs. enable() passes ScheduleUpdateArgs(enabled=True); the backend resets consecutive_failures=0 and last_fire_error=NULL when enabled=True is set.

schedule_id instance-attribute

schedule_id: UUID

actor instance-attribute

actor: str

cron_expr instance-attribute

cron_expr: str

timezone instance-attribute

timezone: str

enabled instance-attribute

enabled: bool

next_fire_at instance-attribute

next_fire_at: datetime

dst_strategy class-attribute instance-attribute

dst_strategy: DstStrategy = 'skip'

name class-attribute instance-attribute

name: str = ''

identity_key class-attribute instance-attribute

identity_key: IdentityKey | None = None

disable async

disable() -> None

Source code in src/taskq/cron.py

async def disable(self) -> None:
    await self._backend.update_schedule(
        self.schedule_id,
        ScheduleUpdateArgs(enabled=False),
    )

enable async

enable() -> None

Source code in src/taskq/cron.py

async def enable(self) -> None:
    await self._backend.update_schedule(
        self.schedule_id,
        ScheduleUpdateArgs(enabled=True),
    )

delete async

delete() -> None

Source code in src/taskq/cron.py

async def delete(self) -> None:
    await self._backend.delete_schedule(self.schedule_id)

ActorConfigDriftError

ActorConfigDriftError(
    actor: str,
    field: Literal["queue", "metadata"],
    registered: str | dict[str, object] | None,
    stored: str | dict[str, object] | None,
)

Bases: TaskQError

One actor whose registered structural config differs from the stored row.

Only queue and metadata are structural — a mismatch there means a stale worker is routing an actor to the wrong place, which is a correctness bug. Capacity fields (max_concurrent, max_pending, result_ttl) are operator-owned and never raise this error; see :func:taskq.worker.startup.sync_actor_config.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    actor: str,
    field: Literal["queue", "metadata"],
    registered: str | dict[str, object] | None,
    stored: str | dict[str, object] | None,
) -> None:
    self.actor = actor
    self.field = field
    self.registered = registered
    self.stored = stored
    super().__init__(
        f"ActorConfigDrift: actor={actor}, field={field}, "
        f"registered={registered!r}, stored={stored!r}"
    )

hint class-attribute instance-attribute

hint = _ACTOR_CONFIG_DRIFT_HINT

actor instance-attribute

actor = actor

field instance-attribute

field = field

registered instance-attribute

registered = registered

stored instance-attribute

stored = stored

ActorConfigDriftList

ActorConfigDriftList(
    drifts: tuple[ActorConfigDriftError, ...],
)

Bases: TaskQError

Collected wrapper raised at worker startup when one or more actors have drift.

Source code in src/taskq/exceptions.py

def __init__(self, drifts: tuple[ActorConfigDriftError, ...]) -> None:
    self.drifts = drifts
    lines = [f"{len(drifts)} actor(s) have config drift:"]
    for d in drifts:
        lines.append(f"  {d}")
    lines.append(self.hint)
    super().__init__("\n".join(lines))

hint class-attribute instance-attribute

hint = _ACTOR_CONFIG_DRIFT_HINT

drifts instance-attribute

drifts = drifts

ActorDeregistrationError

ActorDeregistrationError(actor: str, detail: str)

Bases: TaskQError

Base for actor deregistration refusals.

Source code in src/taskq/exceptions.py

def __init__(self, actor: str, detail: str) -> None:
    self.actor = actor
    super().__init__(f"Cannot deregister actor {actor!r}: {detail}")

actor instance-attribute

actor = actor

ActorHasActiveJobsError

ActorHasActiveJobsError(
    actor: str,
    active_count: int,
    status_counts: dict[str, int],
    *,
    force: bool = False,
)

Bases: ActorDeregistrationError

Non-terminal jobs reference the actor.

Carries the count and per-status breakdown of the blocking jobs so the caller can decide whether to cancel them first or use force=True.

When force is True, the message reflects that running jobs cannot be cancelled by force=True — the caller must wait for them to finish or cancel them individually first.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    actor: str,
    active_count: int,
    status_counts: dict[str, int],
    *,
    force: bool = False,
) -> None:
    self.active_count = active_count
    self.status_counts = status_counts
    if force:
        detail = (
            f"{active_count} running job(s) still reference this actor"
            f" (breakdown: {status_counts}). Running jobs cannot be"
            f" cancelled by force=True \u2014 wait for them to finish or"
            f" cancel them individually first."
        )
    else:
        detail = (
            f"{active_count} non-terminal job(s) still reference this actor"
            f" (breakdown: {status_counts}). Cancel them first or pass"
            f" force=True to cancel pending/scheduled jobs automatically."
        )
    super().__init__(actor, detail)

active_count instance-attribute

active_count = active_count

status_counts instance-attribute

status_counts = status_counts

ActorHasEnabledSchedulesError

ActorHasEnabledSchedulesError(
    actor: str, schedule_ids: list[str]
)

Bases: ActorDeregistrationError

Enabled cron schedules reference the actor.

Carries the schedule IDs so the caller can disable or delete them first.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    actor: str,
    schedule_ids: list[str],
) -> None:
    self.schedule_ids = schedule_ids
    detail = (
        f"{len(schedule_ids)} enabled cron schedule(s) reference this actor"
        f" (ids: {schedule_ids}). Disable or delete them first or pass"
        f" force=True to disable them automatically."
    )
    super().__init__(actor, detail)

schedule_ids instance-attribute

schedule_ids = schedule_ids

ActorNotFoundError

ActorNotFoundError(actor: str)

Bases: ActorDeregistrationError

The actor_config row does not exist — nothing to deregister.

Currently raised only by :func:deregister_actor. Other ops (get, set_capacity) return None for missing rows.

Source code in src/taskq/exceptions.py

def __init__(self, actor: str) -> None:
    super().__init__(actor, "no stored actor_config row for this actor")

BackpressureError

BackpressureError(
    actor: str,
    pending: int = 0,
    max_pending: int | None = None,
)

Bases: TaskQError

Base class for synchronous enqueue-time backpressure signals.

Subclassed by SingletonCollisionError (singleton collision) and used directly by max_pending enforcement. The caller decides whether to retry, fail, or wait; the library does not block on capacity.

Source code in src/taskq/exceptions.py

def __init__(self, actor: str, pending: int = 0, max_pending: int | None = None) -> None:
    self.actor = actor
    self.pending = pending
    self.max_pending = max_pending
    super().__init__(
        f"BackpressureError: actor={actor}, pending={pending}, max_pending={max_pending}"
    )

actor instance-attribute

actor = actor

pending instance-attribute

pending = pending

max_pending instance-attribute

max_pending = max_pending

BatchAbortedError

BatchAbortedError(
    batch_id: UUID,
    consecutive_failures: int,
    threshold: int | None,
)

Bases: TaskQError

A batch was aborted because consecutive failures exceeded the threshold.

Running jobs are NOT cancelled by the abort — only pending and scheduled jobs are cancelled. Running jobs continue to completion. This matches the post-terminal-write hook design: the hook runs after the terminal write, so a job that was dispatched before the abort triggered will run to completion.

Source code in src/taskq/exceptions.py

def __init__(self, batch_id: UUID, consecutive_failures: int, threshold: int | None) -> None:
    self.batch_id = batch_id
    self.consecutive_failures = consecutive_failures
    self.threshold = threshold
    displayed_threshold = threshold if threshold is not None else 0
    super().__init__(
        f"batch {batch_id} aborted after {consecutive_failures} consecutive failures "
        f"(threshold={displayed_threshold})"
    )

batch_id instance-attribute

batch_id = batch_id

consecutive_failures instance-attribute

consecutive_failures = consecutive_failures

threshold instance-attribute

threshold = threshold

DependencyCycle

DependencyCycle(cycle_path: list[str])

Bases: TaskQError

A cycle was detected in the provider graph.

Source code in src/taskq/exceptions.py

def __init__(self, cycle_path: list[str]) -> None:
    if len(cycle_path) < 2:
        raise ValueError(
            f"cycle_path must contain at least 2 entries (got {len(cycle_path)!r})"
        )
    self.cycle_path = list(cycle_path)
    super().__init__(f"dependency cycle: {' -> '.join(cycle_path)}")

cycle_path instance-attribute

cycle_path = list(cycle_path)

DIError

Bases: TaskQError

Base for DI engine errors not covered by startup-validation.

Raised by the solver at resolution time for malformed annotations (e.g. multiple Scope markers in one Annotated parameter) or unresolvable forward references in actor signatures. Distinct from MissingProvider / ScopeViolation / DependencyCycle, which are raised at startup validation.

EmptyBatchError

EmptyBatchError(batch_id: UUID, expected: int, actual: int)

Bases: TaskQError

A batch has fewer jobs than the expected minimum.

This can happen when jobs were pruned before wait_for_batch ran, or when expected_size was set but jobs were never created. Pass on_empty="ok" to wait_for_batch to suppress the no-batch-row variant of this error.

Source code in src/taskq/exceptions.py

def __init__(self, batch_id: UUID, expected: int, actual: int) -> None:
    self.batch_id = batch_id
    self.expected = expected
    self.actual = actual
    super().__init__(
        f"batch {batch_id} has {actual} jobs, expected at least {expected}"
        + (
            ' — jobs may have been pruned; pass on_empty="ok" to suppress'
            " the no-batch-row variant"
            if actual == 0
            else ""
        )
    )

batch_id instance-attribute

batch_id = batch_id

expected instance-attribute

expected = expected

actual instance-attribute

actual = actual

EmptyFilterError

EmptyFilterError()

Bases: TaskQError

Raised when cancel_where is called with a filter that has no predicates.

A filter with no queue, status, actor, identity_key, batch_id, tags, or active predicate would match every job in the table — almost certainly a bug. The guardrail is intentionally loud: the caller must add at least one predicate or explicitly bypass with allow_empty_filter=True.

Source code in src/taskq/exceptions.py

def __init__(self) -> None:
    super().__init__(
        "cancel_where requires at least one filter predicate "
        "(queue, status, actor, identity_key, batch_id, tags, or active); "
        "an empty filter would cancel the entire table. "
        "Pass allow_empty_filter=True to override this guardrail."
    )

IllegalStateTransition

IllegalStateTransition(
    job_id: JobId,
    from_status: JobStatus,
    to_status: JobStatus,
)

Bases: TaskQError

Attempted to transition a job to a status not reachable from its current status.

Best-effort fast-path check only; the SQL WHERE clause is the authoritative serialization gate for concurrent writes.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    job_id: "JobId",
    from_status: "JobStatus",
    to_status: "JobStatus",
) -> None:
    self.job_id = job_id
    self.from_status = from_status
    self.to_status = to_status
    super().__init__(
        f"job {self.job_id} cannot transition from {self.from_status} to {self.to_status}"
    )

job_id instance-attribute

job_id = job_id

from_status instance-attribute

from_status = from_status

to_status instance-attribute

to_status = to_status

JobFailed

JobFailed(row: JobRow)

Bases: TaskQError

:meth:JobHandle.wait saw a non-success terminal state.

Carries the row so callers can inspect status, error_class, error_message, and error_traceback. Distinct from :class:ResultUnavailable (which means terminal but no result stored) and from the original actor exception (which is recorded on the row, not raised).

Source code in src/taskq/exceptions.py

def __init__(self, row: "JobRow") -> None:
    self.row = row
    super().__init__(
        f"job {row.id} ended in {row.status!r}"
        + (f": {row.error_class}: {row.error_message}" if row.error_class else ""),
    )

row instance-attribute

row = row

MaxPendingExceededError

MaxPendingExceededError(
    actor: str, current_count: int, max_pending: int
)

Bases: BackpressureError

Raised when an actor's max_pending queue-depth limit is reached.

current_count is the count of pending+scheduled jobs at the time of the pre-flight check. max_pending is the configured limit. The caller decides whether to retry, fail, or wait; the library does not block on capacity.

Source code in src/taskq/exceptions.py

def __init__(self, actor: str, current_count: int, max_pending: int) -> None:
    self.current_count = current_count
    super().__init__(actor, pending=current_count, max_pending=max_pending)

current_count instance-attribute

current_count = current_count

MissingProvider

MissingProvider(*, type_name: str, required_by: str)

Bases: TaskQError

A type was injected but no provider is registered.

Source code in src/taskq/exceptions.py

def __init__(self, *, type_name: str, required_by: str) -> None:
    self.type_name = type_name
    self.required_by = required_by
    super().__init__(f"no provider registered for {type_name} (required by {required_by})")

type_name instance-attribute

type_name = type_name

required_by instance-attribute

required_by = required_by

PartialBatchError

PartialBatchError(
    *,
    succeeded_count: int,
    failed_items: list[tuple[int, Exception]],
    total: int,
)

Bases: TaskQError

Raised when an autonomous enqueue_batch partially fails.

Items enqueued before the first failure are committed; remaining items are not inserted. succeeded_count is the number of items that were successfully enqueued. failed_items maps the index of each failed item to its exception. total is the original batch size.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    *,
    succeeded_count: int,
    failed_items: list[tuple[int, Exception]],
    total: int,
) -> None:
    self.succeeded_count = succeeded_count
    self.failed_items = failed_items
    self.total = total
    super().__init__(
        f"PartialBatchError: {succeeded_count}/{total} succeeded, "
        f"{len(failed_items)} failed at indices: {[i for i, _ in failed_items]}"
    )

succeeded_count instance-attribute

succeeded_count = succeeded_count

failed_items instance-attribute

failed_items = failed_items

total instance-attribute

total = total

PayloadValidationError

PayloadValidationError(
    detail: str,
    *,
    actor: str | None = None,
    payload_schema_ver: str | None = None,
    validation_errors: list[dict[str, object]]
    | None = None,
)

Bases: TaskQError

Pydantic validation failed at enqueue or dispatch.

At enqueue: raised before the row is inserted ('fail at the door'). At dispatch: causes the job to transition to 'failed' with error_class='PayloadValidationError'. Non-retryable in both cases regardless of the actor's retry policy.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    detail: str,
    *,
    actor: str | None = None,
    payload_schema_ver: str | None = None,
    validation_errors: list[dict[str, object]] | None = None,
) -> None:
    self.actor = actor
    self.payload_schema_ver = payload_schema_ver
    self.validation_errors: list[dict[str, object]] = validation_errors or []
    super().__init__(detail)

actor instance-attribute

actor = actor

payload_schema_ver instance-attribute

payload_schema_ver = payload_schema_ver

validation_errors instance-attribute

validation_errors: list[dict[str, object]] = (
    validation_errors or []
)

ProgressTooLarge

ProgressTooLarge(limit: int, actual: int)

Bases: TaskQError

Raised when progress data payload exceeds the configured size limit.

limit is the configured cap in bytes (WorkerSettings.progress_data_max_bytes). actual is the serialised byte length of the data dict that was rejected. Non-retryable: the caller must reduce the payload before retrying.

Source code in src/taskq/exceptions.py

def __init__(self, limit: int, actual: int) -> None:
    self.limit = limit
    self.actual = actual
    super().__init__(f"Progress data payload {actual}B exceeds limit {limit}B")

limit instance-attribute

limit = limit

actual instance-attribute

actual = actual

ReservationUnavailable

ReservationUnavailable(
    bucket_name: str,
    retry_after: timedelta,
    *,
    source: Literal[
        "reservation", "rate_limit"
    ] = "reservation",
)

Bases: TaskQError

A ConcurrencyReservation slot could not be acquired.

When the upstream RateLimitDecision.retry_after is None, callers MUST substitute DEFAULT_RESERVATION_BACKOFF. When it is timedelta(0) (allowed decisions) callers MUST pass it through unchanged — do NOT use a truthiness coalesce (x or DEFAULT_RESERVATION_BACKOFF) because timedelta(0) is falsy and would be wrongly replaced.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    bucket_name: str,
    retry_after: timedelta,
    *,
    source: Literal["reservation", "rate_limit"] = "reservation",
) -> None:
    if retry_after < timedelta(0):
        raise ValueError(f"retry_after must be non-negative, got {retry_after!r}")
    super().__init__(f"no reservation slot in {bucket_name!r}")
    self.bucket_name = bucket_name
    self.retry_after = retry_after
    self.source = source

bucket_name instance-attribute

bucket_name = bucket_name

retry_after instance-attribute

retry_after = retry_after

source instance-attribute

source = source

ResultTooLarge

Bases: TaskQError

Terminal result exceeded WorkerSettings.result_max_bytes.

Non-retryable: the actor already ran to completion and a re-run returns the same oversized value, so retrying only burns the remaining attempts (re-running the actor's side effects each time) before the job fails anyway. Classified alongside PayloadValidationError in :meth:taskq.retry.RetryClassifier.classify.

ResultUnavailable

ResultUnavailable(row: JobRow)

Bases: TaskQError

:meth:JobHandle.wait saw a terminal state but no usable result.

Causes: - result TTL expired before the call; - actor returned None while R is non-None (treated as schema mismatch, not a value); - row stored result=NULL for a non-success status.

Carries the row for inspection.

Source code in src/taskq/exceptions.py

def __init__(self, row: "JobRow") -> None:
    self.row = row
    super().__init__(f"job {row.id} has no stored result")

row instance-attribute

row = row

RetryAfter

RetryAfter(
    delay: timedelta, *, consume_budget: bool = True
)

Bases: TaskQError

Schedule retry at specific delay. Consumes retry budget by default.

Source code in src/taskq/exceptions.py

def __init__(self, delay: timedelta, *, consume_budget: bool = True) -> None:
    if delay < timedelta(0):
        raise ValueError(f"delay must be non-negative, got {delay!r}")
    super().__init__(f"retry after {delay}")
    self.delay = delay
    self.consume_budget = consume_budget

delay instance-attribute

delay = delay

consume_budget instance-attribute

consume_budget = consume_budget

SchemaNotMigratedError

SchemaNotMigratedError(schema: str)

Bases: TaskQError

Backend raised UndefinedTableError — the TaskQ schema is missing.

Translated by the client layer (:mod:taskq.client._jobs) from an asyncpg.exceptions.UndefinedTableError on the enqueue/get/list/cancel paths, so operators see an actionable message instead of a raw asyncpg traceback. The original exception is chained via __cause__.

Source code in src/taskq/exceptions.py

def __init__(self, schema: str) -> None:
    self.schema = schema
    super().__init__(
        f"TaskQ schema {schema!r} is missing or not migrated. "  # noqa: S608  # Why: human-readable error message, not a SQL query; ruff's SQL-injection heuristic false-positives on the word "schema" near f-string interpolation.
        f"Run `taskq migrate up` to create/update it, or set "
        f"TASKQ_MIGRATE_ON_START=true to migrate automatically at worker startup."
    )

schema instance-attribute

schema = schema

ScopedIdempotencyMigrationPendingError

ScopedIdempotencyMigrationPendingError(
    *,
    actor: str | None = None,
    idempotency_key: str | None = None,
    idempotency_scope: str | None = None,
    detail: str | None = None,
)

Bases: TaskQError

idempotency_scope was used, but the schema has not yet had 01.00.03_01_post_idempotency_scope_drop_old_index.sql applied.

Between applying 01.00.03_01_pre_idempotency_scope.sql and its post counterpart (the rolling-deploy window every worker's schema passes through), BOTH the old global jobs_idempotency_key_uniq index (on idempotency_key alone) and the new composite jobs_idempotency_scope_key_uniq index (on (idempotency_scope, idempotency_key)) exist simultaneously — this is deliberate, see the "PHASE OBLIGATIONS" comment in the pre migration file, and is what keeps pre-this-release code's unscoped ON CONFLICT (idempotency_key) working unmodified during the window.

The cost of that safety: enqueuing the same idempotency_key under two different idempotency_scope values satisfies the new composite index's ON CONFLICT target (no conflict there — the (scope, key) pair is new) but still violates the still-present old global index, which is not covered by that ON CONFLICT target. PostgreSQL raises UniqueViolationError for a conflict against a non-arbiter unique index unconditionally — the library deliberately does NOT catch that and silently fall back to a different scope's row, because doing so would return the wrong job for the scope the caller actually asked for, silently, which is a worse failure mode than a loud, explicit error for a purely transitional migration-window condition. Raised instead of letting the raw asyncpg.UniqueViolationError propagate.

Any call — scoped or unscoped — is affected whenever its idempotency_key already exists under a different scope: an unscoped call that reuses a key first written under a non-default scope raises this error just as a scoped call reusing an unscoped key does (verified against live PostgreSQL). Only brand-new keys and same-scope repeats are unaffected — a repeated key under the same scope (including two unscoped calls, which share the default '' scope) conflicts identically against both indexes for the exact same row, which ON CONFLICT DO NOTHING on the composite index resolves cleanly.

Resolution: confirm every worker is running the release that shipped idempotency_scope, then apply taskq migrate up --phase post (or a plain taskq migrate up) to drop the old index and activate scoped dedupe — or avoid passing idempotency_scope until that migration has run.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    *,
    actor: str | None = None,
    idempotency_key: str | None = None,
    idempotency_scope: str | None = None,
    detail: str | None = None,
) -> None:
    self.actor = actor
    self.idempotency_key = idempotency_key
    self.idempotency_scope = idempotency_scope
    self.detail = detail
    if idempotency_key is not None:
        what = (
            f"idempotency_key={idempotency_key!r} is already enqueued under a "
            "different idempotency_scope (this call: "
        )
        if actor is not None:
            what += f"actor={actor!r}, "
        what += f"idempotency_scope={idempotency_scope!r})"
    else:
        what = (
            "one or more items in the batch reuse an idempotency_key that "
            "already exists under a different idempotency_scope"
        )
    message = (
        f"enqueue rejected: {what}. This schema has not yet had "
        "01.00.03_01_post_idempotency_scope_drop_old_index.sql applied, so the "
        "legacy global jobs_idempotency_key_uniq index still enforces "
        "idempotency_key uniqueness across ALL scopes, and cross-scope key reuse "
        "is rejected rather than silently deduped against the wrong scope's job. "
        "No row was inserted or modified. To resolve: confirm every worker is on "
        "this release, then run `taskq migrate up --phase post` to activate "
        "scoped dedupe. Until then, do not reuse an idempotency_key under more "
        "than one scope (including the default '' scope) -- this fires in either "
        "direction, scoped-then-unscoped included."
    )
    if detail is not None:
        message = f"{message} (postgres detail: {detail})"
    super().__init__(message)

actor instance-attribute

actor = actor

idempotency_key instance-attribute

idempotency_key = idempotency_key

idempotency_scope instance-attribute

idempotency_scope = idempotency_scope

detail instance-attribute

detail = detail

ScopeViolation

ScopeViolation(
    *,
    from_scope: Scope,
    to_scope: Scope,
    type_name: str,
    dependent: str,
)

Bases: TaskQError

A provider depends on a shorter-lived scope than its own.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    *,
    from_scope: Scope,
    to_scope: Scope,
    type_name: str,
    dependent: str,
) -> None:
    self.from_scope = from_scope
    self.to_scope = to_scope
    self.type_name = type_name
    self.dependent = dependent
    super().__init__(
        f"{from_scope.name}-scoped {dependent} depends on {to_scope.name}-scoped {type_name}"
    )

from_scope instance-attribute

from_scope = from_scope

to_scope instance-attribute

to_scope = to_scope

type_name instance-attribute

type_name = type_name

dependent instance-attribute

dependent = dependent

SingletonCollisionError

SingletonCollisionError(
    actor: str,
    blocking_job_id: UUID | None = None,
    retry_after: timedelta | None = None,
)

Bases: BackpressureError

Raised when a singleton actor already has a job in pending/scheduled/running.

blocking_job_id is the UUID of the existing job from the Layer 1 pre-flight query; it is None when raised from the Layer 2 UniqueViolationError catch (the race path) because no pre-flight row was fetched.

retry_after is computed from the blocking job's schedule_to_close when available. It is None when the blocking job has no schedule_to_close set, or when raised from the Layer 2 catch path.

The heartbeat_interval * 4 fallback is intentionally NOT implemented — retry_after is computed from schedule_to_close only. Callers who need a poll cadence when retry_after is None should poll on their own schedule (research.md Gap 1, resolution path (a)). Reason: heartbeat_interval is not available at the backend enqueue boundary; propagating it would require enlarging the backend constructor surface and is out of scope.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    actor: str,
    blocking_job_id: UUID | None = None,
    retry_after: timedelta | None = None,
) -> None:
    self.blocking_job_id = blocking_job_id
    self.retry_after = retry_after
    super().__init__(actor)

blocking_job_id instance-attribute

blocking_job_id = blocking_job_id

retry_after instance-attribute

retry_after = retry_after

Snooze

Snooze(delay: timedelta)

Bases: TaskQError

Job returns control with new scheduled_at; does not consume retry budget.

Source code in src/taskq/exceptions.py

def __init__(self, delay: timedelta) -> None:
    if delay < timedelta(0):
        raise ValueError(f"delay must be non-negative, got {delay!r}")
    super().__init__(f"snooze for {delay}")
    self.delay = delay

delay instance-attribute

delay = delay

SubEnqueueError

SubEnqueueError(
    failed_items: list[tuple[EnqueueArgs, Exception]],
)

Bases: TaskQError

Raised by flush_buffer() when one or more buffered sub-job enqueues fail after parent commit.

failed_items carries each failed EnqueueArgs and the exception that caused the enqueue to fail. The parent job has already been marked succeeded — this exception signals that child jobs were lost.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    failed_items: "list[tuple[EnqueueArgs, Exception]]",
) -> None:
    self.failed_items = failed_items
    super().__init__(
        f"SubEnqueueError: {len(failed_items)} sub-job(s) failed to enqueue after parent commit"
    )

failed_items instance-attribute

failed_items = failed_items

TaskQError

Bases: Exception

Base for all library-raised exceptions.

WorkerOwnershipMismatch

WorkerOwnershipMismatch(
    job_id: UUID, expected: UUID, actual: UUID | None
)

Bases: TaskQError

Terminal write predicate failed: job exists but is owned by a different worker.

Source code in src/taskq/exceptions.py

def __init__(
    self,
    job_id: UUID,
    expected: UUID,
    actual: UUID | None,
) -> None:
    self.job_id = job_id
    self.expected = expected
    self.actual = actual
    super().__init__(f"job {self.job_id} owned by {actual}, expected {expected}")

job_id instance-attribute

job_id = job_id

expected instance-attribute

expected = expected

actual instance-attribute

actual = actual

ErrorReporter

Bases: Protocol

Vendor-neutral hook for routing terminal job failures to external systems.

Implementations capture the error and job row, then forward to a vendor-specific backend (Sentry, Datadog, a DLQ, etc.). The library calls :meth:report when a job reaches a terminal failure state — either because retries were exhausted or because the error was non-retryable.

The call is wrapped in a try/except by :func:invoke_error_reporter; a failing reporter never crashes the worker. Reporter failures are counted on the taskq.error_reporter.failures counter with a reporter_type attribute.

Register an :class:ErrorReporter instance as a DI provider (registry.register_value(ErrorReporter, Scope.PROCESS, my_reporter)) or pass it directly to the worker bootstrap.

report async

report(job: JobRow, exception: BaseException) -> None

Source code in src/taskq/obs/error_reporter.py

async def report(self, job: JobRow, exception: BaseException) -> None: ...

NullErrorReporter

Default no-op :class:ErrorReporter — silently drops all reports.

Used when no vendor-specific error routing is configured. Instances are stateless and safe to share.

report async

report(job: JobRow, exception: BaseException) -> None

Source code in src/taskq/obs/error_reporter.py

async def report(self, job: JobRow, exception: BaseException) -> None:
    return None

ProgressEvent

Bases: BaseModel

Point-in-time progress snapshot published to Redis for SSE/stream fanout.

Covers both kind="progress" (incremental update) and kind="state_change" (terminal or status transition) events. The exclude_none=True flag on :meth:model_dump_json suppresses null fields so the JSON payload stays compact on the wire.

model_config class-attribute instance-attribute

model_config = ConfigDict(frozen=True)

v class-attribute instance-attribute

v: int = 1

kind instance-attribute

kind: Literal['progress', 'state_change']

job_id instance-attribute

job_id: UUID

actor instance-attribute

actor: str

ts instance-attribute

ts: datetime

seq instance-attribute

seq: int

status instance-attribute

status: str

step class-attribute instance-attribute

step: int | None = None

percent class-attribute instance-attribute

percent: float | None = None

detail class-attribute instance-attribute

detail: str | None = None

data class-attribute instance-attribute

data: dict[str, object] | None = None

terminal class-attribute instance-attribute

terminal: bool = False

Fail

Bases: BaseModel

Fail decision: the job will not be retried.

model_config class-attribute instance-attribute

model_config = ConfigDict(frozen=True)

error_class instance-attribute

error_class: str

retryable instance-attribute

retryable: bool

JobRetryState

Bases: NamedTuple

Projection of JobRow columns consumed by the retry classifier.

schedule_to_close is no longer an input to classification — the SQL deadline guard in mark_failed_or_retry is the single deadline arbiter (one arbiter per predicate; see docs/architecture.md §Clock Domains). The field is retained on the projection for observability and hook consumers. start_to_close is reserved for per-attempt timeout enforcement at the consumer level (asyncio.wait_for); not used by the classifier.

attempt instance-attribute

attempt: int

max_attempts instance-attribute

max_attempts: int

retry_kind instance-attribute

retry_kind: RetryKind

schedule_to_close instance-attribute

schedule_to_close: datetime | None

start_to_close instance-attribute

start_to_close: timedelta | None

Retry

Bases: BaseModel

Retry decision: reschedule the job after retry_delay.

The delay — not a computed timestamp — is the decision payload: the backend derives scheduled_at = now() + retry_delay and the scheduled/pending status from its own clock (single arbiter, immune to app↔DB clock skew). retry_delay=0 means "retry immediately" (lands pending).

model_config class-attribute instance-attribute

model_config = ConfigDict(frozen=True)

retry_delay instance-attribute

retry_delay: timedelta

RetryClassifier

Pure classifier that maps an exception + policy to a RetryDecision.

The classifier decides retry-kind and backoff only — it is deliberately NOT a deadline arbiter. schedule_to_close is arbitrated by the SQL guard in mark_failed_or_retry (single arbiter, the backend's clock); a Python-side pre-check computed from the worker's clock would disagree with it under app↔DB skew and kill jobs early (or rubber-stamp them).

classify staticmethod

classify(
    policy: RetryPolicy,
    non_retryable_exceptions: tuple[
        type[BaseException], ...
    ],
    exception: BaseException,
    attempt: int,
    *,
    max_retry_backoff: timedelta = DEFAULT_MAX_RETRY_BACKOFF,
    override: RetryOverride | None = None,
) -> RetryDecision

Source code in src/taskq/retry.py

@staticmethod
def classify(
    policy: RetryPolicy,
    non_retryable_exceptions: tuple[type[BaseException], ...],
    exception: BaseException,
    attempt: int,
    *,
    max_retry_backoff: timedelta = DEFAULT_MAX_RETRY_BACKOFF,
    override: RetryOverride | None = None,
) -> RetryDecision:
    if isinstance(exception, non_retryable_exceptions):
        return Fail(error_class=type(exception).__name__, retryable=False)

    if isinstance(exception, PayloadValidationError):
        return Fail(error_class="PayloadValidationError", retryable=False)

    # Why: the actor already ran to completion — the failure is the size
    # of the value it returned, which a re-run reproduces exactly. Left
    # retryable, a single oversized result burns every remaining attempt
    # (re-running the actor's side effects each time) before landing in
    # 'failed' anyway.
    if isinstance(exception, ResultTooLarge):
        return Fail(error_class="ResultTooLarge", retryable=False)

    if isinstance(exception, ValidationError):
        return Fail(error_class="PayloadValidationError", retryable=False)

    effective_kind = (
        override.kind if override is not None and override.kind is not None else policy.kind
    )
    override_delay = override.delay if override is not None else None

    if effective_kind == "non_retryable":
        return Fail(error_class=type(exception).__name__, retryable=False)

    if effective_kind == "transient":
        if attempt < policy.max_attempts:
            return RetryClassifier._retry_decision(
                policy,
                attempt,
                max_retry_backoff=max_retry_backoff,
                override_delay=override_delay,
            )
        return Fail(error_class=type(exception).__name__, retryable=False)

    # effective_kind == "indefinite"
    return RetryClassifier._retry_decision(
        policy,
        attempt,
        max_retry_backoff=max_retry_backoff,
        override_delay=override_delay,
    )

RetryOverride

Bases: BaseModel

Per-exception override returned by an actor's retry_classifier hook.

Both fields are optional; None means "use the actor's static RetryPolicy/computed backoff for this field." Returning a RetryOverride with only kind set lets one exception type branch into different retry behaviour per occurrence — e.g. an HTTP 429 response goes indefinite while a 404 response on the same exception type goes non_retryable. Returning one with only delay set lets the actor honour a server-provided retry-after duration instead of the policy's computed exponential/linear backoff, while max_retry_backoff still applies as a safety ceiling so a malicious or malformed header cannot strand a job.

model_config class-attribute instance-attribute

model_config = ConfigDict(frozen=True)

kind class-attribute instance-attribute

kind: RetryKind | None = None

delay class-attribute instance-attribute

delay: timedelta | None = None

RetryPolicy

Bases: BaseModel

Policy controlling retry behaviour for an actor.

model_config class-attribute instance-attribute

model_config = ConfigDict(frozen=True)

kind class-attribute instance-attribute

kind: RetryKind = 'transient'

max_attempts class-attribute instance-attribute

max_attempts: int = 3

time_budget class-attribute instance-attribute

time_budget: timedelta | None = None

backoff class-attribute instance-attribute

backoff: Literal["exponential", "linear", "fixed"] = (
    "exponential"
)

base class-attribute instance-attribute

base: timedelta = timedelta(seconds=5)

cap class-attribute instance-attribute

cap: timedelta = timedelta(hours=1)

jitter class-attribute instance-attribute

jitter: float = 0.2

OIDCSettings

Bases: DotEnvConfig

OIDC SSO configuration (loaded from TASKQ_OIDC_* env vars).

env_prefix class-attribute instance-attribute

env_prefix = 'TASKQ_OIDC_'

issuer class-attribute instance-attribute

issuer: str = Field(
    default="",
    description="OIDC discovery issuer URL (e.g. https://login.microsoftonline.com/{tenant}/v2.0).",
)

client_id class-attribute instance-attribute

client_id: str = Field(
    default="",
    description="OAuth2 client ID registered at the IdP.",
)

client_secret class-attribute instance-attribute

client_secret: str = Field(
    default="", description="OAuth2 client secret."
)

redirect_uri class-attribute instance-attribute

redirect_uri: str = Field(
    default="",
    description="Must match the app registration's configured redirect URI.",
)

session_secret class-attribute instance-attribute

session_secret: str = Field(
    default="",
    description="Signing key for session cookies; use >=32 bytes of random data. Rotate to invalidate all sessions.",
)

session_max_age_seconds class-attribute instance-attribute

session_max_age_seconds: int = Field(
    default=28800,
    ge=60,
    description="Session lifetime (s). Default 8h.",
)

scope class-attribute instance-attribute

scope: str = Field(
    default="openid profile email",
    description="OIDC scopes. Add 'Group.Read.All' for the Entra overage group_resolver (Graph API /me/memberOf).",
)

group_claim class-attribute instance-attribute

group_claim: str | None = Field(
    default=None,
    description="ID token claim name for groups (e.g. 'groups', 'roles'). None = authentication-only authorization.",
)

allowed_groups class-attribute instance-attribute

allowed_groups: str = Field(
    default="",
    description="Comma-separated group allowlist.",
)

allowed_groups_set property

allowed_groups_set: frozenset[str]

SAMLSettings

Bases: DotEnvConfig

SAML SSO configuration (loaded from TASKQ_SAML_* env vars).

env_prefix class-attribute instance-attribute

env_prefix = 'TASKQ_SAML_'

entity_id class-attribute instance-attribute

entity_id: str = Field(
    default="", description="SP entity ID."
)

acs_url class-attribute instance-attribute

acs_url: str = Field(
    default="",
    description="Assertion Consumer Service URL.",
)

idp_entity_id class-attribute instance-attribute

idp_entity_id: str = Field(
    default="", description="IdP entity ID."
)

idp_sso_url class-attribute instance-attribute

idp_sso_url: str = Field(
    default="", description="IdP SSO endpoint."
)

idp_x509_cert class-attribute instance-attribute

idp_x509_cert: str = Field(
    default="", description="IdP signing certificate (PEM)."
)

sp_x509_cert class-attribute instance-attribute

sp_x509_cert: str | None = Field(
    default=None,
    description="SP cert (signed requests / encrypted assertions).",
)

sp_private_key class-attribute instance-attribute

sp_private_key: str | None = Field(
    default=None, description="SP private key (PEM)."
)

session_secret class-attribute instance-attribute

session_secret: str = Field(
    default="",
    description="Signing key for session cookies.",
)

session_max_age_seconds class-attribute instance-attribute

session_max_age_seconds: int = Field(
    default=28800,
    ge=60,
    description="Session lifetime (s). Default 8h.",
)

group_attribute class-attribute instance-attribute

group_attribute: str | None = Field(
    default=None,
    description="SAML attribute name for groups.",
)

allowed_groups class-attribute instance-attribute

allowed_groups: str = Field(
    default="",
    description="Comma-separated group allowlist.",
)

allowed_groups_set property

allowed_groups_set: frozenset[str]

TaskQSettings

Bases: DotEnvConfig

Top-level TaskQ runtime configuration.

env_prefix class-attribute instance-attribute

env_prefix = 'TASKQ_'

pg_dsn class-attribute instance-attribute

pg_dsn: PostgresDsn = Field(
    default=PostgresDsn(
        "postgresql://taskq:taskq@localhost:5432/taskq"
    ),
    description="Direct (non-PgBouncer) DSN. LISTEN/NOTIFY and advisory locks need a session.",
)

schema_name class-attribute instance-attribute

schema_name: str = Field(
    default="taskq",
    validator=_schema_name_validator,
    description="Postgres schema for all TaskQ tables.",
)

redis_url class-attribute instance-attribute

redis_url: RedisDsn | None = Field(
    default=None,
    description="Optional Redis URL. Required for real-time progress fanout.",
)

environment class-attribute instance-attribute

environment: str | None = Field(
    default=None,
    description="TASKQ_ENVIRONMENT. Deployment environment label. The unauthenticated-admin WARNING ('admin-ui-no-auth') fires in EVERY environment whenever the admin UI is served without auth_dependency. 'dev' and 'development' additionally skip the fail-closed RuntimeError (the WARNING then notes the absence is the dev exemption); any other value (or None/empty) fails closed when admin_ui_require_auth is True.",
)

admin_max_sse_connections class-attribute instance-attribute

admin_max_sse_connections: int = Field(
    default=50,
    ge=1,
    description="TASKQ_ADMIN_MAX_SSE_CONNECTIONS. Maximum concurrent SSE connections the admin UI will serve. Used to size the connection-limit semaphore.",
)

progress_max_sse_connections class-attribute instance-attribute

progress_max_sse_connections: int = Field(
    default=50,
    ge=1,
    description="TASKQ_PROGRESS_MAX_SSE_CONNECTIONS. Maximum concurrent per-job progress SSE streams this process will serve. Each holds a Redis pubsub subscription and an asyncio task for as long as the client stays connected, so an uncapped endpoint is a resource-exhaustion surface on the app hosting the pipeline.",
)

admin_host class-attribute instance-attribute

admin_host: str = Field(
    default="0.0.0.0",
    description="TASKQ_ADMIN_HOST. Bind address for ``taskq ui serve``.",
)

admin_port class-attribute instance-attribute

admin_port: int = Field(
    default=8080,
    ge=1,
    le=65535,
    description="TASKQ_ADMIN_PORT. Bind port for ``taskq ui serve``.",
)

admin_url class-attribute instance-attribute

admin_url: str = Field(
    default="http://localhost:8080",
    description="TASKQ_ADMIN_URL. Public base URL of the admin UI as seen from a browser. Used by the example trigger app to construct redirect URLs after enqueueing. In a shared-container deployment this is the external address of the admin process (e.g. http://localhost:8001). Override when admin and trigger app are on different hosts or ports.",
)

admin_ui_polling_interval_seconds class-attribute instance-attribute

admin_ui_polling_interval_seconds: float = Field(
    default=2.0,
    ge=0.1,
    description="TASKQ_ADMIN_UI_POLLING_INTERVAL_SECONDS. How often the admin UI polls PG in polling/degraded mode. Injected as poll_interval_ms into every template.",
)

admin_worker_liveness_seconds class-attribute instance-attribute

admin_worker_liveness_seconds: int = Field(
    default=30,
    ge=1,
    description="TASKQ_ADMIN_WORKER_LIVENESS_SECONDS. How recently a worker must have written last_seen_at to count as alive in the admin UI: it drives the 'queue has pending jobs but no alive worker' banner and the leader's watchdog_healthy verdict. Must comfortably exceed TASKQ_HEARTBEAT_INTERVAL (default 10 s), so the default 30 s is three beats; a deployment that lengthens the heartbeat, or whose PG is cross-region, has to raise this or every healthy worker reads as dead. Measured by Postgres against clock_timestamp(), never by the admin process's own clock.",
)

admin_ui_allow_rate_limit_reset class-attribute instance-attribute

admin_ui_allow_rate_limit_reset: bool = Field(
    default=False,
    description="TASKQ_ADMIN_UI_ALLOW_RATE_LIMIT_RESET. When True, the admin UI shows a reset button on the rate-limits page and serves the POST /rate-limits/{bucket_name}/reset endpoint. Default False for safety - prevents accidental resets in production.",
)

admin_ui_require_auth class-attribute instance-attribute

admin_ui_require_auth: bool = Field(
    default=True,
    description="TASKQ_ADMIN_UI_REQUIRE_AUTH. When True (the default), create_router raises RuntimeError if auth_dependency is None in a non-dev environment, failing closed. Set to False to suppress the error and allow an unauthenticated admin UI in non-dev (not recommended - only for air-gapped or localhost-only deployments).",
)

admin_ui_frame_ancestors class-attribute instance-attribute

admin_ui_frame_ancestors: str = Field(
    default="none",
    validator=_frame_ancestors_validator,
    description="TASKQ_ADMIN_UI_FRAME_ANCESTORS. Who may frame admin pages: 'none' (the default, nobody) or 'self' (the admin UI's own origin, for a host app that embeds the admin UI in its own dashboard). Emitted as both 'Content-Security-Policy: frame-ancestors ...' and the legacy 'X-Frame-Options' (DENY / SAMEORIGIN). CSRF is no defence against UI redress: the framed page is the real, authenticated, same-origin page, so a tricked click carries a valid token.",
)

admin_ui_secure_cookies class-attribute instance-attribute

admin_ui_secure_cookies: bool = Field(
    default=True,
    description="TASKQ_ADMIN_UI_SECURE_COOKIES. Sets the 'Secure' flag on the admin UI's CSRF cookie. A configured value, not one inferred from request.url.scheme: behind a TLS-terminating edge (Azure Application Gateway, App Service) the app sees plain http, so an inferred flag is silently dropped on a connection the browser reached over HTTPS. Set False only for local http dev, where a Secure cookie is rejected by the browser and the admin UI stops working.",
)

admin_actions_enabled class-attribute instance-attribute

admin_actions_enabled: bool = Field(
    default=False,
    description="TASKQ_ADMIN_ACTIONS_ENABLED. When True, the admin UI permits state-changing actions: run schedule now, enable/disable/skip a schedule, retry job, cancel job. Default False - prevents on-demand triggering of registered business logic, and silent suppression of scheduled work, via the admin UI without explicit opt-in. Separate from auth_dependency, which controls read access to all admin routes.",
)

pg_credential_provider class-attribute instance-attribute

pg_credential_provider: str | None = Field(
    default=None,
    description="TASKQ_PG_CREDENTIAL_PROVIDER. Module:attr reference to a PgCredentialProvider (e.g. myapp.auth:make_provider) - an instance, a zero-arg factory returning one, or the provider class. Every Postgres pool and dedicated connection is then built through it, so SIGHUP / TASKQ_RELOAD_INTERVAL rotate real credentials. The CLI options --pg-credential-provider (taskq worker / migrate / ui serve) override it. The ref is resolved, not validated, at load time: the import lives in the CLI so a bad ref exits 1 with a pointed message instead of a settings traceback. See docs/guides/managed-identities.md.",
)

redis_credential_provider class-attribute instance-attribute

redis_credential_provider: str | None = Field(
    default=None,
    description="TASKQ_REDIS_CREDENTIAL_PROVIDER. Module:attr reference to a RedisCredentialProvider, in the same shapes as pg_credential_provider. Requires TASKQ_REDIS_URL. Overridden by --redis-credential-provider.",
)

sso_backend class-attribute instance-attribute

sso_backend: str = Field(
    default="none",
    validator=_sso_backend_validator,
    description="TASKQ_SSO_BACKEND. Selects the SSO backend for the admin UI: 'none' (default, unauthenticated/BYO-auth), 'oidc' (taskq[oidc]), or 'saml' (taskq[saml]). See docs/guides/sso.md.",
)

health_token class-attribute instance-attribute

health_token: str = Field(
    default="",
    description="TASKQ_HEALTH_TOKEN. Bearer token for machine-to-machine access to health/metrics endpoints. When set, health and metrics routes require a matching 'Authorization: Bearer <token>' header. Leave empty for unauthenticated cluster-internal access - but see health_require_token, which fails closed on an empty token outside dev.",
)

health_require_token class-attribute instance-attribute

health_require_token: bool = Field(
    default=True,
    description="TASKQ_HEALTH_REQUIRE_TOKEN. When True (the default), taskq ui serve raises RuntimeError if health_token is empty in a non-dev environment, failing closed. Set to False to suppress the error and allow unauthenticated health/metrics endpoints in non-dev (e.g. when relying on network policy / cluster-internal-only access instead of a bearer token - note that many k8s liveness/readiness probes don't send auth headers by default, so enabling the token may require updating the probe config too).",
)

migrate_on_start class-attribute instance-attribute

migrate_on_start: bool = Field(
    default=False,
    description="TASKQ_MIGRATE_ON_START. When True, apply pending migrations before the admin UI accepts its first request. Aborts startup if migrations fail. Consumed ONLY by `taskq ui serve` -- the worker ignores it (and warns when it is set), because N worker replicas racing to migrate is the concurrent-migration hazard migrations are supposed to avoid. Migrate from a pre-deploy job or init container.",
)

example_host class-attribute instance-attribute

example_host: str = Field(
    default="0.0.0.0",
    description="TASKQ_EXAMPLE_HOST. Bind address for the example trigger app (uvicorn). Only consumed by the example app; ignored by the worker and admin UI.",
)

example_port class-attribute instance-attribute

example_port: int = Field(
    default=8000,
    ge=1,
    le=65535,
    description="TASKQ_EXAMPLE_PORT. Bind port for the example trigger app (uvicorn). Only consumed by the example app; ignored by the worker and admin UI.",
)

idempotency_key_max_bytes class-attribute instance-attribute

idempotency_key_max_bytes: int = Field(
    default=MAX_IDEMPOTENCY_KEY_BYTES,
    ge=1,
    le=IDEMPOTENCY_KEY_BYTES_CEILING,
    description="TASKQ_IDEMPOTENCY_KEY_MAX_BYTES. Maximum UTF-8 byte length of idempotency_key, and of idempotency_scope, each. Bytes rather than characters because the real bound is the composite unique index jobs_idempotency_scope_key_uniq: a btree v4 entry cannot exceed 2704 bytes, counted encoded. The ceiling keeps scope + key + index-tuple overhead under that, so no value here can turn a valid enqueue into a raw Postgres 'index row size ... exceeds btree version 4 maximum'. Raise it when keys are derived from URLs, composite business keys or opaque vendor cursors.",
)

oidc property

oidc: OIDCSettings

Lazily loaded OIDC sub-config (TASKQ_OIDC_* env vars).

Backed by dotenvmodel's cached() singleton: the environment is read on first access and the same instance returned thereafter.

Reload (e.g. on SIGHUP): call OIDCSettings.cached().reload() to re-read the environment and mutate the shared instance in place - every holder observes the new values - or OIDCSettings.reset_cached() to force the next access to re-load. Tests that change TASKQ_OIDC_* mid-process must do the same (or use cached_override()).

saml property

saml: SAMLSettings

Lazily loaded SAML sub-config (TASKQ_SAML_* env vars).

See :attr:oidc for the singleton/caching semantics and the SIGHUP-style reload recipe (SAMLSettings.cached().reload() / SAMLSettings.reset_cached()).

load classmethod

load(
    env: str | None = None,
    *,
    override: bool | None = None,
    env_dir: Path | str | None = None,
    read_dotfiles: bool | None = None,
    read_environ: bool | None = None,
    load_local: bool | None = None,
) -> Self

Load settings via dotenvmodel's cascading .env discovery.

All parameters are forwarded to DotEnvConfig.load unchanged (resolution: explicit argument > DOTENV_* env var > default). override=None keeps dotenvmodel's default precedence — the process environment beats .env files; pass override=True or set DOTENV_OVERRIDE=true to make .env files win instead. read_dotfiles=False / read_environ=False disable the .env cascade / the process environment respectively, per dotenvmodel's documented symmetry.

dotenvmodel logs a WARNING ("No .env files found in ") on every call when no .env file is present - noisy on every CLI invocation in projects that configure purely via real environment variables. read_dotfiles=False is not the answer: it silences the warning but disables the .env cascade entirely, a documented core TaskQ feature. This override instead installs a :class:_NoEnvFilesWarningFilter on the dotenvmodel logger for the duration of the call, dropping only that one warning; every other dotenvmodel warning (e.g. an invalid DOTENV_* knob value) remains visible.

WorkerSettings.load inherits this override via MRO, so worker startup gets the same quiet load.

Source code in src/taskq/settings.py

@classmethod
def load(
    cls,
    env: str | None = None,
    *,
    override: bool | None = None,
    env_dir: Path | str | None = None,
    read_dotfiles: bool | None = None,
    read_environ: bool | None = None,
    load_local: bool | None = None,
) -> Self:
    """Load settings via dotenvmodel's cascading ``.env`` discovery.

    All parameters are forwarded to ``DotEnvConfig.load`` unchanged
    (resolution: explicit argument > ``DOTENV_*`` env var > default).
    ``override=None`` keeps dotenvmodel's default precedence — the
    process environment beats ``.env`` files; pass ``override=True``
    or set ``DOTENV_OVERRIDE=true`` to make ``.env`` files win instead.
    ``read_dotfiles=False`` / ``read_environ=False`` disable the
    ``.env`` cascade / the process environment respectively, per
    dotenvmodel's documented symmetry.

    dotenvmodel logs a WARNING ("No .env files found in <cwd>") on
    every call when no ``.env`` file is present - noisy on every CLI
    invocation in projects that configure purely via real environment
    variables. ``read_dotfiles=False`` is not the answer: it silences
    the warning but disables the ``.env`` cascade entirely, a
    documented core TaskQ feature. This override instead installs a
    :class:`_NoEnvFilesWarningFilter` on the ``dotenvmodel`` logger
    for the duration of the call, dropping only that one warning;
    every other dotenvmodel warning (e.g. an invalid ``DOTENV_*``
    knob value) remains visible.

    ``WorkerSettings.load`` inherits this override via MRO, so worker
    startup gets the same quiet load.
    """
    dotenv_logger = logging.getLogger("dotenvmodel")
    no_env_files = _NoEnvFilesWarningFilter()
    dotenv_logger.addFilter(no_env_files)
    try:
        return super().load(
            env=env,
            override=override,
            env_dir=env_dir,
            read_dotfiles=read_dotfiles,
            read_environ=read_environ,
            load_local=load_local,
        )
    finally:
        dotenv_logger.removeFilter(no_env_files)

WorkerSettings

Bases: TaskQSettings

Worker-specific configuration with three-pool sizing and dual-DSN support.

Extends :class:TaskQSettings with pool-size knobs, dual-DSN fields, and the validated lock_lease >= 4 * heartbeat_interval invariant.

pg_dsn_direct class-attribute instance-attribute

pg_dsn_direct: PostgresDsn | None = Field(
    default=None,
    description="TASKQ_PG_DSN_DIRECT; falls back to pg_dsn when absent. Bypasses PgBouncer - used by dispatcher_pool, heartbeat_pool, notify_conn, and leader_conn.",
)

pg_dsn_pooled class-attribute instance-attribute

pg_dsn_pooled: PostgresDsn | None = Field(
    default=None,
    description="TASKQ_PG_DSN_POOLED; falls back to pg_dsn when absent. May route through PgBouncer transaction mode - used by worker_pool only.",
)

dispatcher_pool_size class-attribute instance-attribute

dispatcher_pool_size: int = Field(
    default=4,
    ge=1,
    description="TASKQ_DISPATCHER_POOL_SIZE. Max connections for the dispatcher pool. Bypasses PgBouncer.",
)

dispatcher_command_timeout class-attribute instance-attribute

dispatcher_command_timeout: float = Field(
    default=5.0,
    ge=1.0,
    description="TASKQ_DISPATCHER_COMMAND_TIMEOUT (seconds). Per-query timeout for the dispatcher pool and the TaskQ-built leader connections (election, cron, monitor), and the single deadline wrapped around each period-1 leader-loop iteration (scheduled_wake, cron): a stalled PG errors the iteration instead of hanging the loop past its staleness budget. Checked at load time when the watchdog is enabled: timeout + the 1.0s leader-loop period must be < max(period x watchdog_tick_grace_factor, watchdog_stale_floor) for the period-1 leader loops (scheduled_wake, cron), so a timeout-capped iteration can never false-trip the stale-loop detector on a healthy worker. The producer loop is not checked (its multi-statement dispatch_batch is not wrapped in a single asyncio.timeout).",
)

dispatch_oversample class-attribute instance-attribute

dispatch_oversample: int = Field(
    default=2,
    ge=1,
    le=1000,
    description="TASKQ_DISPATCH_OVERSAMPLE. Multiplier for per-actor candidate gathering in the dispatch SQL. Each LATERAL reads residual x oversample candidates. Higher values absorb more identity collisions and multi-producer contention. Default 2 (tolerates 50% dupe identities). Set 1 when no identity_key is used and single-producer.",
)

dispatch_scope_by_home_queue class-attribute instance-attribute

dispatch_scope_by_home_queue: bool = Field(
    default=False,
    description="TASKQ_DISPATCH_SCOPE_BY_HOME_QUEUE. When True, restrict per_actor_capacity to actors whose home queue (actor_config.queue) the worker subscribes to. Lowers per-cycle probe count at the cost of not dispatching enqueue(queue=...) override jobs whose actor's home queue is not subscribed. Default False (override-safe).",
)

heartbeat_pool_size class-attribute instance-attribute

heartbeat_pool_size: int = Field(
    default=4,
    ge=1,
    description="TASKQ_HEARTBEAT_POOL_SIZE. Max connections for the heartbeat pool. Bypasses PgBouncer.",
)

heartbeat_command_timeout class-attribute instance-attribute

heartbeat_command_timeout: float = Field(
    default=2.0,
    gt=0.0,
    description="TASKQ_HEARTBEAT_COMMAND_TIMEOUT (seconds). Per-query timeout for the heartbeat pool, deliberately tighter than dispatcher_command_timeout: a beat that takes longer than the tick cannot keep a lock lease alive, so failing it fast is the point. Raise it on a loaded or cross-region Postgres — max_heartbeat_failures consecutive timeouts self-terminate the worker, and before this was a setting the 2 s literal made that unavoidable. Must be > 0: asyncpg reads 0 as 'no timeout', which turns a stalled beat into a hang.",
)

max_concurrency class-attribute instance-attribute

max_concurrency: int = Field(
    default=8,
    ge=1,
    description="TASKQ_MAX_CONCURRENCY. Upper bound on concurrent jobs. worker_pool max_size = int(max_concurrency * 1.5).",
)

heartbeat_interval class-attribute instance-attribute

heartbeat_interval: float = Field(
    default=10.0,
    ge=0.5,
    description="TASKQ_HEARTBEAT_INTERVAL (seconds). Period between heartbeat ticks.",
)

lock_lease class-attribute instance-attribute

lock_lease: float = Field(
    default=60.0,
    ge=1.0,
    description="TASKQ_LOCK_LEASE (seconds). Time before a held lock is reclaimed by the recovery sweep. Must be >= 4 * heartbeat_interval.",
)

max_heartbeat_failures class-attribute instance-attribute

max_heartbeat_failures: int = Field(
    default=3,
    ge=1,
    description="TASKQ_MAX_HEARTBEAT_FAILURES. Consecutive heartbeat failures before the worker self-terminates.",
)

sweep_interval class-attribute instance-attribute

sweep_interval: float = Field(
    default=30.0,
    ge=1.0,
    description="TASKQ_SWEEP_INTERVAL (seconds). Period between leader sweep loop iterations — reclaim_expired_locks, sweep_expired_results, cleanup_stale_workers, and idle keyed-ref eviction. Lower values reduce recovery latency for crashed workers at the cost of more frequent PG queries.",
)

queue_depth_interval class-attribute instance-attribute

queue_depth_interval: float = Field(
    default=15.0,
    ge=1.0,
    description="TASKQ_QUEUE_DEPTH_INTERVAL (seconds). Period between queue-depth metrics sampling iterations.",
)

reservation_slots_interval class-attribute instance-attribute

reservation_slots_interval: float = Field(
    default=15.0,
    ge=1.0,
    description="TASKQ_RESERVATION_SLOTS_INTERVAL (seconds). Period between reservation-slot metrics sampling iterations.",
)

stranded_jobs_interval class-attribute instance-attribute

stranded_jobs_interval: float = Field(
    default=60.0,
    ge=1.0,
    description="TASKQ_STRANDED_JOBS_INTERVAL (seconds). Period between stranded-jobs (pending jobs whose actor has no actor_config) warning checks.",
)

termination_grace_period class-attribute instance-attribute

termination_grace_period: float = Field(
    default=75.0,
    ge=5.0,
    description="TASKQ_TERMINATION_GRACE_PERIOD (seconds). Total wall-clock budget from SIGTERM to forced exit; the shutdown watchdog counts it down from the first shutdown signal. Must satisfy cancellation_grace + cleanup_grace < termination_grace - 5, and should cover the modelled worst case cancellation_grace + cleanup_grace + the ~27s bounded-close teardown tail (see WorkerSettings.worst_case_shutdown_seconds) — the default does: 30 + 10 + 27 = 67s, with headroom for the ~72s sibling-crash path that model understates. A value below the worst case still loads but logs shutdown-budget-exceeds-termination-grace at startup. Size the pod/container grace (terminationGracePeriodSeconds / stop_grace_period) from the same worst case, not from this setting alone.",
)

cancellation_grace_period class-attribute instance-attribute

cancellation_grace_period: float = Field(
    default=30.0,
    ge=0.0,
    description="TASKQ_CANCELLATION_GRACE_PERIOD (seconds). Cooperative cancel phase duration.",
)

cleanup_grace_period class-attribute instance-attribute

cleanup_grace_period: float = Field(
    default=10.0,
    ge=0.0,
    description="TASKQ_CLEANUP_GRACE_PERIOD (seconds). Force-cancel cleanup grace.",
)

reclaim_event_visibility_delay class-attribute instance-attribute

reclaim_event_visibility_delay: float = Field(
    default=RECLAIM_EVENT_VISIBILITY_DELAY.total_seconds(),
    ge=0.0,
    description="TASKQ_RECLAIM_EVENT_VISIBILITY_DELAY (seconds). Trailing-watermark margin poll_reclaim_events()/TaskQ.watch_reclaims() apply before returning a job_events row, so an out-of-commit-order sibling with a lower event_id has time to appear first (see docs/architecture.md's crash-reclaim section). Correctness assumes every job_events writer transaction commits within this margin of its INSERT; raise it if sweeps run under heavy lock contention or against very large batches, lower it if latency matters more and writes are known to be fast. A writer that exceeds the margin can cause a silently missed event - this is a real, not merely theoretical, risk under misconfiguration.",
)

max_retry_backoff class-attribute instance-attribute

max_retry_backoff: timedelta = Field(
    default=DEFAULT_MAX_RETRY_BACKOFF,
    description="TASKQ_MAX_RETRY_BACKOFF (interval). Global ceiling on retry backoff per attempt - caps the per-actor RetryPolicy.cap so a misconfigured actor (e.g. cap=timedelta(days=365)) cannot strand jobs for an unreasonably long time. Default 24 h: conservative, matches one standard on-call rotation, and mirrors Dramatiq's DEFAULT_MAX_BACKOFF philosophy ",
)

default_start_to_close class-attribute instance-attribute

default_start_to_close: timedelta | None = Field(
    default=None,
    validator=_positive_timedelta,
    description="TASKQ_DEFAULT_START_TO_CLOSE (interval). Worker-side fallback per-attempt execution timeout, applied only when a job has no start_to_close of its own (neither passed at enqueue time nor declared as an @actor(start_to_close=...) default). None (the default) means unbounded - matches existing behaviour, opt-in only. Set this to give every actor on this worker a safety-net wall-clock budget per attempt, preventing a hung or infinite-looping actor from occupying a coroutine slot forever, without having to configure start_to_close on every individual actor. Precedence (highest wins): per-enqueue start_to_close > @actor(start_to_close=...) > this setting. This does not affect schedule_to_close, which is a separate, unrelated deadline for the job's *overall* retry budget across all attempts - start_to_close bounds a single attempt's wall-clock time.",
)

rate_limit_pg_fallback_enabled class-attribute instance-attribute

rate_limit_pg_fallback_enabled: bool = Field(
    default=True,
    description="TASKQ_RATE_LIMIT_PG_FALLBACK_ENABLED. When False, Redis errors propagate instead of triggering PG fallback.",
)

max_keyed_reservations class-attribute instance-attribute

max_keyed_reservations: int = Field(
    default=10000,
    ge=1,
    description="TASKQ_MAX_KEYED_RESERVATIONS. Guardrail on the number of distinct keyed-reservation entries tracked in memory. When the limit is reached, new keyed reservations raise ReservationUnavailable. Tune to your workload's expected key cardinality.",
)

max_keyed_rate_limits class-attribute instance-attribute

max_keyed_rate_limits: int = Field(
    default=10000,
    ge=1,
    description="TASKQ_MAX_KEYED_RATE_LIMITS. Guardrail on the number of distinct keyed-rate-limit entries tracked in memory. When the limit is reached, new keyed rate limits raise ReservationUnavailable. Independent from max_keyed_reservations, which governs keyed reservations only. Tune to your workload's expected key cardinality.",
)

metrics_port class-attribute instance-attribute

metrics_port: int = Field(
    default=9090,
    ge=1,
    le=65535,
    description="TASKQ_METRICS_PORT. Bind port for the standalone Prometheus metrics server (taskq health metrics --port). The in-process FastAPI mount ignores this field.",
)

health_enabled class-attribute instance-attribute

health_enabled: bool = Field(
    default=True,
    description="TASKQ_HEALTH_ENABLED. Enable the Unix-socket health server.",
)

health_socket_path class-attribute instance-attribute

health_socket_path: str = Field(
    default="/tmp/taskq_health.sock",
    description="TASKQ_HEALTH_SOCKET_PATH. Unix socket path for the health server.",
)

health_pg_ping_timeout class-attribute instance-attribute

health_pg_ping_timeout: float = Field(
    default=0.2,
    ge=0.0,
    description="TASKQ_HEALTH_PG_PING_TIMEOUT. Seconds to wait for dispatcher_pool.acquire() in the readiness PG ping. Default 200ms .",
)

health_tasks_enabled class-attribute instance-attribute

health_tasks_enabled: bool = Field(
    default=False,
    description="TASKQ_HEALTH_TASKS_ENABLED. Expose the privileged /tasks asyncio stack-dump endpoint on the Unix health socket. Off by default: the dump reveals code structure, file paths, and task names (never locals or payload values). Enabling it also tightens the socket to owner-only (no group/other access). Unix socket only — never mounted on the admin UI surface.",
)

health_host class-attribute instance-attribute

health_host: str = Field(
    default="0.0.0.0",
    description="TASKQ_HEALTH_HOST. Bind address for the optional TCP health listener. Only used when health_port is set. Defaults to all interfaces because Azure Container Apps and Kubernetes probe the replica over the pod network; narrow it to 127.0.0.1 when a local sidecar is the only prober.",
)

health_port class-attribute instance-attribute

health_port: int | None = Field(
    default=None,
    ge=0,
    le=65535,
    description="TASKQ_HEALTH_PORT. TCP port for the HTTP health listener serving /live and /ready. Unset (the default) means no TCP listener at all — setting a port is the opt-in. Required on Azure Container Apps, whose probes support only httpGet/tcpSocket and cannot reach a Unix socket (there is no exec probe type). The Unix socket keeps working either way. If the port cannot be bound the worker fails to start rather than run with probes silently dead. 0 binds an ephemeral port (tests only).",
)

health_request_timeout class-attribute instance-attribute

health_request_timeout: float = Field(
    default=2.0,
    gt=0.0,
    description="TASKQ_HEALTH_REQUEST_TIMEOUT. Seconds allowed for a probe to send its whole request line and headers before the connection is dropped unanswered. Bounds a drip-feed client that would otherwise hold a connection open forever by staying just inside a per-line timeout. Keep it at or below the shortest probe timeoutSeconds you configure.",
)

health_max_header_bytes class-attribute instance-attribute

health_max_header_bytes: int = Field(
    default=16 * 1024,
    gt=0,
    description="TASKQ_HEALTH_MAX_HEADER_BYTES. Cap on a probe request's accumulated request line plus headers. Pairs with health_request_timeout to bound a peer that sends many small lines fast enough to stay inside the deadline. 16 KiB is far above any real probe request, which carries a path and a handful of headers.",
)

health_readiness_check_timeout class-attribute instance-attribute

health_readiness_check_timeout: float = Field(
    default=5.0,
    gt=0.0,
    description="TASKQ_HEALTH_READINESS_CHECK_TIMEOUT. Seconds each check registered via taskq.worker.health.register_readiness_check may take before it counts as a readiness failure. Defaults to 5s, matching the Azure Container Apps default readiness probe timeoutSeconds, so a wedged check fails the probe rather than outliving it.",
)

watchdog_enabled class-attribute instance-attribute

watchdog_enabled: bool = Field(
    default=True,
    description="TASKQ_WATCHDOG_ENABLED. Master switch for the in-worker watchdog detectors (shutdown deadline, stale loop ticks, sibling contract, event-loop lag). A detector trip dumps the asyncio task stacks and force-exits non-zero so the supervisor restarts the worker instead of leaving it wedged.",
)

watchdog_loop_lag_budget class-attribute instance-attribute

watchdog_loop_lag_budget: float = Field(
    default=30.0,
    gt=0.0,
    description="TASKQ_WATCHDOG_LOOP_LAG_BUDGET (seconds). How long the event loop may go without scheduling before the lag watchdog trips. Deliberately far beyond any legitimate pause (GC, a slow tick) because the trip is terminal. Tier 2 of the lag detector; see watchdog_loop_lag_warn_budget for the non-terminal tier 1.",
)

watchdog_loop_lag_warn_budget class-attribute instance-attribute

watchdog_loop_lag_warn_budget: float = Field(
    default=5.0,
    gt=0.0,
    description="TASKQ_WATCHDOG_LOOP_LAG_WARN_BUDGET (seconds). Non-terminal tier-1 event-loop lag threshold: faulthandler thread dump + metric + deferred asyncio task-stack dump. Never exits; the terminal tier is watchdog_loop_lag_budget.",
)

watchdog_loop_lag_startup_grace class-attribute instance-attribute

watchdog_loop_lag_startup_grace: float = Field(
    default=30.0,
    ge=0.0,
    description="TASKQ_WATCHDOG_LOOP_LAG_STARTUP_GRACE (seconds). Grace before the lag watchdog arms, covering import-heavy startup, DI bootstrap, and first dispatch. Anchored to thread start; the lag detector also arms early once the first loop liveness tick lands.",
)

watchdog_tick_grace_factor class-attribute instance-attribute

watchdog_tick_grace_factor: float = Field(
    default=5.0,
    gt=0.0,
    description="TASKQ_WATCHDOG_TICK_GRACE_FACTOR. Multiplier on a loop's iteration period before its liveness tick is declared stale (floor 10s). Generous on purpose: a terminal detector must never fire on a merely loaded host.",
)

watchdog_dump_interval class-attribute instance-attribute

watchdog_dump_interval: float = Field(
    default=5.0,
    gt=0.0,
    description="TASKQ_WATCHDOG_DUMP_INTERVAL (seconds). Interval between straggler logs (names + await sites of still-alive siblings) while a shutdown is in progress.",
)

watchdog_dump_after_fraction class-attribute instance-attribute

watchdog_dump_after_fraction: float = Field(
    default=0.5,
    gt=0.0,
    lt=1.0,
    description="TASKQ_WATCHDOG_DUMP_AFTER_FRACTION. Fraction of the shutdown deadline that must be consumed before straggler dumps begin (0.5 = only in the back half of the budget). A drain inside its front half is within expectations and stays quiet; one countdown-start record is always logged so the window is never blind. Must be < 1: at 1.0 the deadline trip would always fire first, silently disabling the dumps.",
)

watchdog_stale_floor class-attribute instance-attribute

watchdog_stale_floor: float = Field(
    default=10.0,
    gt=0.0,
    description="TASKQ_WATCHDOG_STALE_FLOOR (seconds). Minimum staleness budget for any loop (period x grace_factor, floored at this value). Guards tiny intervals against false trips under host starvation — a terminal detector must never fire on load.",
)

watchdog_check_interval class-attribute instance-attribute

watchdog_check_interval: float = Field(
    default=1.0,
    gt=0.0,
    description="TASKQ_WATCHDOG_CHECK_INTERVAL (seconds). Poll cadence for the stale-tick sweep and the loop-lag watchdog thread.",
)

poll_interval class-attribute instance-attribute

poll_interval: float = Field(
    default=1.0,
    gt=0,
    description="TASKQ_POLL_INTERVAL (seconds). Producer loop fallback polling cadence when the NOTIFY listener is unavailable.",
)

notify_health_check_interval class-attribute instance-attribute

notify_health_check_interval: float = Field(
    default=5.0,
    gt=0,
    description="TASKQ_NOTIFY_HEALTH_CHECK_INTERVAL (seconds). How often _health_check_loop issues SELECT 1 on notify_conn. Detection latency before reconnect is at most this interval.",
)

notify_reconnect_backoff_initial class-attribute instance-attribute

notify_reconnect_backoff_initial: float = Field(
    default=1.0,
    gt=0,
    description="TASKQ_NOTIFY_RECONNECT_BACKOFF_INITIAL (seconds). Initial exponential backoff delay before the first reconnect retry. Cap is 30 s (factor 2 per attempt). Backoff sequence: 1, 2, 4, 8, 16, 30.",
)

notify_listener_setup_timeout class-attribute instance-attribute

notify_listener_setup_timeout: float = Field(
    default=10.0,
    validator=_positive_finite_float,
    description="TASKQ_NOTIFY_LISTENER_SETUP_TIMEOUT (seconds). Bounds each ``add_listener`` call during NOTIFY listener setup and reconnect - a half-open PG connection that accepts TCP but stalls on the LISTEN handshake would otherwise wedge the notify loop forever. On timeout the connection is closed (bounded) and the reconnect retry loop is entered (or the initial setup raises).",
)

notify_enabled class-attribute instance-attribute

notify_enabled: bool = Field(
    default=True,
    description="TASKQ_NOTIFY_ENABLED. When True, the worker uses LISTEN/NOTIFY for near-zero-latency dispatch wakeups with poll interval as fallback. When False, the worker uses poll-only dispatch.",
)

notify_poll_interval class-attribute instance-attribute

notify_poll_interval: float = Field(
    default=5.0,
    ge=0.5,
    description="TASKQ_NOTIFY_POLL_INTERVAL (seconds). Fallback poll cadence when NOTIFY is enabled (rarely reached - NOTIFY handles the common case). Use poll_interval when NOTIFY is disabled.",
)

reload_interval class-attribute instance-attribute

reload_interval: float | None = Field(
    default=None,
    gt=0,
    description="TASKQ_RELOAD_INTERVAL (seconds). When set, the worker periodically triggers a credential hot-reload (the same path as SIGHUP) with no external signal required - the rotation path for platforms without SIGHUP (e.g. Windows) and for hands-off scheduled rotation (e.g. ~720s for AWS IAM's 15-minute tokens). None disables the timer; SIGHUP and deps.request_reload() still work. Only factory-backed resources are rebuilt; DSN/static credentials are unaffected.",
)

reload_factory_timeout class-attribute instance-attribute

reload_factory_timeout: float = Field(
    default=30.0,
    gt=0,
    description="TASKQ_RELOAD_FACTORY_TIMEOUT (seconds). Bounds each individual factory call during a credential hot-reload - a hung token endpoint is marked failed for that resource instead of wedging the reload coordinator (and all future SIGHUPs).",
)

queues class-attribute instance-attribute

queues: list[str] = Field(
    default_factory=lambda: ["default"],
    validator=_queue_names_validator,
    description="TASKQ_QUEUES. Comma-separated list of queue names this worker will consume from.",
)

worker_label class-attribute instance-attribute

worker_label: str | None = Field(
    default=None,
    validator=_worker_label_validator,
    description="TASKQ_WORKER_LABEL. Human-readable label stored in the workers table for correlation with workgroup supervisors and external monitoring. When omitted the column is NULL; hostname and pid columns provide identification.",
)

workgroup_instance class-attribute instance-attribute

workgroup_instance: str | None = Field(
    default=None,
    validator=_workgroup_instance_validator,
    description="TASKQ_WORKGROUP_INSTANCE. UUIDv7 identifying the workgroup orchestrator that launched this worker. Used for cross-process correlation.",
)

pool_max_inactive_lifetime class-attribute instance-attribute

pool_max_inactive_lifetime: float = Field(
    default=300.0,
    ge=0.0,
    description="TASKQ_POOL_MAX_INACTIVE_LIFETIME (seconds). asyncpg max_inactive_connection_lifetime - closes connections idle longer than this threshold. Set to 3600.0 to match a typical SQLAlchemy pool_recycle=3600 setting when running alongside an SQLAlchemy-based service. Applied to dispatcher_pool, heartbeat_pool, and worker_pool.",
)

otel_enabled class-attribute instance-attribute

otel_enabled: bool = Field(
    default=True,
    description="TASKQ_OTEL_ENABLED. When False, the library suppresses all span and metric creation but operations still succeed .",
)

exception_message_max_chars class-attribute instance-attribute

exception_message_max_chars: int = Field(
    default=2000,
    ge=100,
    description="TASKQ_EXCEPTION_MESSAGE_MAX_CHARS. Bound on exception message text on spans and logs, after scrubbing. Matches the admin UI's traceback bound so there is one number for how much error text is kept, not two. Truncation appends the dropped character count, so an operator can see text was cut and raise this. Raise it when an actor formats large context into its messages; the stack trace is a separate field and is not bounded by this.",
)

exception_redaction_enabled class-attribute instance-attribute

exception_redaction_enabled: bool = Field(
    default=True,
    description="TASKQ_EXCEPTION_REDACTION_ENABLED. When True (the default), Postgres 'DETAIL:' lines are dropped from exception text before it reaches spans and logs, because they quote caller-supplied row values (idempotency_key, identity_key, fairness_key routinely hold tenant or subject identifiers). Set to False for advanced debugging to ship the raw text, including those row values, to every configured telemetry backend; the worker logs a startup WARNING while it is off. URI credential masking (scheme://user:***@host) is NOT affected by this setting and is always applied - no debugging case justifies sending a password to a telemetry vendor.",
)

worker_group class-attribute instance-attribute

worker_group: str = Field(
    default="default",
    description="TASKQ_WORKER_GROUP. Consumer group name emitted as messaging.consumer.group.name on CONSUMER spans .",
)

log_format class-attribute instance-attribute

log_format: str = Field(
    default="json",
    validator=_log_format_validator,
    description="TASKQ_LOG_FORMAT. json|console. Selects JSONRenderer or ConsoleRenderer in setup_logging.",
)

log_level class-attribute instance-attribute

log_level: str = Field(
    default="INFO",
    validator=_log_level_validator,
    description="TASKQ_LOG_LEVEL. Root logger level.",
)

prune_schedule_utc class-attribute instance-attribute

prune_schedule_utc: str = Field(
    default="03:00",
    validator=_hh_mm_validator,
    description="TASKQ_PRUNE_SCHEDULE_UTC. HH:MM (UTC) for the daily prune run. Ignored when prune_cron_expr is set.",
)

prune_cron_expr class-attribute instance-attribute

prune_cron_expr: str | None = Field(
    default=None,
    validator=_cron_expr_validator,
    description="TASKQ_PRUNE_CRON_EXPR. Full 5-field cron expression. When set, takes precedence over prune_schedule_utc.",
)

prune_batch_size class-attribute instance-attribute

prune_batch_size: int = Field(
    default=DEFAULT_PRUNE_BATCH_SIZE,
    ge=1,
    description="TASKQ_PRUNE_BATCH_SIZE. Rows to delete per batch.",
)

prune_retention_period class-attribute instance-attribute

prune_retention_period: timedelta = Field(
    default=DEFAULT_PRUNE_RETENTION,
    validator=_non_negative_timedelta,
    description="TASKQ_PRUNE_RETENTION_PERIOD. Global fallback retention. timedelta(0) means archive all terminal jobs immediately (valid). Negative values raise ConstraintViolationError at settings load.",
)

prune_retention_succeeded class-attribute instance-attribute

prune_retention_succeeded: timedelta = Field(
    default=timedelta(days=30),
    validator=_non_negative_timedelta,
    description="TASKQ_PRUNE_RETENTION_SUCCEEDED.",
)

prune_retention_failed class-attribute instance-attribute

prune_retention_failed: timedelta = Field(
    default=timedelta(days=90),
    validator=_non_negative_timedelta,
    description="TASKQ_PRUNE_RETENTION_FAILED.",
)

prune_retention_cancelled class-attribute instance-attribute

prune_retention_cancelled: timedelta = Field(
    default=timedelta(days=30),
    validator=_non_negative_timedelta,
    description="TASKQ_PRUNE_RETENTION_CANCELLED.",
)

prune_retention_abandoned class-attribute instance-attribute

prune_retention_abandoned: timedelta = Field(
    default=timedelta(days=90),
    validator=_non_negative_timedelta,
    description="TASKQ_PRUNE_RETENTION_ABANDONED. Also used for crashed jobs (no separate prune_retention_crashed field).",
)

archive_retention_period class-attribute instance-attribute

archive_retention_period: timedelta = Field(
    default=timedelta(days=365),
    validator=_non_negative_timedelta,
    description="TASKQ_ARCHIVE_RETENTION_PERIOD. How long archived jobs are retained in jobs_archive before hard-deletion. Default 1 year. timedelta(0) is valid. Negative values raise ConstraintViolationError.",
)

archive_expiry_schedule_utc class-attribute instance-attribute

archive_expiry_schedule_utc: str = Field(
    default="04:00",
    validator=_hh_mm_validator,
    description="TASKQ_ARCHIVE_EXPIRY_SCHEDULE_UTC. HH:MM (UTC) for the daily archive expiry sweep. Default 04:00, 1 hour after the prune sweep.",
)

archive_expiry_cron_expr class-attribute instance-attribute

archive_expiry_cron_expr: str | None = Field(
    default=None,
    validator=_cron_expr_validator,
    description="TASKQ_ARCHIVE_EXPIRY_CRON_EXPR. Full 5-field cron expression. When set, takes precedence over archive_expiry_schedule_utc.",
)

force_update_actor_config class-attribute instance-attribute

force_update_actor_config: bool = Field(
    default=False,
    description="When True, sync_actor_config silently overwrites a stored actor_config row whose queue or metadata differ from the registered values. When False (the default), that structural drift raises ActorConfigDriftList and the worker refuses to start. Capacity fields (max_concurrent, max_pending, result_ttl) are unaffected by this flag: once a row exists, the stored value is always authoritative and is never overwritten by the registered @actor(...) literal, regardless of force. Use `taskq actor-config set` to change a stored capacity value. Env var: TASKQ_FORCE_UPDATE_ACTOR_CONFIG.",
)

progress_coalesce_interval class-attribute instance-attribute

progress_coalesce_interval: float = Field(
    default=0.5,
    ge=0.1,
    description="TASKQ_PROGRESS_COALESCE_INTERVAL (seconds). How long the periodic flush loop waits between writing coalesced progress state to Postgres. Redis publishes are not throttled by this setting - each ctx.progress() call publishes immediately (fire-and-forget). Lower values increase PG write frequency; minimum 0.1 s.",
)

progress_data_max_bytes class-attribute instance-attribute

progress_data_max_bytes: int = Field(
    default=16384,
    ge=1024,
    le=1048576,
    description="TASKQ_PROGRESS_DATA_MAX_BYTES. Maximum serialised byte length of the ``data`` dict in a single progress call. Payloads exceeding this limit raise ProgressTooLarge . Range: 1 KiB - 1 MiB; default 16 KiB.",
)

progress_publish_global class-attribute instance-attribute

progress_publish_global: bool = Field(
    default=True,
    description="TASKQ_PROGRESS_PUBLISH_GLOBAL. When True (the default), progress events are additionally published to a schema-wide global fanout channel (in addition to the per-job channel). When False, events are only published to the per-job Redis channel. Does not affect Postgres flushing.",
)

result_max_bytes class-attribute instance-attribute

result_max_bytes: int = Field(
    default=MAX_RESULT_BYTES,
    ge=1024,
    le=1048576,
    description="TASKQ_RESULT_MAX_BYTES. Maximum serialised byte length of a job's terminal result dict. A larger result raises ResultTooLarge, which is non-retryable — the actor already ran, so a re-run returns the same oversized value. Range: 1 KiB - 1 MiB (the same ceiling as progress_data_max_bytes, so the durable payload can be configured as large as the transient one); default 64 KiB. Raise it only with the row size in mind: unlike progress data, the result is stored for the job's result_ttl.",
)

cron_catch_up_window class-attribute instance-attribute

cron_catch_up_window: timedelta = Field(
    default=timedelta(hours=1),
    validator=_non_negative_timedelta,
    description="TASKQ_CRON_CATCH_UP_WINDOW. Missed firings within this window are caught up sequentially; older misses are skipped.",
)

cron_auto_disable_threshold class-attribute instance-attribute

cron_auto_disable_threshold: int = Field(
    default=3,
    ge=1,
    description="TASKQ_CRON_AUTO_DISABLE_THRESHOLD. Consecutive failures before a schedule is auto-disabled.",
)

idle_settle_window class-attribute instance-attribute

idle_settle_window: float = Field(
    default=2.0,
    ge=0.0,
    description="TASKQ_IDLE_SETTLE_WINDOW (seconds). Time the drain monitor waits after queues appear empty before declaring drained. Only used when --until-idle is active.",
)

idle_poll_interval class-attribute instance-attribute

idle_poll_interval: float = Field(
    default=1.0,
    ge=0.1,
    description="TASKQ_IDLE_POLL_INTERVAL (seconds). How often the drain monitor checks queue depth. Only used when --until-idle is active.",
)

idle_max_runtime class-attribute instance-attribute

idle_max_runtime: float | None = Field(
    default=None,
    gt=0,
    description="TASKQ_IDLE_MAX_RUNTIME (seconds). Maximum wall-clock time for until-idle mode. When exceeded, exit code 4. None = no limit. Only used when --until-idle is active.",
)

resolved_pg_dsn_direct property

resolved_pg_dsn_direct: PostgresDsn

Direct DSN guaranteed non-None after :meth:post_load.

Why a property: pg_dsn_direct: PostgresDsn | None carries the environment-shape that distinguishes "user did not set TASKQ_PG_DSN_DIRECT" (None, fallback to pg_dsn) from "user set it explicitly". Once :meth:post_load has applied the fallback, the field is always non-None - but pyright cannot prove that across method boundaries. This property re-asserts the invariant at every call site, eliminating the need for assert or cast at call sites that read the DSN.

Raises :class:RuntimeError if accessed before :meth:post_load ran (signals a programming error: WorkerSettings() constructor must always go through :meth:load / :meth:load_from_dict).

resolved_pg_dsn_pooled property

resolved_pg_dsn_pooled: PostgresDsn

Pooled DSN guaranteed non-None after :meth:post_load.

See :attr:resolved_pg_dsn_direct for the rationale.

worker_pool_size property

worker_pool_size: int

Derived pool size for worker_pool: int(max_concurrency * 1.5).

worst_case_shutdown_seconds property

worst_case_shutdown_seconds: float

Modelled worst-case wall clock from SIGTERM to process exit.

The shutdown phase graces plus the bounded-close tail that unwinds after them (see :func:taskq._close.worst_case_teardown_tail).

This is deliberately NOT enforced by post_load. The cross-field validator there rejects a config outright, and a budget below this dead-backend worst case is a legitimate operator choice (a tight dev deployment that would rather be SIGKILLed mid-unwind than wait out a hung close). The shipped default covers the model — raising the validator to reject sub-worst-case budgets would take that choice away. The number is surfaced as a startup warning instead, and docs/guides/deployment.md documents the pod-grace formula.

shutdown_budget_is_sufficient property

shutdown_budget_is_sufficient: bool

Whether termination_grace_period covers the modelled worst case.

post_load

post_load() -> list[ValidationError] | None

Apply DSN fallback and validate cross-field invariants after loading.

Runs automatically on every load path (load(), load_from_dict(), reload(), and nested config loading), including under validate=False - consistent with the per-field validator hooks (transformation is part of loading, not validation). No WorkerSettings.load / load_from_dict override is needed; the base DotEnvConfig._load_fields invokes this hook itself.

Returns list[ValidationError] so failures integrate with dotenvmodel's uniform error hierarchy: a single returned error is raised unchanged (its exact type preserved), several aggregate into MultipleValidationErrors. Catch DotEnvModelError (the common base) to cover both single and aggregate cases - MultipleValidationErrors is a DotEnvModelError but not a ValidationError, so except ValidationError alone misses the multi-invariant case. ValidationError suffices only when at most one invariant can fire (e.g. a single field constraint).

Source code in src/taskq/settings.py

def post_load(self) -> list[ValidationError] | None:
    """Apply DSN fallback and validate cross-field invariants after loading.

    Runs automatically on every load path (``load()``,
    ``load_from_dict()``, ``reload()``, and nested config loading),
    including under ``validate=False`` - consistent with the per-field
    ``validator`` hooks (transformation is part of loading, not
    validation). No ``WorkerSettings.load`` / ``load_from_dict``
    override is needed; the base ``DotEnvConfig._load_fields`` invokes
    this hook itself.

    Returns ``list[ValidationError]`` so failures integrate with
    dotenvmodel's uniform error hierarchy: a single returned error is
    raised unchanged (its exact type preserved), several aggregate
    into ``MultipleValidationErrors``. Catch ``DotEnvModelError`` (the
    common base) to cover both single and aggregate cases -
    ``MultipleValidationErrors`` is a ``DotEnvModelError`` but not a
    ``ValidationError``, so ``except ValidationError`` alone misses the
    multi-invariant case. ``ValidationError`` suffices only when at
    most one invariant can fire (e.g. a single field constraint).
    """
    errors: list[ValidationError] = []

    # DSN fallback: if split DSNs were not provided, resolve to pg_dsn.
    # After this, pg_dsn_direct and pg_dsn_pooled are always non-None.
    if self.pg_dsn_direct is None:
        self.pg_dsn_direct = self.pg_dsn
    if self.pg_dsn_pooled is None:
        self.pg_dsn_pooled = self.pg_dsn

    # lock_lease invariant: "Tolerates 3 missed heartbeats before reclamation."
    if self.lock_lease < 4 * self.heartbeat_interval:
        errors.append(
            ValidationError(
                field_name="lock_lease",
                value=self.lock_lease,
                error_msg=(
                    f"lock_lease ({self.lock_lease}) must be >= 4 * heartbeat_interval "
                    f"({4 * self.heartbeat_interval})"
                ),
            )
        )

    # Cancellation + cleanup grace must fit within termination_grace_period.
    # termination_grace_period may be added by a subclass; the getattr guard
    # tolerates its absence when this base validation runs first.
    termination_grace = getattr(self, "termination_grace_period", None)
    if (
        termination_grace is not None
        and self.cancellation_grace_period + self.cleanup_grace_period
        >= termination_grace - 5.0
    ):
        errors.append(
            ValidationError(
                field_name="cancellation_grace_period",
                value=self.cancellation_grace_period,
                error_msg=(
                    f"cancellation_grace_period ({self.cancellation_grace_period}) + "
                    f"cleanup_grace_period ({self.cleanup_grace_period}) must be < "
                    f"termination_grace_period - 5.0 ({termination_grace - 5.0})"
                ),
            )
        )

    # Cancellation grace + cleanup grace must be less than lock_lease.
    if self.cancellation_grace_period + self.cleanup_grace_period >= self.lock_lease:
        errors.append(
            ValidationError(
                field_name="cancellation_grace_period",
                value=self.cancellation_grace_period,
                error_msg=(
                    f"cancellation_grace_period ({self.cancellation_grace_period}) + "
                    f"cleanup_grace_period ({self.cleanup_grace_period}) must be < "
                    f"lock_lease ({self.lock_lease})"
                ),
            )
        )

    # Bounded-loop staleness invariant: the period-1 leader loops
    # (scheduled_wake, cron) are wrapped in asyncio.timeout, so their
    # worst-case tick gap is timeout + period. That gap must fit the
    # loop's own budget max(period * watchdog_tick_grace_factor,
    # watchdog_stale_floor) or detector 2 force-exits a healthy worker
    # mid-degradation (measured: timeout 10.0 against budget 10.0
    # produced an 11s tick gap and a trip at age 10.008s). Only checked
    # when the watchdog is armed: with watchdog_enabled=False detector 2
    # is never spawned, and a stale tick only costs a transient NotReady,
    # which is not worth blocking boot over.
    #
    # The producer loop is deliberately NOT checked here: it is not
    # wrapped in asyncio.timeout (dispatch_batch is a multi-statement
    # transaction — BEGIN + resolve_queue_modes + dispatch CTE + INSERTs
    # + COMMIT, each bounded separately by the pool's command_timeout),
    # so the timeout + period model does not hold. The actual worst-case
    # gap is k * timeout + period for k statements, which the invariant
    # cannot express without knowing k at settings-load time.
    if self.watchdog_enabled:
        loop_label = "leader loops"
        period = 1.0
        budget = max(period * self.watchdog_tick_grace_factor, self.watchdog_stale_floor)
        if budget <= period + 1.0:
            # 1.0 = dispatcher_command_timeout's own ge= minimum: no
            # legal timeout can satisfy the gap, so the budget side
            # is what the operator must change.
            errors.append(
                ValidationError(
                    field_name="watchdog_stale_floor",
                    value=self.watchdog_stale_floor,
                    error_msg=(
                        f"the {loop_label} staleness budget max({period} x "
                        f"watchdog_tick_grace_factor, watchdog_stale_floor) "
                        f"({budget}) must exceed dispatcher_command_timeout's "
                        f"1.0s minimum + the {period}s loop period"
                    ),
                )
            )
        elif self.dispatcher_command_timeout + period >= budget:
            errors.append(
                ValidationError(
                    field_name="dispatcher_command_timeout",
                    value=self.dispatcher_command_timeout,
                    error_msg=(
                        f"dispatcher_command_timeout ({self.dispatcher_command_timeout}) "
                        f"+ {period}s {loop_label} period must be < the loop's "
                        f"staleness budget max(period x watchdog_tick_grace_factor, "
                        f"watchdog_stale_floor) ({budget})"
                    ),
                )
            )

    # Lag-watchdog lease invariant: a stalled event loop must die (the
    # terminal lag watchdog trips at watchdog_loop_lag_budget) before
    # its leases can expire (lock_lease), otherwise the leader sweep
    # reclaims LIVE jobs' locks mid-stall and the worker wakes from the
    # stall to find its work reassigned. The heartbeat_interval term is
    # the worst-case age the last beat can carry when the stall starts,
    # so the trip is guaranteed to land inside the lease. Only checked
    # when the watchdog is armed: with watchdog_enabled=False no
    # terminal lag detector exists, and stall-vs-lease ordering is a
    # deployment concern, not a load-time guarantee (same gating as the
    # bounded-loop invariant above).
    if self.watchdog_enabled and (
        self.watchdog_loop_lag_budget + self.heartbeat_interval >= self.lock_lease
    ):
        errors.append(
            ValidationError(
                field_name="watchdog_loop_lag_budget",
                value=self.watchdog_loop_lag_budget,
                error_msg=(
                    f"watchdog_loop_lag_budget ({self.watchdog_loop_lag_budget}) + "
                    f"heartbeat_interval ({self.heartbeat_interval}) must be < "
                    f"lock_lease ({self.lock_lease}): a stalled event loop must die "
                    f"(the terminal lag watchdog) before its leases expire, or the "
                    f"leader sweep reclaims LIVE jobs' locks mid-stall. Keep the lag "
                    f"budget comfortably inside lock_lease — both knobs must move "
                    f"together."
                ),
            )
        )

    # Lag budget vs check interval coherence: the lag detector samples
    # the loop once per watchdog_check_interval and schedules the beat
    # it measures from the same poll, so a healthy loop's observed lag
    # is ~check_interval by construction. A budget at or below the
    # sampling period therefore trips on health, not stalls (measured:
    # budget 1.0 against the 1.0s default check interval force-exits an
    # idle worker on its first armed poll). Same watchdog gating as the
    # lease invariant above.
    if self.watchdog_enabled and (
        self.watchdog_loop_lag_budget <= self.watchdog_check_interval
    ):
        errors.append(
            ValidationError(
                field_name="watchdog_loop_lag_budget",
                value=self.watchdog_loop_lag_budget,
                error_msg=(
                    f"watchdog_loop_lag_budget ({self.watchdog_loop_lag_budget}) "
                    f"must be > watchdog_check_interval "
                    f"({self.watchdog_check_interval}): the detector samples the "
                    f"loop once per check interval, so a budget at or below its "
                    f"own sampling period trips on a healthy loop's beat cadence. "
                    f"Raise the budget (keeping it inside lock_lease) or lower "
                    f"watchdog_check_interval."
                ),
            )
        )

    return errors or None

FakeClock

FakeClock(start: datetime)

Deterministic clock for tests.

Accepts a start datetime (typically datetime(2025, 1, 1, tzinfo=UTC)). now() returns the current internal time; move_to and advance let tests control the clock explicitly. monotonic() returns elapsed seconds from _EPOCH so that elapsed-time guards see a non-zero starting value.

Source code in src/taskq/testing/clock.py

def __init__(self, start: datetime) -> None:
    self._now = start

now

now() -> datetime

Source code in src/taskq/testing/clock.py

def now(self) -> datetime:
    return self._now

move_to

move_to(when: datetime) -> None

Set the clock to when.

Source code in src/taskq/testing/clock.py

def move_to(self, when: datetime) -> None:
    """Set the clock to *when*."""
    self._now = when

advance

advance(delta: timedelta) -> None

Add delta to the clock.

Source code in src/taskq/testing/clock.py

def advance(self, delta: timedelta) -> None:
    """Add *delta* to the clock."""
    self._now = self._now + delta

monotonic

monotonic() -> float

Elapsed seconds since _EPOCH — consistent with now().

Same wall-clock position always returns the same float; never decreases within a test.

Source code in src/taskq/testing/clock.py

def monotonic(self) -> float:
    """Elapsed seconds since ``_EPOCH`` — consistent with ``now()``.

    Same wall-clock position always returns the same float; never
    decreases within a test.
    """
    return (self._now - _EPOCH).total_seconds()

validate_actor_payload

validate_actor_payload(
    payload_type: type[BaseModel],
    raw_payload: dict[str, object] | BaseModel,
    actor: str | None = None,
) -> BaseModel

Validate a raw payload dict (or existing BaseModel) against the actor's payload model.

Wraps pydantic.ValidationError as :class:~taskq.exceptions.PayloadValidationError (non-retryable) so the retry classifier fails the job immediately instead of retrying a deterministic validation failure.

Error details are sanitized via include_url=False, include_input=False to prevent attacker-controlled field values from being persisted to the jobs row or surfaced in the web admin.

Parameters:

Name	Type	Description	Default
payload_type	type[BaseModel]	The actor's Pydantic payload model class.	required
raw_payload	dict[str, object] | BaseModel	The raw dict[str, object] from the job row, or an existing BaseModel to re-validate against payload_type.	required
actor	str | None	The actor name, for error context.	None

Returns:

Type	Description
BaseModel	The validated BaseModel instance.

Raises:

Type	Description
PayloadValidationError	If validation fails.

Source code in src/taskq/_validation.py

def validate_actor_payload(
    payload_type: type[BaseModel],
    raw_payload: dict[str, object] | BaseModel,
    actor: str | None = None,
) -> BaseModel:
    """Validate a raw payload dict (or existing BaseModel) against the actor's payload model.

    Wraps ``pydantic.ValidationError`` as
    :class:`~taskq.exceptions.PayloadValidationError` (non-retryable) so
    the retry classifier fails the job immediately instead of retrying
    a deterministic validation failure.

    Error details are sanitized via ``include_url=False,
    include_input=False`` to prevent attacker-controlled field values
    from being persisted to the jobs row or surfaced in the web admin.

    Args:
        payload_type: The actor's Pydantic payload model class.
        raw_payload: The raw ``dict[str, object]`` from the job row, or
            an existing ``BaseModel`` to re-validate against
            ``payload_type``.
        actor: The actor name, for error context.

    Returns:
        The validated ``BaseModel`` instance.

    Raises:
        PayloadValidationError: If validation fails.
    """
    try:
        return payload_type.model_validate(raw_payload)
    except ValidationError as exc:
        errs: list[dict[str, object]] = exc.errors(include_url=False, include_input=False)  # type: ignore[assignment]  # Why: pydantic v2 ErrorDetails is a TypedDict (subtype of dict[str, Any]); assignment to list[dict[str,object]] is safe at runtime but pyright cannot prove covariance
        raise PayloadValidationError(
            f"Payload validation failed for actor {actor!r}: {exc.title}",
            actor=actor,
            validation_errors=errs,
        ) from exc

enrich_pg_dsn

enrich_pg_dsn(dsn: str, credential: PgCredential) -> str

Apply credential to dsn and return a self-contained DSN string.

The credential is written into the DSN userinfo (percent-encoded), replacing any existing userinfo password - and replacing the userinfo user when credential.username is set (Vault dynamic DB creds). This is the only slot that is guaranteed to take effect: asyncpg's resolver applies userinfo before query parameters (both behind if user is None / if password is None guards), so a query-string user= / password= is silently ignored whenever the DSN already carries userinfo. A stale password= query parameter is dropped (always shadowed by the userinfo password); a user= query parameter is dropped only when the userinfo carries a user to shadow it - a query-carried user with no userinfo user is the effective principal and is preserved.

sslmode=require is added only when the DSN has no explicit sslmode, so stronger modes (verify-full) are never downgraded.

Prefer the factory builders (:func:make_pg_pool_factory / :func:make_dedicated_conn_factory) where possible - they pass the credential as keyword arguments instead, keeping the token out of the DSN string entirely.

Source code in src/taskq/auth.py

def enrich_pg_dsn(dsn: str, credential: PgCredential) -> str:
    """Apply *credential* to *dsn* and return a self-contained DSN string.

    The credential is written into the DSN **userinfo** (percent-encoded),
    replacing any existing userinfo password - and replacing the userinfo
    user when ``credential.username`` is set (Vault dynamic DB creds).
    This is the only slot that is guaranteed to take effect: asyncpg's
    resolver applies userinfo *before* query parameters (both behind
    ``if user is None`` / ``if password is None`` guards), so a
    query-string ``user=`` / ``password=`` is silently ignored whenever
    the DSN already carries userinfo. A stale ``password=`` query
    parameter is dropped (always shadowed by the userinfo password);
    a ``user=`` query parameter is dropped only when the userinfo
    carries a user to shadow it - a query-carried user with no userinfo
    user is the effective principal and is preserved.

    ``sslmode=require`` is added only when the DSN has no explicit
    sslmode, so stronger modes (``verify-full``) are never downgraded.

    Prefer the factory builders (:func:`make_pg_pool_factory` /
    :func:`make_dedicated_conn_factory`) where possible - they pass the
    credential as keyword arguments instead, keeping the token out of
    the DSN string entirely.
    """
    parsed = urlparse(str(dsn))
    query = parse_qs(parsed.query, keep_blank_values=True)
    query.pop("password", None)

    if "@" in parsed.netloc:
        auth, _, hostspec = parsed.netloc.partition("@")
    else:
        auth, hostspec = "", parsed.netloc
    user, _, _old_password = auth.partition(":")
    if credential.username is not None:
        user = quote(credential.username, safe="")
    if user:
        # The userinfo will carry a user, which shadows any query user= in
        # asyncpg's resolver - drop the stale query copy. When the userinfo
        # has NO user (credential.username unset, none in the DSN), a query
        # user= is the effective principal and must be preserved.
        query.pop("user", None)
    netloc = f"{user}:{quote(credential.password, safe='')}@{hostspec}"

    query.setdefault("sslmode", ["require"])
    new_query = urlencode(query, doseq=True)
    return urlunparse(parsed._replace(netloc=netloc, query=new_query))

ensure_sslmode_require

ensure_sslmode_require(dsn: str) -> str

Add sslmode=require to dsn unless an sslmode is already set.

An explicit sslmode is never overridden - in particular stronger modes (verify-ca / verify-full) must not be downgraded: require skips certificate verification, which would expose the very token this module injects to a MITM.

Public because anyone assembling a credential-bearing DSN by hand needs exactly this rule and must not re-derive it: a token path that silently connects without TLS puts the credential on the wire. The factory builders in this module apply it for you; reach for it directly only on the DSN paths they do not cover (a raw asyncpg.connect, a migration connection, a DSN handed to another library).

sslmode=disable is an explicit choice and is preserved - that is how a test container or a Unix-socket deployment opts out.

Source code in src/taskq/auth.py

def ensure_sslmode_require(dsn: str) -> str:
    """Add ``sslmode=require`` to *dsn* unless an sslmode is already set.

    An explicit sslmode is never overridden - in particular stronger
    modes (``verify-ca`` / ``verify-full``) must not be downgraded:
    ``require`` skips certificate verification, which would expose the
    very token this module injects to a MITM.

    Public because anyone assembling a credential-bearing DSN by hand needs
    exactly this rule and must not re-derive it: a token path that silently
    connects without TLS puts the credential on the wire. The factory
    builders in this module apply it for you; reach for it directly only on
    the DSN paths they do not cover (a raw ``asyncpg.connect``, a migration
    connection, a DSN handed to another library).

    ``sslmode=disable`` is an explicit choice and is preserved - that is how
    a test container or a Unix-socket deployment opts out.
    """
    parsed = urlparse(str(dsn))
    query = parse_qs(parsed.query, keep_blank_values=True)
    if "sslmode" in query:
        return str(dsn)
    query["sslmode"] = ["require"]
    return urlunparse(parsed._replace(query=urlencode(query, doseq=True)))

make_dedicated_conn_factory

make_dedicated_conn_factory(
    dsn: str,
    provider: PgCredentialProvider,
    *,
    command_timeout: float | None = None,
    setup: Callable[[Connection], Awaitable[None]]
    | None = None,
    server_settings: dict[str, str] | None = None,
    connection_class: type[Connection] | None = None,
) -> ConnFactory

Build a :data:~taskq.connections.ConnFactory backed by provider.

Used for the worker's notify_conn / leader_conn or :class:taskq.TaskQ's pg_conn_factory. Like :func:make_pg_pool_factory, the credential is passed as keyword arguments (precedence over userinfo and query params; the token never appears in the DSN string), and password= is an async callable that asyncpg awaits per physical connection.

A dedicated connection is opened once and then held for the life of the worker, so the callable normally fires exactly once - but these are precisely the long-lived connections a credential expiry kills, and the callable is what makes every re-open (a LISTEN connection reconnecting after the server drops it, or reload_credentials rebuilding it) authenticate with a fresh credential rather than the one captured when the factory was first invoked.

command_timeout is forwarded to asyncpg.connect as the default per-operation timeout. The worker's DSN-built notify_conn / leader_conn carry dispatcher_command_timeout; pass it here too so a credential-provider deployment does not silently drop the bound that keeps a wedged query from stalling leader election.

setup is forwarded to asyncpg.connect and runs once after the connection is established (e.g. registering type codecs, setting session GUCs). For a dedicated connection this is equivalent to init on a pool - there is no acquire/reuse cycle.

server_settings is forwarded to asyncpg.connect and applied as session-level GUCs at connect time (e.g. {"statement_timeout": "30s", "search_path": "app"}).

connection_class is forwarded to asyncpg.connect and sets the :class:asyncpg.Connection subclass for this connection. Use it to install custom codecs or override connection methods.

Source code in src/taskq/auth.py

def make_dedicated_conn_factory(
    dsn: str,
    provider: PgCredentialProvider,
    *,
    command_timeout: float | None = None,
    setup: Callable[[asyncpg.Connection], Awaitable[None]] | None = None,
    server_settings: dict[str, str] | None = None,
    connection_class: type[asyncpg.Connection] | None = None,
) -> ConnFactory:
    """Build a :data:`~taskq.connections.ConnFactory` backed by *provider*.

    Used for the worker's ``notify_conn`` / ``leader_conn`` or
    :class:`taskq.TaskQ`'s ``pg_conn_factory``. Like
    :func:`make_pg_pool_factory`, the credential is passed as keyword
    arguments (precedence over userinfo and query params; the token
    never appears in the DSN string), and ``password=`` is an async
    callable that asyncpg awaits per physical connection.

    A dedicated connection is opened once and then held for the life of
    the worker, so the callable normally fires exactly once - but these
    are precisely the long-lived connections a credential expiry kills,
    and the callable is what makes every *re-open* (a LISTEN connection
    reconnecting after the server drops it, or ``reload_credentials``
    rebuilding it) authenticate with a fresh credential rather than the
    one captured when the factory was first invoked.

    *command_timeout* is forwarded to ``asyncpg.connect`` as the default
    per-operation timeout. The worker's DSN-built ``notify_conn`` /
    ``leader_conn`` carry ``dispatcher_command_timeout``; pass it here too
    so a credential-provider deployment does not silently drop the bound
    that keeps a wedged query from stalling leader election.

    *setup* is forwarded to ``asyncpg.connect`` and runs once after the
    connection is established (e.g. registering type codecs, setting
    session GUCs). For a dedicated connection this is equivalent to
    *init* on a pool - there is no acquire/reuse cycle.

    *server_settings* is forwarded to ``asyncpg.connect`` and applied as
    session-level GUCs at connect time (e.g.
    ``{"statement_timeout": "30s", "search_path": "app"}``).

    *connection_class* is forwarded to ``asyncpg.connect`` and sets the
    :class:`asyncpg.Connection` subclass for this connection. Use it to
    install custom codecs or override connection methods.
    """
    import asyncpg

    async def factory() -> asyncpg.Connection:
        # Fetched once to resolve `user=` and fail fast; see make_pg_pool_factory.
        credential = await provider.get_pg_credential()
        kwargs: dict[str, Any] = {
            "dsn": ensure_sslmode_require(dsn),
            "password": _make_pg_password_callable(
                provider, pinned_username=credential.username, role="dedicated_conn"
            ),
        }
        if credential.username is not None:
            kwargs["user"] = credential.username
        if command_timeout is not None:
            kwargs["command_timeout"] = command_timeout
        if setup is not None:
            kwargs["setup"] = setup
        if server_settings is not None:
            kwargs["server_settings"] = server_settings
        if connection_class is not None:
            kwargs["connection_class"] = connection_class
        return await asyncpg.connect(**kwargs)

    return factory

make_pg_pool_factory

make_pg_pool_factory(
    dsn: str,
    provider: PgCredentialProvider,
    *,
    min_size: int = 1,
    max_size: int = 4,
    max_inactive_connection_lifetime: float = 300.0,
    command_timeout: float | None = None,
    init: Callable[[Connection], Awaitable[None]]
    | None = None,
    setup: Callable[[Connection], Awaitable[None]]
    | None = None,
    server_settings: dict[str, str] | None = None,
    connection_class: type[Connection] | None = None,
) -> PoolFactory

Build a :data:~taskq.connections.PoolFactory backed by provider.

Each invocation fetches a fresh :class:PgCredential from provider and calls asyncpg.create_pool with the credential as keyword arguments - password= always, user= when the credential carries a username. Keyword arguments take precedence over both DSN userinfo and query parameters in asyncpg's resolver, so a stale credential baked into dsn can never shadow the fresh one, and the token never appears in the DSN string. The pool is owned by the worker (entered on its AsyncExitStack).

Token refresh: password= is passed as an async callable, which asyncpg invokes and awaits once per physical connection - the connections opened at pool creation, those opened later by pool growth, and the replacements opened after max_inactive_connection_lifetime recycles an idle connection. Every new connection therefore authenticates with a freshly fetched credential, and no external rotation is required. This matters because Postgres authenticates at connect time only: a credential resolved once and reused as a fixed string keeps working on already-open connections while every new connection fails, roughly one token-lifetime after deploy.

SIGHUP (see taskq.worker.deps.reload_credentials) still works and is no longer required for token refresh. It remains the way to force a full pool rebuild - and the only way to pick up a changed username, since asyncpg resolves user= once per pool and accepts a callable only for password=. A provider that rotates its username (e.g. Vault dynamic database credentials) raises a RuntimeError naming this constraint rather than pairing a fresh password with a stale username.

Per-connection setup: init is forwarded verbatim to asyncpg.create_pool and runs once per new physical connection - on the connections opened at pool creation, on connections opened later by pool growth, and again on replacements opened after max_inactive_connection_lifetime recycles an idle connection. That lifecycle is exactly why this setup (registering type codecs - e.g. pgvector.asyncpg.register_vector - preparing statements, setting session GUCs) cannot be done correctly after pool creation: a connection configured by hand is silently replaced under load or after an idle period. The only per-connection work this factory does of its own is the credential refresh described above (an asyncpg password= callback, not an init hook), so a caller-supplied init is the only hook of its kind: it is passed through unwrapped and can never silently replace internal setup.

Per-acquire setup: setup is forwarded to asyncpg.create_pool and runs every time a connection is acquired from the pool (via pool.acquire()), not just on new-connection creation. Use it for per-checkout work that must run even when a pooled connection is reused - e.g. resetting search_path or verifying session state. Unlike init, setup runs on every acquire, so keep it lightweight. Both init and setup can be provided simultaneously.

server_settings is forwarded to asyncpg.create_pool and applied as session-level GUCs on every new connection (e.g. {"statement_timeout": "30s", "search_path": "app"}). Useful for per-pool configuration that must be set at connection time.

connection_class is forwarded to asyncpg.create_pool and sets the :class:asyncpg.Connection subclass used by the pool. Use it to install custom codecs or override connection methods across the entire pool.

Source code in src/taskq/auth.py

def make_pg_pool_factory(
    dsn: str,
    provider: PgCredentialProvider,
    *,
    min_size: int = 1,
    max_size: int = 4,
    max_inactive_connection_lifetime: float = 300.0,
    command_timeout: float | None = None,
    init: Callable[[asyncpg.Connection], Awaitable[None]] | None = None,
    setup: Callable[[asyncpg.Connection], Awaitable[None]] | None = None,
    server_settings: dict[str, str] | None = None,
    connection_class: type[asyncpg.Connection] | None = None,
) -> PoolFactory:
    """Build a :data:`~taskq.connections.PoolFactory` backed by *provider*.

    Each invocation fetches a fresh :class:`PgCredential` from *provider*
    and calls ``asyncpg.create_pool`` with the credential as keyword
    arguments - ``password=`` always, ``user=`` when the credential
    carries a username. Keyword arguments take precedence over both DSN
    userinfo and query parameters in asyncpg's resolver, so a stale
    credential baked into *dsn* can never shadow the fresh one, and the
    token never appears in the DSN string. The pool is owned by the
    worker (entered on its ``AsyncExitStack``).

    Token refresh: ``password=`` is passed as an **async callable**, which
    asyncpg invokes and awaits once per *physical* connection - the
    connections opened at pool creation, those opened later by pool
    growth, and the replacements opened after
    ``max_inactive_connection_lifetime`` recycles an idle connection. Every
    new connection therefore authenticates with a freshly fetched
    credential, and no external rotation is required. This matters because
    Postgres authenticates at connect time only: a credential resolved once
    and reused as a fixed string keeps working on already-open connections
    while every new connection fails, roughly one token-lifetime after
    deploy.

    ``SIGHUP`` (see ``taskq.worker.deps.reload_credentials``) still works
    and is no longer *required* for token refresh. It remains the way to
    force a full pool rebuild - and the only way to pick up a **changed
    username**, since asyncpg resolves ``user=`` once per pool and accepts
    a callable only for ``password=``. A provider that rotates its username
    (e.g. Vault dynamic database credentials) raises a ``RuntimeError``
    naming this constraint rather than pairing a fresh password with a
    stale username.

    Per-connection setup: *init* is forwarded verbatim to
    ``asyncpg.create_pool`` and runs **once per new physical connection**
    - on the connections opened at pool creation, on connections opened
    later by pool growth, and again on replacements opened after
    ``max_inactive_connection_lifetime`` recycles an idle connection.
    That lifecycle is exactly why this setup (registering type codecs -
    e.g. ``pgvector.asyncpg.register_vector`` - preparing statements,
    setting session GUCs) cannot be done correctly after pool creation:
    a connection configured by hand is silently replaced under load or
    after an idle period. The only per-connection work this factory does
    of its own is the credential refresh described above (an asyncpg
    ``password=`` callback, not an ``init`` hook), so a caller-supplied
    *init* is the only hook of its kind: it is passed through unwrapped
    and can never silently replace internal setup.

    Per-acquire setup: *setup* is forwarded to ``asyncpg.create_pool``
    and runs **every time a connection is acquired from the pool**
    (via ``pool.acquire()``), not just on new-connection creation. Use
    it for per-checkout work that must run even when a pooled connection
    is reused - e.g. resetting ``search_path`` or verifying session
    state. Unlike *init*, *setup* runs on every acquire, so keep it
    lightweight. Both *init* and *setup* can be provided simultaneously.

    *server_settings* is forwarded to ``asyncpg.create_pool`` and applied
    as session-level GUCs on every new connection (e.g.
    ``{"statement_timeout": "30s", "search_path": "app"}``). Useful for
    per-pool configuration that must be set at connection time.

    *connection_class* is forwarded to ``asyncpg.create_pool`` and sets
    the :class:`asyncpg.Connection` subclass used by the pool. Use it to
    install custom codecs or override connection methods across the
    entire pool.
    """
    import asyncpg  # Why: deferred so this module is import-safe without asyncpg at module load.

    async def factory() -> asyncpg.Pool:
        # Fetched once here to resolve `user=` (not callable in asyncpg) and to
        # fail fast at pool construction on a broken provider, rather than
        # deferring the first failure to the first connection attempt. The
        # password itself goes in as a callable so it is re-fetched per
        # physical connection.
        credential = await provider.get_pg_credential()
        kwargs: dict[str, Any] = {
            "dsn": ensure_sslmode_require(dsn),
            "password": _make_pg_password_callable(
                provider, pinned_username=credential.username, role="pool"
            ),
            "min_size": min_size,
            "max_size": max_size,
            "max_inactive_connection_lifetime": max_inactive_connection_lifetime,
        }
        if credential.username is not None:
            kwargs["user"] = credential.username
        if command_timeout is not None:
            kwargs["command_timeout"] = command_timeout
        if init is not None:
            kwargs["init"] = init
        if setup is not None:
            kwargs["setup"] = setup
        if server_settings is not None:
            kwargs["server_settings"] = server_settings
        if connection_class is not None:
            kwargs["connection_class"] = connection_class
        pool = await asyncpg.create_pool(**kwargs)
        assert pool is not None  # asyncpg returns None only for record_class paths
        return pool

    return factory

make_redis_client_factory

make_redis_client_factory(
    url: str | None,
    provider: RedisCredentialProvider,
    **client_kwargs: Any,
) -> RedisFactory

Build a :data:~taskq.connections.RedisFactory backed by provider.

url is the Redis URL without credentials. The factory attaches a redis-py CredentialProvider that delegates to provider, so reconnects re-fetch the credential automatically. Use a rediss:// (TLS) URL - with a plain redis:// URL the bearer token is sent unencrypted, and the factory logs a warning.

If url is None the factory raises :class:RuntimeError when called (matches the worker's "Redis not configured" contract).

Source code in src/taskq/auth.py

def make_redis_client_factory(
    url: str | None,
    provider: RedisCredentialProvider,
    **client_kwargs: Any,
) -> RedisFactory:
    """Build a :data:`~taskq.connections.RedisFactory` backed by *provider*.

    ``url`` is the Redis URL **without** credentials. The factory attaches
    a redis-py ``CredentialProvider`` that delegates to *provider*, so
    reconnects re-fetch the credential automatically. Use a ``rediss://``
    (TLS) URL - with a plain ``redis://`` URL the bearer token is sent
    unencrypted, and the factory logs a warning.

    If ``url`` is ``None`` the factory raises :class:`RuntimeError` when
    called (matches the worker's "Redis not configured" contract).
    """
    import redis.asyncio as redis_async  # type: ignore[import-not-found]  # Why: optional [redis] extra; required at call time.
    from redis.credentials import (
        CredentialProvider,  # type: ignore[import-not-found]  # Why: optional [redis] extra; required at call time.
    )

    class _CredentialProviderAdapter(CredentialProvider):
        """redis-py ``CredentialProvider`` → TaskQ ``RedisCredentialProvider``.

        redis-py's async connection calls ``get_credentials_async`` (not
        ``get_credentials``) on every (re)connect - the base class's
        ``get_credentials_async`` only exists for backward compatibility
        and delegates to the *sync* ``get_credentials``, so it must be
        overridden here for the credential to actually rotate.
        """

        def get_credentials(self) -> tuple[str, str]:
            raise NotImplementedError(
                "_CredentialProviderAdapter only supports the async redis client; "
                "get_credentials_async is called instead."
            )

        async def get_credentials_async(self) -> tuple[str, str]:
            cred = await provider.get_redis_credential()
            return cred.username, cred.password

    adapter = _CredentialProviderAdapter()

    async def factory() -> Any:
        if url is None:
            raise RuntimeError(
                "Redis URL is not configured but a Redis credential-provider "
                "factory was provided. Set TASKQ_REDIS_URL or pass url= explicitly."
            )
        if urlparse(url).scheme == "redis":
            logger.warning(
                "redis-credential-over-plaintext",
                scheme="redis",
                note=(
                    "redis:// sends the credential provider's bearer token "
                    "unencrypted; use rediss:// (TLS) instead."
                ),
            )
        client_kwargs.setdefault("decode_responses", False)
        return redis_async.Redis.from_url(
            url,
            credential_provider=adapter,
            **client_kwargs,
        )

    return factory

apply_batch_terminal_outcome async

apply_batch_terminal_outcome(
    backend: Backend,
    job: JobRow,
    outcome: AttemptOutcome,
    *,
    loop_conn: Connection | None = None,
) -> None

Apply batch policy after a job reaches a terminal write.

Called after every terminal write by the consumer and the in-memory runner. For non-batched jobs (no metadata.batch_id) this returns immediately — zero overhead.

• "succeeded": resets the consecutive-failure counter. If no jobs remain non-terminal, marks the batch complete.
• "failed": increments the consecutive-failure counter. If the threshold is reached, aborts the batch and logs batch-aborted. If not aborted and no jobs remain non-terminal, marks the batch complete.
• "cancelled" / "crashed": counts non-terminal jobs. If none remain, marks the batch complete.
• "snoozed" / "reservation_denied" / "rate_limit_denied" / "scheduled": returns immediately — the job is rescheduled, not terminal.

Best-effort semantics (M7): the increment/reset/abort/complete writes are best-effort. A crash between the terminal job write and the counter increment loses that increment — the failure count is under-counted by one. The next terminal failure re-triggers the check and increments again, so a consistently failing batch still aborts (just one failure later than it would have). The stale-batch sweep is the safety net for batch STATUS (it transitions stuck active/aborted rows to terminal) but it cannot recover lost failure counts — a crash gap means the consecutive-failure streak is permanently broken, potentially preventing an abort that should have fired.

Source code in src/taskq/batch.py

async def apply_batch_terminal_outcome(
    backend: Backend,
    job: JobRow,
    outcome: AttemptOutcome,
    *,
    loop_conn: "asyncpg.Connection | None" = None,
) -> None:
    """Apply batch policy after a job reaches a terminal write.

    Called after every terminal write by the consumer and the in-memory
    runner.  For non-batched jobs (no ``metadata.batch_id``) this returns
    immediately — zero overhead.

    - ``"succeeded"``: resets the consecutive-failure counter.  If no
      jobs remain non-terminal, marks the batch complete.
    - ``"failed"``: increments the consecutive-failure counter.  If the
      threshold is reached, aborts the batch and logs ``batch-aborted``.
      If not aborted and no jobs remain non-terminal, marks the batch
      complete.
    - ``"cancelled"`` / ``"crashed"``: counts non-terminal jobs.  If none
      remain, marks the batch complete.
    - ``"snoozed"`` / ``"reservation_denied"`` / ``"rate_limit_denied"`` /
      ``"scheduled"``: returns immediately — the job is rescheduled, not
      terminal.

    **Best-effort semantics (M7):** the increment/reset/abort/complete
    writes are best-effort.  A crash between the terminal job write and
    the counter increment loses that increment — the failure count is
    under-counted by one.  The next terminal failure re-triggers the
    check and increments again, so a consistently failing batch still
    aborts (just one failure later than it would have).  The stale-batch
    sweep is the safety net for batch **STATUS** (it transitions stuck
    active/aborted rows to terminal) but it **cannot** recover lost
    failure **counts** — a crash gap means the consecutive-failure streak
    is permanently broken, potentially preventing an abort that should
    have fired.
    """
    raw_bid = job.metadata.get("batch_id")
    if raw_bid is None:
        return
    batch_id = UUID(str(raw_bid))

    if outcome in ("snoozed", "reservation_denied", "rate_limit_denied", "scheduled"):
        return

    if outcome == "succeeded":
        remaining = await backend.reset_batch_failures(batch_id, connection=loop_conn)
        if remaining == 0:
            await backend.complete_batch(batch_id, connection=loop_conn)
        return

    if outcome == "failed":
        count, threshold, remaining = await backend.increment_batch_failures(
            batch_id, connection=loop_conn
        )
        if threshold is not None and count >= threshold:
            await backend.abort_batch(batch_id, connection=loop_conn)
            _logger.info(
                "batch-aborted",
                batch_id=str(batch_id),
                consecutive_failures=count,
                threshold=threshold,
                job_id=str(job.id),
            )
        elif remaining == 0:
            await backend.complete_batch(batch_id, connection=loop_conn)
        return

    # outcome is "cancelled" or "crashed" — the only remaining
    # terminal outcomes in AttemptOutcome that are not handled above.
    if outcome in ("cancelled", "crashed"):
        remaining = await backend.count_batch_non_terminal(batch_id, connection=loop_conn)
        if remaining == 0:
            await backend.complete_batch(batch_id, connection=loop_conn)
        return

    assert_never(outcome)

wait_for_batch async

wait_for_batch(
    db: Connection | Pool,
    batch_id: UUID,
    *,
    schema: str = "taskq",
    snooze_interval: timedelta = timedelta(seconds=10),
    snooze_via_exception: bool = True,
    expect_at_least: int | None = None,
    on_empty: Literal["error", "ok"] = "error",
    exclude_job_id: UUID | None = None,
) -> BatchCompletionStatus

Convenience helper for the fan-out-then-finalize pattern.

Queries batch children by batch_id using the GIN-indexed WHERE metadata @> $1::jsonb predicate.

Inside an actor (snooze_via_exception=True, the default): - If any children are in-flight, raises Snooze(snooze_interval). The consumer transitions the job to scheduled; the actor is retried after snooze_interval without consuming retry budget. - If all children are terminal, returns BatchCompletionStatus.

Outside an actor (snooze_via_exception=False): - Blocks via asyncio.sleep(snooze_interval) in a loop until all children are terminal, then returns BatchCompletionStatus. - Use this form from scripts and integration tests where no consumer is present to translate a Snooze into a rescheduled job.

expect_at_least raises :class:~taskq.exceptions.EmptyBatchError when fewer than the expected number of jobs are present and none are in flight. on_empty controls the behaviour when the batch has zero jobs and no batches row exists: "error" (default) raises :class:~taskq.exceptions.EmptyBatchError; "ok" returns the empty status. exclude_job_id omits a specific job from the count — when not set, the batch row's finalizer_job_id is used automatically.

If the batch row has status = 'aborted' and all jobs are terminal, raises :class:~taskq.exceptions.BatchAbortedError.

snooze_interval is clamped to a minimum of 1 second. schema must match the schema used when the PostgresBackend was constructed (default "taskq").

Source code in src/taskq/batch.py

async def wait_for_batch(
    db: "asyncpg.Connection | asyncpg.Pool",
    batch_id: UUID,
    *,
    schema: str = "taskq",
    snooze_interval: timedelta = timedelta(seconds=10),
    snooze_via_exception: bool = True,
    expect_at_least: int | None = None,
    on_empty: Literal["error", "ok"] = "error",
    exclude_job_id: UUID | None = None,
) -> BatchCompletionStatus:
    """Convenience helper for the fan-out-then-finalize pattern.

    Queries batch children by batch_id using the GIN-indexed
    ``WHERE metadata @> $1::jsonb`` predicate.

    Inside an actor (snooze_via_exception=True, the default):
      - If any children are in-flight, raises Snooze(snooze_interval).
        The consumer transitions the job to scheduled; the actor is
        retried after snooze_interval without consuming retry budget.
      - If all children are terminal, returns BatchCompletionStatus.

    Outside an actor (snooze_via_exception=False):
      - Blocks via asyncio.sleep(snooze_interval) in a loop until all
        children are terminal, then returns BatchCompletionStatus.
      - Use this form from scripts and integration tests where no consumer
        is present to translate a Snooze into a rescheduled job.

    ``expect_at_least`` raises :class:`~taskq.exceptions.EmptyBatchError`
    when fewer than the expected number of jobs are present and none are
    in flight.  ``on_empty`` controls the behaviour when the batch has
    zero jobs and no ``batches`` row exists: ``"error"`` (default) raises
    :class:`~taskq.exceptions.EmptyBatchError`; ``"ok"`` returns the
    empty status.  ``exclude_job_id`` omits a specific job from the
    count — when not set, the batch row's ``finalizer_job_id`` is used
    automatically.

    If the batch row has ``status = 'aborted'`` and all jobs are
    terminal, raises :class:`~taskq.exceptions.BatchAbortedError`.

    snooze_interval is clamped to a minimum of 1 second.
    ``schema`` must match the schema used when the PostgresBackend was
    constructed (default ``"taskq"``).
    """
    if not _IDENT_RE.match(schema):
        raise ValueError(f"invalid schema identifier: {schema!r}")

    import asyncpg as _asyncpg

    if snooze_interval < MIN_SNOOZE_INTERVAL:
        original = snooze_interval
        snooze_interval = MIN_SNOOZE_INTERVAL
        _logger.warning(
            "snooze-interval-clamped",
            original=str(original),
            clamped=str(snooze_interval),
        )

    containment = dumps_str({"batch_id": str(batch_id)})

    _batch_row_sql = (
        f"SELECT id, queue, status, expected_size, consecutive_failures, "  # noqa: S608  # Why: schema validated against _IDENT_RE immediately above.
        f"failure_threshold, finalizer_job_id, originating_actor, "
        f"created_at, completed_at, metadata "
        f'FROM "{schema}".batches WHERE id = $1'
    )

    async def _fetch_batch_row(
        conn: "asyncpg.Connection",
    ) -> BatchRow | None:
        try:
            rec = await conn.fetchrow(_batch_row_sql, batch_id)
        except _asyncpg.exceptions.UndefinedTableError:
            return None
        if rec is None:
            return None
        return _batch_row_from_record(rec)

    async def _fetch_and_decide(
        conn: "asyncpg.Connection",
    ) -> BatchCompletionStatus:
        batch_row = await _fetch_batch_row(conn)

        exclusion_id = exclude_job_id
        if exclusion_id is None and batch_row is not None:
            exclusion_id = batch_row.finalizer_job_id

        if exclusion_id is not None:
            sql = _WAIT_FOR_BATCH_SQL.format(schema=schema) + " AND id <> $2"
            row = await conn.fetchrow(sql, containment, exclusion_id)
        else:
            sql = _WAIT_FOR_BATCH_SQL.format(schema=schema)
            row = await conn.fetchrow(sql, containment)

        if row is None:
            status = BatchCompletionStatus(
                total=0,
                pending=0,
                succeeded=0,
                failed=0,
                cancelled=0,
                crashed=0,
                abandoned=0,
            )
        else:
            status = BatchCompletionStatus(
                total=int(row["total"]),
                pending=int(row["in_flight"]),
                succeeded=int(row["succeeded"]),
                failed=int(row["failed"]),
                cancelled=int(row["cancelled"]),
                crashed=int(row["crashed"]),
                abandoned=int(row["abandoned"]),
            )

        status = decide_batch_status(
            batch_id=batch_id,
            batch_row=batch_row,
            status=status,
            snooze_interval=snooze_interval,
            expect_at_least=expect_at_least,
            on_empty=on_empty,
            snooze_via_exception=snooze_via_exception,
        )

        # Snooze for pending > 0 (batch not aborted) — the decision
        # function already handles the aborted-but-in-flight case
        # (raises Snooze or returns status depending on snooze_via_exception).
        # Here we handle the normal pending case.
        if status.pending > 0 and snooze_via_exception:
            raise Snooze(snooze_interval)

        return status

    async def _fetch() -> BatchCompletionStatus:
        if isinstance(db, _asyncpg.Pool):
            async with db.acquire() as conn:  # type: ignore[reportArgumentType]  # Why: Pool.acquire() returns PoolConnectionProxy; pyright stubs model it as incompatible with Connection but it is runtime-compatible
                return await _fetch_and_decide(conn)  # type: ignore[reportArgumentType]  # Why: PoolConnectionProxy is a runtime-compatible Connection proxy; pyright stubs model it as incompatible
        return await _fetch_and_decide(db)

    status = await _fetch()

    if status.pending > 0 and not snooze_via_exception:
        while status.pending > 0:
            await asyncio.sleep(snooze_interval.total_seconds())
            status = await _fetch()

    return status

register_cron

register_cron(schedule: CronScheduleSpec) -> None

Add schedule to the module-level registry.

Validates the cron expression at call time. Raises :class:ValueError on bad expression or mutually exclusive fields. Duplicate calls append again — deduplication is the caller's responsibility (the DB (actor, name) UNIQUE constraint is the authoritative gate at startup time).

Source code in src/taskq/scheduler.py

def register_cron(schedule: CronScheduleSpec) -> None:
    """Add *schedule* to the module-level registry.

    Validates the cron expression at call time.  Raises :class:`ValueError`
    on bad expression or mutually exclusive fields.  Duplicate calls append
    again — deduplication is the caller's responsibility (the DB
    ``(actor, name)`` UNIQUE constraint is the authoritative gate at
    startup time).
    """
    _validate_spec(schedule)
    _CRON_REGISTRY.append(schedule)

cron()

taskq.cron is both a submodule (src/taskq/cron.py) and, via from taskq.cron import cron, the name of a re-exported function on the taskq package. This name collision means the cron() function does not render under the taskq package-level directive above — mkdocstrings resolves taskq.cron to the submodule. The explicit directive below documents the function itself; see the Cron Scheduling guide for usage.

cron

cron(
    expression: str,
    actor: str,
    *,
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    name: str = "",
    identity_key: IdentityKey | None = None,
    timezone: str = "UTC",
    dst_strategy: DstStrategy = "skip",
    enabled: bool = True,
) -> CronScheduleSpec

Declare a cron schedule and auto-register it.

Validates expression via croniter.is_valid(); raises :class:ValueError on invalid expressions. Raises :class:ValueError if both payload_factory and static_payload are provided.

The returned :class:CronScheduleSpec is registered via :func:~taskq.scheduler.register_cron at decoration time so decorated schedules are auto-discovered at worker startup without any explicit register_cron() call.

Startup auto-discovery is create-only, skip-on-conflict. Existing cron_schedules rows are never modified by the decorator registration pass. If a @cron decorator's parameters change after the schedule was first registered, the operator must manually update or delete and recreate the schedule.

Parameters:

Name	Type	Description	Default
dst_strategy	DstStrategy	How to handle DST gaps and overlaps. skip (default) advances past gaps, uses the first occurrence in overlaps. firstof explicitly selects the earlier wall-clock time in overlaps. allof fires at both occurrences in overlaps (the caller receives two datetimes from compute_next_fire_after).	'skip'

Source code in src/taskq/cron.py

def cron(
    expression: str,
    actor: str,
    *,
    payload_factory: str | None = None,
    static_payload: dict[str, object] | None = None,
    name: str = "",
    identity_key: IdentityKey | None = None,
    timezone: str = "UTC",
    dst_strategy: DstStrategy = "skip",
    enabled: bool = True,
) -> CronScheduleSpec:
    """Declare a cron schedule and auto-register it.

    Validates *expression* via ``croniter.is_valid()``; raises
    :class:`ValueError` on invalid expressions.  Raises
    :class:`ValueError` if both *payload_factory* and *static_payload*
    are provided.

    The returned :class:`CronScheduleSpec` is registered via
    :func:`~taskq.scheduler.register_cron` at decoration time so
    decorated schedules are auto-discovered at worker startup without
    any explicit ``register_cron()`` call.

    Startup auto-discovery is **create-only, skip-on-conflict**.  Existing
    ``cron_schedules`` rows are never modified by the decorator
    registration pass.  If a ``@cron`` decorator's parameters change
    after the schedule was first registered, the operator must manually
    update or delete and recreate the schedule.

    Args:
        dst_strategy: How to handle DST gaps and overlaps.
            ``skip`` (default) advances past gaps, uses the first
            occurrence in overlaps. ``firstof`` explicitly selects the
            earlier wall-clock time in overlaps. ``allof`` fires at
            both occurrences in overlaps (the caller receives two
            datetimes from ``compute_next_fire_after``).
    """
    if not croniter.is_valid(expression):
        raise ValueError(f"Invalid cron expression: {expression!r}")
    if payload_factory is not None and static_payload is not None:
        raise ValueError(
            "payload_factory and static_payload are mutually exclusive; "
            "provide one or the other, not both"
        )
    spec = CronScheduleSpec(
        actor=actor,
        cron_expr=expression,
        timezone=timezone,
        dst_strategy=dst_strategy,
        payload_factory=payload_factory,
        static_payload=static_payload,
        name=name,
        identity_key=identity_key,
        enabled=enabled,
    )
    from taskq.scheduler import register_cron

    register_cron(spec)
    return spec