Testing¶
InMemoryBackend, FakeClock, pytest fixtures, and test assertions.
testing ¶
Test-only helpers for TaskQ: FakeClock, InMemoryBackend, and stub utilities.
Every symbol here lives in taskq.testing — never in taskq.backend — so
production code does not pull in test-only helpers.
run_until_drained, tick_cancel_polling, and register_cancel_event
are methods on the re-exported :class:InMemoryBackend class. The runner
logic lives in :mod:taskq.testing._runner.
Pytest fixtures are NOT re-exported here — they live in
:mod:taskq.testing.fixtures and are imported by
:mod:tests.conftest directly. This avoids importing pytest /
asyncpg at the taskq.testing top level.
JobContext is re-exported for convenience; the eventual public
JobContext will replace this test-scoped version.
OTel test utilities (ListSpanExporter, setup_tracer, setup_meter)
are NOT re-exported here — import them from taskq.testing.otel directly.
They require the [otel] extra (opentelemetry-sdk).
DEFAULT_ACTORS
module-attribute
¶
DEFAULT_ACTORS: tuple[str, ...] = (
"actor_a",
"actor_b",
"actor_c",
"A",
"C",
"X",
"test_actor",
"_progress_redis_hundred",
"_progress_redis_single",
"_progress_redis_three",
)
__all__
module-attribute
¶
__all__ = [
"DEFAULT_ACTORS",
"EmptyPayload",
"FakeBackend",
"FakeClock",
"InMemoryBackend",
"JobContext",
"JobTriple",
"StubActorConfig",
"WarningSpy",
"as_backend",
"assert_attempt",
"assert_has_event",
"assert_has_otel_event",
"assert_has_span",
"assert_job_status",
"assert_job_terminal",
"assert_transition_sequence",
"create_pending_job",
"create_running_job",
"create_worker",
"create_workered_running_job",
"default_actor_config",
"error_info",
"get_job_triple",
"make_enqueue_args",
"make_integration_settings",
"make_integration_settings_dict",
"make_job_row",
"parse_detail",
"pg_now",
"reset_schema",
"seed_actors",
"setup_running_job",
"truncate_schema",
"wait_for",
"wait_for_job_status",
"wait_for_leader",
]
EmptyPayload ¶
Bases: BaseModel
FakeBackend ¶
FakeBackend(
*,
mark_snoozed_return: Literal[
"scheduled", "failed", "noop"
] = "scheduled",
mark_retry_after_return: Literal[
"scheduled",
"failed:DeadlineExceeded",
"failed:MaxAttemptsExceeded",
"noop",
] = "scheduled",
)
Minimal backend recording method calls for assertions.
Source code in src/taskq/testing/actor.py
supports_transactional_simulation
class-attribute
instance-attribute
¶
mark_succeeded_calls
instance-attribute
¶
mark_failed_or_retry_calls
instance-attribute
¶
enqueue
async
¶
enqueue_with_conn
async
¶
dispatch_batch
async
¶
heartbeat_jobs
async
¶
extend_reservation_leases
async
¶
mark_succeeded
async
¶
mark_succeeded(
job_id: UUID,
worker_id: UUID,
result: dict[str, object] | None,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> bool
Source code in src/taskq/testing/actor.py
mark_succeeded_with_conn
async
¶
mark_succeeded_with_conn(
conn: object,
job_id: UUID,
worker_id: UUID,
result: dict[str, object] | None,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> bool
Source code in src/taskq/testing/actor.py
mark_failed_or_retry
async
¶
mark_failed_or_retry(
job_id: UUID,
worker_id: UUID,
error_info: ErrorInfo,
next_scheduled_at: datetime | None,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> JobRow
Source code in src/taskq/testing/actor.py
mark_cancelled
async
¶
mark_cancelled(
job_id: UUID,
worker_id: UUID,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> bool
Source code in src/taskq/testing/actor.py
write_cancel_escalation
async
¶
mark_abandoned
async
¶
mark_snoozed
async
¶
mark_snoozed(
job_id: UUID,
worker_id: UUID,
delay: timedelta,
*,
metadata_update: dict[str, object] | None = None,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
outcome: AttemptOutcome = "snoozed",
) -> Literal["scheduled", "failed", "noop"]
Source code in src/taskq/testing/actor.py
mark_retry_after
async
¶
mark_retry_after(
job_id: UUID,
worker_id: UUID,
delay: timedelta,
*,
consume_budget: bool = True,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> Literal[
"scheduled",
"failed:DeadlineExceeded",
"failed:MaxAttemptsExceeded",
"noop",
]
Source code in src/taskq/testing/actor.py
write_attempt
async
¶
get_attempts
async
¶
get_events
async
¶
write_cancel_request
async
¶
poll_cancel_flags
async
¶
scheduled_to_pending
async
¶
deadline_sweep
async
¶
reclaim_expired_locks
async
¶
get
async
¶
list_jobs
async
¶
count_pending_jobs
async
¶
enqueue_batch
async
¶
subscribe_wake ¶
create_schedule
async
¶
list_schedules
async
¶
update_schedule
async
¶
StubActorConfig
dataclass
¶
StubActorConfig(
retry: RetryPolicy,
non_retryable_exceptions: tuple[
type[Exception], ...
] = (),
retry_classifier: RetryClassifierHook | None = None,
on_retry_exhausted: OnRetryExhausted | None = None,
on_retry_exhausted_timeout: float = 3.0,
on_success: OnSuccess | None = None,
on_success_timeout: float = 3.0,
)
FakeClock ¶
Deterministic clock for tests.
Accepts a start datetime (typically
datetime(2025, 1, 1, tzinfo=UTC)). now() returns the current
internal time; move_to and advance let tests control the clock
explicitly. monotonic() returns elapsed seconds from _EPOCH
so that elapsed-time guards see a non-zero starting value.
Source code in src/taskq/testing/clock.py
InMemoryBackend ¶
InMemoryBackend(
clock: Clock,
cancellation_grace_period: timedelta = timedelta(
seconds=30
),
cleanup_grace_period: timedelta = timedelta(seconds=30),
rng: Random | None = None,
*,
actor_configs: Iterable[ActorConfig] | None = None,
)
Deterministic, in-memory backend for unit tests.
All state is held as per-instance attributes: no module-level
mutable state, no class-level caches. Two InMemoryBackend instances
created in the same test session are fully isolated.
Single-threaded by contract — do not share across threads or event loops. Intra-coroutine re-entry within a single event loop is acceptable; cross-thread use is a caller bug.
Source code in src/taskq/testing/in_memory.py
advance_clock_to ¶
register_stub ¶
register_stub(
actor_name: str,
fn: StubFn,
*,
retry: RetryPolicy | None = None,
non_retryable_exceptions: tuple[
type[BaseException], ...
] = (),
retry_classifier: RetryClassifierHook | None = None,
on_retry_exhausted: OnRetryExhausted | None = None,
on_retry_exhausted_timeout: float = 3.0,
on_success: OnSuccess | None = None,
on_success_timeout: float = 3.0,
payload_type: type[BaseModel] | None = None,
) -> None
Source code in src/taskq/testing/in_memory.py
register_cancel_event ¶
get_events
async
¶
enqueue
async
¶
enqueue_with_conn
async
¶
enqueue_batch
async
¶
enqueue_batch_fast
async
¶
register_actor_config ¶
register_actor_config(
*,
actor: str,
max_concurrent: int | None = None,
queue: str = "default",
metadata: dict[str, object] | None = None,
) -> None
Source code in src/taskq/testing/in_memory.py
register_actor_configs ¶
set_queue_mode ¶
dispatch_batch
async
¶
heartbeat_jobs
async
¶
Source code in src/taskq/testing/in_memory.py
extend_reservation_leases
async
¶
Source code in src/taskq/testing/in_memory.py
mark_succeeded
async
¶
mark_succeeded(
job_id: JobId,
worker_id: UUID,
result: dict[str, object] | None,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> bool
Source code in src/taskq/testing/in_memory.py
mark_succeeded_with_conn
async
¶
mark_succeeded_with_conn(
conn: object,
job_id: JobId,
worker_id: UUID,
result: dict[str, object] | None,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> bool
Source code in src/taskq/testing/in_memory.py
mark_failed_or_retry
async
¶
mark_failed_or_retry(
job_id: JobId,
worker_id: UUID,
error_info: ErrorInfo,
next_scheduled_at: datetime | None,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> JobRow
Source code in src/taskq/testing/in_memory.py
mark_cancelled
async
¶
mark_cancelled(
job_id: JobId,
worker_id: UUID,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> bool
Source code in src/taskq/testing/in_memory.py
write_cancel_escalation
async
¶
mark_abandoned
async
¶
mark_snoozed
async
¶
mark_snoozed(
job_id: JobId,
worker_id: UUID,
delay: timedelta,
*,
metadata_update: dict[str, object] | None = None,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
outcome: AttemptOutcome = "snoozed",
) -> Literal["scheduled", "failed", "noop"]
Source code in src/taskq/testing/in_memory.py
mark_retry_after
async
¶
mark_retry_after(
job_id: JobId,
worker_id: UUID,
delay: timedelta,
*,
consume_budget: bool = True,
progress_seq: int = 0,
progress_state: dict[str, object] | None = None,
) -> Literal[
"scheduled",
"failed:DeadlineExceeded",
"failed:MaxAttemptsExceeded",
"noop",
]
Source code in src/taskq/testing/in_memory.py
write_attempt
async
¶
get_attempts
async
¶
write_cancel_request
async
¶
Source code in src/taskq/testing/in_memory.py
poll_cancel_flags
async
¶
Source code in src/taskq/testing/in_memory.py
retry_job
async
¶
Source code in src/taskq/testing/in_memory.py
scheduled_to_pending
async
¶
deadline_sweep
async
¶
reclaim_expired_locks
async
¶
archive_terminal_jobs ¶
archive_terminal_jobs(
retention: timedelta,
archive_retention: timedelta,
*,
statuses: frozenset[str] | None = None,
) -> PruneResult
Source code in src/taskq/testing/in_memory.py
expire_archived_jobs ¶
get_archived
async
¶
get
async
¶
list_jobs
async
¶
count_pending_jobs
async
¶
subscribe_wake ¶
subscribe_cancel_wake ¶
tick_cancel_polling
async
¶
run_until_drained
async
¶
create_schedule
async
¶
Source code in src/taskq/testing/in_memory.py
list_schedules
async
¶
Source code in src/taskq/testing/in_memory.py
update_schedule
async
¶
Source code in src/taskq/testing/in_memory.py
JobContext
dataclass
¶
JobContext(
job_id: JobId,
actor: str,
queue: str,
attempt: int,
payload: P,
cancel_event: Event,
worker_id: UUID,
jobs: SubJobEnqueuer,
log: BoundLogger,
deps: dict[str, object] | None = None,
abort_requested: Event = threading.Event(),
)
Test-scoped context with deps for fixture-injected dependencies.
Field shape mirrors :class:taskq.context.JobContext (the production
class) and adds deps. The bound on P matches production —
payload is always a :class:pydantic.BaseModel. Tests that pass raw
dicts as payload should validate them through a wrapper model
(:class:taskq.testing.in_memory._PassthroughPayload is the
permissive default for register_stub callers).
JobTriple ¶
WarningSpy ¶
Records how many times warning() was called, without sniffing args.
as_backend ¶
default_actor_config ¶
assert_attempt ¶
assert_attempt(
attempts: Sequence[object],
index: int,
*,
outcome: str | None = None,
error_class: str | None = None,
attempt_num: int | None = None,
) -> object
Assert on the attempt row at index.
Source code in src/taskq/testing/assertions.py
assert_has_event ¶
assert_has_event(
events: Sequence[Record],
kind: str,
*,
from_state: str | None = None,
to_state: str | None = None,
) -> asyncpg.Record
Find at least one event matching kind and optional state filters.
Source code in src/taskq/testing/assertions.py
assert_has_otel_event ¶
assert_has_otel_event(
exporter: _SpanExporter,
span_name: str,
event_name: str,
*,
from_state: str | None = None,
to_state: str | None = None,
) -> object
Find an OTel event by span and event name; assert state attributes if provided.
Source code in src/taskq/testing/assertions.py
assert_has_span ¶
assert_has_span(
exporter: _SpanExporter,
name: str,
*,
kind: object = None,
status: object = None,
) -> ReadableSpan
Find a span by name on the exporter; assert kind/status if provided.
Source code in src/taskq/testing/assertions.py
assert_job_status ¶
assert_job_status(
row: Record | None,
status: str,
*,
error_class: str | None = None,
attempt: int | None = None,
finished: bool | None = None,
) -> asyncpg.Record
Assert a job row has the expected status and optional fields.
Returns the row (guaranteed non-None on success) so callers can chain attribute/subscript access without pyright narrowing issues.
Source code in src/taskq/testing/assertions.py
assert_job_terminal ¶
assert_job_terminal(
row: Record | None,
status: str,
*,
error_class: str | None = None,
) -> asyncpg.Record
Assert a job is in a terminal status with finished_at set.
Source code in src/taskq/testing/assertions.py
assert_transition_sequence ¶
assert_transition_sequence(
events: Sequence[object],
expected: Sequence[tuple[str | None, str | None]],
) -> None
Assert the (from_state, to_state) sequence from state_change events matches expected.
Source code in src/taskq/testing/assertions.py
parse_detail ¶
Normalize a detail value (dict, JSON string, or other) to a dict.
Source code in src/taskq/testing/assertions.py
pg_now
async
¶
Return PG's clock_timestamp() — the realtime clock the server uses.
Use this instead of datetime.now(UTC) when a test needs to compute
cutoffs/margins that are compared against rows written via SQL: the
Python wall clock and PG's realtime clock can diverge enough under
parallel load to make Python-computed margins flaky.
Source code in src/taskq/testing/assertions.py
wait_for
async
¶
Wait for an asyncio.Event with test-failure semantics on timeout.
Source code in src/taskq/testing/assertions.py
wait_for_job_status
async
¶
wait_for_job_status(
backend: _AssertBackend,
job_id: JobId,
status: str,
*,
timeout: float = 2.0,
poll_interval: float = 0.05,
) -> JobRow
Poll backend.get until the job reaches the expected status.
Source code in src/taskq/testing/assertions.py
wait_for_leader
async
¶
Wait for the leader event on WorkerDeps with test-failure semantics.
Source code in src/taskq/testing/assertions.py
error_info ¶
Shorthand for ErrorInfo with error_traceback=None.
Source code in src/taskq/testing/jobs.py
make_enqueue_args ¶
make_enqueue_args(
*,
actor: str = "test_actor",
queue: str = "default",
payload: dict[str, object] | None = None,
idempotency_key: str | None = None,
identity_key: str | None = None,
scheduled_at: datetime | None = None,
max_attempts: int = 3,
retry_kind: RetryKind = "transient",
priority: int = 0,
schedule_to_close: datetime | None = None,
metadata: dict[str, object] | None = None,
tags: tuple[str, ...] | None = None,
) -> EnqueueArgs
Build EnqueueArgs with sensible defaults.
scheduled_at defaults to 1 second in the past (relative to the
Python wall clock) so freshly-enqueued test jobs can never classify
as a future-"scheduled" job: the enqueue SQL compares against PG's
clock_timestamp(), and Python's monotonic/wall clock can diverge
from PG's realtime clock enough under parallel load that a "now"
computed here reads as still-future by the time the row lands.
Source code in src/taskq/testing/jobs.py
make_job_row ¶
make_job_row(
*,
attempt: int = 1,
max_attempts: int = 3,
retry_kind: RetryKind = "transient",
schedule_to_close: datetime | None = None,
identity_key: IdentityKey | None = None,
trace_id: str | None = None,
span_id: str | None = None,
start_to_close: timedelta | None = None,
heartbeat_timeout: timedelta | None = None,
cancel_phase: int | CancelPhase | None = None,
status: JobStatus = "running",
priority: int = 0,
error_class: str | None = None,
error_message: str | None = None,
payload: dict[str, object] | None = None,
queue: str = "default",
actor: str = "test_actor",
progress_seq: int = 0,
) -> JobRow
Build a JobRow with sensible defaults.
Source code in src/taskq/testing/jobs.py
create_pending_job
async
¶
create_pending_job(
conn: _Conn,
schema: str,
job_id: UUID | None = None,
*,
schedule_to_close: datetime | None = None,
status: str = "pending",
scheduled_at: datetime | None = None,
) -> UUID
Source code in src/taskq/testing/pg.py
create_running_job
async
¶
create_running_job(
conn: _Conn,
schema: str,
worker_id: UUID,
job_id: UUID | None = None,
*,
cancel_phase: int = 0,
max_attempts: int = 3,
retry_kind: str = "transient",
attempt: int = 1,
cancel_requested_at: datetime | None = None,
lock_expires_at: datetime | None = None,
schedule_to_close: datetime | None = None,
with_events: bool = True,
) -> UUID
Source code in src/taskq/testing/pg.py
create_workered_running_job
async
¶
create_workered_running_job(
conn: _Conn,
schema: str,
*,
worker_id: UUID | None = None,
**job_kwargs: Any,
) -> tuple[UUID, UUID]
Create a worker row and a running job row, returning (worker_id, job_id).
Passthrough wrapper: creates a worker (generating a UUID if none provided),
then creates a running job belonging to that worker. All extra keyword
arguments are forwarded to :func:create_running_job.
Source code in src/taskq/testing/pg.py
get_job_triple
async
¶
Source code in src/taskq/testing/pg.py
reset_schema
async
¶
Truncate all dynamic tables then seed default actor_config rows.
Tests needing a custom actor set can pass actors=[...];
tests that need an empty actor_config can pass actors=[].
Source code in src/taskq/testing/pg.py
seed_actors
async
¶
Insert actor_config rows for the given actors (or DEFAULT_ACTORS).
ON CONFLICT (actor) DO NOTHING makes this safe to call
alongside custom seed data — it never overwrites existing rows.
Source code in src/taskq/testing/pg.py
setup_running_job
async
¶
setup_running_job(
conn: _Conn,
schema: str,
*,
worker_id: UUID | None = None,
job_id: UUID | None = None,
attempt: int = 1,
max_attempts: int = 3,
retry_kind: str = "transient",
cancel_phase: int = 0,
cancel_requested_at: datetime | None = None,
lock_expires_at: datetime | None = None,
schedule_to_close: datetime | None = None,
with_events: bool = True,
) -> tuple[UUID, UUID]
Create a worker row and a running job row in one call.
Returns (worker_id, job_id). Delegates to
:func:create_workered_running_job.
Source code in src/taskq/testing/pg.py
truncate_schema
async
¶
Truncate all dynamic tables in FK-safe order using CASCADE.
Leaves schema_migrations intact. Safe to call repeatedly.
Source code in src/taskq/testing/pg.py
make_integration_settings ¶
Construct WorkerSettings with fast intervals for integration tests.
make_integration_settings_dict ¶
Return the raw dict passed to WorkerSettings.load_from_dict.