Dependency Injection¶
ProviderRegistry, Scope, and lifecycle types.
di ¶
Public DI declaration surface.
__all__
module-attribute
¶
ProviderRegistry ¶
Mutable provider registry with seal guard and edge capture.
Structurally satisfies the taskq._di.types.ProviderRegistry
Protocol without explicit inheritance, the Protocol is
runtime_checkable for isinstance checks, but the concrete class
does not inherit from it to avoid forcing runtime_checkable
constraints on the implementation.
Source code in src/taskq/_di/registry.py
providers
property
¶
Shallow copy preventing external mutation.
Why: the internal map is dict[type, ProviderEntry[object]] per the erasure boundary. Returning a copy prevents callers from mutating registry state.
register_value ¶
Source code in src/taskq/_di/registry.py
register_factory ¶
Source code in src/taskq/_di/registry.py
register_class ¶
register_class(
type_: type[T],
scope: Scope,
*,
lifecycle: ProviderLifecycle | None = None,
) -> None
Source code in src/taskq/_di/registry.py
has_provider ¶
get ¶
Source code in src/taskq/_di/registry.py
shadow_derived_providers ¶
Every non-value provider whose dependency closure reaches one of
shadow_types (see :func:taskq._di._shadow.shadow_derived_providers).
Memoized per shadow_types once the registry is sealed: the closure is a pure function of the (then immutable) provider graph, and the per-slot dispatch path asks for it on every job. Before the seal a registration can still change the answer, so the walk runs uncached. The memo is keyed by the worker's own slot wiring (one key per distinct shadowed-type set), never by user input, so it needs no bound.
Source code in src/taskq/_di/registry.py
validate ¶
validate(
actors: list[ActorRef[Any, Any]] | None = None,
rate_limit_registry: RateLimitRegistry | None = None,
) -> None
Walk all providers and actors; raise on first error; seal on success.
Pure graph-walk over registration-time metadata. Never invokes a
factory, calls a resolver, or performs await, the entire algorithm
is synchronous introspection on _providers and _dep_edges.
actors: explicit list of ActorRef instances whose DI parameter annotations are walked for MissingProvider checks and plan-cache population. Defaults to None (no actor walk, only provider→provider edges are validated).
rate_limit_registry: when provided, each actor's rate_limits
and reservations lists are checked against the registry's
dicts. Unknown names raise MissingProvider at startup.
When None (default), the name-check phase is
skipped entirely.
Why: ActorRef[Any, Any] is the sanctioned erasure¶
boundary for the heterogeneous actor registry, each ActorRef has¶
different P and R type parameters, and validate() does not need¶
per-actor narrowing. The same erasure is used at the existing API¶
boundary in worker/run.py:283 and cli.py:98.¶
Source code in src/taskq/_di/registry.py
LifecycleDetectionWarning ¶
Bases: UserWarning
Emitted for DI usage hazards detected during DI operations.
Specific timing varies per emission site and is documented at each emission. Fires for redundant call-site Scope overrides , sync close detection on registered classes, and other drift-prone API patterns. Subclass of UserWarning so pytest captures it under default filters via pytest.warns(LifecycleDetectionWarning).
Scope ¶
Bases: IntEnum
DI scope lifetime. Higher value = narrower scope.
PROCESS: worker process startup -> exit (singletons). THREAD: worker thread spawn -> thread close (placeholder for multi-thread worker). LOOP: worker loop start -> loop close (pools, long-lived clients). TRANSIENT: fresh per injection point - no cache.
ProviderLifecycle ¶
Bases: Enum
Provider lifecycle shapes for resolution-time dispatch.
registry ¶
Concrete ProviderRegistry, registration API and seal mechanics.
The public surface (register_value / register_factory / register_class,
has_provider, get[T], providers, validate) matches the
published block. The validate() method delegates the five-phase startup
validation algorithm to _di._validate.run_validation.
ProviderRegistry ¶
Mutable provider registry with seal guard and edge capture.
Structurally satisfies the taskq._di.types.ProviderRegistry
Protocol without explicit inheritance, the Protocol is
runtime_checkable for isinstance checks, but the concrete class
does not inherit from it to avoid forcing runtime_checkable
constraints on the implementation.
Source code in src/taskq/_di/registry.py
providers
property
¶
Shallow copy preventing external mutation.
Why: the internal map is dict[type, ProviderEntry[object]] per the erasure boundary. Returning a copy prevents callers from mutating registry state.
register_value ¶
Source code in src/taskq/_di/registry.py
register_factory ¶
Source code in src/taskq/_di/registry.py
register_class ¶
register_class(
type_: type[T],
scope: Scope,
*,
lifecycle: ProviderLifecycle | None = None,
) -> None
Source code in src/taskq/_di/registry.py
has_provider ¶
get ¶
Source code in src/taskq/_di/registry.py
shadow_derived_providers ¶
Every non-value provider whose dependency closure reaches one of
shadow_types (see :func:taskq._di._shadow.shadow_derived_providers).
Memoized per shadow_types once the registry is sealed: the closure is a pure function of the (then immutable) provider graph, and the per-slot dispatch path asks for it on every job. Before the seal a registration can still change the answer, so the walk runs uncached. The memo is keyed by the worker's own slot wiring (one key per distinct shadowed-type set), never by user input, so it needs no bound.
Source code in src/taskq/_di/registry.py
validate ¶
validate(
actors: list[ActorRef[Any, Any]] | None = None,
rate_limit_registry: RateLimitRegistry | None = None,
) -> None
Walk all providers and actors; raise on first error; seal on success.
Pure graph-walk over registration-time metadata. Never invokes a
factory, calls a resolver, or performs await, the entire algorithm
is synchronous introspection on _providers and _dep_edges.
actors: explicit list of ActorRef instances whose DI parameter annotations are walked for MissingProvider checks and plan-cache population. Defaults to None (no actor walk, only provider→provider edges are validated).
rate_limit_registry: when provided, each actor's rate_limits
and reservations lists are checked against the registry's
dicts. Unknown names raise MissingProvider at startup.
When None (default), the name-check phase is
skipped entirely.
Why: ActorRef[Any, Any] is the sanctioned erasure¶
boundary for the heterogeneous actor registry, each ActorRef has¶
different P and R type parameters, and validate() does not need¶
per-actor narrowing. The same erasure is used at the existing API¶
boundary in worker/run.py:283 and cli.py:98.¶
Source code in src/taskq/_di/registry.py
scopes ¶
Concrete ScopeContainer with LIFO teardown and factory-shape dispatch.
Implements the log-and-continue teardown policy: each teardown callback
runs in its own try/except; failures are logged at ERROR; remaining
teardowns always fire. Teardowns are kept in the container's own LIFO
list rather than an AsyncExitStack, whose aclose() re-raises the
first exception and swallows the rest.
ScopeContainer ¶
Concrete scope-lifetime container owning a cache and a teardown list.
The container is responsible for ALL factory invocation, caching, and teardown registration. The solver engine NEVER calls a factory directly and NEVER registers a teardown.
Source code in src/taskq/_di/scopes.py
last_cache_hit
property
¶
Whether the most recent get_or_create returned a cached value.
has_teardown_work
property
¶
Whether :meth:aclose has anything left to run.
False once every registered teardown has run and the pinned
SYNC_GENERATOR executor (if one was ever opened) is shut down ,
i.e. exactly when aclose() would return without awaiting
anything, which lets a per-job caller skip scheduling it.
get_or_create
async
¶
Resolve type_ via entry, creating and caching the instance if needed.
Source code in src/taskq/_di/scopes.py
aclose
async
¶
Close the container with the log-and-continue teardown policy.
Source code in src/taskq/_di/scopes.py
ProcessScope ¶
Bases: ScopeContainer
PROCESS-lifetime scope, worker process startup → exit.
Source code in src/taskq/_di/scopes.py
bootstrap
async
¶
Resolve all PROCESS-scoped providers.
Why: no try/except around get_or_create, earlier providers' teardowns are already registered on self._teardowns; the caller runs aclose() on unwind, which iterates whatever teardowns were registered before the failure. Partial-bootstrap leaks are not possible: every successful get_or_create has its teardown queued before the next get_or_create starts.
Source code in src/taskq/_di/scopes.py
shutdown
async
¶
ThreadScope ¶
Bases: ScopeContainer
THREAD-lifetime scope, placeholder for multi-thread workers (trivially empty in M3).
Source code in src/taskq/_di/scopes.py
bootstrap
async
¶
Resolve all THREAD-scoped providers (empty in M3 single-thread deployment).
Source code in src/taskq/_di/scopes.py
shutdown
async
¶
LoopScope ¶
Bases: ScopeContainer
LOOP-lifetime scope, worker loop start → loop close.
Source code in src/taskq/_di/scopes.py
bootstrap
async
¶
bootstrap(
registry: ProviderRegistry,
process_scope: ProcessScope,
thread_scope: ThreadScope | None = None,
) -> None
Resolve all LOOP-scoped providers.
Why: process_scope / thread_scope parameters are accepted for API completeness even though bootstrap doesn't read them directly, wider scopes' instances are reachable through the _resolver callable (which holds the registry + container map).
Source code in src/taskq/_di/scopes.py
shutdown
async
¶
resolved_cache ¶
Return a read-only view of the LOOP-scope resolved values.
The returned mapping is a live view onto the underlying cache ,
providers resolved after this method is called (e.g., lazy
providers) become visible without re-fetching, and replacements
made via :meth:replace_value become visible to subsequent
.get() reads. Callers MUST NOT attempt to mutate the mapping;
doing so raises TypeError.
Stability invariant: once bootstrap returns, the registered
set of LOOP-scope provider types is FROZEN for the lifetime of
the loop. asyncpg.Connection values are stable references ,
callers that read the mapping repeatedly (such as
SubJobEnqueuer.enqueue) assume the connection identity does
not change between dispatches, because the consumer's
per-dispatch transaction lifecycle requires the same connection
across both transaction-open and transaction-close.
asyncpg.Pool values are the one sanctioned exception: they
may be replaced mid-loop via :meth:replace_value (credential
hot-reload, see taskq.worker.deps.reload_credentials).
Pools are stateless acquisition factories, so swapping the
reference does not violate any transaction-identity invariant;
consumers that read the mapping per dispatch (rather than
capturing it once) always see the current pool.
Source code in src/taskq/_di/scopes.py
replace_value ¶
Replace the cached instance for type_.
The sanctioned mid-loop swap for VALUE providers whose resource
was hot-swapped externally, currently asyncpg.Pool after a
SIGHUP credential reload (taskq.worker.deps.reload_credentials
drains and closes the old pool; without this, DI-injected
db: asyncpg.Pool consumers would keep using a closed pool).
The old value is NOT torn down here, its lifecycle belongs to
the swapper. Only the cache entry is replaced; the (sealed)
registry is untouched. Raises :class:KeyError when type_ has
no cached value, replacing an absent entry would silently
insert a value no provider declared.
Source code in src/taskq/_di/scopes.py
get_or_create
async
¶
Source code in src/taskq/_di/scopes.py
LoopScopeSlotView ¶
LoopScopeSlotView(
inner: LoopScope,
slot_values: Mapping[type, object],
*,
invocation_runner: ScopeContainer,
shadow_derived: frozenset[type],
)
Per-actor-invocation view of the LOOP scope for one consumer slot.
The worker's per-slot path acquires a dedicated connection per job (a
LOOP-scope asyncpg.Connection registered plus max_concurrency >
1); the transaction that wraps the actor runs on that connection.
This view shadows the LOOP-shared cache for exactly the types its
mapping carries, for exactly one build_actor_scope body, so the
actor, and every nested LOOP-scoped resolution its dependencies
trigger through the same scope-containers map, resolves the slot's
instance instead of the one object registered at Scope.LOOP.
Without the shadow, every concurrent slot's actor received the SAME
registered connection and interleaved operations on it: asyncpg
permits one operation per connection, so healthy actors raised
InterfaceError (misattributed to the actor, retry budget burned),
and the actor's own writes sat outside its slot's transaction
(each job's work must run in isolation within its
transaction; sharing a connection across concurrent jobs violates
that isolation and causes writes to land outside any job's boundary).
The shadow reaches one level further than the mapping itself: a LOOP-scoped FACTORY whose dependency closure reaches a shadowed type (a helper that injects the connection, "any object derived from a shared connection is shared the same way", the same rule one level removed) is re-resolved PER INVOCATION through the invocation's TRANSIENT runner instead of the LOOP cache. The scope's bootstrap eagerly resolved that factory through the REAL containers, so the cached singleton holds the ONE registered connection; handing it to concurrent slots' actors would rebuild the exact sharing the view exists to prevent. Re-resolution runs the factory's own parameters through the same shadowed scope-containers map (nested LOOP-scoped dependencies included), produces an instance this invocation alone owns, and lands any lifecycle teardown on the invocation's TRANSIENT teardown, the derived object lives and dies with the actor call. Providers whose closure never touches a shadowed type keep the LOOP singleton: the view does not per-invocation-ize unrelated loop-lifetime resources.
The view is a pure read-through otherwise: it never caches, never invokes a factory on its own account, and never registers a teardown, a mapped instance's lifecycle belongs to the wiring that supplied it (the dispatch acquire/release exit stack that owns the slot connection), and a re-resolved instance's to the invocation runner. Every type NOT in the mapping and NOT shadow-derived resolves through the real LOOP container unchanged, and nothing outside this actor invocation can observe the shadow: the LOOP cache itself is never touched.
Mapped values must be live instances; None is not a meaningful
mapping (it is treated as absent, exactly like the LOOP cache's own
miss shape).
Source code in src/taskq/_di/scopes.py
last_cache_hit
property
¶
Whether the most recent get_or_create served a mapped or cached value.
get_or_create
async
¶
Source code in src/taskq/_di/scopes.py
ResolvedActorScope
dataclass
¶
The yielded value of build_actor_scope.
ctx , the JobContext supplied by the consumer (passthrough). di_kwargs , the DI-resolved kwargs dict for the actor function.
Usage::
async with build_actor_scope(...) as resolved:
await run_actor(job, resolved.ctx, **resolved.di_kwargs)
make_resolver ¶
make_resolver(
registry: ProviderRegistry,
scope_containers: dict[Scope, ScopeContainer],
) -> _Resolver
Construct the resolver callable expected by ScopeContainer.init.
The resolver invokes solve_dependencies with the registry and scope containers. The closure captures scope_containers by reference so the resolver sees containers added after its construction (the dict is populated incrementally as each scope container is bootstrapped).
Source code in src/taskq/_di/scopes.py
build_actor_scope
async
¶
build_actor_scope(
*,
registry: ProviderRegistry,
process_scope: ProcessScope,
thread_scope: ThreadScope,
loop_scope: LoopScope,
actor_func: Callable[..., Awaitable[object]],
actor_name: str,
passthrough_kwargs: dict[str, object],
loop_slot_values: Mapping[type, object] | None = None,
) -> AsyncGenerator[ResolvedActorScope, None]
Per-invocation actor scope: opens TRANSIENT stack, resolves DI kwargs.
Yields ResolvedActorScope(ctx, di_kwargs). On exit (regardless of outcome), closes the TRANSIENT stack in LIFO order via the log-and-continue teardown policy.
passthrough_kwargs MUST contain the JobContext (key "ctx") and the validated payload (key "payload"). The consumer constructs both per-job and supplies them as passthrough; the registry's graph walk is configured to skip these parameter names, so they are never resolved from providers.
loop_slot_values maps LOOP-registered types to the instances THIS
consumer slot owns for this one actor invocation (the dispatch path
maps asyncpg.Connection to the job's slot connection). Each
mapped type resolves to the slot's instance, through a
:class:LoopScopeSlotView that shadows the LOOP cache for this
invocation only, actor parameters and nested LOOP-scoped
dependencies alike, so a LOOP-registered connection never reaches
two concurrent slots' actors, and a LOOP-scoped
factory DERIVED from a shadowed type (a helper holding the
connection) resolves per invocation instead of serving the
bootstrap singleton that baked the registered connection in. None
/empty keeps the plain LOOP container: the single-slot worker, whose
transaction connection IS the registered connection.
Source code in src/taskq/_di/scopes.py
693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 | |