Credential Providers¶
Vendor-neutral credential provider Protocols (PgCredentialProvider,
RedisCredentialProvider), the PgCredential / RedisCredential carriers,
the make_* factory builders, the ReloadSchedule a factory-built pool is
rebuilt on, and enrich_pg_dsn. Provider-specific
implementations live in the taskq[aad], taskq[aws],
and taskq[vault] extras. See the
Managed Identities guide for usage.
auth ¶
Vendor-neutral credential providers and connection factories.
This module provides the reusable primitives for rotating-credential
Postgres and Redis connections - the abstract interfaces that any auth
provider (Azure Entra ID, AWS IAM RDS, HashiCorp Vault, a custom OAuth
flow, a secrets manager, …) plugs into. Provider-specific implementations
live in the taskq[aad], taskq[aws], and taskq[vault] extras;
users with other providers implement :class:PgCredentialProvider /
:class:RedisCredentialProvider directly and get all the factory
builders for free.
See the managed-identities deployment guide (docs/guides/managed-identities.md).
Design¶
- :class:
PgCredentialProvider- async Protocol returning a :class:PgCredential(a password, optionally a fresh username). AAD and AWS IAM RDS return a token-as-password; Vault dynamic DB creds return a fresh username + password pair. - :class:
RedisCredentialProvider- async Protocol returning a :class:RedisCredential(username + password). AAD returns the managed-identity object ID + token. - :func:
make_pg_pool_factory/ :func:make_dedicated_conn_factory/ :func:make_redis_client_factory- accept any provider implementing the Protocol and return the zero-arg async factories that :class:~taskq.connections.WorkerConnectionsconsumes. Credentials are passed to asyncpg asuser=/password=keyword arguments (which take precedence over both DSN userinfo and query parameters), so the token never appears in the DSN string. - :func:
enrich_pg_dsn- shared DSN helper for callers that need a self-contained DSN string: the credential is written into the DSN userinfo (the only slot asyncpg's resolver never shadows) andsslmode=requireis added only when no sslmode is already set.
Credential refresh¶
Both transports re-fetch on every physical (re)connect, so no external rotation schedule is needed:
- Postgres -
password=is handed to asyncpg as an async callable, which asyncpg awaits once per physical connection (pool creation, pool growth, and replacements aftermax_inactive_connection_lifetimerecycles an idle connection). - Redis - reconnects re-fetch via the redis-py
CredentialProvideradapter.
The one thing that cannot refresh in place is a username-bearing
pair: asyncpg resolves user= once per pool / connection and accepts
a callable only for password=, and a dynamic username is only valid
with the password issued alongside it. Providers that issue a fresh
username per credential (Vault dynamic database credentials) therefore
pin the pair for the pool's life - the callable hands asyncpg the pair's
password - and rotate on the pool rebuild that SIGHUP /
TASKQ_RELOAD_INTERVAL / taskq.worker.deps.reload_credentials
performs. The rebuild cadence is a :class:ReloadSchedule: the
operator's explicit interval when one is set, otherwise derived from the
lease the provider granted (PgCredential.lease_duration) at
:data:LEASE_RELOAD_FRACTION of the TTL, so a pair is replaced with a
full half-life to spare. Every pool builder records the leases it is
issued on its schedule, and every consumer that rebuilds pools (the
worker, taskq ui serve, :class:taskq.TaskQ) reads the interval
from it.
__all__
module-attribute
¶
__all__ = [
"LEASE_RELOAD_FRACTION",
"PgCredential",
"PgCredentialProvider",
"RedisCredential",
"RedisCredentialProvider",
"ReloadSchedule",
"build_worker_connections",
"credential_provider_of",
"enrich_pg_dsn",
"ensure_sslmode_require",
"make_dedicated_conn_factory",
"make_pg_pool_factory",
"make_redis_client_factory",
"reload_schedule_of",
]
LEASE_RELOAD_FRACTION
module-attribute
¶
Fraction of a granted lease TTL at which a pinned pair is rebuilt.
Half the TTL leaves a full half-life for the rebuild to fail and be retried
before the issuer revokes the pair: a rebuild that fails at T + TTL/2
still has until T + TTL before every reconnect on the old pool starts
failing authentication - the renew-at-half-life rule lease-issuing
secret stores apply to their own renewals.
PgCredential
dataclass
¶
A Postgres credential issued by a rotating-credential provider.
password is always required (a token or dynamic password).
username, when set, overrides the DSN's userinfo user - needed by
providers that issue a fresh username alongside the password (e.g.
Vault dynamic DB creds). When None, the DSN's existing user is
preserved.
lease_duration is how long, in seconds, the issuer will honour this
credential - the Vault lease TTL. It is the bound a rebuild schedule
has to beat for a username-bearing pair (see :class:ReloadSchedule),
and None when the issuer does not say (token providers, whose
tokens refresh per connection and never need one).
RedisCredential
dataclass
¶
PgCredentialProvider ¶
Bases: Protocol
Provides rotating Postgres credentials on demand.
Implementations fetch a fresh token / dynamic username+password each
call. Called by :func:make_pg_pool_factory /
:func:make_dedicated_conn_factory once at pool / connection
construction (to resolve user= and fail fast). A credential with
username unset is then re-fetched for every physical
connection asyncpg opens thereafter - not on each acquire(), which
hands back an already-authenticated connection from the pool - so
token providers are expected to cache and only hit the issuing service
when the cached token is near expiry. A credential that carries a
username is an issued pair used for the pool's life and is fetched
again only when the pool is rebuilt.
RedisCredentialProvider ¶
Bases: Protocol
Provides rotating Redis credentials on demand.
Implementations fetch a fresh (username, token/password) each call.
Called by :func:make_redis_client_factory on every reconnect via
the redis-py CredentialProvider adapter.
ReloadSchedule
dataclass
¶
How often pools built through a credential provider are rebuilt.
A username-bearing credential pins its pool to one issued pair (see
:func:_make_pg_password_callable), so the only rotation is a pool
rebuild, and the rebuild has to happen before the issuer revokes the
pair. This object is the single source of that cadence:
configuredis the operator's explicit interval (TASKQ_RELOAD_INTERVALon the worker andtaskq ui serve,reload_interval=on :class:taskq.TaskQ). It always wins.- Otherwise the interval is derived from the granted lease: the
pool builders record
PgCredential.lease_durationhere as each credential is issued, and :attr:intervalis the shortest lease seen so far scaled by :data:LEASE_RELOAD_FRACTION. Shortest, not latest, because Vault caps a lease at the issuing token's remaining TTL - a lease that came back shorter than the last is the bound that now has to be beaten. Nonewhen nothing is configured and no issued credential carried a lease - there is nothing to schedule, and a username-bearing credential built against such a schedule warns at build time (pg-lease-pair-pinned-without-reload).
sources composes schedules: a consumer that rebuilds several
factories (the worker's role pools and dedicated connections) reads
one schedule whose lease is the shortest across all of them, and whose
configured is its own. The composite is live - a lease recorded on
a source after composition is seen through it.
One schedule is normally shared by every factory a consumer rebuilds
(:func:build_worker_connections does this); a factory built without
one declares its own, which :func:reload_schedule_of returns so a
consumer handed an opaque factory can still adopt it.
lease_duration
property
¶
Shortest lease TTL recorded here or on any source, in seconds.
pins_pair
property
¶
Whether a username-bearing credential was issued through this schedule or any source - i.e. whether anything actually needs the rebuild.
interval
property
¶
Seconds between rebuilds: configured, else the derived
lease-based interval, else None.
derived
property
¶
True when :attr:interval comes from a granted lease rather
than an explicit setting.
__post_init__ ¶
record ¶
Note a credential a factory on this schedule was just issued.
Source code in src/taskq/auth.py
ensure_sslmode_require ¶
Add sslmode=require to dsn unless an sslmode is already set.
An explicit sslmode is never overridden - in particular stronger
modes (verify-ca / verify-full) must not be downgraded:
require skips certificate verification, which would expose the
very token this module injects to a MITM.
Public because anyone assembling a credential-bearing DSN by hand needs
exactly this rule and must not re-derive it: a token path that silently
connects without TLS puts the credential on the wire. The factory
builders in this module apply it for you; reach for it directly only on
the DSN paths they do not cover (a raw asyncpg.connect, a migration
connection, a DSN handed to another library).
sslmode=disable is an explicit choice and is preserved - that is how
a test container or a Unix-socket deployment opts out.
Because an explicit verify-ca/verify-full is passed through
untouched, it reaches asyncpg needing a CA bundle that this helper does
not supply: asyncpg and libpq do NOT fall back to the system trust store
for a verifying sslmode, they look for ~/.postgresql/root.crt and
raise ClientConfigurationError while parsing connection arguments
(before any socket is opened) when it is absent - which is the normal
case in a container. Such a DSN must carry its own sslrootcert=
(or set PGSSLROOTCERT); see the sslmode note in
docs/guides/managed-identities.md. Deliberately not defaulted here:
picking a trust root is a security decision and the correct bundle is
environment-specific.
Source code in src/taskq/auth.py
enrich_pg_dsn ¶
Apply credential to dsn and return a self-contained DSN string.
The credential is written into the DSN userinfo (percent-encoded),
replacing any existing userinfo password - and replacing the userinfo
user when credential.username is set (Vault dynamic DB creds).
This is the only slot that is guaranteed to take effect: asyncpg's
resolver applies userinfo before query parameters (both behind
if user is None / if password is None guards), so a
query-string user= / password= is silently ignored whenever
the DSN already carries userinfo. A stale password= query
parameter is dropped (always shadowed by the userinfo password);
a user= query parameter is dropped only when the userinfo
carries a user to shadow it - a query-carried user with no userinfo
user is the effective principal and is preserved.
sslmode=require is added only when the DSN has no explicit
sslmode, so stronger modes (verify-full) are never downgraded.
Prefer the factory builders (:func:make_pg_pool_factory /
:func:make_dedicated_conn_factory) where possible - they pass the
credential as keyword arguments instead, keeping the token out of
the DSN string entirely.
Source code in src/taskq/auth.py
reload_schedule_of ¶
The :class:ReloadSchedule a pool / connection factory declares.
Every factory :func:make_pg_pool_factory and
:func:make_dedicated_conn_factory return declares the schedule it
records leases on; a consumer handed an opaque factory (TaskQ(
pool_factory=...), a hand-assembled WorkerConnections) reads it
here to derive its rebuild cadence. None for a factory built some
other way - such a factory has no lease to report.
Source code in src/taskq/auth.py
credential_provider_of ¶
The credential provider a pool / connection / Redis factory was built
over, or None for a factory built some other way.
Every factory :func:make_pg_pool_factory, :func:make_dedicated_conn_factory
and :func:make_redis_client_factory return declares the provider it
fetches credentials from. A consumer that owns the factory's resources
(open_worker_deps) reads it here to release the provider at teardown
once every pool and client built through it is closed - the provider's
own resources (the Entra ID providers' lazily created credential session)
must outlive the connections that authenticate with it. None for a
factory built some other way: such a factory declares no provider, and
its consumer has nothing to release.
Source code in src/taskq/auth.py
make_pg_pool_factory ¶
make_pg_pool_factory(
dsn: str,
provider: PgCredentialProvider,
*,
min_size: int = 1,
max_size: int = 4,
max_inactive_connection_lifetime: float = 300.0,
command_timeout: float | None = None,
statement_cache_size: int = DEFAULT_STATEMENT_CACHE_SIZE,
max_cached_statement_lifetime: int = DEFAULT_MAX_CACHED_STATEMENT_LIFETIME,
init: Callable[[Connection], Awaitable[None]]
| None = None,
setup: Callable[[Connection], Awaitable[None]]
| None = None,
server_settings: dict[str, str] | None = None,
connection_class: type[Connection] | None = None,
reload_schedule: ReloadSchedule | None = None,
) -> PoolFactory
Build a :data:~taskq.connections.PoolFactory backed by provider.
Each invocation fetches a fresh :class:PgCredential from provider
and calls asyncpg.create_pool with the credential as keyword
arguments - password= always, user= when the credential
carries a username. Keyword arguments take precedence over both DSN
userinfo and query parameters in asyncpg's resolver, so a stale
credential baked into dsn can never shadow the fresh one, and the
token never appears in the DSN string. The pool is owned by the
worker (entered on its AsyncExitStack).
Token refresh: password= is passed as an async callable, which
asyncpg invokes and awaits once per physical connection - the
connections opened at pool creation, those opened later by pool
growth, and the replacements opened after
max_inactive_connection_lifetime recycles an idle connection. For
a token credential (username unset: Entra ID, AWS IAM RDS) every
new connection therefore authenticates with a freshly fetched token,
and no external rotation is required. This matters because Postgres
authenticates at connect time only: a credential resolved once and
reused as a fixed string keeps working on already-open connections
while every new connection fails, roughly one token-lifetime after
deploy.
A username-bearing credential (Vault dynamic database credentials)
is one issued pair: asyncpg resolves user= once per pool, so the
pool is pinned to that username and every physical connection
authenticates with the pair's password - re-fetching would burn a
lease per connection for a password the pinned user cannot use. Such
a pool rotates when the factory is re-invoked: SIGHUP /
TASKQ_RELOAD_INTERVAL (taskq.worker.deps.reload_credentials)
rebuilds it on a fresh pair, on the cadence reload_schedule carries
(below). Reload is also the way to force a full pool rebuild for a
token credential (dropping sessions opened under a revoked token).
Per-connection setup: init is forwarded verbatim to
asyncpg.create_pool and runs once per new physical connection
- on the connections opened at pool creation, on connections opened
later by pool growth, and again on replacements opened after
max_inactive_connection_lifetime recycles an idle connection.
That lifecycle is exactly why this setup (registering type codecs -
e.g. pgvector.asyncpg.register_vector - preparing statements,
setting session GUCs) cannot be done correctly after pool creation:
a connection configured by hand is silently replaced under load or
after an idle period. The only per-connection work this factory does
of its own is the credential refresh described above (an asyncpg
password= callback, not an init hook), so a caller-supplied
init is the only hook of its kind: it is passed through unwrapped
and can never silently replace internal setup.
Per-acquire setup: setup is forwarded to asyncpg.create_pool
and runs every time a connection is acquired from the pool
(via pool.acquire()), not just on new-connection creation. Use
it for per-checkout work that must run even when a pooled connection
is reused - e.g. resetting search_path or verifying session
state. Unlike init, setup runs on every acquire, so keep it
lightweight. Both init and setup can be provided simultaneously.
server_settings is forwarded to asyncpg.create_pool and applied
as session-level GUCs on every new connection (e.g.
{"statement_timeout": "30s", "search_path": "app"}). Useful for
per-pool configuration that must be set at connection time.
statement_cache_size / max_cached_statement_lifetime are forwarded
to asyncpg.create_pool and default to TaskQ's tuning (the
taskq.connections module constants, 512 entries, 1 h lifetime ,
which are also the TaskQSettings field defaults). asyncpg's own
defaults (100 / 300 s) thrash on TaskQ's read paths, so a
provider-backed pool must get the same cache treatment as the
DSN-built ones. Call sites with a
:class:~taskq.settings.WorkerSettings in scope pass the values
resolved through :func:taskq.connections.statement_cache_kwargs so
the TASKQ_STATEMENT_CACHE_SIZE / TASKQ_MAX_CACHED_STATEMENT_LIFETIME
env vars apply to provider-backed pools too.
connection_class is forwarded to asyncpg.create_pool and sets
the :class:asyncpg.Connection subclass used by the pool. Use it to
install custom codecs or override connection methods across the
entire pool.
reload_schedule is the :class:ReloadSchedule this pool is rebuilt
on. Each build records the credential it was issued there - its
lease_duration when the issuer reports one - so a consumer with no
explicit interval rebuilds at :data:LEASE_RELOAD_FRACTION of the
granted TTL. Pass one schedule to every factory a consumer rebuilds
together (:func:build_worker_connections does) so they share the
shortest lease; omitted, the factory declares a schedule of its own,
readable with :func:reload_schedule_of. A username-bearing
credential built against a schedule that can derive nothing (no
configured interval, no reported lease) logs
pg-lease-pair-pinned-without-reload naming
TASKQ_RELOAD_INTERVAL, because reconnects will fail authentication
once the lease expires and nothing here can prevent it. It is a
warning, never a refusal: an operator rotating the lease externally
can ignore it.
Source code in src/taskq/auth.py
596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 | |
make_dedicated_conn_factory ¶
make_dedicated_conn_factory(
dsn: str,
provider: PgCredentialProvider,
*,
command_timeout: float | None = None,
setup: Callable[[Connection], Awaitable[None]]
| None = None,
server_settings: dict[str, str] | None = None,
connection_class: type[Connection] | None = None,
reload_schedule: ReloadSchedule | None = None,
) -> ConnFactory
Build a :data:~taskq.connections.ConnFactory backed by provider.
Used for the worker's notify_conn / leader_conn or
:class:taskq.TaskQ's pg_conn_factory. Like
:func:make_pg_pool_factory, the credential is passed as keyword
arguments (precedence over userinfo and query params; the token
never appears in the DSN string), and password= is an async
callable that asyncpg awaits per physical connection - a fresh token
for a token credential, the issued pair's password for a
username-bearing one.
A dedicated connection is opened once and then held for the life of
the worker, so the callable normally fires exactly once - but these
are precisely the long-lived connections a credential expiry kills,
and the callable is what makes a re-open by asyncpg (a LISTEN
connection reconnecting after the server drops it) authenticate with
the current token rather than the one captured when the factory was
first invoked. reload_credentials re-invokes the factory itself,
which is where a username-bearing pair is replaced.
command_timeout is forwarded to asyncpg.connect as the default
per-operation timeout. The worker's DSN-built notify_conn /
leader_conn carry dispatcher_command_timeout; pass it here too
so a credential-provider deployment does not silently drop the bound
that keeps a wedged query from stalling leader election.
setup runs once inside the factory, immediately after
asyncpg.connect establishes the connection (e.g. registering type
codecs, setting session GUCs) - asyncpg.connect takes no setup
parameter, so the factory applies the hook itself, and a hook failure
terminates the connection rather than returning one half-configured.
For a dedicated connection this is equivalent to
init on a pool - there is no acquire/reuse cycle. The hook is also
declared on the returned factory (see
:func:taskq.connections.with_connection_init), so when this factory
provides the worker's LOOP-scope asyncpg.Connection registration
the per-slot transaction pool inherits it - the codec family that a
bare set_type_codec on one live connection silently loses above
max_concurrency = 1.
server_settings is forwarded to asyncpg.connect and applied as
session-level GUCs at connect time (e.g.
{"statement_timeout": "30s", "search_path": "app"}).
connection_class is forwarded to asyncpg.connect and sets the
:class:asyncpg.Connection subclass for this connection. Use it to
install custom codecs or override connection methods.
reload_schedule is the :class:ReloadSchedule the connection is
rebuilt on, exactly as for :func:make_pg_pool_factory: pass the
consumer's shared schedule for a connection that lives as long as the
process (the worker's notify_conn / leader_conn), so the lease
it is issued counts toward the derived cadence. Omitted, the factory
is taken to be one-shot - a migration connection the caller opens,
uses and closes - and records its lease on a schedule of its own
without the pinned-pair warning, since there is no long-lived
connection to rotate.
Source code in src/taskq/auth.py
753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 | |
make_redis_client_factory ¶
make_redis_client_factory(
url: str | None,
provider: RedisCredentialProvider,
**client_kwargs: Any,
) -> RedisFactory
Build a :data:~taskq.connections.RedisFactory backed by provider.
url is the Redis URL without credentials. The factory attaches
a redis-py CredentialProvider that delegates to provider, so
reconnects re-fetch the credential automatically. Use a rediss://
(TLS) URL - with a plain redis:// URL the bearer token is sent
unencrypted, and the factory logs a warning.
If url is None the factory raises :class:RuntimeError when
called (matches the worker's "Redis not configured" contract).
Source code in src/taskq/auth.py
build_worker_connections ¶
build_worker_connections(
settings: WorkerSettings,
*,
pg_provider: PgCredentialProvider | None = None,
redis_provider: RedisCredentialProvider | None = None,
pg_dsn: str | None = None,
pg_dsn_direct: str | None = None,
pg_dsn_pooled: str | None = None,
redis_url: str | None = None,
) -> WorkerConnections
Build the full set of provider-backed factories for one worker.
Every Postgres role the worker opens (dispatcher / heartbeat / worker
pools, the notify_conn LISTEN connection and the leader_conn
advisory-lock connection) plus the Redis client, sized and timed out
exactly as :func:taskq.worker.deps.open_worker_deps sizes its
DSN-built equivalents - so switching a deployment to a credential
provider changes how it authenticates, never its connection budget
or its timeouts.
This is what makes the credential path reachable from the taskq
worker console script (--pg-credential-provider /
TASKQ_PG_CREDENTIAL_PROVIDER): every role is factory-backed, so
SIGHUP / TASKQ_RELOAD_INTERVAL rebuild all of them through the
provider. A role left on the DSN fallback would be silently
un-rotatable - reload_credentials skips roles with no factory - so
this builder deliberately covers all of them or raises.
Explicit endpoints¶
pg_dsn / pg_dsn_direct / pg_dsn_pooled / redis_url override where the factories point, while every pool size and timeout still comes from settings. Pass pg_dsn to send all five Postgres roles at one server; pass the _direct / _pooled pair to keep a pgbouncer split. They are mutually exclusive - a call that sets both is ambiguous about which wins.
Why this exists: an application that already knows where TaskQ's tables
live otherwise had to restate that in TASKQ_PG_DSN purely to reach
this builder, duplicating one fact across two config systems (the class
of bug where the two copies disagree about the schema). The alternative
it reached for instead - one hand-built make_pg_pool_factory passed
to all three pool roles - silently discards the per-role budget this
function exists to apply: TaskQ resolves each role separately, so every
role gets a full pool_max-sized pool rather than
dispatcher_pool_size / heartbeat_pool_size / worker_pool_size.
Overriding the endpoint keeps the budget.
Raises ValueError when no provider is given, when pg_dsn is
combined with pg_dsn_direct / pg_dsn_pooled, or when
redis_provider is set with no Redis URL available from either
redis_url or settings: a Redis provider that quietly did nothing
is the failure mode this wiring exists to remove.
Source code in src/taskq/auth.py
951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 | |