AshHooks.Delivery (AshHooks v2.0.3)

Drives one durable outbound delivery through signing, HTTP, and result persistence. It uses your resource modules and configured HTTP adapter. AshHooks.Worker adds an Oban trigger; this driver also works without Oban.

The delivery row owns retry policy: the driver refuses to re-send :succeeded/:dead_letter rows, waits on next_attempt_at, counts attempts against the ceiling, and honors Retry-After (bounded). Oban is the durable trigger — {:snooze, s} re-drives later without exhausting the job (snooze extends max_attempts; only this module's ceiling decides dead-letter).

Every attempt runs in an owned monitored process under one finite deadline. The budget includes endpoint reads, secret resolution, signing, destination validation, DNS/connect/HTTP work, response handling, and the result write. The durable send lease is derived from that deadline plus a bounded finalization allowance, and every result write is fenced by the row's full primary key, source, attempt token, sending state, and live lease.

Lease construction and comparisons use the configured application clock (:now) or DateTime.utc_now/0. Deploy executors with synchronized clocks; clock skew can shorten or extend admission windows, so this module does not claim a database-clock distributed timing guarantee.

Response classification:

2xx → :succeeded (status + allowlisted content-type summary — no body bytes by default; a per-call snippet_capture: true config persists a bounded, redacted body marked [captured])
408/429/5xx → retryable, Retry-After when present (integer seconds or HTTP-date; clamped [1, retry_after_cap_seconds]), else backoff
410 → :disable_pending, then conditional endpoint disable and :dead_letter
other 4xx → :dead_letter (client errors do not self-heal)
3xx → :dead_letter (redirect_refused — never followed)
transport error / secret-resolution failure → retryable backoff
send-time SSRF refusal / disabled-or-gone endpoint → :dead_letter

Backoff: min(base · 2^min(attempts, 16), max_backoff) seconds plus :rand.uniform(delay) jitter, re-clamped — always ≥ 1 second.

The 410 auto-disable is a durable two-step system write. A live attempt first stores :disable_pending with the endpoint configuration snapshot; recovery then conditionally disables that same configuration and finalizes the row. A changed endpoint is never disabled by an older 410. One 410 disables the matching endpoint as a system action (authorize?: false). The write has no application actor and is unattributed in a versioned register. Without application notifications, delivery stops silently from the tenant's perspective. Attach [:ash_hooks, :delivery, :disable] to your operator or notification surface; it carries the tenant and endpoint IDs. The endpoint's status or mapped switch also exposes the durable disabled state.

Summary

Functions

prune(deliv_mod, opts)

Retention hook: deletes TERMINAL delivery rows (:succeeded, :dead_letter) older than older_than, by the resource's inserted_at (add Ash timestamps() to the resource and its migration). Non-terminal rows are never deleted. Returns {:ok, deleted_count}, or {:error, error} when the resource lacks inserted_at — the same error contract as AshHooks.Ingress.prune/2.

redact(body)

The package snippet floor (ADR-0005, amended 2026-08-22) — what opt-in-captured bodies pass through before persistence: NFKC normalization (fullwidth homoglyph markers fold), a bounded-fixpoint decode chain (percent ×2 + JSON \u per-escape, re-run until stable — a \u0025 escape can materialize % only after the percent layers), the separator-tolerant marker patterns, the ≥16-char union-alphabet entropy rule, a control-byte strip, and the 2048 cap. Un-redaction is impossible by construction; invalid UTF-8 collapses to [binary].

run(args, config)

Drives one delivery from the worker's JSON-safe args. Current jobs carry delivery_pk, delivery_resource, endpoint_resource, endpoint_id, event_uuid, dispatch_source, dispatch_route, and an optional tenant. String keys are Oban's JSON round-trip shape; atom keys are tolerated. Legacy jobs carrying only endpoint_id and event_uuid remain readable. The tenant arg is the enqueue seam's serialized row tenant (the worker reads it off the multitenancy attribute at enqueue time); run/2 re-parses it through the resource's tenancy pipeline by passing it as the tenant on every data call — a worker restart or replica handoff recovers full tenant context from args alone.

summarize(status, headers)

The DEFAULT response-snippet summary (ADR-0005's 2026-08-22 amendment): a fixed grammar over the status and one ALLOWLISTED content-type token — never body bytes, never a body-derived digest (a hash is correlation material that explains nothing).

Functions

prune(deliv_mod, opts)

@spec prune(module(), keyword()) :: {:ok, non_neg_integer()} | {:error, term()}

:tenant scopes the sweep (required on multitenant resources — a tenant-less global sweep over them is the named {:error, :tenant_required} before any data access). :batch_size defaults to 500 and accepts integers from 1 through 5,000; invalid values return {:error, :invalid_batch_size}.

redact(body)

@spec redact(term()) :: String.t() | nil

run(args, config)

@spec run(map(), keyword()) :: :ok | {:snooze, pos_integer()} | {:error, term()}

Returns :ok (terminal or attempted-to-terminal), {:snooze, seconds} (retry later), or {:error, term} for a broken trigger (row missing → :ok; the durable row is the record — a missing row is a completed or reaped delivery, not a failure). A tenant-less trigger against multitenant resources is {:error, :tenant_required} before any data access. A trigger whose resource, source, route, or complete primary key does not match the fetched row fails explicitly.

Required config keys are :deliveries, :endpoints, and :secret_resolver. :attempt_timeout defaults to 25,000 ms and :finalization_allowance to 5,000 ms. Retry defaults are 10 attempts, 2-second base backoff, 3,600-second maximum backoff, and an 86,400-second Retry-After cap. Omitted or nil retry options use these defaults. :http, :http_opts, :tenant_aware_secrets, :snippet_capture, :snippet_redactor, and :now customize the documented seams.

Set :max_attempts here to configure the delivery row's ceiling. The worker macro calls this option :delivery_max_attempts; its own :max_attempts configures Oban's job attempts. Other retry option names are the same.

summarize(status, headers)

@spec summarize(integer() | nil, keyword() | list() | nil) :: String.t()

"200 json token=application/json"   # the type was allowlisted
"200 text token=other"              # anything else collapses to other

The status is an integer, the kind comes from a fixed vocabulary (json | html | text | xml | binary | other), and the token is either an exact allowlist member or the literal other — a hostile Content-Type header cannot smuggle material into the ledger through this string.