AshHooks guided tour

Copy Markdown View Source

You will learn

  • Verify webhooks from any sender by configuration: an HMAC header, a timestamped structured header with a replay window, an Ed25519 signature, a signed JWS body, and a sender that canonicalizes form fields
  • Ingest through the ledger: deduplicate a replay, reject a tamper, refuse a stale timestamp, and rotate a secret without dropping deliveries
  • Classify events with your own table and run an idempotent handler
  • Verify directly, without a ledger, and see what an invalid configuration reports
  • Send an outbound webhook to an existing WebHook Tester inbox and verify its captured bytes
  • Scope a multi-tenant deployment: one tenant's dispatch never reaches another's rows
  • Fit the extensions to a domain that reserves payload, owns an active switch, and uses atom-valued subscription types
  • Observe verification and delivery through telemetry
  • Prune old rows with the retention hooks
  • Implement and check a host custodian with real cryptography and permanent tombstones
  • Issue, roll, reissue, deliver with, and retire endpoint keys through a key registry
  • Create inbound source records, resolve and verify their keys, and pause new requests
  • Apply request-scoped JSON policy and pause subscription fanout

This notebook uses a real SQLite database and a WebHook Tester receiver, an open-source request-capture server. The notebook checks captured request bytes, headers, and signatures. Before evaluating cells, provision a receiver from a Git checkout with docker compose -f ops/local/webhook-tester.compose.yaml up -d, or reuse an existing receiver. Probe http://127.0.0.1:52871/api/version; set LOCAL_WEBHOOK_TESTER_PORT for both Compose and the notebook to choose another free loopback port. See the local receiver guide. Do not restart another user's receiver or delete capture data. The notebook starts no server. PostgreSQL row security requires PostgreSQL; this SQLite notebook cannot exercise its policies. Use the row-security guide for repository binding, table policies, and per-tenant jobs. Uses ash_hooks 3.3.0. The tour plays both sides of each webhook: it signs requests the way a sender would and verifies them the way your application does. Real senders, a production queue, and your HTTP endpoint come from your own application.

Setup

Mix.install(
  [
    {:ash, ">= 3.37.2 and < 4.0.0-0"},
    {:ash_hooks, "== 3.3.0"},
    {:ash_sqlite, "~> 0.3.1"},
    {:jason, "~> 1.2"}
  ],
  consolidate_protocols: false
)
# Set Ash's string length counting mode before compiling any resource.
# This notebook is its own application.
# :codepoints is Ash's recommended value: max_length bounds value size
# and matches how SQL data layers count.
Application.put_env(:ash, :default_string_length_count, :codepoints)
"3.3.0" = Application.spec(:ash_hooks, :vsn) |> to_string()

# Keys for the tour's senders. In your application these come from your
# secret store through the inbound `secret` sources, never from literals.
#
# Shared secrets for the HMAC senders:
Application.put_env(:ash_hooks, :tour_orders_secret, :crypto.strong_rand_bytes(32))
Application.put_env(:ash_hooks, :tour_orders_previous_secret, :crypto.strong_rand_bytes(32))
Application.put_env(:ash_hooks, :tour_payments_secret, :crypto.strong_rand_bytes(32))
Application.put_env(:ash_hooks, :tour_sms_secret, :crypto.strong_rand_bytes(20))

# An Ed25519 key pair: the sender keeps the private key, you configure the
# public key (here as base64 of its 32 raw bytes).
{partner_public, partner_private} = :crypto.generate_key(:eddsa, :ed25519)
Application.put_env(:ash_hooks, :tour_partner_public_key, Base.encode64(partner_public))

# An RSA key pair for a sender that signs a JWS body. You configure its
# public key as PEM.
envelope_private = :public_key.generate_key({:rsa, 2048, 65_537})
{:RSAPrivateKey, _, modulus, exponent, _, _, _, _, _, _, _} = envelope_private

envelope_public_pem =
  :public_key.pem_encode([
    :public_key.pem_entry_encode(:SubjectPublicKeyInfo, {:RSAPublicKey, modulus, exponent})
  ])

Application.put_env(:ash_hooks, :tour_envelope_public_key, envelope_public_pem)
IO.puts(envelope_public_pem)
defmodule Tour.Repo do
  use AshSqlite.Repo, otp_app: :ash_hooks

  @impl true
  def write_transactions?, do: true

  # SQLite 3.38 supplies JSON operators and includes RETURNING for atomic writes.
  @impl true
  def min_sqlite_version, do: %Version{major: 3, minor: 38, patch: 0}
end

Describe your senders

Each sender gets one provider module. use AshHooks.Provider, verify: [...] describes how the sender signs its requests; ash_hooks checks the options when the module compiles and generates the verification callbacks. The options come from the sender's documentation: the header, the algorithm, how the signature and secret are encoded, what is signed, and where the timestamp is. The Verify any sender guide has recipes for common shapes, and AshHooks.Provider.Scheme documents every option.

defmodule Tour.VerifiedEvent do
  defstruct [:type, :payload]
end

# 1. An HMAC-SHA256 of the raw body, lowercase hex, in one header.
#    This provider also classifies events and runs a handler.
defmodule Tour.Orders do
  use AshHooks.Provider,
    verify: [
      algorithm: :hmac_sha256,
      signature: {:header, "x-tour-signature"},
      encoding: :hex
    ]

  # Map the sender's type strings to your event atoms with an explicit
  # table, never String.to_atom on request data (that would let a hostile
  # sender grow your atom table). Unknown types fail closed.
  @types %{"order.paid" => :order_paid}

  @impl true
  def parse_event_type(%{"type" => type}) do
    case Map.fetch(@types, type) do
      {:ok, atom} -> {:ok, atom}
      :error -> {:error, :unknown_event_type}
    end
  end

  def parse_event_type(_payload), do: {:error, :malformed_payload}

  # Your application's action. It must be safe to run twice for the same
  # event: the ledger deduplicates deliveries, but a crash between this
  # call and the ledger update re-runs it on redelivery.
  @impl true
  def handle_event(type, payload) do
    Agent.update(Tour.Handled, fn ids -> Enum.uniq([payload["id"] | ids]) end)
    {:ok, %Tour.VerifiedEvent{type: type, payload: payload}}
  end
end

# 2. A structured header carrying a timestamp and one or more signatures:
#    Each v1 field holds a hexadecimal signature.
#    The signed input is "timestamp.body"; any listed v1 that verifies is
#    accepted, so the sender can rotate keys.
defmodule Tour.Payments do
  use AshHooks.Provider,
    verify: [
      algorithm: :hmac_sha256,
      signature: {:structured_header, "x-tour-payments", signature_key: "v1"},
      timestamp: {:signature_field, "t"},
      signed_input: [:timestamp, ".", :body],
      tolerance_seconds: 300
    ]
end

# 3. A public-key signature: Ed25519 over the raw body, base64 in a header.
#    The secret source returns the sender's public key.
defmodule Tour.Partner do
  use AshHooks.Provider,
    verify: [
      algorithm: :ed25519,
      signature: {:header, "x-partner-signature"},
      encoding: :base64,
      key_encoding: :base64
    ]
end

# 4. The whole body is a compact JWS (an RS256 JWT). The algorithm is pinned
#    here and never read from the token; the verified payload is what the
#    ledger stores.
defmodule Tour.Envelope do
  use AshHooks.Provider,
    verify: [
      algorithm: :rsa_pkcs1_sha256,
      signature: :jws_body
    ]
end

# 5. A sender that canonicalizes before signing: HMAC-SHA1 over the public
#    URL followed by every form field's name and value, sorted by name. A host
#    function builds that string; returning anything but a binary fails.
defmodule Tour.Sms do
  use AshHooks.Provider,
    verify: [
      algorithm: :hmac_sha1,
      signature: {:header, "x-sms-signature"},
      encoding: :base64,
      signed_input: {__MODULE__, :signed_input}
    ]

  def signed_input(raw_body, %{request_uri: url}) when is_binary(url) do
    fields =
      raw_body
      |> URI.decode_query()
      |> Enum.sort()
      |> Enum.map(fn {name, value} -> name <> value end)

    url <> Enum.join(fields)
  end

  def signed_input(_raw_body, _ctx), do: nil
end

JSON policy and durable provider context

A ledger's json_decoder MFA receives bytes and returns a decoded map or list. This host policy refuses a field the application does not accept. Ingress uses it for JWS protected headers, claims, and delivered payloads. A host-defined decode_payload/1 also implements /2 using the supplied decoder; generated JWS decoders honor it directly. The callback conformance helper belongs in consumer tests.

The optional provider callback arities receive AshHooks.Provider.DeliveryContext. Handling gets persisted IDs; parsing and identity get tenant, source, cast scope, and alias. Recovery rebuilds these from the receipt, without request headers.

defmodule Tour.JsonPolicy do
  def decode(bytes) do
    case Jason.decode(bytes) do
      {:ok, %{"forbidden" => _}} -> {:error, :disallowed_field}
      result -> result
    end
  end
end

defmodule Tour.ManagedOrders do
  use AshHooks.Provider,
    verify: [algorithm: :hmac_sha256, signature: {:header, "x-managed-signature"}]

  @impl true
  def decode_payload(bytes), do: Jason.decode(bytes)

  @impl true
  def decode_payload(bytes, json_decoder), do: json_decoder.(bytes)

  @impl true
  def parse_event_type(_payload, %AshHooks.Provider.DeliveryContext{source: :managed_orders}),
    do: {:ok, :order_paid}

  @impl true
  def event_identity(%{"id" => id}, %AshHooks.Provider.DeliveryContext{}), do: {:ok, id}

  def event_identity(_payload, %AshHooks.Provider.DeliveryContext{}),
    do: {:error, :malformed_payload}

  @impl true
  def handle_event(type, payload, %AshHooks.Provider.DeliveryContext{} = context) do
    true = context.provider_alias == :orders
    true = context.scope.account_id == "acct-42"
    true = not is_nil(context.delivery_id)
    true = context.external_event_id == payload["id"]
    {:ok, %AshHooks.Provider.Event{type: type, payload: payload}}
  end
end
# Educational SQLite custodian owned by this notebook's host application.
# It uses real cryptography and durable SQL tombstones, not test support.
# It stores material in a disposable local database. A deployed host must
# secure and encrypt its key store and govern access to its callbacks.
defmodule Tour.Custodian do
  @behaviour AshHooks.Custody

  def create_store! do
    sql!("""
    CREATE TABLE tour_custody_keys (
      ref TEXT PRIMARY KEY, tenant TEXT, kind TEXT, material TEXT,
      destroyed INTEGER NOT NULL CHECK (destroyed IN (0, 1)))
    """)
  end

  @impl true
  def store(ref, material, tenant, kind) do
    transaction(fn ->
      if valid?(material, kind) do
        row = bind(ref, tenant, label(kind), Base.encode64(material), 0)

        case row do
          [_, _, _, 1] ->
            {:error, :ref_destroyed}

          [^tenant, stored_kind, stored, 0] ->
            if stored_kind == label(kind) and stored == Base.encode64(material),
              do: :ok,
              else: {:error, :ref_exists}

          _ ->
            {:error, :ref_exists}
        end
      else
        case sql!("SELECT destroyed FROM tour_custody_keys WHERE ref = ?", [ref]).rows do
          [[1]] -> {:error, :ref_destroyed}
          [[0]] -> {:error, :ref_exists}
          _ -> {:error, :invalid_material}
        end
      end
    end)
  end

  @impl true
  def destroy(ref, tenant) do
    transaction(fn ->
      case bind(ref, tenant, nil, nil, 1) do
        [^tenant, _, _, _] ->
          sql!(
            "UPDATE tour_custody_keys SET destroyed = 1, material = NULL, kind = NULL WHERE ref = ?",
            [ref]
          )

          :ok

        _ ->
          {:error, :tenant_mismatch}
      end
    end)
  end

  @impl true
  def sign(ref, tenant, kind, input) do
    with {:ok, material} <- fetch(ref, tenant, kind) do
      case kind do
        {:sender_hmac, _} ->
          {:error, :not_signable}

        {:standard_webhooks, :v1a} ->
          {:ok, :crypto.sign(:eddsa, :none, input, [material, :ed25519])}

        {_, digest} ->
          {:ok, :crypto.mac(:hmac, digest(digest), material, input)}
      end
    end
  end

  @impl true
  def verify(ref, tenant, kind, input, signature) do
    with {:ok, material} <- fetch(ref, tenant, kind) do
      valid =
        case kind do
          {:standard_webhooks, :v1a} ->
            {public, _} = :crypto.generate_key(:eddsa, :ed25519, material)

            byte_size(signature) == 64 and
              :crypto.verify(:eddsa, :none, input, signature, [public, :ed25519])

          {_, digest} ->
            expected = :crypto.mac(:hmac, digest(digest), material, input)

            byte_size(signature) == byte_size(expected) and
              :crypto.hash_equals(expected, signature)
        end

      if valid, do: :ok, else: {:error, :invalid_signature}
    end
  end

  @impl true
  def public_key(ref, tenant, kind) do
    with {:ok, material} <- fetch(ref, tenant, kind) do
      if kind == {:standard_webhooks, :v1a} do
        {public, _} = :crypto.generate_key(:eddsa, :ed25519, material)
        {:ok, public}
      else
        {:error, :no_public_half}
      end
    end
  end

  @impl true
  def fingerprint(ref, tenant, kind) do
    with {:ok, material} <- fetch(ref, tenant, kind) do
      case kind do
        {:standard_webhooks, version} ->
          bytes =
            if version == :v1a do
              {public, _} = :crypto.generate_key(:eddsa, :ed25519, material)
              public
            else
              material
            end

          digest =
            :crypto.hash(
              :sha256,
              "ash_hooks key fingerprint v1\0standard_webhooks.#{version}\0" <> bytes
            )

          {:ok, binary_part(digest, 0, 16)}

        _ ->
          {:error, :no_fingerprint}
      end
    end
  end

  # The upsert locks the global reference through comparison and destruction.
  # SQLite serializes concurrent writers on this real store.
  defp bind(ref, tenant, kind, material, destroyed) do
    sql!(
      """
      INSERT INTO tour_custody_keys (ref, tenant, kind, material, destroyed) VALUES (?, ?, ?, ?, ?)
      ON CONFLICT (ref) DO UPDATE SET ref = excluded.ref
      RETURNING tenant, kind, material, destroyed
      """,
      [ref, tenant, kind, material, destroyed]
    ).rows
    |> hd()
  end

  defp fetch(ref, tenant, kind) do
    case sql!(
           "SELECT tenant, kind, material FROM tour_custody_keys WHERE ref = ? AND destroyed = 0",
           [ref]
         ).rows do
      [[^tenant, stored_kind, material]] ->
        if stored_kind == label(kind),
          do: {:ok, Base.decode64!(material)},
          else: {:error, :kind_mismatch}

      _ ->
        {:error, :unknown_ref}
    end
  end

  defp valid?(material, {:standard_webhooks, :v1}),
    do: is_binary(material) and byte_size(material) in 24..64

  defp valid?(material, {:standard_webhooks, :v1a}),
    do: is_binary(material) and byte_size(material) == 32

  defp valid?(material, {:sender_hmac, digest}) when digest in [:sha1, :sha256, :sha384, :sha512],
    do: is_binary(material) and byte_size(material) in 1..16_384

  defp valid?(material, {:legacy_hmac, :sha256}),
    do: is_binary(material) and byte_size(material) > 0

  defp valid?(_, _), do: false
  defp label({family, version}), do: "#{family}:#{version}"
  defp digest(:v1), do: :sha256
  defp digest(:sha1), do: :sha
  defp digest(value), do: value
  defp sql!(query, params \\ []), do: Tour.Repo.query!(query, params, log: false)

  defp transaction(fun) do
    {:ok, result} = Tour.Repo.transaction(fun, log: false)
    result
  end
end

defmodule Tour.KeyRegistry do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.KeyRegistry]

  sqlite do
    table("tour_key_registry")
    repo(Tour.Repo)
  end

  key_registry do
    issue_timeout(900)
  end
end

defmodule Tour.CustodyEndpoint do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.Endpoint]

  sqlite do
    table("tour_custody_endpoints")
    repo(Tour.Repo)
  end

  endpoint do
    custodian(Tour.Custodian)
    key_registry(Tour.KeyRegistry)
    key_lifecycle(:on)
    legacy_key_lifecycle(:on)
    default_overlap_seconds(60)
    max_overlap_seconds(3600)
  end

  actions do
    defaults([:read, create: :*])
  end
end

defmodule Tour.Source do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.InboundSource]

  sqlite do
    table("tour_sources")
    repo(Tour.Repo)
  end

  inbound_source do
    ledger(Tour.Ledger)
    source_attribute(:source)
    scope([{:account_id, :string, []}])
    providers(orders: Tour.ManagedOrders, partner: Tour.Partner)
    custodian(Tour.Custodian)
    key_registry(Tour.KeyRegistry)
    default_overlap_seconds(60)
    max_overlap_seconds(3600)
  end

  actions do
    defaults([:read, create: :*])
  end
end

defmodule Tour.Ledger do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks, AshHooks.InboundDelivery]

  sqlite do
    table("tour_ledgers")
    repo(Tour.Repo)
  end

  inbound_delivery do
    scope_identity([:account_id])
    record_provider_alias(:on)
    json_decoder({Tour.JsonPolicy, :decode, []})
  end

  attributes do
    attribute(:account_id, :string, allow_nil?: false)
    timestamps()
  end

  actions do
    defaults([:read])
  end

  # This declaration names a provider directly; `source_record` can instead
  # select one from an allowlist. The name identifies the source in the ledger.
  # Secrets are sources, never literals.
  webhooks do
    inbound :managed_orders do
      source_record(Tour.Source)
    end

    inbound :orders do
      provider(Tour.Orders)
      secret fn -> {:ok, Application.fetch_env!(:ash_hooks, :tour_orders_secret)} end
      # the outgoing key during a rotation: both keys verify until you remove it
      previous_secret fn ->
        {:ok, Application.fetch_env!(:ash_hooks, :tour_orders_previous_secret)}
      end

      # the sender's event id is the deduplication identity
      event_id(&__MODULE__.event_id/1)
    end

    inbound :payments do
      provider(Tour.Payments)
      secret fn -> {:ok, Application.fetch_env!(:ash_hooks, :tour_payments_secret)} end
      # overrides the provider's tolerance_seconds for this source
      replay_window_seconds(120)
    end

    inbound :partner do
      provider(Tour.Partner)
      secret fn -> {:ok, Application.fetch_env!(:ash_hooks, :tour_partner_public_key)} end
    end

    inbound :envelope do
      provider(Tour.Envelope)
      secret fn -> {:ok, Application.fetch_env!(:ash_hooks, :tour_envelope_public_key)} end
    end

    inbound :sms do
      provider(Tour.Sms)
      body_format(:form)
      secret fn -> {:ok, Application.fetch_env!(:ash_hooks, :tour_sms_secret)} end
    end
  end

  def event_id(%{"id" => id}) when is_binary(id), do: {:ok, id}
  def event_id(_payload), do: :error
end

defmodule Tour.Endpoint do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.Endpoint]

  sqlite do
    table("tour_endpoints")
    repo(Tour.Repo)
  end

  actions do
    defaults([:read, :create, :update])
    default_accept(:*)
  end

  endpoint do
    # per-endpoint response capture: the four columns exist only when
    # declared (a migration is required to adopt them)
    response_capture_attributes(:on)
  end
end

defmodule Tour.Subscription do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.Subscription]

  sqlite do
    table("tour_subscriptions")
    repo(Tour.Repo)
  end

  actions do
    defaults([:read, :create, :update])
    default_accept(:*)
  end

  subscription do
    endpoint_resource(Tour.Endpoint)
    pausable(:on)
  end
end

defmodule Tour.Delivery do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.OutboundDelivery]

  sqlite do
    table("tour_deliveries")
    repo(Tour.Repo)
  end

  attributes do
    timestamps()
  end

  actions do
    defaults([:read])
  end

  outbound_delivery do
    # resend/replay bookkeeping (migrate the four columns when you adopt)
    resend_tracking(:on)
  end
end

defmodule Tour.Attempt do
  # The attempt history: one durable row per delivery attempt (time,
  # outcome, classified reason, HTTP status, latency, the signing key
  # REFERENCE). Written by the delivery runtime when the worker config
  # carries `attempts:`, providing the audit trail behind delivery logs and health.
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.OutboundAttempt]

  sqlite do
    table("tour_attempts")
    repo(Tour.Repo)
  end

  actions do
    defaults([:read])
  end
end

defmodule Tour.Order do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks]

  sqlite do
    table("tour_orders")
    repo(Tour.Repo)
  end

  attributes do
    uuid_primary_key(:id)
  end

  actions do
    defaults([:read, :create])
  end

  webhooks do
    outbound :order_paid do
      subscriptions(Tour.Subscription)
      deliveries(Tour.Delivery)
    end
  end
end

defmodule Tour.Domain do
  use Ash.Domain, otp_app: nil, validate_config_inclusion?: false

  resources do
    resource(Tour.KeyRegistry)
    resource(Tour.CustodyEndpoint)
    resource(Tour.Source)
    resource(Tour.Ledger)
    resource(Tour.Endpoint)
    resource(Tour.Subscription)
    resource(Tour.Delivery)
    resource(Tour.Attempt)
    resource(Tour.Order)
    resource(Tour.TenantEndpoint)
    resource(Tour.TenantSubscription)
    resource(Tour.TenantDelivery)
    resource(Tour.TenantOrder)
  end
end
Application.put_env(:ash_hooks, Tour.Repo,
  database: Path.join(System.tmp_dir!(), "ash_hooks_tour_#{Ash.UUID.generate()}.sqlite3"),
  pool_size: 1,
  log: false
)

database_path = Application.fetch_env!(:ash_hooks, Tour.Repo)[:database]

System.at_exit(fn _status ->
  try do
    if pid = Process.whereis(Tour.Repo), do: Supervisor.stop(pid)
  catch
    :exit, _reason -> :ok
  after
    for suffix <- ["", "-shm", "-wal"], do: File.rm(database_path <> suffix)
  end
end)

{:ok, _} = Tour.Repo.start_link()
%{rows: [[sqlite_version]]} = Tour.Repo.query!("SELECT sqlite_version()")
IO.puts("Installed SQLite: #{sqlite_version}; required feature floor: 3.38.0")

The tables

Tour.Repo.query!("""
CREATE TABLE tour_ledgers (
  id TEXT PRIMARY KEY, provider TEXT NOT NULL, provider_alias TEXT,
  external_event_id TEXT NOT NULL,
  external_event_type TEXT, payload TEXT NOT NULL, payload_digest TEXT NOT NULL,
  status TEXT NOT NULL, fencing_token INTEGER NOT NULL DEFAULT 0,
  lease_expires_at TEXT, error_class TEXT, attempts INTEGER NOT NULL DEFAULT 0,
  account_id TEXT NOT NULL, inserted_at TEXT, updated_at TEXT)
""")

Tour.Repo.query!(
  "CREATE UNIQUE INDEX tour_ledgers_unique ON tour_ledgers (provider, external_event_id, account_id)"
)

Tour.Repo.query!("""
CREATE TABLE tour_endpoints (
  id TEXT PRIMARY KEY, url TEXT NOT NULL, status TEXT NOT NULL DEFAULT 'enabled',
  secret_ref TEXT NOT NULL, previous_secret_ref TEXT, legacy_secret_ref TEXT,
  legacy_previous_secret_ref TEXT,
  response_capture INTEGER NOT NULL DEFAULT 0,
  response_capture_max_bytes INTEGER NOT NULL DEFAULT 2048,
  response_capture_retention_seconds INTEGER NOT NULL DEFAULT 2592000,
  response_header_allowlist TEXT)
""")

Tour.Repo.query!("""
CREATE TABLE tour_subscriptions (
  id TEXT PRIMARY KEY, event_types TEXT NOT NULL, endpoint_id TEXT NOT NULL, signing_mode TEXT,
  paused INTEGER NOT NULL DEFAULT 0)
""")

Tour.Repo.query!("""
CREATE TABLE tour_deliveries (
  id TEXT PRIMARY KEY, event_uuid TEXT NOT NULL, event_type TEXT NOT NULL,
  payload BLOB NOT NULL, endpoint_id TEXT NOT NULL, subscription_id TEXT,
  signing_mode TEXT, status TEXT NOT NULL DEFAULT 'pending',
  attempts INTEGER NOT NULL DEFAULT 0, response_status INTEGER,
  response_snippet TEXT, last_error TEXT, next_attempt_at TEXT,
  dispatch_source TEXT NOT NULL DEFAULT 'v1:direct:unbound',
  dispatch_route TEXT NOT NULL DEFAULT 'v1:route:unbound',
  attempt_token TEXT, send_lease_expires_at TEXT,
  enqueue_token TEXT, enqueue_lease_expires_at TEXT,
  endpoint_snapshot JSONB,
  resend_baseline INTEGER NOT NULL DEFAULT 0,
  resend_count INTEGER NOT NULL DEFAULT 0,
  last_resend_by TEXT, last_resend_at TEXT,
  inserted_at TEXT, updated_at TEXT)
""")

Tour.Repo.query!(
  "CREATE UNIQUE INDEX tour_deliveries_unique ON tour_deliveries (endpoint_id, event_uuid)"
)

Tour.Repo.query!("""
CREATE TABLE tour_attempts (
  id TEXT PRIMARY KEY, delivery_pk TEXT NOT NULL, endpoint_id TEXT NOT NULL,
  event_uuid TEXT NOT NULL, event_type TEXT NOT NULL, attempt INTEGER NOT NULL,
  outcome TEXT NOT NULL, reason TEXT, response_status INTEGER,
  response_snippet TEXT, response_headers TEXT, latency_ms INTEGER,
  signing_key_ref TEXT, test INTEGER NOT NULL DEFAULT 0,
  attempted_at TEXT NOT NULL)
""")

Tour.Repo.query!(
  "CREATE UNIQUE INDEX tour_attempts_unique ON tour_attempts (delivery_pk, attempt)"
)

Tour.Repo.query!("CREATE TABLE IF NOT EXISTS tour_orders (id TEXT PRIMARY KEY)")

:ok

Custody: a host-owned key store

The custodian above implements all six AshHooks.Custody callbacks over a real SQLite table. Kinds are checked before computing, tenants bind globally, and store/destroy ordering retains permanent tombstones. The conformance template checks the contract with real HMAC and Ed25519 signatures. Each test creates and drops a fresh store. The tour then creates its own store, whose tombstones live until this disposable database is removed.

ExUnit.start(autorun: false)

defmodule Tour.CustodyContract do
  use AshHooks.Custody.Conformance, custodian: Tour.Custodian

  setup do
    Tour.Custodian.create_store!()
    on_exit(fn -> Tour.Repo.query!("DROP TABLE tour_custody_keys") end)
    :ok
  end
end

defmodule Tour.ProviderContract do
  use AshHooks.Provider.Conformance, provider: Tour.ManagedOrders
end

%{failures: 0} = ExUnit.run()
Tour.Custodian.create_store!()

Tour.Repo.query!("""
CREATE TABLE tour_key_registry (
  ref TEXT PRIMARY KEY, kind BLOB NOT NULL, owner_resource TEXT NOT NULL,
  owner_key TEXT, state TEXT NOT NULL DEFAULT 'issued', custody_tenant TEXT,
  issued_at TEXT NOT NULL, destroy_after TEXT, attempts INTEGER NOT NULL DEFAULT 0,
  next_attempt_at TEXT, destroy_generation INTEGER NOT NULL DEFAULT 0,
  last_error TEXT, inserted_at TEXT NOT NULL, updated_at TEXT NOT NULL)
""")

Tour.Repo.query!("""
CREATE TABLE tour_custody_endpoints (
  id TEXT PRIMARY KEY, url TEXT NOT NULL, status TEXT NOT NULL DEFAULT 'enabled',
  secret_ref TEXT NOT NULL, previous_secret_ref TEXT, legacy_secret_ref TEXT,
  legacy_previous_secret_ref TEXT, secret_kind BLOB, previous_secret_kind BLOB,
  legacy_secret_kind BLOB, legacy_previous_secret_kind BLOB,
  secret_created_at TEXT, secret_first_used_at TEXT, secret_retired_at TEXT,
  previous_secret_created_at TEXT, previous_secret_first_used_at TEXT,
  previous_secret_retired_at TEXT,
  legacy_secret_created_at TEXT, legacy_secret_first_used_at TEXT,
  legacy_previous_secret_created_at TEXT, legacy_previous_secret_first_used_at TEXT,
  legacy_previous_secret_retired_at TEXT)
""")

Tour.Repo.query!("""
CREATE TABLE tour_sources (
  id TEXT PRIMARY KEY, source TEXT NOT NULL, account_id TEXT NOT NULL, provider TEXT NOT NULL,
  secret_ref TEXT, previous_secret_ref TEXT, secret_kind BLOB, previous_secret_kind BLOB,
  public_key BLOB, public_key_kind BLOB, paused INTEGER NOT NULL DEFAULT 0,
  secret_created_at TEXT, secret_first_used_at TEXT, previous_secret_created_at TEXT,
  previous_secret_first_used_at TEXT, previous_secret_retired_at TEXT)
""")

Tour.Repo.query!("CREATE UNIQUE INDEX tour_sources_unique ON tour_sources (source, account_id)")
:ok

The registry is an Ash resource on the owner's write repository with matching tenancy. It injects issue and transition actions; hosts authorize operators and schedule sweeps. issue_timeout must exceed the custodian's store timeout. The key guide explains outage retries, destroy_failed telemetry, pending/2, rearm/3, and pruning.

Observe verification and delivery through telemetry

# exact event names: :telemetry.execute/3 matches exact names only
:telemetry.attach_many(
  "tour",
  [
    [:ash_hooks, :ingress, :verify],
    [:ash_hooks, :ingress, :dedup],
    [:ash_hooks, :ingress, :claim],
    [:ash_hooks, :dispatch, :enqueue_failed],
    [:ash_hooks, :delivery, :attempt],
    [:ash_hooks, :delivery, :result],
    [:ash_hooks, :delivery, :backoff],
    [:ash_hooks, :delivery, :dead_letter],
    [:ash_hooks, :delivery, :disable]
  ],
  fn event, _measurements, metadata, _ ->
    IO.puts("EVENT #{inspect(event)} #{inspect(metadata)}")
  end,
  nil
)

:ok

Inbound: verify, deduplicate, reject a tamper

In an HTTP application, forward the webhook route to AshHooks.Plug with the ledger resource and either a fixed source: or an endpoint resolve: MFA. Configure AshHooks.BodyReader on preceding Plug.Parsers; the plug uses its exact cached bytes, or reads the body itself when no parser ran. See the get-started tutorial for Phoenix wiring and tenant resolution. This executable tour uses the lower-level AshHooks.Ingress.ingest/4 with the exact raw body and request headers. One call verifies the signature, stores the payload, deduplicates, claims the row under a lease, runs your handler, and records the outcome.

# A record showing that a duplicate delivery does not run the completed handler again.
{:ok, _} = Agent.start_link(fn -> [] end, name: Tour.Handled)

# The sender's side: an HMAC-SHA256 of the exact bytes, lowercase hex.
sign_orders = fn body, key ->
  :crypto.mac(:hmac, :sha256, key, body) |> Base.encode16(case: :lower)
end

orders_secret = Application.fetch_env!(:ash_hooks, :tour_orders_secret)
body = Jason.encode!(%{"id" => "evt-tour-1", "type" => "order.paid"})
scope = %{"account_id" => "acct-42"}
ctx = %{headers: %{"x-tour-signature" => sign_orders.(body, orders_secret)}, scope: scope}

# First delivery: verified, persisted, claimed, handled, marked processed.
{:ok, :created, first} = AshHooks.Ingress.ingest(Tour.Ledger, :orders, body, ctx)
IO.inspect({first.status, first.external_event_type}, label: "first delivery")
:processed = first.status

# The same delivery again: the unique index answers. It is a duplicate, and
# the handler does not run a second time.
{:ok, :duplicate, _} = AshHooks.Ingress.ingest(Tour.Ledger, :orders, body, ctx)
["evt-tour-1"] = Agent.get(Tour.Handled, & &1)
IO.puts("duplicate recognized; the handler ran once")

# One changed byte in the body: rejected before anything is stored.
{:error, %{__struct__: AshHooks.Errors.Invalid.InvalidSignature}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :orders, body <> " ", ctx)

# No signature header at all: rejected the same way, never a crash.
{:error, %{__struct__: AshHooks.Errors.Invalid.InvalidSignature}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :orders, body, %{headers: %{}, scope: scope})

IO.puts("tampered and unsigned requests rejected")

Event types your handler does not know

unknown = Jason.encode!(%{"id" => "evt-tour-2", "type" => "order.cancelled"})

unknown_ctx = %{
  headers: %{"x-tour-signature" => sign_orders.(unknown, orders_secret)},
  scope: scope
}

# A verified request with an unknown type is recorded as a permanent
# failure, so the sender is acknowledged and does not retry forever. Put
# these rows on your operator surface.
{:ok, :created, unknown_row} = AshHooks.Ingress.ingest(Tour.Ledger, :orders, unknown, unknown_ctx)
IO.inspect({unknown_row.status, unknown_row.error_class}, label: "unknown type")
{:failed_permanent, "unknown_event_type"} = {unknown_row.status, unknown_row.error_class}

Rotate a secret without dropping deliveries

# The sender still signs with the outgoing key. Because the declaration
# carries `previous_secret`, the request verifies with either key until you
# remove that option.
previous = Application.fetch_env!(:ash_hooks, :tour_orders_previous_secret)
rotated = Jason.encode!(%{"id" => "evt-tour-3", "type" => "order.paid"})
rotated_ctx = %{headers: %{"x-tour-signature" => sign_orders.(rotated, previous)}, scope: scope}

{:ok, :created, %{status: :processed}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :orders, rotated, rotated_ctx)

IO.puts("a request signed with the previous key verified during the overlap")

A timestamp and a replay window

payments_secret = Application.fetch_env!(:ash_hooks, :tour_payments_secret)

payments_header = fn body, timestamp, key ->
  mac = :crypto.mac(:hmac, :sha256, key, "#{timestamp}.#{body}") |> Base.encode16(case: :lower)
  # a second v1 from a key the receiver does not have: any match verifies
  stale_key_mac = String.duplicate("0", 64)
  "t=#{timestamp},v1=#{stale_key_mac},v1=#{mac}"
end

charge = Jason.encode!(%{"type" => "charge.succeeded", "amount" => 1200})
now = System.system_time(:second)

# Headers can also arrive as an ordered list, the shape of conn.req_headers.
# Providers then see every line in ctx.header_list, and repeated lines are
# joined with ", " in ctx.headers.
fresh_ctx = %{
  headers: [
    {"content-type", "application/json"},
    {"x-tour-payments", payments_header.(charge, now, payments_secret)}
  ],
  scope: scope
}

{:ok, :created, payment} = AshHooks.Ingress.ingest(Tour.Ledger, :payments, charge, fresh_ctx)

# Configured providers that keep the generated parse_event_type callback
# classify verified requests as :webhook; the sender's event name stays in the payload.
IO.inspect({payment.status, payment.external_event_type, payment.payload["type"]},
  label: "payment"
)

# A correctly signed request from an hour ago is outside the 120-second
# window declared on this source: a replay, refused.
old = now - 3600

old_ctx = %{
  headers: [{"x-tour-payments", payments_header.(charge, old, payments_secret)}],
  scope: scope
}

{:error, %{__struct__: AshHooks.Errors.Invalid.StaleTimestamp}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :payments, charge, old_ctx)

# A fresh timestamp never rescues a bad signature: that is still an invalid signature.
forged_ctx = %{
  headers: %{"x-tour-payments" => "t=#{now},v1=#{String.duplicate("a", 64)}"},
  scope: scope
}

{:error, %{__struct__: AshHooks.Errors.Invalid.InvalidSignature}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :payments, charge, forged_ctx)

IO.puts("stale timestamp refused; forged signature refused")

A public-key signature

shipment = Jason.encode!(%{"shipment" => "shp_1", "status" => "delivered"})
partner_signature = :crypto.sign(:eddsa, :none, shipment, [partner_private, :ed25519])

partner_ctx = %{
  headers: %{"x-partner-signature" => Base.encode64(partner_signature)},
  scope: scope
}

{:ok, :created, %{status: :processed}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :partner, shipment, partner_ctx)

# A signature from any other key fails: only the configured public key verifies.
{_other_public, other_private} = :crypto.generate_key(:eddsa, :ed25519)
imposter = :crypto.sign(:eddsa, :none, shipment, [other_private, :ed25519])
imposter_ctx = %{headers: %{"x-partner-signature" => Base.encode64(imposter)}, scope: scope}

{:error, %{__struct__: AshHooks.Errors.Invalid.InvalidSignature}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :partner, shipment, imposter_ctx)

IO.puts("Ed25519: the sender's key verifies, another key does not")

A signed JWS body

b64url = &Base.url_encode64(&1, padding: false)

jws = fn header, claims ->
  input = b64url.(Jason.encode!(header)) <> "." <> b64url.(Jason.encode!(claims))
  input <> "." <> b64url.(:public_key.sign(input, :sha256, envelope_private))
end

claims = %{
  "event" => "member.joined",
  "member" => "m_7",
  "exp" => System.system_time(:second) + 300
}

token = jws.(%{"alg" => "RS256", "typ" => "JWT"}, claims)

# No signature header: the body is the signed envelope. The ledger stores the
# verified claims.
{:ok, :created, envelope} =
  AshHooks.Ingress.ingest(Tour.Ledger, :envelope, token, %{headers: %{}, scope: scope})

IO.inspect(envelope.payload, label: "stored JWS payload")
"member.joined" = envelope.payload["event"]

# A token that names another algorithm is rejected, whatever its signature:
# the algorithm comes from your configuration, never from the token.
unsigned =
  b64url.(Jason.encode!(%{"alg" => "none"})) <> "." <> b64url.(Jason.encode!(claims)) <> "."

{:error, %{__struct__: AshHooks.Errors.Invalid.InvalidSignature}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :envelope, unsigned, %{headers: %{}, scope: scope})

# A validly signed token past its exp claim is refused as stale.
expired = jws.(%{"alg" => "RS256"}, %{claims | "exp" => System.system_time(:second) - 60})

{:error, %{__struct__: AshHooks.Errors.Invalid.StaleTimestamp}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :envelope, expired, %{headers: %{}, scope: scope})

IO.puts("JWS: pinned algorithm, alg none refused, exp enforced")

A sender that canonicalizes form fields

# This sender posts a form and signs the public URL it called plus the
# sorted fields. Pass that URL as request_uri: behind a proxy, build it from
# a base URL you configure, never from forwarded headers.
url = "https://hooks.example.test/sms?account=acct-42"
form = "To=%2B15550100&From=%2B15550199&Body=hello"
sms_secret = Application.fetch_env!(:ash_hooks, :tour_sms_secret)
signed = url <> "Bodyhello" <> "From+15550199" <> "To+15550100"
sms_signature = :crypto.mac(:hmac, :sha, sms_secret, signed) |> Base.encode64()

sms_ctx = %{headers: %{"x-sms-signature" => sms_signature}, request_uri: url, scope: scope}
{:ok, :created, sms} = AshHooks.Ingress.ingest(Tour.Ledger, :sms, form, sms_ctx)

# body_format :form decoded the verified bytes into the stored payload.
IO.inspect(sms.payload, label: "stored form payload")
"hello" = sms.payload["Body"]

# The same bytes posted to a different URL do not verify.
{:error, %{__struct__: AshHooks.Errors.Invalid.InvalidSignature}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :sms, form, %{
    sms_ctx
    | request_uri: "https://evil.example.test/sms"
  })

What to answer the sender

ingest/4 resultMeaningAnswer
{:ok, _, %{status: s}} with s in [:processed, :failed_permanent, :superseded]terminal: done, or permanently failed (on your operator surface)2xx
{:ok, _, _row} with another statusprocessing is pending, already running, or retryable; return 5xx to request redelivery5xx requests a retry
{:error, %InvalidSignature{}}, %StaleTimestamp{}, %MalformedPayload{}not authentic, a replay, or not decodable4xx
{:error, %NoWebhookSecret{}} or {:error, :source_paused}your configuration or a deliberate pause5xx requests a retry; recovery depends on the sender's retry policy

Inbound source records

This source selects its provider from a code-defined alias allowlist. Issuance commits a registry issue before custody receives material, then attaches it with the owner in one transaction. Run these helpers outside enclosing transactions. The disclosure is returned once; this tour plays the sender using it without printing it. A deployed host delivers it securely to its sender.

{:ok, source, source_disclosure} =
  AshHooks.InboundSource.create_source(
    Tour.Source,
    %{source: :managed_orders, account_id: "acct-42", provider: :orders},
    generate: {:standard_webhooks, :v1}
  )

managed_body = Jason.encode!(%{"id" => "managed-1", "type" => "order.paid"})

managed_signature = fn "whsec_" <> encoded, bytes ->
  :crypto.mac(:hmac, :sha256, Base.decode64!(encoded), bytes) |> Base.encode16(case: :lower)
end

{:ok, %{__struct__: AshHooks.Inbound.ResolvedSource} = resolved} =
  AshHooks.Ingress.resolve_source(Tour.Ledger, :managed_orders, scope: scope)

Tour.ManagedOrders = resolved.provider
:orders = resolved.provider_alias
false = resolved.paused?
[key] = resolved.keys
true = key.ref == source.secret_ref

{:ok, :created, %{status: :processed, provider_alias: :orders}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :managed_orders, managed_body, %{
    scope: scope,
    resolved_source: resolved,
    headers: %{"x-managed-signature" => managed_signature.(source_disclosure, managed_body)}
  })

{:ok, source, next_source_disclosure} =
  AshHooks.InboundSource.roll_secret(source,
    generate: {:standard_webhooks, :v1},
    overlap_seconds: 60
  )

# Resolve per request, never cache a resolution across requests or rotations.
{:ok, overlap} = AshHooks.Ingress.resolve_source(Tour.Ledger, :managed_orders, scope: scope)
2 = length(overlap.keys)

for disclosure <- [source_disclosure, next_source_disclosure] do
  :ok =
    Tour.ManagedOrders.verify_signature(
      managed_body,
      %{headers: %{"x-managed-signature" => managed_signature.(disclosure, managed_body)}},
      overlap.keys
    )
end

{:ok, source} = AshHooks.InboundSource.pause(source)

{:ok, %{paused?: true} = paused_source} =
  AshHooks.Ingress.resolve_source(Tour.Ledger, :managed_orders, scope: scope)

{:error, :source_paused} =
  AshHooks.Ingress.ingest(Tour.Ledger, :managed_orders, managed_body, %{
    scope: scope,
    resolved_source: paused_source,
    headers: %{"x-managed-signature" => managed_signature.(next_source_disclosure, managed_body)}
  })

{:ok, source} = AshHooks.InboundSource.unpause(source)
{:ok, source, recovered_disclosure} = AshHooks.InboundSource.reissue_secret(source)
true = source.previous_secret_ref == key.ref
{:ok, source} = AshHooks.InboundSource.expire_previous_secret(source)
{:ok, %{destroyed: 2, destroy_failed: 0}} = AshHooks.KeyRegistry.sweep(Tour.KeyRegistry)

{:ok, active_source} = AshHooks.Ingress.resolve_source(Tour.Ledger, :managed_orders, scope: scope)
[active_key] = active_source.keys

:ok =
  Tour.ManagedOrders.verify_signature(
    managed_body,
    %{
      headers: %{"x-managed-signature" => managed_signature.(recovered_disclosure, managed_body)}
    },
    active_key
  )

{:error, %{__struct__: AshHooks.Errors.Invalid.InvalidSignature}} =
  Tour.ManagedOrders.verify_signature(
    managed_body,
    %{
      headers: %{
        "x-managed-signature" => managed_signature.(next_source_disclosure, managed_body)
      }
    },
    active_key
  )

# The JSON policy refuses a correctly signed but disallowed payload.
# With no decoded event identity, ingress refuses it before creating a receipt.
forbidden_body = Jason.encode!(%{"id" => "managed-policy", "forbidden" => true})

{:error, %{__struct__: AshHooks.Errors.Invalid.MalformedPayload}} =
  AshHooks.Ingress.ingest(Tour.Ledger, :managed_orders, forbidden_body, %{
    scope: scope,
    headers: %{"x-managed-signature" => managed_signature.(recovered_disclosure, forbidden_body)}
  })

IO.puts("source resolved, rotated, paused, resumed, reissued, and verified under JSON policy")

Source receipts retain the authenticating alias for recovery, independently of later source pause, rotation, or deletion. Existing declarations can migrate with adopt_provider_alias/4; the key guide covers cutover ordering and bounded claims. Sender-issued HMAC imports use material: and kind:; only standard v1 is source-generated. Public senders keep their private keys and register typed public keys outside custody with register_public_key/3.

Verify without a ledger

# Every provider's verify_signature/3 can be called directly, for example in
# your own tests. Direct calls may pin the clock with :now_ms.
:ok = Tour.Orders.verify_signature(body, %{headers: ctx.headers}, orders_secret)

# The configuration also works without a module.
:ok =
  AshHooks.Provider.Scheme.verify(body, %{headers: ctx.headers}, orders_secret,
    algorithm: :hmac_sha256,
    signature: {:header, "x-tour-signature"}
  )

# The public primitives also support schemes you implement yourself.
:ok =
  AshHooks.Provider.Signature.hmac(body, ctx.headers["x-tour-signature"], orders_secret,
    digest: :sha256
  )

:ok =
  AshHooks.Provider.Signature.ed25519(shipment, Base.encode64(partner_signature), [partner_public])

{:ok, _claims_json} = AshHooks.Provider.Signature.jws(token, [envelope_public_pem], alg: "RS256")

# A list of keys verifies with any of them: rotation for direct calls.
:ok =
  AshHooks.Provider.Signature.hmac(rotated, rotated_ctx.headers["x-tour-signature"], [
    orders_secret,
    previous
  ])

IO.puts("direct verification agrees with ingress")
# Options are checked when the provider compiles. A typo is a compile error
# that names the problem, not a provider that silently rejects everything.
try do
  Code.compile_string("""
  defmodule Tour.Misconfigured do
    use AshHooks.Provider,
      verify: [algorithm: :hmac_sha256, signature: {:header, "x-sig"}, encoding: :hexadecimal]
  end
  """)
rescue
  error -> IO.puts("compile error: " <> Exception.message(error))
else
  _ -> raise "an invalid encoding must not compile"
end

Outbound: deliver to the real receiver

port = String.to_integer(System.get_env("LOCAL_WEBHOOK_TESTER_PORT", "52871"))
receiver_base = "http://127.0.0.1:#{port}"
receiver_session = Ash.UUID.generate()

# Remove the disposable receiver session even if a later cell fails.
System.at_exit(fn _status ->
  :httpc.request(
    :delete,
    {to_charlist(receiver_base <> "/api/session/" <> receiver_session), []},
    [],
    []
  )
end)

{:ok, %{status: 200}} =
  AshHooks.Http.Bounded.request(
    :get,
    receiver_base <> "/api/version",
    %{},
    "",
    validate_destination: false
  )

IO.puts("using the existing receiver on #{port}")

# Registration rejects private/loopback URLs by design; ash_hooks
# treats webhook destinations as untrusted input. This demo needs a
# trusted local receiver, so we register the endpoint with a public URL and
# repoint the stored row for this local inspection; production
# endpoints keep their public URLs.
endpoint =
  Ash.create!(
    Tour.Endpoint,
    %{
      url: "https://partner.example.test/hook",
      secret_ref: "tour-endpoint",
      # this endpoint opts IN to response capture per row (bounded bytes,
      # redaction floor, allowlisted headers only)
      response_capture: true,
      response_header_allowlist: ["content-type"]
    },
    authorize?: false
  )

Tour.Repo.query!("UPDATE tour_endpoints SET url = ? WHERE id = ?", [
  receiver_base <> "/" <> receiver_session,
  endpoint.id
])

subscription =
  Ash.create!(Tour.Subscription, %{endpoint_id: endpoint.id, event_types: ["order_paid"]},
    authorize?: false
  )

{:ok, event} = AshHooks.Event.new(type: :order_paid, payload: Jason.encode!(%{"order" => 1}))

# No enqueue configured: dispatch persists the durable rows only
# (the deferred pattern, where an Oban-backed app wires enqueue: here)
{:ok, results} = AshHooks.dispatch(Tour.Order, :order_paid, event)
IO.inspect(Enum.map(results, & &1.status), label: "dispatch results")

Pause new subscription fanout

pausable :on injects a nonnullable boolean, default false. Your authorized update action accepts paused. Pause suppresses new delivery rows, while the already persisted delivery from the preceding cell still runs below.

subscription = Ash.update!(subscription, %{paused: true}, authorize?: false)
{:ok, suppressed_event} = AshHooks.Event.new(type: :order_paid, payload: "{}")
{:ok, []} = AshHooks.dispatch(Tour.Order, :order_paid, suppressed_event)
1 = Ash.count!(Tour.Delivery, authorize?: false)
# Drive the delivery machine directly (the shape an Oban worker uses).
# Two options permit this trusted loopback receiver: `ssrf_check` replaces the
# send-time destination re-check, and http_opts' `validate_destination:
# false` relaxes the HTTP adapter's own pin; both exist for tests and
# diagnostics; production configs keep both guards on.
outbound_secret = AshHooks.Signing.generate_secret()

config = [
  deliveries: Tour.Delivery,
  endpoints: Tour.Endpoint,
  attempts: Tour.Attempt,
  secret_resolver: fn "tour-endpoint" ->
    {:ok, outbound_secret}
  end,
  max_attempts: 3,
  base_backoff_seconds: 1,
  max_backoff_seconds: 60,
  retry_after_cap_seconds: 100,
  ssrf_check: fn url -> url == receiver_base <> "/" <> receiver_session end,
  http_opts: [validate_destination: false],
  now: fn -> DateTime.utc_now() |> DateTime.truncate(:second) end
]

[row] = Ash.read!(Tour.Delivery, authorize?: false)

:ok =
  AshHooks.Delivery.run(
    %{"endpoint_id" => row.endpoint_id, "event_uuid" => row.event_uuid},
    config
  )

final = Ash.reload!(row, authorize?: false)
{:succeeded, 200} = {final.status, final.response_status}
IO.inspect({final.status, final.response_status, final.response_snippet}, label: "delivered")

{:ok, %{status: 200, body: captures}} =
  AshHooks.Http.Bounded.request(
    :get,
    receiver_base <> "/api/session/" <> receiver_session <> "/requests",
    %{},
    "",
    validate_destination: false,
    max_body_bytes: 1_048_576
  )

matching_captures =
  Enum.filter(Jason.decode!(captures), fn request ->
    Enum.any?(
      request["headers"],
      &(&1["name"] |> String.downcase() == "webhook-id" and &1["value"] == event.id)
    )
  end)

[capture] = matching_captures
captured_bytes = Base.decode64!(capture["request_payload_base64"])
true = captured_bytes == event.payload
captured_headers = Map.new(capture["headers"], &{String.downcase(&1["name"]), &1["value"]})

{:ok, %{id: verified_id}} =
  AshHooks.Signing.verify(captured_bytes, captured_headers, outbound_secret)

true = verified_id == event.id
IO.puts("receiver retained exact bytes and a valid signature")

# A completed row is harmless to drive again: no second network request.
:ok =
  AshHooks.Delivery.run(
    %{"endpoint_id" => row.endpoint_id, "event_uuid" => row.event_uuid},
    config
  )

{:ok, %{status: 200, body: captures_after}} =
  AshHooks.Http.Bounded.request(
    :get,
    receiver_base <> "/api/session/" <> receiver_session <> "/requests",
    %{},
    "",
    validate_destination: false,
    max_body_bytes: 1_048_576
  )

1 = length(Jason.decode!(captures_after))
IO.puts("terminal redrive retained exactly one receiver request")

# One attempt row was recorded beside the delivery: outcome, HTTP status,
# latency, the signing key REFERENCE, and, because this endpoint opted in
# per row, the redacted body and only allowlisted response headers.
[attempt] = Ash.read!(Tour.Attempt, authorize?: false)
:succeeded = attempt.outcome
200 = attempt.response_status
true = is_integer(attempt.latency_ms) and attempt.latency_ms >= 0
"tour-endpoint" = attempt.signing_key_ref
true = String.starts_with?(attempt.response_snippet, "[captured]")
# only allowlisted names survive when the receiver sends any (this one
# answers empty-bodied: no headers to keep)
true = attempt.response_headers == nil or is_map(attempt.response_headers)
IO.puts("attempt history recorded the send")

# The delivery log answers from the rows, keyset-paginated.
{:ok, %{records: [logged], next_cursor: nil}} =
  AshHooks.Delivery.list(Tour.Delivery, endpoint_id: endpoint.id)

true = logged.id == row.id

{:ok, %{records: [history], next_cursor: nil}} =
  AshHooks.OutboundAttempt.list(Tour.Attempt, endpoint_id: endpoint.id)

:succeeded = history.outcome
IO.puts("delivery log and attempt history answer from the rows")

IO.puts("""
Response bodies stay off by default everywhere: without the endpoint's
per-row opt-in, attempt rows carry the status + content-type summary only
This run leaves the delivery row's snippet as a summary. A direct run with
`snippet_capture: true` can store a bounded, redacted body there.
""")
subscription = Ash.update!(subscription, %{paused: false}, authorize?: false)
false = subscription.paused
# Existing event identity still deduplicates; resume restores its fanout route.
{:ok, [_result]} = AshHooks.dispatch(Tour.Order, :order_paid, event)
1 = Ash.count!(Tour.Delivery, authorize?: false)

Resend and replay: the same event id, again

# A resend re-arms the TERMINAL row: same webhook-id (the receiver
# deduplicates), a fresh retry ladder, and who asked recorded on the row.
{:ok, %{status: :resent, delivery: re_armed}} =
  AshHooks.Delivery.resend(final,
    requested_by: "tour-operator",
    endpoints: Tour.Endpoint
  )

:pending = re_armed.status
0 = re_armed.attempts - re_armed.resend_baseline
"tour-operator" = re_armed.last_resend_by
1 = re_armed.resend_count

# Bulk replay counts before it touches anything (the re-armed row is
# pending; that is the replays-eligible set here)
{:ok, %{count: 1}} =
  AshHooks.Delivery.replay(Tour.Delivery,
    endpoints: Tour.Endpoint,
    status: [:pending],
    requested_by: "tour-operator",
    dry_run: true
  )

# The second sweep re-arms nothing more (the only row is pending,
# not terminal; live rows are left to their schedule)
{:ok, %{resent: 0, skipped: 0}} =
  AshHooks.Delivery.replay(Tour.Delivery,
    endpoints: Tour.Endpoint,
    status: [:dead_letter],
    requested_by: "tour-operator",
    per_second: 10
  )

IO.puts(
  "resend recorded who asked; replay counted matching rows and left nonmatching rows unchanged"
)

Multi-tenancy: one tenant's event, one tenant's rows

# The same machine, scoped by an explicit tenant: declare attribute
# multitenancy on the touched resources (in your app: the subscription,
# the endpoint, and both ledgers) and pass :tenant to every call. Ash
# scopes every query and write to the tenant; the package additionally
# fails closed with a named error when a multitenant resource is touched
# without one.
defmodule Tour.TenantEndpoint do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.Endpoint]

  sqlite do
    table("tour_tenant_endpoints")
    repo(Tour.Repo)
  end

  attributes do
    attribute(:org_id, :string, allow_nil?: false)
  end

  multitenancy do
    strategy(:attribute)
    attribute(:org_id)
  end

  actions do
    defaults([:read, :create, :update])
    default_accept(:*)
  end
end

defmodule Tour.TenantSubscription do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.Subscription]

  sqlite do
    table("tour_tenant_subscriptions")
    repo(Tour.Repo)
  end

  attributes do
    attribute(:org_id, :string, allow_nil?: false)
  end

  multitenancy do
    strategy(:attribute)
    attribute(:org_id)
  end

  actions do
    defaults([:read, :create])
    default_accept(:*)
  end

  subscription do
    endpoint_resource(Tour.TenantEndpoint)
  end
end

defmodule Tour.TenantDelivery do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.OutboundDelivery]

  sqlite do
    table("tour_tenant_deliveries")
    repo(Tour.Repo)
  end

  attributes do
    attribute(:org_id, :string, allow_nil?: false)
    timestamps()
  end

  multitenancy do
    strategy(:attribute)
    attribute(:org_id)
  end

  actions do
    defaults([:read])
  end
end

defmodule Tour.TenantOrder do
  use Ash.Resource,
    domain: Tour.Domain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks]

  sqlite do
    table("tour_tenant_orders")
    repo(Tour.Repo)
  end

  attributes do
    uuid_primary_key(:id)
  end

  actions do
    defaults([:read, :create])
  end

  webhooks do
    outbound :order_paid do
      subscriptions(Tour.TenantSubscription)
      deliveries(Tour.TenantDelivery)
    end
  end
end
# Tenant-bearing tables: the multitenancy attribute prefixes the unique
# identity; "have I seen this event for this endpoint" is per-tenant
Tour.Repo.query!("""
CREATE TABLE tour_tenant_endpoints (
  id TEXT PRIMARY KEY, url TEXT NOT NULL, status TEXT NOT NULL DEFAULT 'enabled',
  secret_ref TEXT NOT NULL, previous_secret_ref TEXT, legacy_secret_ref TEXT,
  legacy_previous_secret_ref TEXT, org_id TEXT NOT NULL)
""")

Tour.Repo.query!("""
CREATE TABLE tour_tenant_subscriptions (
  id TEXT PRIMARY KEY, event_types TEXT NOT NULL, endpoint_id TEXT NOT NULL,
  signing_mode TEXT, org_id TEXT NOT NULL)
""")

Tour.Repo.query!("""
CREATE TABLE tour_tenant_deliveries (
  id TEXT PRIMARY KEY, event_uuid TEXT NOT NULL, event_type TEXT NOT NULL,
  payload BLOB NOT NULL, endpoint_id TEXT NOT NULL, subscription_id TEXT,
  signing_mode TEXT, status TEXT NOT NULL DEFAULT 'pending',
  attempts INTEGER NOT NULL DEFAULT 0, response_status INTEGER,
  response_snippet TEXT, last_error TEXT, next_attempt_at TEXT,
  dispatch_source TEXT NOT NULL DEFAULT 'v1:direct:unbound',
  dispatch_route TEXT NOT NULL DEFAULT 'v1:route:unbound',
  attempt_token TEXT, send_lease_expires_at TEXT,
  enqueue_token TEXT, enqueue_lease_expires_at TEXT,
  endpoint_snapshot JSONB,
  org_id TEXT NOT NULL, inserted_at TEXT, updated_at TEXT)
""")

Tour.Repo.query!(
  "CREATE UNIQUE INDEX tour_tenant_deliveries_unique ON tour_tenant_deliveries (org_id, endpoint_id, event_uuid)"
)

Tour.Repo.query!("CREATE TABLE IF NOT EXISTS tour_tenant_orders (id TEXT PRIMARY KEY)")
:ok
# Two tenants, each with an endpoint subscribed to order_paid
for org <- ["org_b", "org_a"] do
  endpoint =
    Ash.create!(
      Tour.TenantEndpoint,
      %{url: "https://#{org}.example.test/hook", secret_ref: "ref-#{org}"},
      tenant: org,
      authorize?: false
    )

  Ash.create!(Tour.TenantSubscription, %{endpoint_id: endpoint.id, event_types: ["order_paid"]},
    tenant: org,
    authorize?: false
  )
end

{:ok, event} = AshHooks.Event.new(type: :order_paid, payload: Jason.encode!(%{"order" => 7}))

# org_a dispatches: the fanout read itself is tenant-scoped; org_b's
# subscription is invisible, so no row, no signature, no bytes for org_b
{:ok, [result]} = AshHooks.dispatch(Tour.TenantOrder, :order_paid, event, tenant: "org_a")
IO.inspect(result.status, label: "org_a dispatch result")

[a_row] = Ash.read!(Tour.TenantDelivery, authorize?: false, tenant: "org_a")
IO.inspect({a_row.org_id, a_row.status}, label: "org_a's row")

[] = Ash.read!(Tour.TenantDelivery, authorize?: false, tenant: "org_b")
IO.puts("org_b's partition: empty; the event never existed for org_b")

# A tenant-less call against a multitenant set fails closed before any
# data access, returning the named error, not a crash:
{:error, :tenant_required} = AshHooks.dispatch(Tour.TenantOrder, :order_paid, event)
IO.puts("tenant-less dispatch: {:error, :tenant_required}")

Retention: prune the terminal rows

# Nothing is older than 30 days; prune finds nothing yet
{:ok, 0} =
  AshHooks.Ingress.prune(Tour.Ledger, older_than: DateTime.add(DateTime.utc_now(), -30, :day))

# Backdate the processed rows past a 30-day TTL and prune again
%{rows: [[processed]]} =
  Tour.Repo.query!("SELECT COUNT(*) FROM tour_ledgers WHERE status = 'processed'")

Tour.Repo.query!("UPDATE tour_ledgers SET inserted_at = ? WHERE status = 'processed'", [
  DateTime.add(DateTime.utc_now(), -31, :day)
  |> DateTime.truncate(:microsecond)
  |> DateTime.to_iso8601()
])

{:ok, ^processed} =
  AshHooks.Ingress.prune(Tour.Ledger, older_than: DateTime.add(DateTime.utc_now(), -30, :day))

IO.puts(
  "Pruned #{processed} terminal rows past the TTL; recent and non-terminal rows are never touched."
)

Fit the extensions to your domain

# A consumer whose domain uses a custom payload column,
# owns an `active` endpoint switch, uses atom-valued subscription types,
# keys rows with
# uuid_v7, and keeps the ledger append-only.
defmodule Tour.FitEndpoint do
  use Ash.Resource,
    domain: Tour.FitDomain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.Endpoint]

  sqlite do
    table("tour_fit_endpoints")
    repo(Tour.Repo)
  end

  attributes do
    uuid_v7_primary_key(:id)
    # the consumer's own switch: the package injects no `status`
    attribute(:active, :boolean, allow_nil?: false, default: true, public?: true)
  end

  actions do
    defaults([:read, :create, :update])
    default_accept(:*)
  end

  endpoint do
    status_attribute(:active)
    enabled_values([true])
    disabled_value(false)
  end
end

defmodule Tour.FitSubscription do
  use Ash.Resource,
    domain: Tour.FitDomain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.Subscription]

  sqlite do
    table("tour_fit_subscriptions")
    repo(Tour.Repo)
  end

  attributes do
    uuid_v7_primary_key(:id)

    # the consumer's closed atom register replaces the injected
    # {:array, :string} default ["*"]; matching handles atoms
    attribute(:event_types, {:array, :atom},
      allow_nil?: false,
      public?: true,
      constraints: [min_length: 1, items: [one_of: [:order_paid, :order_shipped]]]
    )
  end

  actions do
    defaults([:read, :create])
    default_accept(:*)
  end

  subscription do
    endpoint_resource(Tour.FitEndpoint)
  end
end

defmodule Tour.FitDelivery do
  use Ash.Resource,
    domain: Tour.FitDomain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks.OutboundDelivery]

  sqlite do
    table("tour_fit_deliveries")
    repo(Tour.Repo)
  end

  attributes do
    uuid_v7_primary_key(:id)
  end

  actions do
    defaults([:read])
  end

  outbound_delivery do
    # the domain reserves `payload`; the exact-bytes column is renamed
    payload_attribute(:event_bytes)
    # append-only ledger: no destroy action is injected at all
    prune_action(:none)
  end
end

defmodule Tour.FitOrder do
  use Ash.Resource,
    domain: Tour.FitDomain,
    data_layer: AshSqlite.DataLayer,
    extensions: [AshHooks]

  sqlite do
    table("tour_fit_orders")
    repo(Tour.Repo)
  end

  attributes do
    uuid_primary_key(:id)
  end

  actions do
    defaults([:read, :create])
  end

  webhooks do
    outbound :order_paid do
      subscriptions(Tour.FitSubscription)
      deliveries(Tour.FitDelivery)
    end
  end
end

defmodule Tour.FitDomain do
  use Ash.Domain, otp_app: nil, validate_config_inclusion?: false

  resources do
    resource(Tour.FitEndpoint)
    resource(Tour.FitSubscription)
    resource(Tour.FitDelivery)
    resource(Tour.FitOrder)
  end
end
Tour.Repo.query!("""
CREATE TABLE tour_fit_endpoints (
  id TEXT PRIMARY KEY, url TEXT NOT NULL, active BOOLEAN NOT NULL DEFAULT 1,
  secret_ref TEXT NOT NULL, previous_secret_ref TEXT, legacy_secret_ref TEXT,
  legacy_previous_secret_ref TEXT)
""")

Tour.Repo.query!("""
CREATE TABLE tour_fit_subscriptions (
  id TEXT PRIMARY KEY, event_types TEXT NOT NULL, endpoint_id TEXT NOT NULL, signing_mode TEXT)
""")

# NO payload column and no destroy action: the sole-store reservation
# and the append-only posture, in the schema itself
Tour.Repo.query!("""
CREATE TABLE tour_fit_deliveries (
  id TEXT PRIMARY KEY, event_uuid TEXT NOT NULL, event_type TEXT NOT NULL,
  event_bytes BLOB NOT NULL, endpoint_id TEXT NOT NULL, subscription_id TEXT,
  signing_mode TEXT, status TEXT NOT NULL DEFAULT 'pending',
  attempts INTEGER NOT NULL DEFAULT 0, response_status INTEGER,
  response_snippet TEXT, last_error TEXT, next_attempt_at TEXT,
  dispatch_source TEXT NOT NULL DEFAULT 'v1:direct:unbound',
  dispatch_route TEXT NOT NULL DEFAULT 'v1:route:unbound',
  attempt_token TEXT, send_lease_expires_at TEXT,
  enqueue_token TEXT, enqueue_lease_expires_at TEXT,
  endpoint_snapshot JSONB)
""")

Tour.Repo.query!(
  "CREATE UNIQUE INDEX tour_fit_deliveries_unique ON tour_fit_deliveries (endpoint_id, event_uuid)"
)

Tour.Repo.query!("CREATE TABLE IF NOT EXISTS tour_fit_orders (id TEXT PRIMARY KEY)")
:ok
fit_endpoint =
  Ash.create!(
    Tour.FitEndpoint,
    %{url: "https://partner.example.test/hook", secret_ref: "fit-ref", active: true},
    authorize?: false
  )

# the atom-typed register matches the event's canonical string type
Ash.create!(Tour.FitSubscription, %{endpoint_id: fit_endpoint.id, event_types: [:order_paid]},
  authorize?: false
)

# a deterministic ID: the re-fire below matches the existing row
{:ok, fit_event} =
  AshHooks.Event.new(
    type: :order_paid,
    payload: Jason.encode!(%{"order" => 9}),
    id: "msg_fit-order-9"
  )

# Persist a deferred row; repeating the stable event ID returns its duplicate.
{:ok, [first]} = AshHooks.dispatch(Tour.FitOrder, :order_paid, fit_event)
{:ok, [second]} = AshHooks.dispatch(Tour.FitOrder, :order_paid, fit_event)

IO.inspect({first.status, second.status}, label: "deferred, then duplicate")
%{status: :deferred} = first
%{status: :duplicate} = second

[fit_row] = Ash.read!(Tour.FitDelivery, authorize?: false)

# uuid_v7 ids (version nibble 7), bytes under the renamed column
IO.inspect(binary_part(fit_row.id, 14, 1), label: "uuid version nibble")
IO.inspect(fit_row.event_bytes, label: "exact bytes under event_bytes")
"7" = binary_part(fit_row.id, 14, 1)
fit_bytes = Jason.encode!(%{"order" => 9})
^fit_bytes = fit_row.event_bytes

# the consumer's own off switch suppresses the fanout entirely
Ash.update!(fit_endpoint, %{active: false}, authorize?: false)

{:ok, results} = AshHooks.dispatch(Tour.FitOrder, :order_paid, fit_event)
IO.inspect(results, label: "dispatch against a deactivated endpoint")
[] = results

# and the retention hook returns an error on the append-only ledger
{:error, prune_error} =
  AshHooks.Delivery.prune(Tour.FitDelivery, older_than: DateTime.utc_now())

IO.inspect(Exception.message(prune_error), label: "prune on append-only")

IO.puts("""
The resource keeps its own column names, generated keys, typed subscriptions,
enable switch, and append-only retention policy.
""")

Endpoint custody issuance, overlap, and disposal

Create a key without a byte resolver. Endpoint references and kinds are stored, while the one-time disclosure stays with the recipient. v1 generates a shared whsec_; v1a discloses only its Ed25519 whpk_ public half. This cell uses the same explicit loopback diagnostic permissions as the earlier receiver exercise.

{:ok, custody_endpoint, first_disclosure} =
  AshHooks.Endpoint.create_with_key(
    Tour.CustodyEndpoint,
    %{url: "https://partner.example.test/custody"},
    generate: {:standard_webhooks, :v1}
  )

# Only for this disposable receiver, as in the earlier outbound cell.
Tour.Repo.query!("UPDATE tour_custody_endpoints SET url = ? WHERE id = ?", [
  receiver_base <> "/" <> receiver_session,
  custody_endpoint.id
])

custody_endpoint = Ash.reload!(custody_endpoint, authorize?: false)

send_custody = fn endpoint ->
  AshHooks.Delivery.test_send(endpoint,
    deliveries: Tour.Delivery,
    type: :custody_tour,
    payload: ~s({"custody":true}),
    ssrf_check: fn url -> url == receiver_base <> "/" <> receiver_session end,
    http_opts: [validate_destination: false]
  )
end

{:ok, %{status: :succeeded}} = send_custody.(custody_endpoint)

{:ok, custody_endpoint, public_disclosure} =
  AshHooks.Endpoint.roll_secret(custody_endpoint,
    generate: {:standard_webhooks, :v1a},
    overlap_seconds: 60
  )

{:ok, %{status: :succeeded, delivery: overlap_delivery}} = send_custody.(custody_endpoint)
:succeeded = overlap_delivery.status

{:ok, %{status: 200, body: custody_captures}} =
  AshHooks.Http.Bounded.request(
    :get,
    receiver_base <> "/api/session/" <> receiver_session <> "/requests",
    %{},
    "",
    validate_destination: false,
    max_body_bytes: 1_048_576
  )

[custody_capture] =
  Enum.filter(Jason.decode!(custody_captures), fn request ->
    Enum.any?(request["headers"], fn header ->
      String.downcase(header["name"]) == "webhook-id" and
        header["value"] == overlap_delivery.event_uuid
    end)
  end)

custody_bytes = Base.decode64!(custody_capture["request_payload_base64"])
^custody_bytes = overlap_delivery.payload
custody_headers = Map.new(custody_capture["headers"], &{String.downcase(&1["name"]), &1["value"]})
{:ok, _} = AshHooks.Signing.verify(custody_bytes, custody_headers, first_disclosure)
{:ok, _} = AshHooks.Signing.verify(custody_bytes, custody_headers, public_disclosure)

opaque =
  struct!(AshHooks.Custody.Key,
    custodian: Tour.Custodian,
    ref: custody_endpoint.secret_ref,
    tenant: nil,
    kind: custody_endpoint.secret_kind
  )

{:ok, fingerprint} = AshHooks.Custody.display_fingerprint(opaque)
{:ok, ^fingerprint} = AshHooks.Signing.display_fingerprint(public_disclosure)
IO.puts("public and custody fingerprints agree: " <> fingerprint)

# A second roll cannot discard an active previous key.
{:error, :overlap_active} =
  AshHooks.Endpoint.roll_secret(custody_endpoint, generate: {:standard_webhooks, :v1})

{:error, {:overlap_exceeds_max, 3600}} =
  AshHooks.Endpoint.schedule_previous_retirement(
    custody_endpoint,
    DateTime.add(DateTime.utc_now(), 7200)
  )

# Reissue immediately replaces current, preserving the separate previous window.
previous_ref = custody_endpoint.previous_secret_ref
{:ok, custody_endpoint, _replacement_public} = AshHooks.Endpoint.reissue_secret(custody_endpoint)
^previous_ref = custody_endpoint.previous_secret_ref

# Retire at an explicit instant, remove ended slots, then dispose outside transactions.
retire_at = DateTime.utc_now()

{:ok, custody_endpoint} =
  AshHooks.Endpoint.schedule_previous_retirement(custody_endpoint, retire_at, now: retire_at)

{:ok, 1} = AshHooks.Endpoint.expire_previous_secrets(Tour.CustodyEndpoint, now: retire_at)
# Two replaced keys plus the abandoned issue from the refused roll.
{:ok, %{destroyed: 3, destroy_failed: 0}} = AshHooks.KeyRegistry.sweep(Tour.KeyRegistry)

{:error, :unknown_ref} =
  Tour.Custodian.sign(previous_ref, nil, {:standard_webhooks, :v1}, "input")

# The legacy slot has its own markers and timeline; standard remains unchanged.
custody_endpoint = Ash.reload!(custody_endpoint, authorize?: false)
standard_ref = custody_endpoint.secret_ref

{:ok, custody_endpoint, _legacy_disclosure} =
  AshHooks.Endpoint.roll_secret(custody_endpoint,
    slot: :legacy,
    generate: {:legacy_hmac, :sha256}
  )

^standard_ref = custody_endpoint.secret_ref
nil = custody_endpoint.legacy_previous_secret_ref
IO.puts("custody delivery, mixed-algorithm overlap, reissue, expiry, and destruction completed")

Schedule both owner expiry and registry sweeps per tenant. Expiry controls slot removal; the signing deadline already excludes an expired key. The registry also recovers timed-out issues and deleted owners. To migrate existing byte slots, keep secret_resolver while running adopt_keys/2 for every row; remove the byte resolver only after all occupied slots carry custody kinds. Then the compile-time key_material: :custody_only setting can refuse new byte sources. The key guide explains imports, canonical tenant equality, fingerprints, and operational recovery.

Clean up

The tour created one disposable SQLite file. Close its repository and remove only that file and its SQLite sidecars when you finish:

database_path = Application.fetch_env!(:ash_hooks, Tour.Repo)[:database]
:ok = Supervisor.stop(Process.whereis(Tour.Repo))
for suffix <- ["", "-shm", "-wal"], do: File.rm(database_path <> suffix)

for key <- [
      :tour_orders_secret,
      :tour_orders_previous_secret,
      :tour_payments_secret,
      :tour_sms_secret,
      :tour_partner_public_key,
      :tour_envelope_public_key,
      Tour.Repo
    ],
    do: Application.delete_env(:ash_hooks, key)

:ok = Agent.stop(Tour.Handled)
:ok = :telemetry.detach("tour")

{:ok, {{_, cleanup_status, _}, _, _}} =
  :httpc.request(
    :delete,
    {to_charlist(receiver_base <> "/api/session/" <> receiver_session), []},
    [],
    []
  )

true = cleanup_status in 200..299
IO.puts("Tour database and receiver session removed.")

Where to go next