AshHooks.EndpointHealth (AshHooks v3.3.0)

Copy Markdown View Source

Computes endpoint health from attempt history and applies an optional sustained-failure disable policy.

Latency percentiles use attempts with a recorded duration, including failed request attempts. First-attempt successes count successful attempt 1 rows. Eventual successes count deliveries with any successful attempt, including attempt 1.

Schedule stats/2 per endpoint and sweep/3 per endpoint resource. The application owns the sweep schedule. The persisted warning time prevents early or overlapping sweeps from shortening the warning interval. The package does not re-enable disabled endpoints. A failed history read makes a sweep incomplete; it cannot be interpreted as evidence of endpoint health.

Summary

Types

One endpoint's trailing-window health.

Functions

Computes one endpoint's health over the trailing window of its attempt rows (requires an AshHooks.OutboundAttempt resource).

Applies the endpoint resource's auto_disable policy to enabled endpoints.

Types

stats()

@type stats() :: %{
  attempts: non_neg_integer(),
  succeeded: non_neg_integer(),
  failed: non_neg_integer(),
  success_rate: float() | nil,
  first_attempt_successes: non_neg_integer(),
  eventual_successes: non_neg_integer(),
  deliveries: non_neg_integer(),
  latency_p50_ms: non_neg_integer() | nil,
  latency_p90_ms: non_neg_integer() | nil,
  latency_p99_ms: non_neg_integer() | nil
}

One endpoint's trailing-window health.

Functions

stats(attempts_mod, opts \\ [])

@spec stats(module(), keyword()) :: {:ok, stats()} | {:error, term()}

Computes one endpoint's health over the trailing window of its attempt rows (requires an AshHooks.OutboundAttempt resource).

Options: :endpoint_id (required), :window_seconds (default 86,400), :now (the application clock), :from/:to (explicit DateTime bounds), and :tenant. When supplying explicit bounds, provide :from; :to is optional. Supplying :to alone returns {:error, :from_required}.

{:ok, %{attempts: 40, success_rate: 0.925, latency_p50_ms: 120}} =
  AshHooks.EndpointHealth.stats(MyApp.Attempt, endpoint_id: endpoint.id)

sweep(endpoints_mod, attempts_mod, opts \\ [])

@spec sweep(module(), module(), keyword()) ::
  {:ok,
   %{
     disabled: non_neg_integer(),
     warned: non_neg_integer(),
     unread: non_neg_integer()
   }}
  | {:error, term()}

Applies the endpoint resource's auto_disable policy to enabled endpoints.

A qualifying endpoint without disable_warned_at gets a warning marker and [:ash_hooks, :endpoint, :disable_warning] telemetry after the marker is stored. A qualifying failure can disable only after warning_interval_seconds (default 60) has elapsed since that marker, including overlapping sweeps. Endpoints below the warning thresholds have their warning marker cleared.

Successful disables emit [:ash_hooks, :endpoint, :disable] with reason :sustained_failure. Counts and telemetry reflect successful writes. A failed disable leaves its warning marker available for a later sweep to retry.

Options: :now and :tenant. Returns {:ok, %{disabled: n, warned: n, unread: n}}. Failed history reads do not stop other endpoints from being considered. Each increments unread and emits [:ash_hooks, :endpoint_health, :read_failed] with measurement %{count: 1} and metadata %{endpoint_id: id, reason: reason}. That endpoint's status and warning marker remain unchanged. A positive unread count means the sweep is incomplete; repair storage and retry. Counts describe committed writes. Endpoint-list read errors return {:error, reason} directly. Persisted warning markers preserve timing across retries.