Retries, timeouts and budgets

What the SDK retries on its own and what it never retries, how long it waits between attempts, and the five options that bound a call - retries, timeout, deadline, maxRequests and signal.

Every method of the SDK is a read, and a read is safe to repeat. So retries are on by default: a failure that might pass is tried again, with a growing wait, before you see an error. You do not write a retry loop. The one exception is a raw request that is not a known read, which is possible only with allowWrites: true: it is sent once.

This page covers what happens after a request fails. Keeping requests inside the vendor’s limit in the first place is the rate limits page.

What happens by default

import { createOutbound } from "outbound-sdk";
import { instantly } from "outbound-sdk/instantly";

import { fakeVendor } from "./fake-vendor.ts";
import { instantClock } from "./instant-clock.ts";

// Two failures, then an empty 200.
const vendor = fakeVendor(503, 503, 200);

const outbound = createOutbound({
  adapter: instantly({ apiKey: "example-key" }),
  fetch: vendor.fetch,
  hooks: {
    onRetry: (event) => {
      console.log(
        `attempt ${event.attempt} failed (${event.reason}), next in ${event.delayMs} ms`
      );
    },
  },
  runtime: instantClock(),
});

// `raw.get` hands back the response body as it is. Here that is `null`.
const body = await outbound.raw.get("/campaigns");

console.log("answer:", body, "after", vendor.count(), "requests");

attempt 1 failed (instantly responded 503), next in 500 ms
attempt 2 failed (instantly responded 503), next in 1000 ms
answer: null after 3 requests

Two attempts failed with a 503, the SDK waited 500 ms and then 1,000 ms, and the third attempt answered. The caller saw one successful call.

How the samples fail

Nothing on this page reaches a vendor. Each sample constructs a real Instantly adapter with a made-up key and passes two things to createOutbound:

• fetch: a stand-in for the vendor’s server. It answers with a status code and an empty body, never a vendor payload.
• runtime: a clock on which a wait ends at once, with the randomness fixed. The waits shown are the ones the SDK asked for. No real time passed.

// A stand-in for the vendor's server, passed as `fetch`. It answers each
// request with the next entry and repeats the last one. An answer is a status
// code with an empty body, never a vendor payload.
export interface WithHeaders {
  readonly headers: Record<string, string>;
  readonly status: number;
}

export type Answer = number | "hang" | "offline" | WithHeaders;

const isStatus = (answer: number | WithHeaders): answer is number =>
  Number.isInteger(answer);

export interface FakeVendor {
  readonly fetch: typeof fetch;
  /** How many requests arrived. */
  readonly count: () => number;
}

// Like the real fetch: it settles only when the caller's signal aborts.
const hang = (signal: AbortSignal | null | undefined): Promise<Response> =>
  // oxlint-disable-next-line promise/avoid-new -- wraps an abort event.
  new Promise((_resolve, reject) => {
    signal?.addEventListener("abort", () => {
      reject(new DOMException("The operation was aborted", "AbortError"));
    });
  });

export const fakeVendor = (...answers: readonly Answer[]): FakeVendor => {
  let requests = 0;

  return {
    count: () => requests,
    fetch: (_input, init) => {
      const answer = answers[Math.min(requests, answers.length - 1)] ?? 200;

      requests += 1;

      if (answer === "hang") {
        return hang(init?.signal);
      }

      if (answer === "offline") {
        return Promise.reject(new TypeError("fetch failed"));
      }

      return Promise.resolve(
        isStatus(answer)
          ? new Response(null, { status: answer })
          : new Response(null, answer)
      );
    },
  };
};

import type { Runtime } from "outbound-sdk";

// A clock for the samples: a wait of any length ends at once and moves the
// clock forward by that length. Backoff, throttle waits and timeouts then
// run in no real time, and `random` is fixed so the delays are the same on
// every run. Pass it as `runtime`. It is meant for tests, not production.
export const instantClock = (): Pick<Runtime, "now" | "random" | "timer"> => {
  let now = Date.now();

  return {
    now: () => now,
    random: () => 0.5,
    timer: (ms, onFire) => {
      const handle = setTimeout(() => {
        now += ms;
        onFire();
      }, 0);

      return () => {
        clearTimeout(handle);
      };
    },
  };
};

The samples call outbound.raw.get, which returns the response body untouched, because an empty body is not a campaign. Retries work the same on every read. Both options are also how you test your own code against a failing vendor: see test without a key.

What is retried

import { createOutbound, OutboundError } from "outbound-sdk";
import { instantly } from "outbound-sdk/instantly";

import { fakeVendor } from "./fake-vendor.ts";
import type { Answer } from "./fake-vendor.ts";
import { instantClock } from "./instant-clock.ts";

const attempt = async (label: string, answer: Answer): Promise<void> => {
  const vendor = fakeVendor(answer);
  const delays: number[] = [];

  const outbound = createOutbound({
    adapter: instantly({ apiKey: "example-key" }),
    fetch: vendor.fetch,
    hooks: {
      onRetry: (event) => {
        delays.push(event.delayMs);
      },
    },
    runtime: instantClock(),
  });

  try {
    await outbound.raw.get("/campaigns");
  } catch (error) {
    if (OutboundError.is(error)) {
      console.log(
        `${label}: ${vendor.count()} request(s), ${error.code},`,
        `permanent ${error.permanent}, waits`,
        delays
      );
    }
  }
};

// Retried: the default is four retries after the first attempt.
await attempt("500", 500);

await attempt("503", 503);

await attempt("408", 408);

await attempt("no connection", "offline");

await attempt("429 with Retry-After: 20", {
  headers: { "retry-after": "20" },
  status: 429,
});

// Never retried.
await attempt("400", 400);

await attempt("401", 401);

await attempt("403", 403);

await attempt("404", 404);

await attempt("422", 422);

500: 5 request(s), Provider, permanent false, waits [ 500, 1000, 2000, 4000 ]
503: 5 request(s), Provider, permanent false, waits [ 500, 1000, 2000, 4000 ]
408: 5 request(s), Provider, permanent false, waits [ 500, 1000, 2000, 4000 ]
no connection: 5 request(s), Provider, permanent false, waits [ 500, 1000, 2000, 4000 ]
429 with Retry-After: 20: 5 request(s), RateLimited, permanent false, waits [ 20000, 20000, 20000, 20000 ]
400: 1 request(s), Invalid, permanent true, waits []
401: 1 request(s), Unauthorized, permanent true, waits []
403: 1 request(s), Unauthorized, permanent true, waits []
404: 1 request(s), NotFound, permanent true, waits []
422: 1 request(s), Invalid, permanent true, waits []

With the default of four retries, a failing call is five requests. A walk with listAll retries each page on its own.

How long it waits

The wait before retry number n is a random time between 0 and the smaller of 60 seconds and 1 second doubled n - 1 times. That is exponential backoff with full jitter: the ceiling doubles each time, and the randomness keeps many clients that failed together from retrying together.

So four retries add at most 15 seconds of waiting, and half that on average.

When the vendor says how long to wait, with a Retry-After header or a field in its response body, the SDK waits at least that long. The 429 in the output above asked for 20 seconds and got it each time. A 429 also slows the instance down for a while afterwards: see rate limits.

The five options

OperationOptions. Each can be set on createOutbound as the default for every call, and again on any one call.

import { createOutbound, OutboundError } from "outbound-sdk";
import type { OutboundOptions } from "outbound-sdk";
import { instantly } from "outbound-sdk/instantly";

import { fakeVendor } from "./fake-vendor.ts";
import type { Answer } from "./fake-vendor.ts";
import { instantClock } from "./instant-clock.ts";

const attempt = async (
  label: string,
  answer: Answer,
  options: Partial<OutboundOptions>
): Promise<void> => {
  const vendor = fakeVendor(answer);

  const outbound = createOutbound({
    adapter: instantly({ apiKey: "example-key" }),
    fetch: vendor.fetch,
    runtime: instantClock(),
    ...options,
  });

  try {
    await outbound.raw.get("/campaigns");
  } catch (error) {
    if (OutboundError.is(error)) {
      console.log(`${label}: ${vendor.count()} request(s), ${error.code}`);
      console.log(`  ${error.message}`);
      console.log(
        `  timedOut ${error.timedOut}, aborted ${error.aborted}, permanent ${error.permanent}`
      );
    }
  }
};

// The vendor never answers. Each attempt is cut off after `timeout`.
await attempt("timeout", "hang", { retries: 1, timeout: 5000 });

// 500 ms of backoff fits in the deadline. The next 1,000 ms does not.
await attempt("deadline", 503, { deadline: 1200 });

// The third request is never sent.
await attempt("maxRequests", 503, { maxRequests: 2 });

// No retries at all.
await attempt("retries: 0", 503, { retries: 0 });

// A signal that is already aborted sends nothing.
await attempt("signal", 503, { signal: AbortSignal.abort() });

timeout: 2 request(s), Provider
  instantly did not answer within 5000 ms
  timedOut true, aborted false, permanent false
deadline: 2 request(s), Provider
  instantly responded 503
  timedOut true, aborted false, permanent false
maxRequests: 2 request(s), RateLimited
  The call used its budget of 2 request(s) to instantly
  timedOut false, aborted false, permanent false
retries: 0: 1 request(s), Provider
  instantly responded 503
  timedOut false, aborted false, permanent false
signal: 0 request(s), Provider
  The call was aborted
  timedOut false, aborted true, permanent true

timeout

How long one attempt may take. An attempt that gets no answer in time is cut off and counts as a failure, which is retried. The clock restarts for each attempt, so timeout: 5000 with four retries can take 25 seconds plus the backoff. When the last attempt also times out, the error is Provider with timedOut: true.

deadline

How long the whole call may take. The SDK checks it before each wait: if the next backoff, or the wait for the rate budget, would end after the deadline, it stops there and throws instead of waiting.

• Stopped before a retry, the error is the last failure (Provider, or RateLimited after a 429) with timedOut: true.
• Stopped while waiting for the rate budget, the error is RateLimited with timedOut: true.

A deadline does not cut off an attempt already in flight. That is timeout. Set both: deadline for how long you are willing to wait in all, timeout for how long one request may hang. On a walk with listAll, the deadline covers the whole walk from its first page.

maxRequests

The most vendor requests one call may send, retries included. When the count is reached the next request is not sent, and the call fails with RateLimited and the message shown above. It exists so one question cannot spend a vendor quota: an insight that reads every campaign, or a retry storm.

On a walk with listAll and on getMany, it covers all the pages or records together. meta.requests on a result is what a call spent, and outbound.estimate is what it would spend.

signal

An AbortSignal stops a call at once: the request in flight is cancelled, a backoff or a rate-budget wait ends, and nothing is retried. The error is Provider with aborted: true and permanent: true. A signal that is already aborted sends no request at all.

Use AbortSignal.timeout(ms) for a hard wall-clock limit that does cut off an attempt in flight, which deadline does not.

retries

A number is how many retries follow the first attempt. 0 turns retries off. An object is a policy:

fullJitterBackoff is exported, so a policy can build on it. A vendor’s Retry-After still wins when it is longer than what your backoff returns.

On the constructor or on one call

import { createOutbound, fullJitterBackoff, OutboundError } from "outbound-sdk";
import { instantly } from "outbound-sdk/instantly";

import { fakeVendor } from "./fake-vendor.ts";
import { instantClock } from "./instant-clock.ts";

const vendor = fakeVendor(503);
const waits: number[] = [];

// Defaults for every call on the instance.
const outbound = createOutbound({
  adapter: instantly({ apiKey: "example-key" }),
  fetch: vendor.fetch,
  hooks: {
    onRetry: (event) => {
      waits.push(event.delayMs);
    },
  },
  retries: 2,
  runtime: instantClock(),
  timeout: 10_000,
});

const run = async <Result>(
  label: string,
  call: () => Promise<Result>
): Promise<void> => {
  const before = vendor.count();

  waits.length = 0;

  try {
    await call();
  } catch (error) {
    if (OutboundError.is(error)) {
      console.log(`${label}: ${vendor.count() - before} request(s), waits`, [
        ...waits,
      ]);
    }
  }
};

await run("constructor default", () => outbound.raw.get("/campaigns"));

// One call can set its own.
await run("retries: 0", () => outbound.raw.get("/campaigns", { retries: 0 }));

// A policy: how many retries, and how long to wait before each.
await run("fixed 250 ms", () =>
  outbound.raw.get("/campaigns", {
    retries: { backoff: () => 250, max: 3 },
  })
);

// The default backoff, for a policy that only changes `max`.
console.log(
  "default backoff at random 0.5:",
  [1, 2, 3, 4, 5, 6, 7, 8]
    .map((attempt) => fullJitterBackoff(attempt, 0.5))
    .join(", ")
);

constructor default: 3 request(s), waits [ 500, 1000 ]
retries: 0: 1 request(s), waits []
fixed 250 ms: 4 request(s), waits [ 250, 250, 250 ]
default backoff at random 0.5: 500, 1000, 2000, 4000, 8000, 16000, 30000, 30000

A value on a call replaces the instance’s default for that call. The others keep their defaults. Every method that reads takes these options: on list and listAll they sit beside the filters, on get and the handle methods they are the last argument.

Watching it happen

A retried call that succeeds looks like any other to the caller. To see retries, use the hooks: onRetry fires before each wait with the attempt, the delay and the reason, and onRequest fires for each attempt with its status. meta.requests on the result counts the attempts too.

What to do when an error reaches you

By then the SDK has retried what could be retried. error.permanent says whether trying again later could work:

• permanent: false (Provider, RateLimited): the vendor is down, slow or limiting. Try again later, after error.retryAfterMs when it is set. The failover plugin can serve the last good answer meanwhile.
• permanent: true: the same call will fail the same way. Fix the key, the id or the input.

See errors for every field.