budget

Cap the vendor requests one call or one whole run may spend, read what is left, and see what a call looks like when it hits the cap.

budget({ maxRequests }) from outbound-sdk/budget caps vendor requests, so one question from an agent or one runaway loop cannot spend a vendor’s whole rate limit. The cap is enforced inside the request engine, request by request. A call that reaches it stops with RateLimited. Retries count, because each one is a vendor request. List it last, so it sits next to the engine and counts only requests that reach the vendor.

Sample

The in-memory adapter makes no vendor request, so a budget has nothing to count on it. These samples run Instantly’s adapter on a fetch that serves two pages of mailboxes from fixtures/instantly/ in the repository. Those files are the vendor’s documented payloads, built from its published schema. They are not live responses.

// A `fetch` that answers Instantly's "list accounts" request with the two
// pages in fixtures/instantly/. Those files are built from Instantly's
// published schema and example values. They are not captured responses.
import { readFile } from "node:fs/promises";

const FIXTURES = new URL("../../../../fixtures/instantly/", import.meta.url);

const page = (name: string): Promise<string> =>
  readFile(new URL(name, FIXTURES), "utf-8");

/** Every request this fetch has answered, as `METHOD path`. */
export const seen: string[] = [];

export const documentedFetch: typeof fetch = async (input) => {
  const url = new URL(input instanceof Request ? input.url : String(input));

  seen.push(`GET ${url.pathname}`);

  const body = await page(
    url.searchParams.has("starting_after")
      ? "listAccounts.page2.json"
      : "listAccounts.json"
  );

  return new Response(body, {
    headers: { "content-type": "application/json" },
  });
};

A budget for the whole run:

import { createOutbound, OutboundError } from "outbound-sdk";
import { budget } from "outbound-sdk/budget";
import { instantly } from "outbound-sdk/instantly";

import { documentedFetch, seen } from "./documented-fetch.ts";

// The in-memory adapter makes no vendor requests, so there is nothing for a
// budget to count. This runs Instantly's adapter on a fetch that serves the
// vendor's documented payloads from disk. Each page is one request.
const outbound = createOutbound({
  adapter: instantly({ workspaceId: "acme" }),
  fetch: documentedFetch,
  // Three requests for the whole run, whatever the calls are.
  plugins: [budget({ maxRequests: 3, per: "run" })],
});

const all = await outbound.senders.listAll().toArray();

console.log(`listed ${all.length} mailboxes over two pages`);

console.log("used:", outbound.requestBudget.used());

console.log("remaining:", outbound.requestBudget.remaining());

// Two more pages do not fit in what is left.
try {
  await outbound.senders.listAll().toArray();
} catch (error) {
  if (!OutboundError.is(error)) {
    throw error;
  }

  console.log(`${error.code}: ${error.message}`);
}

console.log("used:", outbound.requestBudget.used());

// Nothing is left, so the next call stops before it reaches the vendor.
try {
  await outbound.senders.list();
} catch (error) {
  if (!OutboundError.is(error)) {
    throw error;
  }

  console.log(`${error.code}: ${error.message}`);
}

console.log("requests that reached the fetch:", seen.length);

outbound.requestBudget.reset();

console.log("after reset:", outbound.requestBudget.remaining());

listed 3 mailboxes over two pages
used: 2
remaining: 1
RateLimited: The run used its budget of 3 request(s) to instantly
used: 3
RateLimited: The run used its budget of 3 request(s) to instantly
requests that reached the fetch: 3
after reset: 3

The first walk spent two requests. The second walk got its first page with the third, then stopped before the fourth. The last call was refused without a request: three requests reached the fetch in total.

Options

One SDK call is one list, one get, one stats, or one whole listAll walk. A walk over ten pages is one call.

import { createOutbound, OutboundError } from "outbound-sdk";
import { budget } from "outbound-sdk/budget";
import { instantly } from "outbound-sdk/instantly";

import { documentedFetch } from "./documented-fetch.ts";

const outbound = createOutbound({
  adapter: instantly({ workspaceId: "acme" }),
  fetch: documentedFetch,
  // `per` defaults to "call": every SDK call gets one request of its own.
  plugins: [budget({ maxRequests: 1 })],
});

const first = await outbound.senders.list();

console.log(
  "one page:",
  first.items.length,
  "mailboxes,",
  first.meta.requests,
  "request"
);

const second = await outbound.senders.list({ cursor: first.cursor });

console.log("next page, a new call:", second.items.length, "mailbox");

// One walk over both pages is one call, and needs two requests.
try {
  await outbound.senders.listAll().toArray();
} catch (error) {
  if (!OutboundError.is(error)) {
    throw error;
  }

  console.log(`${error.code}: ${error.message}`);
}

// Constructing the plugin with a cap that is not a positive integer fails.
try {
  budget({ maxRequests: 0 });
} catch (error) {
  if (!OutboundError.is(error)) {
    throw error;
  }

  console.log(`${error.code}: ${error.message}`);
}

one page: 2 mailboxes, 1 request
next page, a new call: 1 mailbox
RateLimited: The call used its budget of 1 request(s) to instantly
Invalid: budget() needs maxRequests to be a positive integer

What it adds

The counter runs in both modes, but it limits anything only with per: "run". With per: "call", used() is a running total that nothing compares with the cap.

The count lives in the plugin object, in this process. outbound.workspace(id) shares it with the instance it came from. Two processes each have their own.

What it changes

It changes no capability and nothing in meta. It lowers the request cap of each call it sees. A tighter maxRequests passed to the call itself, or set on the instance, still applies: the lower of the two wins.

per: "call" has the same effect as maxRequests on createOutbound. The plugin is the way to get a run-wide cap and a counter.

With other plugins

• Inside cache, a cache hit never reaches the budget, so a run with nothing left can still be served from the cache. Listed before cache, a spent run budget refuses a read the cache could have answered.
• clients and other plugins that make their own reads through the chain spend from the same call’s allowance.
• failover does not cover a budget stop. The cap is yours, not a vendor failure, so a call stopped by it fails with RateLimited even when failover’s store holds an answer for that read. See failover.
• A per-run budget is not the vendor’s rate limit. The rate limiter still paces every request.

When it fails

The error has no retryAfterMs: no vendor asked for a wait. error.permanent is false, as for any RateLimited.