Build a rate-limit-aware API client

Northwind’s scripts call a handful of third-party APIs, and every one of them rate-limits. The naive approach — a bare UrlFetchApp.fetch — works until the API returns a 429 Too Many Requests, at which point the whole script throws and the job dies halfway through. A transient 503 does the same.

This automation is a drop-in replacement for UrlFetchApp.fetch that handles the failures worth retrying. It distinguishes a permanent error (a 404, a bad request) from a temporary one (a 429, a 500), honours the API’s Retry-After header when there is one, and otherwise backs off exponentially. Use it anywhere a script talks to an API you do not control.

What you’ll need

• An Apps Script project that makes outbound HTTP calls with UrlFetchApp.
• The base URL and any headers for the API you are calling — this client wraps the request, it does not replace your endpoint logic.
• Nothing else — there is no Sheet, no key, and no setup beyond pasting the function in.

The script

fetchWithRetry takes the same arguments as UrlFetchApp.fetch, plus a cap on attempts. It returns the successful response or throws once retries run out.

// Hard ceiling on a single back-off wait, in milliseconds.
const MAX_BACKOFF_MS = 60_000;

// Base delay for exponential back-off, in milliseconds.
const BASE_BACKOFF_MS = 500;

// HTTP status returned when an API is rate-limiting the caller.
const STATUS_RATE_LIMITED = 429;

/**
 * Fetches a URL, retrying on rate-limit (429) and server (5xx) errors.
 * Permanent client errors (other 4xx) throw immediately — retrying them
 * would only waste attempts.
 *
 * @param {string} url The URL to fetch.
 * @param {Object} [options] UrlFetchApp options (method, headers, payload).
 * @param {number} [maxAttempts=5] How many times to try before giving up.
 * @returns {HTTPResponse} The successful response.
 */
function fetchWithRetry(url, options = {}, maxAttempts = 5) {
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    // muteHttpExceptions lets us inspect the status instead of throwing.
    const res = UrlFetchApp.fetch(url, { ...options, muteHttpExceptions: true });
    const code = res.getResponseCode();

    // 1. Success — anything below 400 is good enough to return.
    if (code < 400) return res;

    // 2. Permanent failure — a 4xx that is not a 429 will not improve.
    if (code !== STATUS_RATE_LIMITED && code < 500) {
      throw new Error('Permanent: ' + code + ' ' + res.getContentText());
    }

    // 3. Retryable failure (429 or 5xx). Prefer the server's own
    //    Retry-After hint; fall back to capped exponential back-off.
    const retryAfter = parseInt(res.getHeaders()['Retry-After'] || '0', 10);
    const wait = retryAfter * 1000
      || Math.min(MAX_BACKOFF_MS, BASE_BACKOFF_MS * Math.pow(2, attempt));

    Logger.log(`Attempt ${attempt} got ${code}; waiting ${wait}ms before retry.`);
    Utilities.sleep(wait);
  }

  // 4. Out of attempts.
  throw new Error('Exhausted retries after ' + maxAttempts + ' attempts');
}

How it works

1. fetchWithRetry loops up to maxAttempts times. Each pass makes one real request with muteHttpExceptions: true, so a bad status returns a response to inspect instead of throwing.
2. Any status below 400 is treated as success and returned straight away.
3. A 4xx that is not 429 is a permanent error — a 404, a malformed request. Retrying it would only burn attempts, so the script throws immediately with the status and body.
4. A 429 or any 5xx is worth retrying. The script first looks for a Retry-After header, which many APIs send to say exactly how long to wait.
5. If there is no Retry-After, it falls back to exponential back-off — BASE_BACKOFF_MS * 2^attempt — capped at MAX_BACKOFF_MS so a long run of failures never sleeps for more than a minute.
6. It sleeps with Utilities.sleep, logs the attempt, and loops.
7. If every attempt fails, it throws once retries are exhausted, so the caller still finds out something went wrong.

Example run

Calling an API that is briefly rate-limiting:

const res = fetchWithRetry('https://api.example.com/v1/orders', {
  method: 'get',
  headers: { Authorization: 'Bearer ...' },
});
Logger.log(res.getContentText());

A run where the API rate-limits twice, then recovers, logs:

Attempt 1 got 429; waiting 2000ms before retry.
Attempt 2 got 429; waiting 4000ms before retry.

The third attempt returns 200 and fetchWithRetry hands back the response. A permanent failure looks different — fetchWithRetry to a missing endpoint throws straight away:

Error: Permanent: 404 {"error":"not found"}

Run it

This is a utility, not a standalone job — you call it from your own code in place of UrlFetchApp.fetch:

1. Paste fetchWithRetry and its config constants into the project that makes API calls.
2. Replace each UrlFetchApp.fetch(url, options) with fetchWithRetry(url, options).
3. Keep your own try/catch around the call to handle the thrown errors — permanent failures and exhausted retries both surface as exceptions.

Watch out for

• Utilities.sleep counts against the runtime limit. Apps Script caps a single execution at about six minutes. A long chain of back-offs can eat that budget — keep maxAttempts and MAX_BACKOFF_MS modest.
• UrlFetchApp has its own daily quota. Consumer accounts get roughly 20,000 calls a day; retries count toward it. A script that retries hard can burn through the quota faster than you expect.
• Not every 5xx is transient. A persistent 500 from a broken endpoint will retry the full maxAttempts and then throw. The retries cost time and quota for no gain — watch the logs for repeated server errors.
• Retry-After can be a date, not seconds. The HTTP spec allows an HTTP-date value. parseInt will read that as 0 and fall through to exponential back-off. That is safe, but parse the date form if an API uses it.
• This retries idempotent calls safely; writes are riskier. Retrying a POST that already succeeded server-side but failed to respond can create a duplicate. For non-idempotent requests, send an idempotency key if the API supports one.