Gemini API Fallback for 429 Errors with Groq in Node.js

A Gemini API fallback becomes important the first time a working chatbot hits a rate limit, times out, or receives a temporary provider error. A single model call can look reliable in local testing and still fail when real visitors arrive, a quota changes, or an upstream service becomes overloaded.

I recently added this pattern to the Ask Mustafa assistant on my portfolio. Gemini is the primary provider, Groq is the automatic fallback, and the server returns which provider answered so I can verify the production path. If the Gemini key is removed later, the same routing code continues with Groq without a frontend change.

This guide explains the practical decisions behind that implementation: which failures should be retried, when a request should move to the second provider, how to keep prompts and conversation history consistent, how to protect keys in Netlify Functions, and how to test failover without waiting for a real outage.

Why a Gemini API fallback is different from a retry

A retry sends the request to Gemini again. A fallback sends it to another provider. Those actions solves different problems, and treating them as the same thing can make an outage worse.

Google documents 429 RESOURCE_EXHAUSTED for rate limits and recommends exponential backoff for retryable errors such as 429, 408, and 5xx responses. A short retry can help when capacity is temporarily unavailable. It does not help when a project has exhausted its daily allowance, billing is blocked, the selected model is unavailable to that project, or the request keeps exceeding the provider timeout.

A user-facing chatbot also has a latency budget. Four long retries may eventually succeed, but a visitor will experience the assistant as broken. I prefer a small, explicit timeout and a limited retry policy, followed by a second provider when the error is genuinely recoverable.

Classify the status before switching providers

• 429: retry briefly when the response indicates a temporary limit, then fall back if the user cannot wait.
• 408, 500, 502, 503, or 504: treat as temporary network or upstream failures. A bounded retry or fallback is reasonable.
• 400: fix the payload, model parameters, or conversation format. Sending the same invalid request to another provider may hide a bug.
• 401 or 403: investigate credentials, project access, billing, or policy restrictions. Do not rotate through providers indefinitely.
• Safety refusal: preserve the refusal. A fallback must never be used to bypass a provider safety decision.

The exact policy depend on the product. A background summary can wait and retry. A support widget needs a quick answer or an honest temporary error. The routing layer should know that difference.

Build one provider contract in Node.js

The frontend should not contain Gemini-specific or Groq-specific logic inside it. It should send one request to a server endpoint and receive one normalized answer. The server owns provider order, authentication, timeouts, error mapping, and observability.

I use a small provider contract with the same inputs for both services: a system prompt, the current question, recent conversation history, environment variables, and a fetch implementation. Each adapter translates those shared inputs into the provider's request format and returns plain answer text.

async function generateAnswer({ systemPrompt, question, history, env, fetchImpl }) {
  const order = env.AI_PRIMARY_PROVIDER === "groq"
    ? ["groq", "gemini"]
    : ["gemini", "groq"];

  const enabled = order.filter((provider) =>
    provider === "gemini" ? env.GEMINI_API_KEY : env.GROQ_API_KEY
  );

  let lastError;
  for (const provider of enabled) {
    try {
      const answer = provider === "gemini"
        ? await callGemini({ systemPrompt, question, history, env, fetchImpl })
        : await callGroq({ systemPrompt, question, history, env, fetchImpl });
      return { answer, provider };
    } catch (error) {
      lastError = normalizeProviderError(provider, error);
      logSafeFailure(lastError.provider, lastError.code);
    }
  }

  throw lastError;
}

This loop is intentionally boring. Reliability code should be easy to read during an incident. Adding a third provider later should mean adding one adapter and one configuration entry, not rewriting the chatbot.

Keep provider differences inside the adapters

Gemini and Groq do not use identical request bodies. Gemini represents conversation content as user and model roles with parts. Groq exposes an OpenAI-compatible chat-completions shape. Their token controls, model names, error bodies, and response structures also differ.

The adapter should absorb those differences. The rest of the application should not depend on candidates[0].content.parts or choices[0].message.content. It should receive a validated non-empty string or a normalized error.

That boundary also protects future migrations. A provider can change a model name or SDK without forcing a change to the browser, prompt builder, or knowledge-selection code.

Preserve behaviour when the provider changes

Successful failover is not only an HTTP 200 response. The fallback answer must still follow the product rules.

For Ask Mustafa, both providers receive the same verified portfolio context, system instructions, user question, and recent history. The prompt blocks fabricated prices, availability, client results, and unsupported experience. Scope Doctor mode has a different structured prompt, but it follows the same provider route.

I keep temperature and output limits conservative, validate the returned text, and apply a final server-side length cap. The fallback should not suddenly produce a long essay when the primary normally returns a short answer.

There is still some model differences. Wording, refusal behaviour, and formatting can change. For important workflows, build a small evaluation set and run it against every enabled provider. A fallback that returns technically valid but misleading output is not reliable.

Secure Gemini and Groq keys in Netlify Functions

API keys belong in the server runtime, never in browser JavaScript, HTML, a public repository, or a generated static bundle. Netlify documents that a variable must include the Functions scope before a serverless function can read it at runtime.

I configure GEMINI_API_KEY and GROQ_API_KEY as secrets for the required deployment contexts. The provider module reads them through process.env. The frontend only calls /.netlify/functions/ask-mustafa; it never sees either key.

Environment configuration needs a deployment check. A variable can exist in dashboard but be missing from the active production function because its scope or deployment context is wrong. After changing secrets, deploy the function and make one controlled production request. Verify the returned provider and inspect the function log without printing the secret.

Log codes, not confidential data

A useful log entry can contain the provider name, normalized error code, request identifier, duration, and whether fallback succeeded. It does not need the API key, authorization header, full system prompt, visitor message, or raw customer documents.

For example, gemini GEMINI_UPSTREAM_429 is enough to start a quota investigation. A separate metric can count fallback activations and latency. If Groq begins answering a large share of traffic, that is an operational signal even when visitors still receive successful responses.

Test the fallback without causing an outage

Provider routing is easy to claim and easy to leave untested. I pass fetchImpl into the adapters so automated tests can supply deterministic responses without calling either real API.

The essential test returns a mocked 429 from Gemini and a valid Groq response. The assertion checks the answer and confirms that the reported provider is Groq. A second test removes the Gemini key entirely and confirms Groq-only mode. Another test confirms Gemini remains first when both keys are available.

mockFetch
  .replyOnce(429, {
    error: { message: "Resource exhausted" }
  })
  .replyOnce(200, {
    choices: [{ message: { content: "Fallback answer" } }]
  });

const result = await generateAnswer(testRequest);

assert.equal(result.provider, "groq");
assert.equal(result.answer, "Fallback answer");

Also test timeouts, empty responses, malformed JSON, both providers failing, and no providers configured. Confirm that a failure returns a controlled 502 or 503 response rather than a stack trace. Rate limiting on your own endpoint should remain separate from upstream provider limits.

After local tests pass, make one production request and confirm the primary provider. Do not repeatedly force failures against a live API just to demonstrate fallback. Mocked failure tests are safer and more repeatable.

Operational risks a fallback does not solve

A second provider improves continuity, but it is not zero downtime. Both provider can fail, the serverless function can time out, DNS can break, and a shared application bug can make both payloads invalid.

• Latency stacking: a nine-second primary timeout followed by a nine-second fallback is too slow. Set a total user-facing budget.
• Cost drift: fallback traffic may use a different pricing model. Track provider usage separately.
• Output drift: compare safety, structure, language quality, and factual behaviour across providers.
• Capacity correlation: two providers can depend on overlapping infrastructure or experience demand spikes at the same time.
• Hidden failures: silent fallback can make the system look healthy while the primary stays broken for days.

Use alerts for sustained fallback rate, not every isolated error. A single transient 503 may be noise. A twenty-minute shift from Gemini to Groq deserves investigation.

Practical takeaway

A reliable Gemini API fallback starts with a narrow provider interface, explicit error classification, short timeouts, safe logs, and automated failure tests. Retry temporary Gemini errors briefly, move to Groq when the user-facing request cannot wait, and preserve the same product rules across both models.

Keep keys in Netlify Functions, return the provider in internal verification data, and monitor how often fallback activates. Most importantly, test Gemini 429 to Groq success before production rather than assuming the loop works.

Use the official Gemini troubleshooting guide, Gemini rate-limit documentation, Groq rate-limit documentation, Groq error reference, and Netlify Functions environment-variable guide for current implementation details.

If you need a production AI assistant with secure provider routing, tested fallbacks and honest failure handling, contact me to discuss the architecture.