---
title: "Configure Fallback Models in AI Gateway | Hostman Docs"
description: "Set up fallback models for AI Gateway API keys. Automatically reroute requests to backup models on errors, timeouts, and rate limits."
---

> For the complete documentation index for AI agents, see [llms.txt](https://hostman.com/llms.txt).

Fallback models let AI Gateway automatically reroute a request to another language model if the primary model returns an error or doesn't respond in time.

You can select up to two fallback models for each AI Gateway API key and set the order in which they're called.

The primary model is defined in each request by the model field. In the key settings, you select only the fallback models. The same chain applies to all requests made with that key.

Fallback models must be text models. Local models aren't supported, and you can't add the same model to the chain twice.

## Add Fallback Models

1.  Go to **AI Services** → **AI Gateway** → **API Keys** and click the key you want to configure.
2.  Open the **Management** tab.
3.  In the **Fallback Models** row, click **Select**. If fallback models are already configured, click **Change**.
4.  In the **Model 2** field, select the first fallback model.
5.  (Optional) In the **Model 3** field, select the second fallback model. 
6.  Click **Save**.

On the **Management** tab, you will now see the selected chain.

## Edit Fallback Models

1.  Go to **AI Services** → **AI Gateway** → **API Keys** and click the key you want to configure.
2.  Open the **Management** tab.
3.  In the **Fallback Models** row, click **Change**.
4.  Select different models in the **Model 2** and **Model 3** fields.
5.  Click **Save**.

## Remove a Fallback Model

1.  Go to **AI Services** → **AI Gateway** → **API Keys** and click the key you want to configure.
2.  Open the **Management** tab.
3.  In the **Fallback Models** row, click **Change**.
4.  Click the cross in the field of the model you want to remove. If you remove **Model 2**, the model selected in **Model 3** moves to **Model 2**.
5.  Click **Save**.

## Disable Fallback

1.  Go to **AI Services** → **AI Gateway** → **API Keys** and click the key you want to configure.
2.  Open the **Management** tab.
3.  In the **Fallback Models** row, click **Change**.
4.  Clear both the **Model 2** and **Model 3** fields.
5.  Click **Save**.

On the **Management** tab, the **Fallback Models** row now shows **Not selected**, and AI Gateway no longer switches models automatically.

## How Fallback Works

AI Gateway processes each request in the following order:

1.  Sends the request to the primary model specified in the model field.
2.  If the primary model doesn't respond successfully, sends the request to the first fallback model (Model 2).
3.  If the first fallback model also returns an error, sends the request to Model 3, if selected.
4.  If no model responds successfully, returns an error.

If a fallback model is the same as the primary model of the request, AI Gateway skips it and moves to the next one. Each fallback model is called at most once per request.

### When Fallback Is Triggered

AI Gateway switches to the next model when:

-   The model provider is unavailable or returns a 5xx error.
-   The request times out.
-   The provider rate-limits the request (429 error).
-   The request exceeds the model's context window.
-   The provider rejects the request as invalid (400 error).

### When Fallback Is Not Triggered

AI Gateway doesn't switch to the next model when:

-   The model returns a successful response, including a refusal. For example, "I can't discuss this topic" isn't treated as an error.
-   The key's limit for the requested model is exhausted. AI Gateway checks the limit before calling the model, so the API returns a 422 error even if the fallback model's limit isn't exhausted. To continue, increase the limit for the requested model or specify a different model in the request.
-   A streaming response has already started. Once the first chunk arrives, AI Gateway doesn't switch models.

## Supported Requests

Fallback works for requests to `/v1/chat/completions` and `/v1/responses`.

Fallback isn't supported for:

-   Embeddings
-   Image generation
-   Speech generation and transcription
-   Anthropic-format requests to `/v1/messages`
-   The legacy `/completions` endpoint

## Billing

A request is billed at the rate of the model that generated the response. A failed attempt by the primary model isn't billed and doesn't count toward its limit.

Token usage counts toward the limit of the model that responded and toward the key's overall limit.

## Identify the Model That Responded

1.  In the API response, check the `model` field. It contains the name of the model that actually processed the request.
2.  Check the `x-hostman-attempted-fallbacks` HTTP header. A value greater than zero means AI Gateway switched to a fallback model. The value indicates that a switch occurred, not how many steps of the chain were used.

For example, if you send a request to `gemini/gemini-3.1-flash-lite`, the response may come from the fallback model Claude 4.5 Haiku. Here's a shortened example of the response:

```js
HTTP/1.1 200 OK
x-hostman-attempted-fallbacks: 1

{
  "model": "claude-haiku-4-5-20251001",
  "choices": [
    {
      "message": {
        "content": "Yes"
      }
    }
  ]
}
```