Text Generation and Embeddings

Last updated July 24, 2026

We recommend using the OpenAI SDK to work with AI Gateway, it removes the need to send raw HTTP requests and makes integration much simpler.

Available SDKs:

The full list of SDKs is available in the OpenAI repository.

To follow this guide, you'll need to create an AI Gateway API key. 

The examples below use Python and the openai library. Install it with pip:

pip install openai

Sending a Request
Copy link

Use the Chat Completions method to send messages. Messages are passed in the messages array.

Each message contains:

  • role: the sender's role (user, assistant, system)
  • content: the message text
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://ai-api.hostman.com/v1"
)

response = client.chat.completions.create(
    model="MODEL_NAME",
    messages=[
        {
            "role": "system",
            "content": "Answer briefly and to the point.",
        },
        {
            "role": "user",
            "content": "Explain what Kubernetes is",
        },
    ],
)

print(response.choices[0].message.content)

Parameters:

  • api_key: your AI Gateway API key. Replace this with your own key.
  • base_url: the base URL for connecting to AI Gateway.
  • model: the name of the model you want to use.
  • messages: an array of messages with roles and text.

Sending a Request With Message History
Copy link

To preserve conversation context, pass previous messages in the messages array:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://ai-api.hostman.com/v1"
)

response = client.chat.completions.create(
    model="MODEL_NAME",
    messages=[
        {
            "role": "system",
            "content": "Reply only in short phrases.",
        },
        {
            "role": "user",
            "content": "What's 2 + 5?",
        },
        {
            "role": "assistant",
            "content": "7",
        },
        {
            "role": "user",
            "content": "Now multiply the result by 2",
        },
    ],
)

print(response.choices[0].message.content)

In this example, previous messages (assistant and user) are included to preserve the conversation context.

Sending a Request (Responses API)
Copy link

The Responses API is a newer way to work with models. It simplifies the request structure and doesn't require building a messages array explicitly.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://ai-api.hostman.com/v1"
)

response = client.responses.create(
    model="MODEL_NAME",
    instructions="Answer briefly and to the point.",
    input="Explain what Kubernetes is"
)

print(response.output_text)

Parameters:

  • model: the name of the model you want to use
  • instructions: instructions for the model (similar to a system prompt)
  • input: the request text

Sending a Request With Message History (Responses API)
Copy link

To preserve conversation context with the Responses API, pass previous_response_id, which is the ID of the previous response.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://ai-api.hostman.com/v1"
)

response = client.responses.create(
    model="MODEL_NAME",
    instructions="Reply only in short phrases.",
    input="What's 2 + 5?"
)

next_response = client.responses.create(
    model="MODEL_NAME",
    instructions="Reply only in short phrases.",
    previous_response_id=response.id,
    input="Now multiply the result by 2"
)

print(next_response.output_text)

In this example, the first request returns a response object containing a unique id. This id is passed as previous_response_id in the next request, letting you continue the conversation without sending the full message history again.

The model parameter must be specified in every request, including follow up calls that use previous_response_id.

Listing Available Models
Copy link

AI Gateway lets you retrieve a list of available models:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://ai-api.hostman.com/v1"
)

models = client.models.list()

for model in models.data:
    print(model.id)

The models.list() method returns a list of models you can use in the model parameter.

Using Embeddings
Copy link

Embeddings convert text into a vector representation. This is useful for semantic search, clustering, or RAG.

AI Gateway provides the openai/text-embedding-3-large model for creating embeddings.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://ai-api.hostman.com/v1"
)

response = client.embeddings.create(
    model="openai/text-embedding-3-large",
    input="Text to vectorize",
)

print(response.data[0].embedding)

The method returns a vector representation of the input text.