OpenAI-compatible API: switching in three lines

Three lines. You point the client's base_url at the new provider, swap the API key, and replace the model name with a tier name. Everything else in a chat completions call, the messages array, streaming, tools, JSON schemas, stays exactly as it is, because OpenAI-compatible means implementing the same HTTP contract rather than merely resembling it.

The interesting part of a migration is not the code. It is what does not port: the endpoints that do not exist yet, the model behaviour that shifts under the same prompt, and the one error code you have probably never handled.

What "OpenAI-compatible" actually means

It means one endpoint's contract: POST /v1/chat/completions, with the same request body and the same response shape, including the server-sent events format for streaming. That contract became the industry's default interface the way S3's API became the default for object storage, and for the same reason: everything was already written against it.

The official SDKs are thin clients over that contract. In OpenAI's Python library the base URL is a constructor argument, so pointing it somewhere else is a supported operation rather than a hack. Your retry logic, your streaming handler and your type definitions all keep working.

What it does not mean is feature parity across the whole platform. Compatible services implement the chat endpoint well and diverge after that, which is why the honest version of this post is mostly a list of divergences.

from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["UNION_API_KEY"],
    base_url="https://api.europeanailab.eu/v1",
)
 
response = client.chat.completions.create(
    model="eu-standard",
    messages=[{"role": "user", "content": prompt}],
    stream=True,
)

The three lines, in practice

Here is the whole change in Python. The Node, Go and .NET SDKs take the same two arguments under different names.

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["UNION_API_KEY"],
    base_url="https://api.europeanailab.eu/v1",
)

response = client.chat.completions.create(
    model="eu-standard",
    messages=[{"role": "user", "content": prompt}],
    stream=True,
)

If your code already reads the base URL from an environment variable, which is worth doing before any migration, this is a config change and not a deploy of new code. That also gives you the cheapest possible rollback: point it back.

What ports, what changes, what is missing

The table you actually want before you plan the work.

Feature	Status	What to do
Chat completions	Works unchanged	Nothing
Streaming (SSE)	Works unchanged	Ask for usage in the stream to keep metering exact
Function calling and tools	Same shapes	Re-test which tool the model picks, not whether it can
Structured output (JSON schema)	Same shapes	Keep validating on your side, always
System prompts	Work	Expect to retune wording, not structure
Model names	Changes	Use a tier: eu-lite, eu-standard, eu-long, eu-pro
Usage and cost	Changes	Read the response headers instead of a dashboard
Embeddings, image, speech	Missing	Keep your current provider for these
Batch API	Missing	On the roadmap, not shipped
Assistants-style stateful API	Missing	Not planned; keep state on your side

Three of those rows are absences. We publish them as a list on the homepage rather than letting you discover them mid-migration, because "compatible" is a word that has been stretched thin by services implementing half the contract and calling it done.

Why model names become tier names

You buy a tier, not a model. eu-standard is the workhorse, eu-long takes a million tokens of context, eu-pro is the heavy reasoner, and each maps to a named open-weight model published with its licence on the AI transparency page.

The reason is churn. Open-weight models are replaced every few months, and hard-coding today's model name into your application means a redeploy every time the field moves. A tier is a promise about capability and price that survives the model underneath being swapped, and when we swap it, we say so rather than quietly changing what your code does.

The licences matter too, and they are short enough to read. Most of what we run is Apache 2.0 or MIT, which is why the same weights can legally be served from European hardware at all.

What changes in the answers, not in the API

This is the part that surprises teams, so budget for it. A provider switch is a model change, and models differ in ways no compatibility layer can hide.

Formatting drifts. A prompt tuned to make one model produce clean markdown may produce a preamble on another. Usually a one-line fix, but it is a fix.

Tool-calling judgement differs. The shape of a function call is identical; the model's willingness to call it is not. Models that call too eagerly and models that answer from memory both exist.

Refusals differ. Some open-weight models duck factual questions about politics or history. We treat that as a defect and reroute once to a model without that blind spot, and tell you in the X-EU-Routed header when we did. That is described in full on the AI transparency page.

The practical answer is a test set. Keep twenty real requests with expected outputs, run them against both providers, and read the diffs yourself. Twenty is enough to catch formatting and tool-choice regressions, and small enough that you will actually do it.

Cost and metering after the switch

Per-token pricing is the same idea everywhere, but where you read the numbers changes. Every response carries what the request cost:

• X-EU-Tokens-In and X-EU-Tokens-Out, the metered tokens.
• X-EU-Cost-EUR, what came off your balance for this request.
• X-EU-Balance-EUR, what is left.
• X-EU-Model-Tier, which tier actually ran it.

That means your own logs can answer the cost question per feature, per customer, per endpoint, without waiting for a monthly invoice to disagree with you. The rates behind those numbers are on the pricing page, and how to think about token cost in general, including why output is dearer than input, is worked through in what an AI API costs per token.

One habit worth carrying across: set max_tokens on every call. It is the single most effective cost control in any per-token API, and it is one line.

Errors, limits and the 402 you have not handled

Your error handling probably covers 429 and 500. Prepaid credit adds one more.

1. 402, insufficient balance. The request is refused with a top-up link, nothing is invoiced afterwards, and your balance never goes below zero. Handle it like an auth error: alert, do not retry in a loop.
2. 429, rate limit or ceiling. Respect Retry-After. Ceilings are yours to set, per key, in euros per day and per month.
3. 502 and 503, upstream unavailable. Retry with backoff. What happens during an outage, and the single-provider limitation behind it, is set out on the status page.
4. Failed requests are free. If no usable answer came back, nothing comes off your balance, and a request rerouted to a weaker model is charged at the lower of the two rates.

The new-account ceiling catches people once: €25 a day for the first 48 hours, whatever your balance. It exists because a leaked key on day one is the most expensive mistake a new integration makes.

Does compatibility actually make you portable?

Partly, and it is worth being precise about which part. The wire contract is portable: any service speaking chat completions can be swapped in behind the same client. That alone removes the crudest form of lock-in, the one where leaving means rewriting your integration layer.

What is not portable is everything you built on top of a specific model. Prompts tuned over months, few-shot examples chosen for one model's quirks, evaluation sets scored against one model's output style, and any threshold you set by watching that model behave. None of it transfers cleanly, and none of it is visible in an architecture diagram.

So treat the model as a dependency with a version, not as a constant. Keep prompts in files rather than scattered through the code, keep the test set in the repo, and write down why a threshold is what it is. Teams that do this switch providers in an afternoon. Teams that do not spend a quarter discovering what their prompts were compensating for.

A migration checklist

Half a day for a small service, a sprint for something with real traffic.

1. Move the base URL into config if it is not already. Everything below is easier once this is true.
2. Pick a tier by workload, not by name. Classification and extraction belong on the cheap tier; only reasoning-heavy work needs the expensive one.
3. Build the test set. Twenty real requests, expected outputs, committed to the repo.
4. Run both providers side by side on a small share of traffic and diff the results.
5. Wire up the cost headers into whatever you already log.
6. Add the 402 path, and a ceiling per key while you are there.
7. Keep the old provider configured for a fortnight. A migration you can undo in one config change is a migration you can do on a Tuesday.

Which provider to run that evaluation against is its own question, and the four realistic options are compared in European AI providers compared. The first ten requests are free and the minimum top-up is €10, so the evaluation costs less than the meeting about whether to do it. The full contract, with copyable curl and Python, is in the API documentation, and where those requests are processed, hop by hop, is in where your prompts go.