When Our AI Provider Retired the Model

In July 2026 several features stopped working at once: AI chat, AI replies in tickets, ticket summaries, the server builder, plus moderation and translation of ideas and bug reports.

The cause was not our code. Our AI provider had retired the model we were using.

Part one: the model is gone

That is the obvious part, and it was fixed quickly. A provider discontinues a model, you enter the successor, done.

The actual incident started afterwards.

Part two: empty replies, without errors

After the swap, replies came back. Only sometimes they were empty. Not truncated, not malformed, but a completely empty reply with no error whatsoever.

The cause lies in a difference the model name does not reveal: the successor is a reasoning model. Before the actual answer it first writes an internal train of thought into a separate field. Both draw from the same token budget.

If the budget is used up by the thinking, a reply comes back that is formally fine: the provider reports "length reached" and the content field is empty. No error, no hint, nothing a try/catch would see.

We reproduced it with a real test: a budget of five tokens is already enough to produce an empty reply on every request.

Part three: the bill

At the most expensive call site, the AI server builder, we had set a generous budget because long answers are produced there. So the thinking did not hit a limit, it simply ran.

The consequences arrived from two directions at once. Requests took so long they ran into timeouts. And they cost several times more, because a complete invisible train of thought was paid for every time. At the end came an error message we had not planned for: insufficient balance.

The fix is one line. Thinking can be disabled in the request body, and the model then behaves like the old one again. Since then that line belongs at every call site, and there are more than a handful in the repository.

Part four: what we found while cleaning up

If you are touching every AI call site anyway, you look at them. That surfaced a second topic, unrelated to the incident but similarly expensive.

Providers charge considerably less for a prompt whose beginning stays unchanged and can be served from a cache. The difference between a hit and a miss is roughly thirtyfold.

You do not get that discount through a flag. We tested it specifically: a field in the request body meant for it had zero effect, the cache applies automatically anyway. The only lever is the order within the prompt, from stable to volatile.

Four places where we had done exactly the opposite:

A volatile value inside the fixed instruction block. The duplicate check for bug reports had the module name in the instruction text. That gave an otherwise identical block its own cache line per module. The value belongs in the variable part, where it changes anyway.

The new thing before the stable thing. The same check sent the new bug report before the up to fifty comparison candidates. That made the API's largest prompt a permanent complete miss.

Per-user data in the fixed part. Public AI chat had the notes about the individual user in the instruction block, so before the shared channel history. That gave every user in the same channel their own beginning, and the shared history was paid for separately per user.

A sliding window over the history. Sending the last N messages on every message shifts the beginning on every message. The complete history is then a permanent miss. The right approach is trimming in blocks, so rarely and substantially.

Plus one detail that looks like nitpicking and is not: a list inside a prompt sorted by only one criterion can change its order on ties. A single swapped line in the middle of the instruction block costs the hit for everything behind it. Sorts inside prompts need an unambiguous tiebreaker.

What we took from it

A model swap is not a configuration change. The name looks like a value in a variable. Behind it sat different behaviour, and the behaviour broke our code, not the interface.

The dangerous failure was the silent one. The old model going away was immediately visible and fixed in minutes. The empty replies came back as formally valid responses and could have stayed that way indefinitely.

Measurability belongs at the call site. We now log the cache hit rate at every one. A correctly built site settles at a high rate after the first request, a badly built one stays at zero. You see that in a log line and nowhere else, because both of them work.

And the uncomfortable conclusion: using somebody else's model means part of your product depends on a decision made elsewhere. That is why, since August, any server can point the AI features at its own endpoint instead of relying on ours.