What happens The application has no notion of reasoning models. ChatCompletionRequest carries model, messages, temperature, topP, n, stream, stop, maxTokens, presencePenalty, frequencyPenalty, logitBias, streamOptions and nothing else, OpenAIChatModel.PATH is chat/completions, and a grep for reasoning / effort / max_completion across the models module returns no hits. Why it matters On the current OpenAI reasoning tiers, reasoning.effort defaults to medium and reasoning tokens are billed at the output rate. Because the effort level cannot be set, every request pays for default-effort reasoning with no way to turn it down, and an effort of none – the setting that would make a cheap reasoning tier behave like a cheap non-reasoning model – is unreachable. For RAG chat this is the wrong default: the task is extracting and summarising text the application already supplies in the context prompt, which needs little reasoning. Concretely, at roughly 1250 input and 350 visible output tokens per question, a tier priced at 0.20 USD per 1M input and 1.20 USD per 1M output costs about 0.67 USD per 1000 questions with no reasoning, but overtakes a 0.40 / 1.60 USD non-reasoning model at roughly 325 reasoning tokens per answer – which default effort clears easily. So picking the cheapest-looking model can cost more, and administrators have no lever. Suggested fix Pass reasoning configuration through to the provider, and surface effort as a field on AI.Models.Code.ModelsClass alongside contextSize, so it can be set per model. Longer term, supporting the Responses API would also give access to reasoning.mode. Related, unverified The application sends max_tokens. Newer reasoning models have generally moved to max_completion_tokens, so that may be a second incompatibility on the same model class – I have not confirmed it against a specific model. See also LLMAI-170, which is the temperature restriction on this same class of models. |
