Repository navigation
Description
Summary
Organization-managed GPT models load all MCP tool definitions upfront, even when tool search is enabled and the model uses the Responses API.
The same underlying model defers tools when accessed through GitHub’s hosted offering or a directly configured external provider using Responses.
This becomes expensive with large tool definitions: a separate organization-managed Sol test consumed 236,905 actual input tokens for a simple “Reply OK” prompt that required no tools.
Environment
- Copilot CLI 1.0.110, macOS arm64.
- Custom model configured centrally through GitHub organization settings, not local BYOK.
- Selected model:
<org>/<key-name>/gpt-6-luna. - Organization configuration:
- Responses API
- Tool calling, Streaming, Vision and Thinking enabled.
- CLI setting:
"toolSearch": true. - MCP server setting:
"deferTools": "auto".
Reproduction
- Configure an organization-managed GPT model using Responses.
- Connect enough MCP tools to exceed the tool-search threshold. Our test used 48 synthetic MCP tools.
- Start a fresh interactive CLI conversation and select the organization-managed model.
- Ask it to call one known synthetic read-only tool.
- Inspect the
turn tool surface resolveddiagnostic. - Repeat with hosted and directly configured models.
Results
Same tool inventory and underlying GPT‑6 Luna model:
| Connection | API | Total tools | Deferred tools |
|---|---|---|---|
| Organization-managed | Responses | 49 | 0 |
| GitHub-hosted | Responses | 49 | 49 |
| Direct external Gateway | Responses | 49 | 49 |
The total includes one native function. All three cases successfully called the requested tool and returned its result.
The organization-managed case reports:
{
"tool_search_enabled": true,
"model_tool_calls_allowed": true,
"opted_out": false,
"client_path": false,
"builtin_path": false,
"generic_path": false,
"deferred_tool_count": 0
}Expected behavior / question
Organization-managed models should support on-demand tool loading where the model and endpoint support it, or clearly explain why tool search is unavailable despite being enabled.
Could the organization-qualified model ID or catalog family: "custom" be preventing selection of the Responses tool-search path? Is there a supported way for an organization administrator to enable this capability?
Related
- GitHub-managed custom model with Vision enabled is exposed as Tools-only and rejects image attachments microsoft/vscode#323699 — Capability propagation problems on the organization-managed model route; concerns Vision, not this exact behavior.
- High-severity 1.0.80+ regression: MCP schemas are eagerly injected, adding 354K startup tokens #4613 — Earlier eager-loading regression, marked fixed in 1.0.105. This organization-managed case reproduces on 1.0.110.
Activity