Model Selection

Every turn, q15 decides which model to call. The decision is not random and not hardcoded — it is a two-stage process of capability filtering followed by deterministic fallback.

How it works

flowchart TD
    Request[User message arrives] --> Infer[Infer required capabilities]
    Infer --> Filter[Filter models by capability match]
    Filter --> HasMatch{Any models match?}
    HasMatch -->|Yes| Try[Try models in configured order]
    HasMatch -->|No| Fallback[Fall back to text-only models]
    Try --> Call[Call first reachable model]
    Call --> Success{Response ok?}
    Success -->|Yes| Done[Return response]
    Success -->|No| Next{More models?}
    Next -->|Yes| Try
    Next -->|No| Error[Return error]
    Fallback --> Try

Capability inference

Before selecting a model, the agent infers what capabilities the request needs. This is not the model deciding — it is the agent harness inspecting the request context:

Capability	Inferred when
text	Always required (baseline)
tool_calling	Tools are available and the request may need them
reasoning	Complex multi-step task, or previous turns used reasoning
image_input	An image was loaded via load_image or attached by the user

The inferred requirements are matched against each model’s declared capabilities. Models that cannot satisfy the requirements are skipped.

Model capabilities

Each model in the config declares its capabilities:

models:
  - name: gpt-5.4
    provider: openai
    capabilities:
      - text
      - tool_calling
      - reasoning

  - name: kimi-k2.5
    provider: moonshot
    capabilities:
      - text
      - tool_calling
      - reasoning
      - image_input

Capabilities are declared by the operator based on what the model actually supports. The agent trusts the config — it does not probe models to verify capabilities at runtime.

Fallback ordering

After capability filtering, the remaining models are tried in the order specified in agent.models:

agent:
  models:
    - deepseek-v4-flash    # tried first
    - deepseek-v4-pro      # tried second if flash fails
    - kimi-k2.5            # tried third

The order is deterministic per turn. The agent does not load-balance or round-robin. It tries the first model, and if that call fails (network error, provider outage, model returns an error), it moves to the next.

This means model ordering is a policy decision:

• Put cheaper models first if you want to minimize cost for simple requests.
• Put more capable models first if you want the best response quality.
• Put local models first if you want to avoid cloud API calls when possible.

Skipped models

When a model is skipped, the reason is logged. Common skip reasons:

Reason	Meaning
Missing capability tool_calling	The request needs tools but this model does not support them
Missing capability image_input	An image is in context but this model cannot see images
Provider unreachable	The provider has no live models (all models from that provider are skipped)

Skipped models are not errors — they are expected behavior. The agent is designed to work with a heterogeneous model pool where not every model supports every capability.

Provider resolution

Models are grouped by provider. When a model is selected, the agent resolves its provider and uses the provider’s configured client:

providers:
  - name: openai
    type: openai-codex

  - name: ollama-cloud
    type: ollama
    base_url: https://ollama.com
    key_env: OLLAMA_API_KEY

Provider types determine how the agent authenticates and connects:

Type	Auth	Use case
openai-codex	auth.json (OAuth)	OpenAI Codex API
openai-compatible	key_env (API key)	Moonshot, any OpenAI-compatible endpoint
ollama	None (local) or key_env (cloud)	Local Ollama or Ollama Cloud

Cognition model selection

Background cognition jobs use a separate model list:

agent:
  models:
    - deepseek-v4-flash
  cognition:
    models:
      - deepseek-v4-pro
      - deepseek-v4-flash

When agent.cognition.models is set, cognition jobs use that list instead of agent.models. Individual jobs can further override with switch_cognition_model.

This separation matters because cognition jobs have different needs than interactive turns. Verification review benefits from a careful model (nemotron-3-ultra), while consolidation can use a fast one (deepseek-v4-flash).

See Cognition for the full background job system.

Model discovery

The agent can discover available models from configured providers at runtime. The discovery system queries each provider’s model list and builds a catalog of available models with their capabilities, context windows, and other metadata.

This catalog is used by the list_providers and list_models tools, which let the agent inspect what is available and switch models during a session.

Discovery happens at agent startup and periodically thereafter. The catalog is held in memory — it does not persist to disk.