dsh-llm-auto-route

dsh-llm-auto-route is a community Cordis plugin for DeepSeek Harness. It discovers which already-configured dsh-llm-pi-ai route should handle a request, explains the decision, and performs failover only before the first visible output.

It is deliberately a routing policy layer. It does not implement HTTP protocols, ship provider SDKs, or register openai, anthropic, deepseek, or other adapter routes. The official @deepseek-ai/dsh-llm-pi-ai plugin remains the owner of those routes.

Community project: this package is not an official DeepSeek Harness package and does not imply DeepSeek endorsement.

Requirements

• Node.js >=22.19.0
• DeepSeek Harness 0.1.0-rc.5 or a compatible 0.1.x release
• @deepseek-ai/dsh-llm-pi-ai configured with the route names you want to select

The package is tested against the npm 0.1.0-rc.6 companion packages while keeping the peer range compatible with rc.5.

Install

pnpm add dsh-llm-auto-route

The official base bundle already contains @deepseek-ai/dsh-llm-pi-ai. If you compose plugins manually, install and load that official adapter before this package. Load the shipped cordis.patch.yml with the normal DeepSeek Harness/Cordis composition command used by your deployment.

The patch adds one plugin named llm-auto-route; it does not add or replace any official adapter route.

Configure the official adapter first

The route names in this plugin must already be registered by dsh-llm-pi-ai. The following is an abbreviated official-adapter configuration:

- id: llm
  name: '@deepseek-ai/dsh-llm-pi-ai'
  config:
    providers:
      deepseek:
        apiKeyEnv: DEEPSEEK_API_KEY
      openai:
        apiKeyEnv: OPENAI_API_KEY
      anthropic:
        apiKeyEnv: ANTHROPIC_API_KEY
      ollama:
        baseURL: http://127.0.0.1:11434/v1
        api: openai-completions
        models:
          - id: llama3.1
            contextWindow: 131072
            maxTokens: 8192
      vllm:
        baseURL: http://127.0.0.1:8000/v1
        api: openai-completions
        models:
          - id: local-model
            contextWindow: 32768
            maxTokens: 4096
      openai-compatible:
        apiKeyEnv: GATEWAY_API_KEY
        baseURL: https://gateway.example.test/v1
        api: openai-completions
        models:
          - id: gateway-model
            contextWindow: 65536
            maxTokens: 8192

dsh-llm-pi-ai owns credentials, model metadata, transport, and stream conversion. The auto-route plugin only sees the route directory and selects one of those route keys.

Automatic selection

An automatic request uses provider: auto, or omits the provider when the plugin configuration's provider token is auto:

const options = {
  provider: 'auto',
  model: 'deepseek-chat',
  messages,
}

The fixed default precedence is:

explicit → provider_env → base_url → model_prefix

• A non-auto provider is always preserved. Explicit requests are never silently rerouted.
• DEEPSEEK_API_KEY, OPENAI_API_KEY, and ANTHROPIC_API_KEY are detection signals only; the key value is never written to a log.
• LLM_BASE_URL selects the generic openai-compatible rule. 11434 identifies Ollama and 8000 identifies a vLLM/OpenAI-compatible local endpoint.
• deepseek-*, gpt-*, o1-*, o3-*, and claude-* provide model-prefix hints.
• A candidate is usable only when the official adapter has registered the same route.
• Equal-priority candidates at the same stage return AMBIGUOUS_ROUTE; the plugin does not guess.
• A route's defaultModel is used only when a matching rule supplies one and the request omits its model.

Every selection can be explained without exposing credentials:

已选择 deepseek/deepseek-chat；原因：发现 DEEPSEEK_API_KEY

Use an explicit provider or a route priority when an environment intentionally contains multiple credentials.

Configuration

The shipped patch contains the default rules. An application can override them through its Cordis configuration:

provider: auto
precedence:
  - explicit
  - provider_env
  - base_url
  - model_prefix
healthCheck:
  mode: adaptive       # off | adaptive | probe
  timeoutMs: 3000
  cacheTtlMs: 30000
failover:
  enabled: true
  maxAttempts: 3
diagnostics: info      # silent | error | info
routes:
  deepseek:
    apiKeyEnv: DEEPSEEK_API_KEY
    defaultModel: deepseek-chat
    priority: 10
  openai-compatible:
    apiKeyEnv: GATEWAY_API_KEY
    baseURLEnv: LLM_BASE_URL
    modelPrefixes: [gateway-]

Route fields are hints, not adapter configuration:

Field	Meaning
provider	Registered route id to return; defaults to the routes key.
apiKeyEnv	Non-empty environment variable used as a provider-env signal and for discovery.
baseURL / baseURLEnv	Exact or user-supplied endpoint hint. baseURLEnv also supports unknown OpenAI-compatible hosts.
baseURLPatterns	Additional normalized URL prefixes.
ports	Local ports that identify this route when a base URL is supplied.
modelPrefixes	Model id prefixes for automatic selection.
defaultModel	Model to use when the request does not name one.
priority	Tie breaker within one matching stage; higher wins.

Do not put an API key directly in route configuration. Use the official adapter's credential reference, normally apiKeyEnv, and keep this plugin's apiKeyEnv aligned with it.

Health checks and failover

adaptive health checks reuse the official dsh-llm model-discovery seam when it is available, with a bounded timeout and an in-memory cache. If a deployment cannot expose discovery, route/model resolution is used as the local fallback and the real stream remains the final availability check. off skips preflight checks. probe asks the discovery seam whenever the adapter exposes it.

Failover is intentionally conservative:

• it is allowed only before text, reasoning, tool-call, or block output has been emitted;
• buffered protocol metadata from a failed attempt is discarded before trying the next route;
• aborts, explicit providers, configuration errors, and requests that already emitted output are not retried;
• a later provider never receives a partial assistant response from an earlier provider.

The plugin observes agent/request, agent/request-error, and llm/stream. It returns a new immutable request configuration and never mutates a frozen request object.

Public API

The package exports AutoRouteConfig, RouteRule, RouteDecision, MatchStage, and the pure helpers normalizeConfig, normalizeBaseURL, and resolveRoute:

import { normalizeConfig, resolveRoute } from 'dsh-llm-auto-route'

const decision = resolveRoute(normalizeConfig(), {
  model: 'deepseek-chat',
  env: { DEEPSEEK_API_KEY: 'present' },
  registeredProviders: new Set(['deepseek']),
})

if (decision.kind === 'matched') {
  console.log(decision.candidate.provider, decision.candidate.model, decision.stage)
}

AutoRouteError.code is stable for AMBIGUOUS_ROUTE, MISSING_MODEL, NO_CANDIDATE, and NO_REGISTERED_ROUTE.

Troubleshooting

NO_REGISTERED_ROUTE — the route exists in this plugin's default hints but not in ctx.llm.listProviders(). Add the same key under the official adapter's providers configuration.

AMBIGUOUS_ROUTE — more than one candidate matched at the same stage and priority. Set provider explicitly, remove an unused environment variable, or give one route a higher priority.

MISSING_CREDENTIAL / INVALID_CREDENTIAL — detection and the official adapter must agree on the environment variable. Check the variable name, not its value in logs or issue reports.

Local gateway is not selected — set LLM_BASE_URL, or provide a URL containing port 11434 (Ollama) or 8000 (vLLM). The official adapter still needs a route with that key and a model catalog.

A response was not retried — this is expected after the first text, reasoning, tool-call, or block output, after cancellation, for an explicit provider, or for a configuration failure. Retrying then could duplicate or splice an assistant response.

Compatibility and project status

This is an independent ecosystem plugin for the DeepSeek Harness developer preview. It follows the current upstream guidance for community plugins, dsh-plugin topics, Discussions announcements, and standalone repositories. It does not open a pull request against the official Harness repository.

Upstream references:

• dsh-llm-pi-ai README
• LLM interface
• Streaming and routing extension points
• Contribution guide

License

MIT. See LICENSE.