dsh-vision

English | 中文

A DeepSeek Harness profile bundle that gives text-only models (DeepSeek V4 Flash, DeepSeek V4 Pro, and any other route whose inputModalities omit image) working vision.

When a request contains image attachments and the target model cannot see them, the plugin describes each image and replaces the image blocks with that text for the provider call only. The session log and the UI keep the original images.

Requirements

• A DeepSeek Harness installation with the dsh CLI available on your PATH.
• For vision-model description: at least one already-added model that declares image in its inputModalities.
• For the OCR fallback: macOS (Vision framework) and Windows (Windows.Media.Ocr) need nothing extra; other platforms need Tesseract installed and on the PATH.

Behaviour

1. Auto (default) — walk already-added models that declare image input, in provider registration order, and use the first one that succeeds. If none are available, fall back to system OCR.
2. Specified model — use only the vision model chosen in Settings. Failures are not retried and never fall back to another model or OCR.
3. Auto + no vision model — system OCR:
   • macOS: Vision framework
   • Windows: Windows.Media.Ocr
   • any platform: Tesseract if installed

Descriptions are cached per attachment + model (or OCR) for the life of the process so later turns do not re-pay the vision call.

Settings

Open Settings → 视觉识别:

• Auto — first available vision model, then system OCR.
• 指定识图模型 — pick one already-added image-capable model. No fallback.

The same values live in $DSH_HOME/settings.yaml under dsh-vision:.

Install

From this directory:

dsh plugin --profile web add .

Restart dsh web so the new bundle layer and the client settings page load.

To remove:

dsh plugin --profile web remove dsh-vision

How it works

The fix has two layers:

1. Declare capability — on startup the host row augments the current text-only model's inputModalities with image (by wrapping ctx.llm.resolveModelInfo). The session.prompt / session.selectModel image-admission gates therefore stop rejecting image messages with "model does not support images", and the read_image tool becomes available to text-only models too.
2. Rewrite — the host row listens on the llm/stream waterfall. When a rewrite is needed the plugin builds a new request (images replaced by descriptions) and calls ctx.llm.stream again. The nested call targets a vision-capable model, so the interceptor lets it through. Frozen agent-loop requests are never mutated.

Native image support is decided with the pre-wrap resolver, so natively image-capable models are never double-described; only models that declare inputModalities as text-only (or omit it) gain the bridged image capability.

Limitations

• Only models that declare image in inputModalities are offered. A gateway that can see images but does not say so will not be selected.
• Specified-model mode fails closed: a missing, text-only, or errored model is not replaced.
• System OCR is text extraction, not a scene description.
• Settings RPCs are loopback-only, same as the rest of the configuration surface.
• Because image is declared, a session that already contains images can switch back to a text-only model; the historical images are described on the next llm/stream pass.

License

MIT