dsh-read-image

简体中文

Plug-and-play image reading for text-only DeepSeek Harness models: pasted images are admitted, projected as [Image #N], and read back through a first-class read_image tool backed by a configurable vision model — no preset changes required.

A DeepSeek Harness plugin that lets non-multimodal models "see" images.

• Pasted images are no longer rejected — text-only routes are declared to accept image input, so the api-proxy admission gate lets them through.
• [Image #N] projection — on text-only routes, image blocks in the model request are replaced in place with [Image #N] text; pixels never reach a text API. Native multimodal routes pass through untouched.
• First-class read_image tool (registered automatically in every session, shadowing the built-in tool of the same name):
  • image_index — read the Nth image in the conversation ([Image #N]);
  • file_path — read an image file from a path (PNG/JPEG/WebP/GIF);
  • prompt / reasoning_effort / timeout_ms / max_tokens / max_thinking_tokens — optional overrides; omitted parameters use the configured defaults (the current real defaults are interpolated into the tool description, so the agent never has to guess);
  • on text-only routes the configured vision model converts the image into a text description; on native multimodal routes the image itself is returned;
  • stateless and repeatable, including re-reading the same image.
  • Native multimodal routes fall back to the built-in tool: when the session's base route itself declares image input (e.g. mimo-v2.5), the custom tool is not registered and no [Image #N] prompt section is injected — the model sees real image blocks and uses the harness's built-in read_image (file_path only, "return the image itself").
• Visual configuration — a "Read image" section in the settings panel (gear icon in the sidebar) edits the vision model and defaults; the browser talks to the host over a typert Remote bridge (readImageConfig), working around the harness's settings allowlist for plugin namespaces.

Installation

Requires DeepSeek Harness (dsh) 0.1.0-rc.6 or later (the harness is in developer preview; a newer release candidate may need a compatibility pass).

Recommended: dsh plugin add

dsh plugin --profile web add github:OoWJZZoO/dsh-read-image

Then restart dsh web. This package ships a cordis.patch.yml via its dsh.bundle manifest, so the profile bundle mechanism composes the plugin row automatically — no manual patch editing.

Manual install

Add the package to the web profile and register the plugin row yourself:

// ~/.dsh/profiles/web/package.json → dependencies
"@deepseek-ai/dsh-read-image": "github:OoWJZZoO/dsh-read-image#v0.1.0"

cd ~/.dsh/profiles/web && pnpm install

Insert one row into ~/.dsh/profiles/web/cordis.patch.yml:

- insert:
    - id: read-image
      name: '@deepseek-ai/dsh-read-image'
      config: {}   # optional: deployment defaults (user layer can override)

Then restart dsh web.

Do not combine the two paths: dsh plugin add already composes the plugin row, so adding the dependency manually and inserting the row would register the plugin twice.

Configuration

Configuration lives in $DSH_HOME/settings.yaml (hot-reloaded, no restart) and can also be edited in Web at Settings → Read image (writes to the user layer, overriding the corresponding settings.yaml keys):

# 1) Declare image input for your vision model (pi-ai route)
llm-pi-ai:
  providers:
    <your-provider>:
      models:
        - id: <your-vision-model>
          # …contextWindow / maxTokens…
          input: [text, image]      # ← required

# 2) This plugin's configuration
dsh-read-image:
  visionProvider: <your-provider>   # the route provider hosting the vision model
  visionModel: <your-vision-model>  # the multimodal model id (must declare input: [text, image])

Key	Default	Description
visionProvider	''	Route provider of the vision model (dropdown fed by the Models page)
visionModel	''	The multimodal model that performs read_image tasks (dropdown under that provider)
defaultPrompt	English step-by-step description prompt (classify → verbatim text to Markdown / visual description)	Used when read_image is called without prompt
defaultReasoningEffort	low	Default reasoning intensity. low is the lowest tier widely accepted and honored; on many adapters off is equivalent to "omit the field" and does not disable thinking on models that default it on, so thinking eats into max_tokens and small budgets come back empty
defaultTimeoutMs	300000	Default vision-call timeout (5 minutes, generous)
defaultMaxThinkingTokens	4096	Default max thinking tokens (separate budget, not counted against output); when thinking exceeds the budget and the output comes back empty, read_image reports an explicit error instead of silently returning nothing
defaultMaxTokens	8192	Default actual output token cap; the API max_tokens sent is this + defaultMaxThinkingTokens (budget is 0 with reasoning_effort=off, passed through unchanged)
guard.enabled	true	Environment self-check switch; false skips the self-check and force-loads the plugin (see Safety)

Why not the settings protocol? The harness allowlists WEB_SETTINGS_NAMESPACES, so a plugin settings.register() namespace only answers settings-not-exposed to the browser. The "Read image" page therefore talks to a host-side typert Remote bridge (readImageConfig.get/set, see lib/config-remote.js) instead — settings.yaml stays the base layer, Web writes go to the user layer, scope.watch hot-syncs the runtime, and headless and Web stay consistent.

Usage

• Paste an image: the text model sees [Image #1]; call read_image image_index=1 to view it.
• read_image file_path=/path/to/image.png reads an image file.
• Pass prompt (a specific question for the vision model), reasoning_effort, timeout_ms, max_tokens, max_thinking_tokens to override the defaults.
• Call it repeatedly, including re-reading the same image.

Safety

This plugin rides on harness internals that can change shape between releases. On startup it runs an environment self-check over every harness contract it touches. If any check fails, the plugin fails safe: it loads nothing and the harness boots normally — the full diagnostics are written to ~/.dsh/logs/dsh-read-image-guard.log and one short notice is logged. To force-load anyway (at your own risk): guard.enabled: false under the dsh-read-image section of settings.yaml.

How it works

Mechanism	Layer	What it does
M1 admission	host	Wraps llm.resolveModelInfo so text-only routes report image input (restored on dispose) → send/switch-model gates let images through
M2 projection	host	Synchronous llm/stream waterfall listener: on text-only routes, image blocks become [Image #N] text, then re-dispatches the harness-deepFreezed request
M3 tool	per-agent	Registers read_image into the agent's own scope at session/created (shadowing the built-in tool) plus a prompt section
Settings	host	Registers the dsh-read-image settings namespace (schema defaults = fallback), hot-syncs runtime config via scope.watch
Config bridge	host	ReadImageConfigService (typert Remote, readImageConfig.get/set) reads/writes the same namespace; auto-discovered by the gateway
Config page	browser	settings.section registers the "Read image" page (order 12); ctx.remote.$mount self-mounts remote.readImageConfig
G1 self-check	host + browser	Total fuse: probes every harness contract at apply; any failure → plugin loads nothing, harness unaffected (see Safety)

• Capability truth table: built from the original resolveModelInfo (never the M1-wrapped one), rebuilt on llm/adapters-updated with startup retries;
• Vision calls reuse ctx.llm (same credentials / retries / logging as the harness), with AbortSignal.any([exec.signal, timeout]) for timeouts;
• Session logs are the single source of truth: image references persist as usual, and only the model-visible boundary performs the [Image #N] replacement.

Development

Contributor guide and the full list of harness-workaround hacks (and the reasons they exist) live in AGENTS.md and docs/known-hacks.md. Tests: node --test test/ (run from the dev-profile copy — see the test file headers).

License

MIT