---

🤔 What is this?

dsh-plugin-vision is a plugin for DeepSeek Harness that enables pure-text LLMs to analyze images.

When your underlying language model doesn't support native image input (e.g., DeepSeek-V3 text-only), this plugin bridges the gap by calling Zhipu AI's GLM-4V-Flash — a free vision model — to analyze images and return text descriptions.

User pastes image → Agent calls vision_analyze → GLM-4V-Flash analyzes → Text description returned

✨ Features

• 🖼️ Analyze local image files (PNG/JPEG/WebP/GIF)
• 🌐 Analyze HTTP(S) URL images
• 🔤 Automatic base64 encoding for local files
• 🆓 Uses Zhipu's free vision model — no payment required
• 🔌 Follows DSH Capability Seam architecture
• 🧩 Drop-in Cordis plugin — works with cordis.yml

📦 Installation

Option 1: Clone and link

git clone https://github.com/co-Elly/dsh-plugin-vision.git
cd dsh-plugin-vision
pnpm install
pnpm build

Then in your DSH project's cordis.yml:

plugins:
  - name: /path/to/dsh-plugin-vision/dist/cordis.js
    config:
      apiKey: "your-zhipu-api-key"

Option 2: npm (coming soon)

pnpm add dsh-plugin-vision

⚙️ Configuration

Get a free API Key

1. Visit Zhipu Open Platform
2. Register and log in
3. Create an API Key in the API Keys page
4. The glm-4v-flash model is free — no payment needed

Environment variables

export ZHIPU_API_KEY="your-api-key"

cordis.yml

plugins:
  - name: dsh-plugin-vision/cordis
    config:
      apiKey: "your-api-key"           # or use $ZHIPU_API_KEY
      baseUrl: "https://api.z.ai/api/paas/v4/chat/completions"  # default
      model: "glm-4v-flash"            # default

🚀 Usage

Once registered, the agent automatically gets the vision_analyze tool:

vision_analyze({
  image_path: "/path/to/screenshot.png",
  question: "What is shown in this chart?"
})

Parameters

Parameter	Type	Required	Description
image_path	string	✅	Image file path or HTTP(S) URL
question	string	❌	Question about the image (default: "请描述这张图片的内容")

Returns

{
  "description": "A bar chart showing monthly revenue..."
}

🏗️ Architecture

This plugin follows DSH's Capability Seam pattern:

┌─────────────────────────────────────────────────┐
│                  dsh-plugin-vision               │
├─────────────┬─────────────────┬─────────────────┤
│  vision.ts  │  vision-glm.ts  │  tool-vision.ts │
│  (Service   │  (Provider:     │  (Consumer:     │
│  Definition)│  GLM-4V-Flash)  │  vision_analyze)│
├─────────────┼─────────────────┼─────────────────┤
│  Defines    │  Calls Zhipu    │  Exposes tool   │
│  ctx.vision │  API to analyze │  to the model   │
│  interface  │  images         │                 │
└─────────────┴─────────────────┴─────────────────┘

Files

File	Role	Description
src/vision.ts	Service Definition	Types and interfaces for the vision capability
src/vision-glm.ts	Provider	GLM-4V-Flash implementation (Zhipu AI)
src/tool-vision.ts	Consumer	vision_analyze tool registration
src/cordis.ts	Plugin Entry	Cordis plugin that wires everything together
src/index.ts	Exports	Public API exports

🗺️ Roadmap

• Core vision capability (Service Definition / Provider / Consumer)
• GLM-4V-Flash provider (free)
• Local file support (base64 encoding)
• HTTP(S) URL support
• Cordis plugin integration
• OpenAI-compatible provider (GPT-4o, Claude 3, etc.)
• Local vision model support (moondream2, LLaVA)
• Batch image analysis
• Image language detection and auto-prompting
• Video frame extraction and analysis
• OCR-specific provider with layout analysis

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

📄 License

MIT

---