dsh-media-tools

Image, video, and speech generation as deepseek-harness tools, backed by OpenRouter. Ships as an out-of-tree profile bundle — no harness fork, no patched core.

What it registers

Tool	Backing endpoint	Result reaches the model as
generate_image	POST /images	the image itself — committed to the attachment store, returned as an image block. Also written to outputDir so a person can open it.
generate_video	POST /videos + polling	a file path
generate_speech	POST /audio/speech	a file path, plus a play link
transcribe_audio	POST /audio/transcriptions	the transcript text — the only tool here that runs inward

That split is the whole design, and it is not arbitrary. The harness content vocabulary (ContentBlockMap) has text, reasoning, image, tool-call, and tool-result — no video, no audio — and a model route may declare only text and image input. So a generated image can re-enter model context and a generated video cannot. Adding a video or audio block would require changing the harness itself plus its adapter, UI, and compaction consumers, which no out-of-tree bundle can do.

The practical consequence: the model never sees the video or hears the audio. It learns the path and the byte count. The rendered tool result stays strictly factual for that reason — the model has no way to check any claim about how an artifact looks or sounds.

Generated images do appear inline in the Web UI, by a route worth understanding before you rely on it. No harness surface renders a tool-result image — the card shows the durable reference, not pixels — and the client's markdown renderer accepts images only from http(s), refusing data: URIs and file:// paths. So this bundle serves each written image on the harness web server behind an unguessable process-local token, and the tool result hands the model that URL with an instruction to include it in its reply.

That last step is the caveat: display depends on the model following that instruction. A model that ignores it leaves you with the file path and no picture. Every image is therefore also written to outputDir with a real extension, and revealImages: true opens each one in your desktop viewer as it is produced — those two paths do not depend on model behaviour.

Install

dsh plugin --profile <name> add @launchmaniac/dsh-media-tools
dsh --profile <name> --dump-config   # verify the layer without booting

That is the recommended route: the npm tarball ships lib/ already built, so nothing executes on your machine at install time and no build allowance is involved.

Installing from GitHub instead

A git install fetches sources, not built artifacts, so pnpm runs this package's prepare script to build lib/. pnpm ≥10 refuses to run a git dependency's build script until you allow it, so the first add fails with the package key it wants. Add it to your profile's pnpm-workspace.yaml:

allowBuilds:
  "@launchmaniac/dsh-media-tools": true

then re-run the add.

npm v12 gates git dependencies separately from build scripts — installing one also needs --allow-git, and a remote URL dependency needs --allow-remote. npm approve-scripts --allow-scripts-pending reviews what is waiting. This is another reason to prefer the npm install above: it is a plain registry dependency and trips none of these.

Treat that allowance as what it is: permission to execute this package's code on your machine at install time, outside any sandbox the agent runs under. Read the source first, and pin a commit so a later push cannot silently change what runs:

dsh plugin --profile <name> add github:launchmaniac/dsh-media-tools#<sha>

If you would rather not grant a build allowance, pnpm pack a tarball from a checkout and dsh plugin --profile <name> add ./launchmaniac-dsh-media-tools-0.1.0.tgz — prebuilt, so no build permission is involved.

Set your key:

export OPENROUTER_API_KEY=sk-or-...

The key is read through the harness credential seam (ctx.credentials) when one is mounted, falling back to the process environment. It is resolved per call and never cached, so a rotated key takes effect on the next tool invocation.

Configuration

Three places, in increasing precedence.

1. Per call, in the conversation. generate_image takes an optional model, so you can say "generate a logo with recraft/recraft-v4-vector" and it applies to that call only.

2. $DSH_HOME/settings.yaml under a media-tools key. This is the live one — a change takes effect on the next tool call, with no restart:

media-tools:
  imageModel: black-forest-labs/flux.2-pro

The Settings dialog's Open configuration file button reaches this file. The bundle registers its schema through installSettingsSection, so the values are validated on write. It does not yet appear as a card in Settings → Plugins: that needs a browser half declared with dsh.client, and the build preset for client bundles is not published outside the harness monorepo.

3. Your profile's cordis.patch.yml — the deployment default everything else layers over:

- id: media-tools
  name: "@launchmaniac/dsh-media-tools"
  config:
    imageModel: bytedance-seed/seedream-4.5
    videoModel: bytedance/seedance-2.5
    # A voice-requiring model must be given one, or every call 400s.
    speechModel: google/gemini-3.1-flash-tts-preview
    speechVoice: Kore
    outputDir: ./media
    videoTimeoutMs: 900000

Key	Default	Meaning
apiKeyEnv	OPENROUTER_API_KEY	Environment variable holding the key
baseURL	https://openrouter.ai/api/v1	API root
attributionTitle	deepseek-harness	Sent as X-Title
attributionUrl	—	Sent as HTTP-Referer
imageModel	bytedance-seed/seedream-4.5	Used by generate_image
videoModel	bytedance/seedance-2.5	Used by generate_video
speechModel	fish-audio/s2.1-pro	Used by generate_speech
speechVoice	—	Default voice, overridable per call
transcribeModel	openai/whisper-1	Used by transcribe_audio; deepgram/nova-3 and openai/gpt-4o-transcribe also work
maxTranscribeBytes	26214400	Largest audio file sent for transcription
outputDir	./media	Where artifacts are written. Relative paths resolve against the session's working directory, so each workspace keeps its own; an absolute path pools everything in one place
revealImages	false	Open each generated image in the desktop viewer
requestTimeoutMs	120000	Per-HTTP-request budget
videoPollIntervalMs	5000	Delay between video job polls
videoTimeoutMs	900000	Total wait for one video job

A patch replaces a row's entire config value rather than merging keys, so an override must restate every key it needs.

Requirements and refusals

generate_image refuses, rather than degrading, in two cases:

• No attachment store mounted. The tool is registered inside ctx.inject(['attachments'], …), so it does not exist without one. @deepseek-ai/dsh-attachment-local ships in @deepseek-ai/dsh-base, so a normal profile has it.
• The calling model does not declare image input. Generating an image the current route cannot carry would put a block into durable session history that the route cannot replay. The refusal names the model and tells you to switch.

Both checks run before any HTTP request, so a refusal never spends a call.

Audio in: transcription

transcribe_audio is the one tool that runs inward. No chat route accepts audio input — ModelModality is text and image only — so a transcript is how recorded speech reaches a model at all. Point it at a file and the model reads what was said.

Three models are served: openai/whisper-1 (default), deepgram/nova-3, and openai/gpt-4o-transcribe. None appear in the ordinary /models listing, the same way the TTS models do not.

This is turn-based, not a conversation. Speaking to the agent live would need microphone capture in the browser, which a plugin cannot reach — that is a harness-side change.

Text to speech: voices and containers

Two things about TTS are worth knowing before you configure it, both learned by running the endpoint rather than reading its docs:

• Most providers require an explicit voice, and voice names are not portable. Gemini TTS rejects a voiceless request and takes names like Kore or Puck (alloy returns a 500). Deepgram takes flux-*-en names and lists the valid set in its error. fish-audio/s2.1-pro synthesizes with no voice at all, which is why it is the shipped default — the tool works out of the box, and you opt into a voice-requiring model deliberately.
• Raw PCM is re-containered as WAV. Both verified providers return audio/pcm with the sample rate only in the media-type parameters. Written as-is those bytes are unplayable, so generate_speech prepends a RIFF/WAVE header (16-bit little-endian, rate and channels taken from the media type) and writes .wav. Non-PCM audio passes through untouched.

Choosing models

GET https://openrouter.ai/api/v1/images/models and .../videos/models list what is currently available for each endpoint. The default /api/v1/models listing returns text-output models only, which is why generation models do not appear there.

Known limitations

• Video and audio writes bypass the ctx.fs seam. Artifacts are written with node:fs straight into outputDir, so they are not subject to the session filesystem sandbox or observation policy. This is bounded by the destination being a single deployment-configured directory and never a model-supplied path, but it is a real gap — see the FIXME in src/output.ts.
• No streaming. Image and speech calls buffer the whole artifact in memory before writing.
• PCM is assumed to be 16-bit. audio/pcm media types carry a rate and a channel count but no sample width. Every provider observed emits signed 16-bit little-endian; one emitting 24- or 32-bit would produce a WAV that plays at the wrong speed rather than failing. See PCM_BITS_PER_SAMPLE in src/output.ts.
• Video content filters reject on the audio track. A generate_audio: true request can fail with a copyright refusal even when the visual prompt is innocuous. Pass generate_audio: false if you keep hitting it.
• Video jobs outlive their timeout. When videoTimeoutMs expires the tool reports a timeout naming the job id; the job keeps running server-side and the id stays pollable.
• Config is read at load, not live. The plugin reads its section once. Wire installSettingsSection from @deepseek-ai/dsh-settings if you want live edits.
• No declaration emit. prepare must work on a git install where peers may not be resolvable, so tsdown runs with dts: false. Run pnpm typecheck in a checkout with peers installed for the real type signal.

Development

pnpm install
pnpm check       # self-check for the pure helpers; needs no key and no peers
pnpm typecheck   # real type signal; needs peers installed
pnpm build

pnpm check exercises the only branching logic no request covers — extension selection, file naming, and replay-safe presenter narrowing. It runs standalone because src/output.ts has no runtime peer imports.

License

MIT