@tma1-ai/dsh-plugin-greptimedb

English | 中文

Send DeepSeek Harness telemetry to GreptimeDB as OpenTelemetry traces, metrics, and logs.

No collector. No sidecar. No fork of DSH. It installs as an ordinary plugin, and every turn, model call, and tool execution becomes a row you can query:

-- Slowest tool calls, with the model that requested them.
SELECT tool.span_name,
       chat."span_attributes.gen_ai.request.model" AS model,
       tool.duration_nano / 1000000 AS ms
FROM opentelemetry_traces AS tool
JOIN opentelemetry_traces AS chat
  ON  chat.trace_id = tool.trace_id
  AND chat."span_attributes.dsh.step" = tool."span_attributes.dsh.step"
  AND chat.span_name LIKE 'chat%'
WHERE tool.span_name LIKE 'execute_tool%'
ORDER BY tool.duration_nano DESC
LIMIT 10;

Install

dsh plugin --profile headless add @tma1-ai/dsh-plugin-greptimedb

The package ships a bundle patch, so that one command wires it into the profile. dsh plugin forwards to whichever pnpm is on your PATH, and a dsh profile directory is its own pnpm workspace root — pnpm 9 refuses to install there and ignores the linker settings dsh writes, so use pnpm 10 or newer. To point it at your own database, override the row in $DSH_HOME/profiles/<name>/cordis.patch.yml:

- id: greptimedb-otel
  name: '@tma1-ai/dsh-plugin-greptimedb'
  config:
    endpoint: https://<host>/v1/otlp
    database: <dbname>
    username: <user>
    password: <password>

A profile patch replaces the row's whole config instead of merging into it, so restate every field you want to keep.

The defaults already point at a local GreptimeDB:

docker run -p 127.0.0.1:4000-4003:4000-4003 \
  -v "$(pwd)/greptimedb_data:/greptimedb_data" \
  --name greptime --rm greptime/greptimedb:v1.2.0-beta.2 standalone start \
  --http-addr 0.0.0.0:4000 --rpc-bind-addr 0.0.0.0:4001 \
  --mysql-addr 0.0.0.0:4002 --postgres-addr 0.0.0.0:4003

That instance serves GreptimeDB's own console at http://localhost:4000/dashboard/ — enough to check the tables and run ad-hoc SQL before bringing up Grafana.

Configuration

Key	Default	Notes
endpoint	(required)	OTLP base URL, e.g. http://localhost:4000/v1/otlp. The plugin appends each signal's /v1/{traces,metrics,logs} suffix; a per-signal path is rejected at load.
database	public	Sent as X-Greptime-DB-Name.
username / password	(none)	Basic auth. Both or neither.
signals	all three	Any subset of traces, metrics, logs. A disabled signal builds no exporter.
content	none	How much payload may leave the process. See What leaves the machine.
serviceName	dsh	OTel service.name.
logTable / traceTable	GreptimeDB defaults	Destination table overrides.
ttl	180d	Retention for the log and trace tables this plugin creates, sent as x-greptime-hints. Also accepts forever. GreptimeDB applies it when it auto-creates the table; an existing table keeps its own until ALTER TABLE. Metric tables are not covered — see Known limitations. Set it empty to send no hint and inherit the database default.
shutdownTimeoutMillis	3000	Deadline for the entire teardown sequence.
metricIntervalMillis	30000	Metric collection period. Must be at least exportTimeoutMillis.
maxExportBatchSize / maxQueueSize	512 / 2048	Batch and buffer bounds.
scheduledDelayMillis / exportTimeoutMillis	5000 / 30000	Export cadence and per-request deadline.

Bad configuration fails at plugin load with the offending field named, not at the first export.

Traces

Turn spans are roots. Chat and tool spans hang off them as siblings, correlated by dsh.step:

invoke_agent dsh              turn/start → turn/end
├── chat deepseek-chat        step/start → assistant/message
├── execute_tool bash         tool/call  → tool/result
└── chat deepseek-chat

Every timestamp comes from the session event it belongs to, not from a clock read while the event is being handled.

A chat span closes on one of four paths, each with a defined end time:

Situation	End time	Status
Model responded	assistant/message	OK
Stream interrupted	assistant/message	OK, plus dsh.response.interrupted
Request failed	that step's step/end	ERROR, with the error type
No end event (crash, teardown)	last event seen	UNSET, plus dsh.span.unclosed

Token accounting

DSH's counts are disjoint: inputTokens is uncached input alone, cache reads and writes are separate fields. gen_ai.usage.input_tokens is the billed total, so the plugin exports:

gen_ai.usage.input_tokens  = inputTokens + cacheReadTokens + cacheWriteTokens
gen_ai.usage.output_tokens = outputTokens          (reasoning tokens included)

The breakdown stays queryable as dsh.usage.uncached_input_tokens, dsh.usage.cache_read_tokens, dsh.usage.cache_write_tokens, and dsh.usage.reasoning_tokens.

Metrics

Instrument	Type	Dimensions
gen_ai.client.token.usage	Histogram	gen_ai.token.type (input/output only), model, provider
gen_ai.client.operation.duration	Histogram	gen_ai.operation.name, model
dsh.token.detail	Histogram	dsh.token.detail_kind (cache_read/cache_write/reasoning)
dsh.tool.invocations	Counter	gen_ai.tool.name, dsh.tool.outcome
dsh.turns / dsh.steps	Counter

Logs

One record per session event. Four attributes become real columns through X-Greptime-Log-Extract-Keys:

SELECT session_id, event_type, turn, step, body
FROM dsh_logs
WHERE session_id = '...' AND event_type = 'tool/result'
ORDER BY timestamp;

assistant/chunk is never exported; the assembled assistant/message carries the same content.

What leaves the machine

content decides this. The default withholds all payloads.

Mode	Exported
none (default)	Structure and accounting: event types, turn and step numbers, token counts, tool names, durations, outcomes, error name and code.
full	Adds user and assistant message content, tool arguments, tool results.
full+prompt	Adds request/header: the complete system prompt and every tool schema.

Three things never leave in any mode: a tool's private meta payload, the internal error.message of a failed turn, and the message and stack of a failed request.

The projection is a positive allowlist, so an event type the plugin does not know — including one a future DSH plugin declares — exports its identity and nothing else.

Dashboards

Five Grafana dashboards ship in grafana/, along with a compose stack that brings up GreptimeDB and Grafana together.

cd grafana && docker compose up -d && open http://localhost:3000

Dashboard	Answers
Overview	What did this cost, how fast was it, how much came from cache
Agent loop	Which tools ran, how often they failed, how many model calls a turn needed
Trace explorer	What happened inside one specific turn, span by span
Log explorer	Every session event, filterable by session, type, and full-text search
Metrics	The same activity through PromQL, for longer retention and sampling-proof percentiles

Every table links onward: a trace id opens that turn's waterfall, a session id jumps between the trace and log views. Every panel query is checked against a live database by node grafana/verify.mjs. See grafana/README.md for the datasource split and grafana/indexes.sql for the indexes these queries want.

With TMA1

TMA1 proxies OTLP into a GreptimeDB it manages. Point endpoint at it and DSH shows up in the OTel GenAI view:

endpoint: http://localhost:14318/v1/otlp

TMA1's tma1_token_usage_1m, cost_1m, latency_1m, and status_1m flow tables derive from span_attributes.gen_ai.*, which this plugin populates by convention.

Development

pnpm test     # unit, profile composition, Loader boot
pnpm smoke    # packaging checks against a freshly packed tarball
GREPTIMEDB_OTLP_ENDPOINT=http://localhost:4000/v1/otlp pnpm test   # adds the live database round trip

Known limitations

• DSH is pre-release and renames and repackages freely before its first tagged release. The peer range is the exact version CI runs against (0.1.1-rc.2); a new DSH release needs a tested bump here.
• The GenAI conventions are experimental. Names come from @opentelemetry/semantic-conventions/incubating and move with it. Spans carry both gen_ai.provider.name and the deprecated gen_ai.system.
• ttl does not reach metric tables. Metrics land on the metric engine, where retention is a property of the physical table. The hint reaches the logical table, which stores and displays it but never enforces it (greptimedb#8951). Set it yourself with ALTER TABLE greptime_physical_table SET 'ttl' = '180d'.
• No per-turn flush. Export follows the batch processors' cadence.
• Shutdown is bounded. Records in flight when shutdownTimeoutMillis expires may be lost at exit.
• Subagent sessions get their own trace, not stitched into the parent's.

License

Apache-2.0