Math_Agent

A math training preset for DeepSeek Harness — not a problem solver.

Rather than solving problems for the trainee, it provides rapid feedback to help sharpen mathematical understanding and build intuition. It features a programmatic zero-leak output guard that mechanically prevents the coach from ever revealing the answer.

Features

• 🧑‍🏫 Coach, Not Solver: Never gives complete solutions, key steps, or final answers. Follows the trainee's reasoning, confirms correct parts, and flags issues.
• 📋 Pre-Session Check: Before each training session, verifies that the problem is well-defined, conditions are sufficient, and there are no errors.
• 🧠 Programmatic Attempt Tracking: Approaches tried by the trainee are recorded as program state via the attempt_update tool (status: in-progress / flawed / validated / incomplete), rendered into the system prompt every step, and rebuilt from the session log after a resume. The coach is nudged in the prompt after several rounds without an update. For longer sessions, attempts can additionally be persisted to notes/attempts.md.
• 🪜 Hint Ladder & Per-Session Hint Policy: Coaching feedback follows a four-level hint ladder (L0 questions → L1 principle-naming → L2 skeleton → L3 partial construction, last resort only). Every L1+ hint is logged via the hint_log tool and counted in the system prompt — a rising count is a signal to slow down. At session start the coach asks the trainee for the hint policy via ask_user_question (strict = L0/L1 only; normal = full ladder) and records it with the hint_policy tool — per-session, resumable, switchable mid-session. The strictness value in agent.cordis.yml is only the default for sessions that never chose (ships as strict); under a strict policy hint_log rejects L2/L3 programmatically.
• 🚦 Summary Discipline: Only interim summaries are allowed before an approach is fully validated; the final summary is unlocked programmatically — the zero-leak guard stands down only once an attempt is recorded as validated through attempt_update, never on the model's own say-so.
• 📚 Final Synthesis: The final review revisits every approach explored during the session, identifies the trainee's own reasoning patterns, and avoids dismissing "flawed" paths outright.
• 🔒 Programmatic Zero-Leak Guard (core highlight): A stream-level guard on llm/stream that inspects the coach's entire reply. When answer clues (numerical values, intervals, correctness judgments, answer forms) are detected, the entire response is replaced — not a single character of the model's original output reaches the trainee. This does not rely on the model's "self-discipline."
• 🗂️ Extensible Knowledge Base Interface: A reserved local/online knowledge-base integration (knowledge-base skill). During final synthesis, understandings backed by facts and data are prioritized.

Directory Structure

math-agent/
├── preset.yml              # Preset metadata (name / description)
├── agent.cordis.yml        # Cordis composition: tools, persona, skills, guard registration
├── attempt-tracker.js      # Training-state plugin: attempt_update + hint_log + hint_policy tools, prompt injection, validated flag, stale-update nudge (folds from the session log)
├── zero-leak-guard.js      # Programmatic zero-leak output guard plugin (loaded via relative path; copied with the preset)
├── zero-leak-guard.test.mjs # Test corpus: leak samples that must block, clean replies that must pass, tracker + unlock behavior
├── README.md               # This file
├── LICENSE                 # MIT License
└── skills/
    ├── coaching-protocol/  # Coaching protocol: workflow, red lines, zero-leak rules
    ├── knowledge-base/     # Knowledge base integration interface (reserved)
    └── final-synthesis/    # Final synthesis workflow and output structure

Installation

Prerequisite: DeepSeek Harness (npx @deepseek-ai/dsh web). This preset is derived from the standard preset and uses the DSH agent-presets mechanism.

git clone https://github.com/RepInS-01/math-agent.git
mkdir -p ~/.dsh/.agent-presets
cp -r math-agent ~/.dsh/.agent-presets/

The preset id is the directory name, so the preset must land as ~/.dsh/.agent-presets/math-agent/ (or ${DSH_HOME}/.agent-presets/math-agent/ if you run with a custom DSH_HOME). Discovery is live: the preset appears in the Web GUI immediately, no DSH restart needed.

Mount validation (run after any modification): in the Web GUI, create a new session and select the Math_Agent preset. A successful selection means the composition mounted; on a mount failure the GUI reverts the selection to the default preset and shows the error. (Programmatically, the same check is agentPresets.standingKeyFor('math-agent'), callable from inside a DSH session.)

Usage

1. In the DeepSeek Harness Web GUI, create a new session and select the Math_Agent preset (id: math-agent).

2. Start a training session with a prompt like:

   I'd like to start a math training exercise. Problem: Let aₙ = √(1 + aₙ₋₁), a₀ = 1. Prove that {aₙ} converges and find its limit. I'm thinking of using monotone convergence but I'm not sure how to prove boundedness.

3. No matter how insistently you ask for the answer — the zero-leak guard stands between you and the model.

How the Zero-Leak Guard Works

• Hooked into the llm/stream waterfall (dispatched process-wide); only activates for requests whose system prompt contains coaching signatures (zero-leak iron rules / math coach). All other sessions pass through untouched.
• The model's full output text is inspected programmatically before delivery. If answer-clue patterns (intervals, correctness judgments, answer forms, decimal values, etc.) match, the entire segment is replaced with a fixed interception message.
• Interception happens at the stream layer: session logs record the interception message, not the leaked text. On the next turn, the model sees its own interception notice and automatically reformulates in compliance.
• The unlock is program state, not model judgment: the attempt-tracker plugin (mounted in the same isolate realm) holds the per-session attempt list. Once an attempt is recorded as validated via the attempt_update tool, the guard stands down so the final synthesis can deliver the complete solution. A missing tracker, an unknown session, or no validated attempt all fall through to blocking — fail-closed in every direction.
• The unlock is program state, not model judgment: the attempt-tracker plugin (mounted in the same isolate realm) holds the per-session attempt list. Once an attempt is recorded as validated via the attempt_update tool, the guard stands down so the final synthesis can deliver the complete solution. A missing tracker, an unknown session, or no validated attempt all fall through to blocking — fail-closed in every direction.
• Compute/search tools are denied under the same lock: a preset-layer tools.guard rejects bash, pwsh, web_search, and run_code until an attempt is validated. The stream inspection cannot see tool output, so the tools that could compute or retrieve the answer are disabled outright — and the denial lifts on the same program state as the final-synthesis unlock.
• Reasoning is hidden during the locked phase: the coach's thinking routinely derives the answer outright, and the client UI renders reasoning rows to the trainee. While no attempt is validated, reasoning deltas collapse to a fixed placeholder and reasoning block payloads are rewritten — in original block order, keeping the persisted message consistent with pi-ai replay state. Reasoning flows again once an approach is validated.
• Pattern definitions are centralized in LEAK_PATTERNS at the top of zero-leak-guard.js and can be extended freely.

Known Boundaries

The programmatic guard blocks enumerable leak forms (numeric values, intervals, judgment words) in the coach's own reply text, and the tool gate covers the compute/search channels. One channel remains persona-constrained rather than guard-enforced:

• Semantic-level hints without numbers (e.g., "this number happens to be the root of the equation you just derived").

When new variants are discovered, simply add them to LEAK_PATTERNS (plus a sample in zero-leak-guard.test.mjs).

Testing

The guard ships with a zero-dependency test corpus (Node's built-in test runner):

node --test zero-leak-guard.test.mjs

Run it after every change to LEAK_PATTERNS, attempt-tracker.js, or the guard's unlock logic. The corpus covers: leak samples that must be blocked, legitimate coaching replies that must pass, the validated-session unlock (and its fail-closed fallbacks), and the attempt tracker's state recording, hint logging (including the strict-policy rejection of L2/L3), stale-update nudge, prompt rendering, and session-log fold. Add every newly discovered leak variant to the block list and every reported false positive to the pass list.

Reload caveat: relative-path preset plugins (./attempt-tracker.js, ./zero-leak-guard.js) are imported through Node's ESM loader with no cache busting. After editing plugin code or agent.cordis.yml, restart the DSH host (dsh web) — re-selecting the preset in the UI is not enough, as re-mounting reuses the cached module. Skill files (skills/**/SKILL.md) are re-read from disk per session and do not need a restart.

Knowledge Base Integration (Reserved)

skills/knowledge-base/SKILL.md defines a unified search interface contract (local directory + online retrieval), currently in reserved state:

• Local: Workspace kb/ directory with Markdown files organized by topic; retrieved via glob + grep.
• Online: web_search retrieval, cited by source URL.
• All results are tagged with confidence (fact / data / reference / heuristic). Final synthesis prioritizes understandings supported by facts and data.

Customization

• Modify persona / coaching protocol: edit the persona section in agent.cordis.yml and skills/coaching-protocol/SKILL.md.
• Modify guard patterns: edit LEAK_PATTERNS in zero-leak-guard.js.
• Always re-check the mount after changes: re-select the preset in the Web GUI (a failed mount reverts the selection to the default), or call agentPresets.standingKeyFor('math-agent') from a DSH session.

Acknowledgments

This project was inspired by the math coaching concept originally proposed by Bilibili creator PiKaChu345. The original creator has not yet released their implementation publicly. This is an independent reimplementation based solely on the publicly described idea. No source code or proprietary materials from the original work were used. All credit for the original concept goes to PiKaChu345.

License

MIT