Research Agent Harness

Build agents that research, reason, trace, and cite.

Research Agent Harness is a plugin-driven agent harness for multi-source research and evidence-grounded answers. It provides the execution loop, tool orchestration, durable state, streaming events, citation validation, and a DeepSeek-inspired web interface that an agentic RAG application needs in production.

It is designed as a foundation rather than a single-purpose chatbot: add a retrieval connector, describe its routing metadata, and let the harness expose it through the same traceable tool protocol.

Why Research Agent Harness

Most RAG demos stop at “retrieve some chunks and ask a model to summarize them.” Research Agent Harness treats the surrounding runtime as a first-class system:

• the Agent Harness controls turns, tool budgets, timeouts, concurrency, retries, graceful finalization, and model protocol adapters;
• the Evidence Ledger normalizes source records into immutable evidence objects instead of trusting prose returned by a tool;
• the Citation Compiler validates model evidence IDs, assigns stable display labels, and serves authorized hover details;
• the Trajectory Ledger persists reasoning, assistant output, tool calls, tool results, and terminal state for reconnectable UI rendering;
• the Cordis plugin layer keeps source connectors and product features independently addable and removable.

The result is an agent whose answers are inspectable, resumable, and tied to the records that support them.

Capabilities

• bounded agent loop with Anthropic Messages and OpenAI-compatible adapters;
• JSON Schema tool contracts with validation, timeouts, and concurrency limits;
• progressively disclosed SKILL.md workflows;
• OpenAlex, official arXiv, and NCBI PubMed retrieval tools;
• append-only SQLite event log with user-bound sessions and background runs;
• reconnectable SSE with cursor replay (Last-Event-ID / after);
• server-owned citation numbering, validation, ACL hooks, and hover metadata;
• standard and deep-research orchestration modes;
• optional deep-thinking policy for more deliberate evidence analysis;
• expandable reasoning and tool traces, session history, and trajectory views;
• graceful partial-answer finalization instead of user-visible budget failures;
• React/Cordis frontend with plugin-based UI composition.

Architecture

User / API / UI ── POST run ──► RunManager (browser-independent task)
      ▲                              │
      │ reconnectable SSE           ▼
      └──── seq cursor ───── append-only events / SQLite
                                     │
                                     ▼
                                AgentHarness
      │
      ├── ContextCompiler ──► skill catalog + citation protocol
      │
      ├── LLM adapter ──────► Anthropic or OpenAI-compatible endpoint
      │          │
      │          ▼ tool calls
      ├── ToolRegistry ─────► validation / timeout / concurrency
      │          │
      │          ├── load_skill
      │          ├── openalex_search
      │          ├── arxiv_search
      │          └── ncbi_pubmed_search
      │
      ├── Evidence Ledger ──► normalized, immutable source records
      │
      └── CitationCompiler ─► {{cite:cit_001}} + hover metadata

The model never owns visible citation numbers. It emits exact evidence references such as [[cite:ev_abc]]; the server validates those IDs, assigns display order, and persists the citation-to-evidence relationship.

Agent modes

The composer exposes two complementary controls:

• Standard keeps retrieval adaptive and favors a concise answer when the question does not require external research;
• Deep Research forces source retrieval, gathers multiple relevant records, and asks the harness to reconcile evidence before answering;
• Deep Thinking is an independent harness policy that asks the model to spend more effort planning, checking evidence, and resolving contradictions.

Deep Research and Deep Thinking are explicit per-run policies, not hidden prompt conventions. They can be combined, and both add observable metadata to the durable run_started event.

Frontend stack

The frontend follows the DeepSeek Harness web stack: React 18, TypeScript 6, Vite 6, Zustand 4.4.7, Immer 10.1.1, TanStack Virtual, pnpm, and CSS Modules. The display layer includes the resizable three-column AppFrame, grouped and searchable sessions, semantic conversation nodes, expandable tool rows, a shared details inspector, and the filtered/virtualized trajectory timeline. It directly uses the MIT-licensed @deepseek-ai/dsh-client-ui-primitives package for Markdown, code highlighting, disclosure rows, JSON inspection, hover cards, icons, and state indicators. The FastAPI/SSE adapter and local workspace sidebar remain project-specific; DeepSeek's Cordis runtime, RPC protocol, Session object layer, and ui-workspace package are intentionally not embedded.

DeepSeek names and brand assets are not used as product identity. Upstream attribution and the MIT notice are in frontend/THIRD_PARTY_NOTICES.md.

Run

cd research-agent-harness
uv sync --extra dev
cp .env.example .env

Edit .env and set LLM_API_KEY. Do not commit .env.

Install and build the frontend:

cd frontend
pnpm install
pnpm build
cd ..

For the Ark Coding Plan endpoint used by this project:

LLM_BASE_URL=https://ark.cn-beijing.volces.com/api/coding
LLM_API_STYLE=anthropic
LLM_AUTH_MODE=auto
LLM_MODEL=deepseek-v4-flash-ga-260731
LLM_API_KEY=your-key

Start the API and UI:

uv run uvicorn app.main:app --host 127.0.0.1 --port 8000 --timeout-graceful-shutdown 5

Open http://localhost:8000. API documentation is at /docs.

Run tests:

uv run pytest -q
cd frontend && pnpm test

Run deterministic unit/integration coverage first, then optional real-model evaluation cases against a running server:

uv run python scripts/run_evals.py --case rag_definition --case hyaluronic_earliest

The real suite reports completion, source-routing precision, citation validity, absence of fallback/budget failures, turns, tool calls, and per-case latency budgets. Case definitions live in evals/cases.yaml; the latest checked result is recorded in evals/REPORT.md.

API

Non-streaming chat

curl http://localhost:8000/api/chat \
  -H 'content-type: application/json' \
  -d '{
    "message": "Find recent papers about agentic RAG for scientific research",
    "require_sources": true,
    "run_mode": "research",
    "thinking_mode": true
  }'

run_mode accepts standard or research; thinking_mode is an optional boolean. The browser composer stores these preferences locally and sends them with the next run.

Streaming chat

The durable protocol separates command submission from event delivery:

POST /api/runs
GET  /api/sessions/{session_id}/stream?user_id=...&after={last_seq}
GET  /api/sessions/{session_id}/events?user_id=...&after={last_seq}
GET  /api/runs/{run_id}?user_id=...
GET  /api/sessions?user_id=...

POST /api/runs returns 202 immediately. The Agent continues if the browser disconnects. Every SSE frame has an id equal to the durable SQLite event sequence. EventSource reconnects with Last-Event-ID; callers can also send after explicitly. History catch-up is subscribed-before-read and sequence-deduplicated, so an event cannot be lost in the handoff from replay to live delivery.

Persisted streaming events include reasoning_delta, assistant_delta, tool_call, tool_result, citation_validation, and terminal run_completed / run_failed. Delta writes are coalesced to reduce SQLite pressure without losing replay fidelity.

The compatibility endpoint POST /api/chat/stream still emits:

• event: persisted harness lifecycle events;
• result: the final ChatResponse;
• error: a terminal error;
• done: stream termination.

Unlike the older implementation, disconnecting this response does not cancel the run.

User and session binding

Run requests accept user_id and session_id. A session is permanently bound to the first user ID that creates it; cross-user reads, continuation, run inspection, and citation resolution return 403. These IDs establish application-level ownership, not authentication—production deployments should derive user_id from an authenticated principal rather than trusting request input.

Citation detail

GET /api/answers/{answer_id}/citations/{citation_id}

This endpoint resolves citation metadata to the underlying evidence. In a real multi-user deployment, put authorization both in source adapters and in this endpoint before returning excerpts.

The default PublicEvidenceAuthorizer only permits evidence whose acl_ref is public. Internal connectors should set an application-specific ACL reference and inject an EvidenceAuthorizer into create_app that evaluates the authenticated request principal.

Adding a tool

Define a Pydantic input contract and an async handler returning ToolExecutionResult:

class SearchInput(BaseModel):
    query: str

async def search(input: BaseModel) -> ToolExecutionResult:
    request = SearchInput.model_validate(input)
    return ToolExecutionResult(content=f"Searched for {request.query}")

registry.register(ToolSpec(
    name="search",
    description="Search an internal source.",
    input_model=SearchInput,
    handler=search,
))

For a retrieval tool, normalize each source record into an Evidence object. Tool result prose is only an observation for the model; the Evidence object is the authoritative citation record.

Adding a skill

Create skills/<name>/SKILL.md:

---
name: my-workflow
description: When and why the agent should use this workflow.
allowed_tools:
  - search
---
# Workflow

Detailed instructions loaded only when the agent calls `load_skill`.

Only skill metadata is placed in the initial system context. Full instructions are progressively disclosed through the load_skill tool.

Citation guarantees and limits

The built-in validator guarantees that:

• every rendered citation references evidence recorded in the current run;
• model-invented evidence IDs are rejected and trigger a repair turn;
• the visible citation label is generated by the server;
• source excerpts, locators, hashes, and retrieval times remain available for audit;
• exact source titles, URIs, and stable object IDs can recover an omitted model tag;
• numeric claims are checked after locale/magnitude normalization (1,600, 1600, 4,400 万, 44 million);
• numeric claims without a nearby citation produce warnings.

The included semantic support check is intentionally conservative and lexical. For high-stakes deployments, add a dedicated entailment verifier or human review before changing partial citations to verified.

External API notes

• NCBI asks E-utilities clients to send an application name and email. Configure NCBI_EMAIL; add NCBI_API_KEY if you need higher request limits.
• OpenAlex is the fast general scholarly index. Temporal queries first form a relevance-ranked candidate set, then sort locally so weak ancient matches do not outrank relevant papers.
• The arXiv API asks clients to avoid rapid repeated calls. The adapter serializes calls and keeps a three-second interval.
• The UI should acknowledge arXiv data use if deployed publicly. This demo keeps all source names factual and does not imply endorsement.

Production hardening checklist

• replace the single-process SQLite connection with PostgreSQL for multiple workers;
• encrypt or redact sensitive event payloads;
• enforce connector-level and citation-detail ACLs;
• put shell/browser tools in an OS or container sandbox;
• add idempotency keys to every side-effecting tool;
• add per-user budgets and rate limiting;
• evaluate retrieval recall, citation coverage, citation entailment, latency, and recovery behavior.

Loop and planning behavior

• The final turn exposes no tools and explicitly asks the model to finish from retrieved evidence.
• Turn and tool-call budgets return the best available cited partial answer; they do not become HTTP 502 errors.
• Independent tool calls in one model response execute concurrently under a configurable semaphore.
• Single-source questions stop retrieval after one successful result set with at least three records; explicit multi-source questions may use up to MAX_RETRIEVAL_ROUNDS.
• Equivalent tool name/argument calls are fingerprinted and skipped within a run.
• Tool registrations carry routing metadata (kind, best_for, parallel_safe) that is compiled into the planning prompt, encouraging source selection rather than speculative fan-out.
• Invalid evidence IDs, missing citations, and numeric conflicts between a claim and its cited record trigger a complete-answer repair turn. If repair cannot complete safely, the server returns a deterministic, citation-backed bibliographic fallback without adding model-memory claims.