简体中文

Memory Vault

A cross-session persistent memory plugin for coding agents: save the experiences, decisions, preferences and pitfalls distilled from conversations to your local machine, and recall them on demand in future sessions.

• Storage: SQLite local database (single source of truth); records, tags and the inverted word table all persisted to disk
• Retrieval: dual-channel fused ranking of keywords (BM25) and semantic vectors (USearch, when available), with time decay support
• Curation: automatic deduplication on write (exact body dedup + semantic approximate dedup, with merge/skip strategies); tidy collapses similar records into summaries in one step to prevent memory bloat
• UI: built-in web viewing interface (pure standard library), browse, search, add, delete, tidy
• Compat: generic markdown memory format import/export, supporting frontmatter and splitting by second-level headings
• Zero-dependency runnable: the default local hash embedder works offline and is deterministically consistent across sessions; optionally plug in sentence-transformers or any OpenAI-compatible embedding API; vector retrieval auto-degrades (USearch → numpy → pure Python)

Quick Start

No dependencies to install; Python ≥ 3.9 is enough:

# 存入一条记忆
python -m memory_vault put "项目架构" "后端采用微服务，服务间通过消息队列通信"

# 混合检索
python -m memory_vault ask "微服务架构"

# 从 markdown 目录批量导入
python -m memory_vault import ./notes --split

# 压缩整理
python -m memory_vault tidy

# 启动 Web 查看界面
python -m memory_vault serve
# 浏览器打开 http://127.0.0.1:8988

# 以 MCP 服务运行（供插件化 harness 加载）
python -m memory_vault mcp

Data is stored by default in ~/.memory-vault/ (vault.sqlite3 and an optional config.json). You can also point it elsewhere with --vault <dir> or the VAULT_DIR environment variable.

Integrating with dsh (plugin harness)

Installing in DSH

dsh plugin --profile demo add github:JohnXu22786/memory-vault

The repo also ships a dsh.bundle (package.json + cordis.patch.yml + index.js). Installing it lays down a Cordis plugin row whose Node bridge drives the Python CLI (everything loads from the same package directory), surfacing the CLI commands as dsh tools vault_put / vault_ask / vault_take / vault_list / vault_drop / vault_tidy / vault_stats / vault_ingest. No npm dependencies are required at load time; Python ≥ 3.9 must be on PATH (override the interpreter with the DSH_MV_PYTHON environment variable). If Python is missing or the package can't be imported, the bridge logs a clear error at startup instead of crashing. For full-fidelity write options (tags / weight / source), use the MCP server below.

The plugin root contains manifest.json; the harness loads it per the following conventions:

Interface	How it starts	Purpose
MCP tools (recommended)	python -m memory_vault mcp (stdio transport)	exposes 8 tools such as vault_put / vault_ask to the agent for reading/writing memory in-session
CLI	python -m memory_vault <command>	scripting, scheduled tidy, batch import/export
Web	python -m memory_vault serve	human browsing and maintenance interface
Skill	read SKILL.md	instruction text guiding the agent on when to write and how to retrieve

Typical harness config sketch (MCP style):

{
  "mcpServers": {
    "memory-vault": {
      "command": ["python", "-m", "memory_vault", "mcp"],
      "env": { "VAULT_DIR": "~/.memory-vault" }
    }
  }
}

After the harness launches the process, it sends the initialize handshake, then discovers tools via tools/list and calls them via tools/call. The protocol is newline-delimited JSON-RPC 2.0 over stdio (MCP standard transport), with no third-party dependencies.

MCP tools at a glance

Tool	Description
vault_put	store a record (auto-dedup; returns action: new/merged/skipped)
vault_ask	hybrid retrieval, returns id/title/score/kw/sem/updated_at
vault_take	fetch full content by id
vault_list	list recent records
vault_drop	delete by id
vault_tidy	tidy up (collapse similar records into summaries)
vault_stats	storage stats (count, vector backend, embedding config)
vault_ingest	batch import from markdown files/directories

Usage Tips

• What to store: project decisions and their reasons, pitfalls hit, user preferences, common commands and conventions, experiment conclusions
• When to retrieve: at the start of a new session, when a task arrives with insufficient context, when a historical topic comes up
• Don't over-store: stable engineering rules belong in AGENTS.md-style documents; memory is meant to hold "context that grew out of real work"

Configuration

The config file defaults to <data-dir>/config.json (JSON); VAULT_* environment variables can override it; command-line arguments take the highest precedence. A full example is in config.example.json.

Section	Key	Default	Description
database	dir	~/.memory-vault	data directory
embedding	provider	local	local (built-in hash) / sentence (sentence-transformers) / api (OpenAI-compatible)
embedding	model	per provider	sentence defaults to all-MiniLM-L6-v2; api defaults to text-embedding-3-small
embedding	dims	128	local embedding dimensions
embedding	api_url / api_key / api_model	empty	api provider endpoint; keys support env://VAR, file:///path forms
search	keyword_weight / semantic_weight	0.4 / 0.6	dual-channel fusion weights (auto-clamped to 0~1)
search	recency_days	180	time-decay half-life (days), 0 disables
search	top_k	8	default number of results
curation	dedup_threshold	0.92	write-time dedup similarity threshold
curation	dedup_mode	merge	merge into the existing record / skip keeping the original
curation	cluster_threshold	0.78	tidy clustering threshold
curation	min_cluster	2	minimum cluster size
curation	digest_member_chars	600	characters retained per member record in the summary
web	host / port	127.0.0.1 / 8988	web interface listen address
web	token	empty	when set, all /api/* requests must carry Authorization: Bearer <token> (the UI prompts and remembers it on first request; ?token= query param also supported)

Environment variables: VAULT_DIR, VAULT_CONFIG, VAULT_EMBED_PROVIDER, VAULT_EMBED_MODEL, VAULT_EMBED_DIMS, VAULT_EMBED_API_URL, VAULT_EMBED_API_KEY, VAULT_EMBED_API_MODEL, VAULT_KW_WEIGHT, VAULT_SEM_WEIGHT, VAULT_RECENCY_DAYS, VAULT_TOP_K, VAULT_DEDUP_THRESHOLD, VAULT_DEDUP_MODE, VAULT_CLUSTER_THRESHOLD, VAULT_MIN_CLUSTER, VAULT_DIGEST_MEMBER_CHARS, VAULT_WEB_HOST, VAULT_WEB_PORT, VAULT_WEB_TOKEN.

Embedding Providers

provider	prerequisite	traits
local (default)	none	zero-dependency, offline, deterministic; limited semantic ability, good for getting started and testing
sentence	pip install sentence-transformers	real local semantic model, best results, fully offline
api	an accessible OpenAI-compatible endpoint	configure api_url + api_key (+ api_model), e.g. https://api.openai.com/v1, a local Ollama-compatible gateway, etc.

If changing the embedding config changes the dimensions, the plugin automatically re-embeds existing records on next use.

Vector Retrieval Backend

Prefers USearch (pip install usearch) approximate nearest neighbor; if not installed it auto-degrades to a numpy exact scan, then to a pure Python scan. SQLite is always the single source of truth; the in-memory index is rebuilt from the database at every startup.

Markdown Compatibility

• Import: import <file-or-directory> [--split]. Recognizes YAML-style frontmatter (title/tags/weight/created/updated), # H1 heading (used as the title and stripped from the body; CRLF line endings supported); --split splits into multiple records by ## H2 headings
• Export: export <dir>, one .md file per record (frontmatter contains id/time/tags), re-importable
• Edge cases: record ids are always generated by the system; the id in frontmatter is only exported as information and ignored on import; tags must not contain commas; leading/trailing whitespace in bodies is trimmed on export
• Designed to interoperate with existing markdown note libraries

CLI Reference

python -m memory_vault init                    # 初始化数据目录
python -m memory_vault put "标题" "正文"        # 存入（正文可省略，此时读标准输入）
echo "正文" | python -m memory_vault put "标题"
python -m memory_vault ask "查询词" -k 5        # 混合检索
python -m memory_vault get <id>                 # 查看单条
python -m memory_vault list -n 20               # 最近列表
python -m memory_vault drop <id>                # 删除
python -m memory_vault tidy                     # 压缩整理
python -m memory_vault import ./notes --split   # 导入 markdown
python -m memory_vault export ./backup          # 导出 markdown
python -m memory_vault info                     # 统计
python -m memory_vault serve                    # Web 界面
python -m memory_vault mcp                      # MCP 服务

All commands support --vault <dir>, --config <file> and --json (machine-readable output). --vault/--config are global options and must come before the subcommand (e.g. python -m memory_vault --vault ~/mv put ...).

Safety note: the web interface listens only on 127.0.0.1 by default. If you need to bind to a non-loopback address (e.g. 0.0.0.0), make sure to also set web.token; the UI has built-in cross-origin write protection (enforced JSON Content-Type + Origin check) and request timeout/concurrency limits.

Web API

Endpoint	Method	Description
/	GET	viewing interface
/api/stats	GET	stats
/api/list?limit=&offset=	GET	recent records
/api/query?q=&k=	GET	hybrid retrieval
/api/put	POST	{title, body, tags, source?, weight?}
/api/delete	POST	{id}
/api/tidy	POST	tidy up
/api/ingest	POST	{path, split?}

Architecture

memory_vault/
├── __main__.py / cli.py   命令行入口（12 个子命令）
├── config.py              配置加载（默认值 <- 文件 <- 环境变量 <- 参数）
├── vault.py               门面：协调存储/嵌入/索引/去重/压缩（进程内锁保证多线程一致）
├── store.py               SQLite 持久层：记录 CRUD、倒排词表、BM25 打分
├── vectors.py             向量索引：USearch -> numpy -> 纯 Python 三级降级
├── embedders.py           嵌入器工厂：local / sentence / api
├── ranking.py             融合排序：双通道 min-max 归一化 + 时间衰减
├── curation.py            去重决策（merge/skip）与摘要构建
├── markdown_io.py         markdown 导入导出（frontmatter / 拆分）
├── webapp.py              内置 Web 界面（http.server + 单页前端）
└── mcp_server.py          MCP stdio 服务（newline-delimited JSON-RPC）

Write flow: put → exact dedup by body (checksum) → semantic approximate dedup (top-k vector search) → persist + update index. Retrieval flow: ask → BM25 keyword score + semantic score → min-max normalized weighted fusion → time decay → rank and output.

Development and Testing

python -m unittest discover -s tests -v   # 129 项测试：存储/检索/去重/压缩/markdown/CLI/MCP/Web/配置
pip install -e .                          # 可选：安装为命令 `vault`

Optional dependencies: pip install usearch (vector acceleration), pip install sentence-transformers (local semantic embeddings).

License

MIT