@r-ayindsh-ttft-hedge
Hedged requests for streaming /v1/messages LLM calls: fires a second attempt when response headers miss a deadline and races them, retries immediately on 5xx/504, passes through 4xx/429, with statistics.
- Stars
- 0
- Language
- TypeScript
- Created
- Oct 15, 2026
- Updated
- Oct 15, 2026
Introduction
dsh-ttft-hedge
TTFT 优化：对发往 LLM 中继 /v1/messages 的流式请求做对冲（hedged request）——deadline 内未收到响应头则并行发第二个请求，先到先用；5xx/504 立即重抽；4xx/429 原样返回。附统计工具与 stats 文件。
由 dsh-super-injector dev_scaffold_plugin 生成。
构建与注入
bash scripts/build.sh
# 注入器环境内：dev_inject_plugin <本目录>