原生双协议
Native dual protocol
OpenAI 与 Anthropic 请求按路径原样透传：上游 URL、请求体与用量解析全程一致，零协议转换、零损坏。
OpenAI & Anthropic pass through format-homogeneous: upstream URL, body and usage parsing all match the path you call — no lossy conversion.
LLM 网关 · Rust + Pingora
LLM GATEWAY · RUST + PINGORA
一个网关，
一个网关，
接入所有模型。
One gateway for
every model.
Hydra 是 Rust + Pingora 构建的 LLM 网关：原生透传 OpenAI 与 Anthropic 协议、租户级认证、加权路由、故障转移、熔断、限流与用量计量——打包进一个 65 MiB 的二进制，零 unsafe。
Hydra is an LLM gateway in Rust + Pingora: native OpenAI & Anthropic passthrough, per-tenant auth, weighted routing, failover, rate limiting and usage metering — packed into a 65 MiB binary with zero unsafe.
// 10 核机器实测 · c=25 · 无真实付费上游
// measured on a 10-core box · c=25 · no paid upstream
实时路由LIVE ROUTING
11,056 单节点峰值 RPSsingle-node peak RPS
0.3ms 网关开销overhead
65MiB 单节点 RSSRSS / node
0RPS
单节点峰值吞吐（c=25，p99 4.39ms）
Single-node peak throughput (c=25, p99 4.39ms)
0ms
单请求网关开销
Per-request gateway overhead
相对 LLM 延迟可忽略
negligible vs. LLM latency
0MiB
满载 RSS（18.6 → 65.4）
RSS under full load (18.6 → 65.4)
16GB 机器的 0.4%
< 0.4% of a 16 GB box
0unsafe
生产代码无 unwrap / panic / unsafe
unsafe / unwrap / panic in production code
287 项测试 · clippy -D warnings 硬门禁
287 tests · clippy -D warnings gate
// 特性// FEATURES
生产网关该有的，这里都有。
Everything a production gateway needs.
从路由到计量，从单机到集群——每一条都经过测试与线上验证。
From routing to metering, single-node to cluster — every one tested and verified.
加权路由
Weighted routing
模型名 → 租户允许的提供方集合，Nginx SWRR 平滑加权轮询，权重可热更新。
Model name → tenant-allowed providers, smooth Nginx SWRR round-robin, hot-reloadable weights.
故障转移 + 熔断
Failover + breaker
候选提供方逐个尝试；连续失败进入死集并由后台探测复活。全 body 已缓冲，重放是 O(1) 引用计数。
Try each candidate in sequence; consecutive failures trip a dead-set revived by background probes. Full body replay is an O(1) refcount.
限流
Rate limiting
内存滑动窗口，按角色匹配租户/密钥/模型/提供方，支持请求数与 Token 数双维度，m/h/d 窗口。
In-memory sliding window per role matched by tenant/key/model/provider, counting both requests and tokens, m/h/d windows.
并发准入
Admission control
按提供方配置并发上限：在途请求、空闲许可与队列深度实时可见，超限排队而非压垮上游。
Per-provider concurrency gates with live in-flight / available / queue visibility — excess waits in queue instead of crushing upstreams.
多租户
Multi-tenant
按域名识别租户，各自独立的认证端点、TLS 证书（SNI）、模型闸门与密钥前缀绑定。
Tenants resolved by domain with their own auth endpoint, SNI TLS certificates, model gate and key-prefix bindings.
密钥静态加密
Keys sealed at rest
提供方 API 密钥 AES-256-GCM 加密落库，管理 API 永不回显明文，启动时密钥缺失即拒启（fail-closed）。
Provider keys sealed with AES-256-GCM; the admin API never returns plaintext and boot fails closed without the key.
认证缓存
Auth cache
租户认证判定缓存 5 分钟，支持即时失效；租户凭自助令牌即可清除本租户缓存（欠费停机/付费恢复）。
Auth verdicts cached 5 min with instant invalidation; tenants can clear their own cache with a self-service token.
精确用量计量
Granular metering
输入 / 缓存 / 输出 Token 分类记录 + TTFT 与转发延迟，落 SQLite 或 ClickHouse，字段无 NULL。
Input / cached / output token split + TTFT and forward latency, into SQLite or ClickHouse — no NULL columns.
集群高可用
Cluster HA
Redis 租约选主 + 快照控制面同步，standby 透明转发写请求；限流/熔断/认证跨节点共享。
Redis lease leader election + snapshot control plane; standby forwards writes transparently; limits, breaker and auth shared cluster-wide.
可观测性
Observability
Prometheus /metrics 免令牌暴露，健康检查 + 集群状态视图，按租户/提供方的用量统计图表。
Token-free Prometheus /metrics, health + cluster views, usage charts by tenant and provider.
内置管理台
Embedded admin UI
无构建步骤的嵌入式控制台：全量 CRUD、OpenAPI 风格 API 文档、熔断/缓存可视化——界面支持中英法德四语。
No-build embedded console: full CRUD, OpenAPI-style API docs, breaker/cache views — the UI speaks zh · en · fr · de.
// 架构// ARCHITECTURE
一个节点，就是完整网关；
One node is already a gateway.
一个节点，就是完整网关；
一群节点，是一个控制面。
One node is already a gateway.
A fleet of them is a control plane.
请求在边缘终结，状态在集群共享。限流、熔断、认证判定在任何节点上都一致；写变更只有一个权威，其余节点透明跟随——规模不是重新设计，是加一台机器。
Requests end at the edge; state is shared across the cluster. Rate limits, breaker verdicts and auth decisions are identical on every node. There is exactly one writer — the rest follow transparently. Scaling is adding a machine, not redesigning.
客户端
Clients
同一入口，任意节点
one entry, any node
leader
active · 唯一写者
active · sole writer
提供方
Providers
OpenAI · Anthropic · …
客户端
Clients
直连数据面
straight to the plane
edge ×N
无状态 · 任意扩缩
stateless · scale freely
提供方
Providers
与 leader 相同的路由
same routes as leader
恰一个写者
Exactly one writer
Redis 租约裁定唯一 active；时间栅栏杜绝双写。故障切换 ≈ 11–18s，数据面与调用方都无感。
A Redis lease crowns a single active; fencing forbids double writes. Failover ≈ 11–18s, invisible to the data plane.
无状态数据面
Stateless data plane
edge 只持 last-known-good 快照与本地缓存，可任意增删；Redis 失联，数据面依旧应答。
Edges carry only last-known-good snapshots and local caches; they come and go freely. Redis goes dark — the plane keeps answering.
一个 Redis，七种用途
One Redis, seven jobs
租约、注册表、共享限流、共享熔断、认证 L2、失效总线……共享是架构属性，不是插件。
Leases, registry, shared limits, shared breaker, auth L2, invalidation bus… sharing is an architectural property, not a plugin.
请求在边缘终结：全量上下文在手，没有半途状态需要跨节点协调——故障重放是常数时间，缓存判定处处一致。
Requests end at the edge: the full context is in hand, no halfway state needs cross-node coordination — failover replay is constant-time, cached verdicts are identical everywhere.
hydra — join the cluster
$ HYDRA_ROLE=edge HYDRA_REDIS_URL=redis://… HYDRA_CLUSTER_TOKEN=… ./hydra # … [edge] bootstrap → snapshot v42 · routes synced · serving :8080 [edge] limits shared · breaker shared · auth L2 hot
// 性能// PERFORMANCE
不解释，直接看数字。
Numbers over adjectives.
左列均为单节点实测（10 核机器，线程化 mock 上游）——单节点已经够快，集群则把这份吞吐乘以节点数。
Left column is measured on a single node (10-core, threaded mock upstream) — fast enough alone, and the cluster multiplies it by the number of nodes.
11,056 RPS
0.3 ms
65 MiB
9.2 / 10
测量方法：
10 核机器 + 线程化 mock 上游，未触达任何真实付费上游。完整方法与 8C16G VPS 容量推算见评测报告。
Methodology: 10-core machine + threaded mock upstream, no real paid provider hit. Full methodology + 8C16G VPS extrapolation in the evaluation report.
正确性门禁：
114 core + 173 server 测试、clippy -D warnings、依赖防火墙（core 零 I/O 依赖）——CI 硬门槛。
Correctness gates: 114 core + 173 server tests, clippy -D warnings, dependency firewall (core has zero I/O deps) — hard CI gate.
横向扩展 ∝ N：数据面无状态，吞吐随节点数近似线性增长——K8s HPA 就绪，加副本就是加吞吐。集群 ≈ 单节点 × N（受上游与网络约束）。
Scale-out ∝ N: the data plane is stateless — throughput grows ~linearly with nodes. K8s HPA-ready: add replicas, add capacity. Cluster ≈ single node × N (upstream- and network-bound).
// 多语言// I18N
管理台会说你的语言。
The admin UI speaks your language.
中文Chinese
Englishdefault
FrançaisFrench
DeutschGerman
// 部署// DEPLOY
30 秒跑起来，几分钟接入生产。
Running in 30 seconds. In production in minutes.
一个二进制，两个端口：8080 代理 / 8081 管理。配置存 SQLite，改完即热加载。
One binary, two ports: 8080 proxy / 8081 admin. Config lives in SQLite and hot-reloads on change.