Inference for gateways and harnesses.
Route your users’ agent traffic to open models built for long runs. Zero data retention, and up to 5M tokens of context billed as 200k–300k.
5M tokens of context. Pay for 200k–300k.
Agent sessions grow with every step. Dynamic context compression packs up to 5M tokens into 200k–300k before it is billed, so your users keep their whole history and you pay for the part that counts. Fewer, more relevant tokens give the model better intelligence density.
- 100k sent100k billed
- 1M sent200k–300k billed
- 5M sent200k–300k billed
Built for agent traffic.
Long runs finish 2x faster at 50–80% lower cost than the same models on standard inference.
Context compression
Stale spans are compressed before they are billed, so long sessions cost 50–80% less.
Prefix and suffix caching
Tokens on both sides of a compressed span stay cached, so each step keeps hitting the cache.
Built for long sessions
Cache-aware routing and load balancing treat 200k+ token sessions as the main workload, so long runs finish 2x faster.
Zero data retention.
Prompts, completions and tool output are never stored, logged or used for training. We keep token counts for billing and nothing else. The commitment is in our terms, so you can pass it on to your users.
Questions
- Agent sessions grow with every step. Our runtime compresses context that stopped mattering and caches the rest, so the same open models finish long runs 2x faster at 50–80% lower cost than on standard inference.
- Per token. Dynamic context compression keeps a session of up to 5M tokens to 200k–300k tokens of context, and that is what you are billed for.
- No. We keep zero data: prompts, completions and tool output are never stored, logged or used for training. We keep token counts for billing. The commitment is in our terms.
- Yes. Every model is served at an OpenAI chat completions endpoint and an Anthropic messages endpoint, with tool calls, streaming and structured output.
- The API serves GLM-5.3 Marathon and DeepSeek V4.1 1017 Marathon in both the OpenAI and Anthropic formats. Dedicated deployments can serve virtually any open model, including your own fine-tunes.
- Yes. Talk to an engineer about volume. Gateways with heavy traffic can move to a Dedicated Deployment: a Single-Tenant API on reserved GPUs with no rate limits.
Give your users the long run.
Talk to an engineer about volume, rate limits and serving our models to your users.