Luminal
An open-source compiler that turns models into native GPU code, sold as serverless or on-prem.
- Founded
- 2025
- Example models
- DeepSeek V4.1 Flash, Ideogram 4.5
By The Subconscious Team · Updated
What is Luminal?
Luminal builds an inference compiler. Where vLLM and SGLang interpret a model at runtime, Luminal compiles it ahead of time into native kernels for GPUs and ASICs. Models get lowered to a small graph of 15 primitive ops, and the compiler searches over fusion, tiling, memory and scheduling choices instead of relying on hand-written rules, which it says can find optimizations like Flash Attention on its own. The compiler is open source in Rust under Apache 2.0 or MIT, runs on CUDA and Metal with ROCm on the roadmap, and works as a torch.compile backend.
The company sells that compiler two ways. Luminal Cloud is serverless inference in early access: upload a PyTorch or Hugging Face model, get a compiled endpoint with automatic batching and scale to zero, and pay per use. A public price list now covers hosted models too: DeepSeek V4.1 Flash at $0.1425 in and $0.57 out, and Ideogram 4.5 images from $0.008 to $0.20 each, with a $50 minimum top-up. The on-prem license adds custom kernel work, dedicated engineers and SLAs. Luminal reports GPT-OSS 120B at 36K tokens per second on 8 H100s, against 28K for TensorRT-LLM and 26K for vLLM. Founders Joe Fioti, Jake Stevens and Matthew Gunton came from Intel, Apple and Amazon, went through Y Combinator in Summer 2025, and raised a $5.3M seed led by Felicis in November 2025.
Luminal pros, cons and use cases
Upsides
- Ahead-of-time compilation can beat runtime engines on throughput for the same model and hardware.
- Open-source compiler, so teams can inspect it or run it themselves before buying.
- Brings your own model, including custom architectures that hosted catalogs skip.
Core use cases
- High-throughput serving of a custom or fine-tuned model on owned GPUs.
- Teams that want an inference engine faster than vLLM without hand-writing kernels.
Downsides
- Very young company; the hosted catalog is just a couple of models so far.
- Speed figures are self-reported aggregate throughput, not independent per-request benchmarks.
Luminal alternatives compared
Pick any row for the full head-to-head.
| Provider | Model access | Flagship models | Speed | Price | Customization | Deployment | Long context | Compare |
|---|---|---|---|---|---|---|---|---|
| Small hosted catalog plus bring your own weights | DeepSeek V4.1 Flash, Ideogram 4.5 | 36K tok/s on GPT-OSS 120B, 8xH100 (vendor) | $0.1425 in, $0.57 out (DeepSeek V4.1 Flash) | Compiles any PyTorch or HF model | Serverless (early access), on-prem license | Unknown | ||
| Open weights | GLM 5.3 Marathon, DeepSeek V4.1 1017 Marathon, Nemotron 3.5 Super VL | 2x faster task completion | 50–80% lower cost; billed on processed tokens | Marathon post-trained variants | Managed API, dedicated, on-prem | 5M+ effective context | Compare | |
| Closed, plus open gpt-oss | GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, GPT-6 Luna | Fast mode; Ultrafast on GPT-6 Astra | $0.10–$10 in, $0.50–$50 out per 1M | N/A | API, Azure OpenAI, Bedrock | 1.05M; 2x input past 272K | Compare | |
| Closed | Claude Fable 5.1, Opus 5.5, Sonnet 5.5, Haiku 5.5 | Fable is the slowest tier | $0.10–$10 in, $0.50–$50 out per 1M | N/A | API, Bedrock, Vertex AI, Microsoft Foundry | 1M; Haiku 5.5 costs more past 100K | Compare | |
| Closed and open, 200+ models | Gemini 4 Argon, Gemini 3.8 Flash, Claude, Gemma | Flash tier built for low latency | Argon $2 in, $10 out until Dec 1, then $4/$20; 3.8 Flash $0.75/$3.75 until 2027 | Custom training on GPUs or TPUs | Managed on Google Cloud | 1M on Gemini 4 Argon and 3.8 Flash | Compare | |
| Closed and open, 100+ models | Claude, GPT-6 Astra, Nova, DeepSeek | Latency-optimized option on some models | ~20–35% above direct; Claude and GPT at parity | Fine-tuning, Custom Model Import | Managed on AWS, AgentCore | Varies by model | Compare | |
| Open weights | Kimi K3, GLM 5.3, DeepSeek V4.1 Flash, Qwen 3.8 Flash | 0.99s TTFT on DeepSeek V4 Pro | Parity with Fireworks and Baseten | LoRA and full SFT; RL via Post-Training | Serverless, dedicated, GPU clusters | 1M on DeepSeek V4 Pro 0813, Kimi K3, GLM 5.3 | Compare | |
| Open weights | DeepSeek V4.1 Flash, GLM 5.3, Kimi K3 | 167–174 tok/s on DeepSeek V4 Pro | Fine-tunes served at base price | SFT, DPO; RL via Training API | Serverless, dedicated GPUs | Full 1M on DeepSeek V4.1 Flash, GLM 5.3 | Compare | |
| Open weights, 15+ curated | GLM 5.3, DeepSeek V4.1 Flash, Kimi K3, gpt-oss 120B | 0.49s TTFT, lowest measured | H100 about $6.50/hr dedicated | Deploy any model with Truss | Model APIs, dedicated, self-host | Varies by model | Compare | |
| Open weights | GPT-OSS 120B, Qwen 3.8 27B | 500–1,000 tok/s | Near the floor on small models | No fine-tuned model hosting | GroqCloud API | Around 131K max | Compare | |
| Open weights | GPT-OSS 120B, Qwen 3.8 27B | ~3,000 tok/s on GPT-OSS 120B | $0.35 in, $0.75 out (GPT-OSS 120B) | Unknown | Shared API, dedicated, partners | 131K on GPT-OSS 120B, 128K on Qwen 3.8 27B | Compare | |
| Open weights | DeepSeek V4 Flash, Llama 3.1 8B | Unknown | From $0.02 per 1M | No managed fine-tuning | Shared API, no contracts | 1M on DeepSeek V4 Pro | Compare | |
| Open weights | GLM 5.3, Kimi K3, DeepSeek V4.1 Flash | Routes to fastest provider by default | Provider rates, no markup | N/A | Serverless router; dedicated Endpoints | Up to 1M, provider-dependent | Compare | |
| Bring your own weights | None hosted | ~1s container boot | Per second; H100 $3.95/hr list | Run any training code | Serverless GPU containers | Depends on the model you deploy | Compare | |
| Open weights | DeepSeek V4 Pro, GLM 5.3, Kimi K3, Kimi K2.7 Code, gpt-oss 120B | Unknown | $0.011 per 1K Neurons; 10K free daily | BYO LoRA on small models (beta) | Serverless on Cloudflare network | 1M on DeepSeek V4 and Kimi K3; 262K on Kimi K2.6 and K2.7 Code | Compare | |
| Closed | Grok 4.7, Grok 4.20, grok-build | ~54 tok/s on Grok 4.6 | $2 in, $6 out (Grok 4.7); 2x past 200K | Unknown | First-party API, Amazon Bedrock | 500K (4.7), 1M (4.20, 4.3) | Compare | |
| Open weights, plus closed Codestral | Mistral Large 4, Small 4, Codestral | Unknown | $0.15–$1.50 in, $0.60–$7.50 out per 1M | Forge (enterprise); fine-tuning API deprecated | API, Azure, Bedrock, Vertex, self-host | 1M on Large 4; 256K on others | Compare | |
| Open weights (MIT) | DeepSeek V4.1 Flash | ~209 tok/s on V4.1 Flash | Off-peak hours at half price | Open weights to fine-tune | First-party API, Hugging Face weights | 1M, 384K max output | Compare | |
| Open weights, custom license | Kimi K3.1 Pro, Kimi K3 | ~33 tok/s on Kimi K3 | $2 in, $10 out (K3.1 Pro); $3 in, $15 out (K3) | Open weights to fine-tune | API, Kimi Code, OpenRouter | 1M | Compare | |
| Open weights (GLM-5.3 License; MIT on Flash and older) | GLM-5.3, GLM-5.3-Flash | ~80 tok/s on GLM-5.3 | $1.40 in, $4.40 out (GLM-5.3); free Flash tier | Open weights; GLM-5.3 License adds review for large hosts | API, GLM Coding Plan | 1M (GLM-5.3) | Compare | |
| Open weights incl. Max (custom license) | Qwen 3.8-Max, Qwen 3.7-Max | ~40 tok/s on Qwen 3.8-Max | $2 in, $6 out international | No fine-tuning on hosted Max | Model Studio on Alibaba Cloud | 1M (Qwen 3.8-Max) | Compare | |
| Closed API; open Muse Spark Open and Glimmer | Muse Wonder, Muse Spark 1.3, Muse Spark Open | ~145–233 tok/s on Muse Spark 1.3 | $1.25–$4 in, $4.25–$20 out; Contributor tier cheaper | Open Muse weights to fine-tune | Meta Model API, Oracle Cloud | 1M | Compare | |
| Closed, plus open Command A+ | Command A+, Command A, Embed 5, Rerank 5 | 375 tok/s on Command A+ W4A4, per Cohere | $0.0375–$2.50 in, $0.15–$10 out per 1M | Enterprise fine-tuning, incl. private | API, Bedrock, Azure, OCI, VPC, on-prem | 256K on Command A; 128K on A+ | Compare | |
| Open weights | MiniMax M3, GPT-OSS 120B, DeepSeek V3.1 | ~800 tok/s on MiniMax M2.7 (SN50, SemiAnalysis) | $0.22 in, $0.59 out (GPT-OSS 120B) | Unknown | SambaCloud, racks for neoclouds | Up to 1M (MiniMax M3, preview) | Compare | |
| Open weights, 60+ models | DeepSeek V4 Pro, Kimi K3, MiniMax M3, GLM | Among top hosts on throughput | From $0.06 per 1M input | Serve uploaded fine-tunes | Token Factory, dedicated, raw GPUs | Varies by model | Compare | |
| Open weights | DeepSeek V4, GLM 5.3, Qwen3.8 27B, Nemotron 3 | Up to 9.9x faster TTFT vs vLLM (vendor claim) | $0.05–$1.74 in, $0.20–$4.40 out per 1M | Serverless LoRA fine-tuning | Serverless, self-serve and tailored dedicated, raw GPUs | Varies by model; cluster-wide KV cache | Compare | |
| Hosted media models | FLUX, Kling, Seedream | Cold starts on less popular endpoints | Per image, per video second, GPU time | LoRA training endpoints | Hosted API, serverless GPUs | Not applicable | Compare | |
| Open weights | DeepSeek V4 Pro, Gemma 4 | ~36 tok/s on DeepSeek V4 Pro | From $0.02 per 1M; batch 50% off | Hot-swappable LoRA adapters | Serverless, GPU cloud, dedicated | Full 1M on DeepSeek V4 Pro | Compare | |
| Open weights, plus proxied closed models | GLM 5.3, Kimi K3, DeepSeek V4 Pro | Unknown | $0.05–$30 in, $0.13–$180 out per 1M; DIEM staking | Unknown | Serverless API, consumer app | 1M on most current models | Compare | |
| Any Hugging Face model | Kimi K3, DeepSeek V4.1 Flash, GLM-5.3, MiniMax M3 | 600ms p99 real-time budget | Per-parameter rates; batch 50% off | Private Hugging Face repos | Serverless, elastic, dedicated, batch | Varies by model | Compare | |
| Open, closed and custom | Schematron, ClipTagger, customer fine-tunes | Batch windows of 24h to 7 days | Free pay-as-you-go; Growth $250/mo; B200 $9.98/hr | Distill traces into custom models | Batch API, gateway, dedicated GPUs | Varies by model | Compare | |
| Open and third-party models | DeepSeek V4 Flash, GLM-5.2, Google Veo | Near bare-metal performance | H100 from $2.60/GPU-hr | Unknown | Shared, autoscaling, reserved GPUs | Varies by model | Compare | |
| Thinking Machines | Open weights | Inkling, Inkling-Small | Unknown | Per 1M tokens by prefill, sample, train; Inkling $1.00 in, $4.05 out | LoRA SFT and RL via Tinker | Training API, beta serverless (Inkling only) | Inkling up to 1M (256K serverless); Tinker 32K–256K | Compare |
| Open weights | Kimi K2.6, GLM-5.3, GPT-OSS 120B | Low-latency default; slower windows cheaper | 30–85% below other hosts by completion window | Customer LoRA fine-tunes | API plus Sailboxes | Varies by model | Compare | |
| Specialist models | morph-v3-fast, morph-v3-large | 10,500+ tok/s Fast Apply | $0.80 in, $1.20 out (morph-v3-fast) | Fine-tuning offered | OpenAI compatible API; self-hosted on Enterprise | Unknown | Compare | |
| Open weights, plus specialist coding models | DeepSeek V4.1 Flash, Kimi K3, GLM 5.3, relace-apply-3 | ~10,000 tok/s apply | $0.07 in, $0.50 out (DeepSeek V4.1 Flash) | Unknown | Hosted API, OpenRouter or self-hosted | 128K on apply; varies by open model | Compare | |
| Decision models | Jev, jev-1.14 | 70–500ms per call | $0.042 per 1M input tokens; output free | Unknown | Early-access API, plus Vercel AI Gateway, OpenRouter and LiteLLM | 64K per request, 32K per question | Compare | |
| Open weights (Apache 2.0 or StepFun license) and API models | Step 5 Preview, Step 3.7 Flash | ~128 tok/s on Step 3.7 Flash | $0.20 in, $1.15 out (Step 3.7 Flash) | Open weights to fine-tune | First-party API, OpenRouter | Up to 1M (Step 5 Preview); 256K on Step 3.7 Flash | Compare | |
| Hosted media models | Seedance 2.5, Qwen-Image-3.0 | Unknown | Images from fractions of a cent | Fine-tuned diffusion checkpoints | Unified API, serverless GPUs, bare metal | Not applicable | Compare | |
| Proprietary coding models, plus hosted open models | KAT-Coder-Pro V2.5, KAT-Coder-Air V2.5 | Unknown | $0.74 in, $2.96 out (Pro V2.5), or KwaiKAT Coding Plan | Unknown | MaaS API, bare metal | 256K | Compare | |
| Open weights | GLM-5.3, DeepSeek V4.1 Flash | ~168 tok/s on GLM-5.3; 2–2.8x vs stock vLLM or SGLang (vendor) | Per token, prepaid credits | Agent-tuned dedicated deployments | Serverless, dedicated | Up to 1M (GLM-5.3, DeepSeek V4.1 Flash) | Compare | |
| Open weights | Nemotron 3.5 Lightning 30B, Qwen 3.8 27B | ~541 tok/s on Nemotron 3.5 Lightning 30B (vendor); cold starts under 2s | From $0.05 in, $0.15 out; coding plans from $10 a month | Uploads up to 50 GB; auto-quantization | Model APIs, agent-built endpoints | Varies by model | Compare | |
| Open weights plus its own Particle-1 tune | Particle-1, DeepSeek V4.1 Flash, GLM 5.3 Flash | ~110 tok/s on Particle-1 (Vercel AI Gateway) | $0.15 in, $0.50 out (GLM 5.3 Flash); $0.25 in, $1 out (Particle-1) | Unknown | Serverless API, Vercel AI Gateway | 1M | Compare | |
| Closed and open, 400+ models | DeepSeek, Qwen, Claude, Gemini, GPT | Unknown | Provider rates; 3–5% top-up fee | Custom deployments | Gateway API, dedicated, BYOK | Varies by model | Compare | |
| Closed weights (Mercury dLLMs) | Mercury 3, Mercury Voice | ~1,070 tok/s on Mercury 3 (vendor) | $0.20 in, $0.70 out (Mercury 3); 50% off at launch | Fine-tuning on request | Serverless API, Bedrock, Azure, private | 1M | Compare | |
| Open weights, uncensored fine-tunes | DeepSeek V4.1 Flash, Dolphin Mistral 24B | Unknown | $0.01 in, $0.07 out (DeepSeek V4.1 Flash) | Own uncensored fine-tunes | Decentralized serverless API | Unknown | Compare | |
| Open weights | DeepSeek V4.1 Flash, GLM 5.3 Flash | 200+ tok/s; fast tier ~2x | $0.06 in, $0.24 out (promo, DeepSeek V4.1 Flash) | N/A | Serverless API | 1M context | Compare | |
| Open weights | MiniMax M2.7, DeepSeek V3.2, GPT-OSS 120B | 1,000+ tok/s on GPT-OSS 120B (vendor) | $0.21–$0.28 in, $0.38–$1.20 out per 1M | Custom LoRA and fine-tuned checkpoints | Serverless API, dedicated racks, bare metal | 192K (MiniMax M2.7) | Compare | |
| Open weights, plus closed GPT and Claude | GLM-5.3, DeepSeek V4.1 Flash, Kimi K3 | 342 tok/s on DeepSeek V4.1 Flash 1017 (Vercel) | $0.04–$2.30 in, $0.21–$11.40 out per 1M (open) | Serving-stack tuning, no fine-tuning | Serverless API, reserved tailored capacity | 1M context on most models | Compare |
Frequently asked questions
What is Luminal?
Luminal builds an inference compiler. Where vLLM and SGLang interpret a model at runtime, Luminal compiles it ahead of time into native kernels for GPUs and ASICs. Models get lowered to a small graph of 15 primitive ops, and the compiler searches over fusion, tiling, memory and scheduling choices instead of relying on hand-written rules, which it says can find optimizations like Flash Attention on its own. The compiler is open source in Rust under Apache 2.0 or MIT, runs on CUDA and Metal with ROCm on the roadmap, and works as a torch.compile backend.
What is Luminal best for?
High-throughput serving of a custom or fine-tuned model on owned GPUs; Teams that want an inference engine faster than vLLM without hand-writing kernels.
How much does Luminal cost?
Luminal pricing at a glance: $0.1425 in, $0.57 out (DeepSeek V4.1 Flash). Rates change often, so check Luminal's pricing page before committing.
What are the downsides of Luminal?
Very young company; the hosted catalog is just a couple of models so far; Speed figures are self-reported aggregate throughput, not independent per-request benchmarks.
What are the best alternatives to Luminal?
Common alternatives include Subconscious, OpenAI, Anthropic, Google Vertex AI, Amazon Bedrock. Each has a head-to-head comparison with Luminal on this site.
Sources: Luminal, Luminal pricing, Luminal on GitHub, Luminal seed round, TechCrunch. Pricing and model lineups change often; figures are a snapshot.