# Groq vs Cohere

> Groq serves a few open models very fast on its LPU. Cohere serves its own enterprise models for RAG, with private and on-prem installs Groq does not offer.

Canonical: https://www.subconscious.dev/compare/groq-vs-cohere · By The Subconscious Team · Updated September 30, 2026

## How they compare

Groq optimizes for speed. Its LPU publishes 1,000 tokens per second on GPT-OSS 20B and 500 on GPT-OSS 120B, with tight tail latency, and its small-model prices sit near the market floor. The catalog is small and capped around 131K context, it hosts no fine-tuned models, and it runs only on GroqCloud. Cohere's Command A has a 256K window at $2.50 in and $10 out, Command R7B costs $0.0375 in, and Command A+ runs 128K. Cohere claims 375 tokens per second on Command A+ in 4-bit form, which is fast for a 218B model but below Groq's published figures on smaller models.

Cohere wins on deployment and customization. It fine-tunes inside customer environments, installs in any VPC or on-prem, and sells through Bedrock, Azure AI Foundry and OCI. Embed 4 and Rerank 4 give it retrieval models Groq lacks, while Groq offers Whisper and Groq Compound for voice and agentic search. Groq's long-term outlook is uncertain after NVIDIA licensed the LPU and hired most of its staff. Command A+ weights are also open under Apache 2.0, so teams can self-host rather than rely on one cloud. For voice agents and tight latency budgets, Groq. For private enterprise search and grounded answers, Cohere.

## What each one does

### Groq

Groq serves open models on its own chip, the LPU, which keeps model weights in on-chip SRAM and runs a deterministic schedule instead of waiting on GPU memory. Groq publishes 1,000 tokens per second on GPT-OSS 20B and 500 on GPT-OSS 120B. Latency stays tight between median and tail, which matters for strict SLAs. The API is OpenAI-compatible and also hosts Whisper for speech to text plus an agentic system called Groq Compound with built-in search and code execution.

### Cohere

Cohere is a Toronto-based lab that sells models and platforms to banks, governments and large enterprises rather than consumers. Its generative line is the Command family. Command A+, released May 20, 2026, is a 218B-parameter mixture-of-experts model with 25B active, published under Apache 2.0 with a 128K context window, and it combines reasoning, vision, translation and tool use in one set of weights. Command A has a 256K window and lists at $2.50 in and $10 out per million tokens, while Command R7B costs $0.0375 in. June 2026 added North Mini Code, a 30B Apache 2.0 coding model, and the lineup also includes Aya multilingual models and Transcribe for speech.

## Which is best, and when

### Choose Groq for

- Voice agents that need instant replies
- Strict tail-latency SLAs
- Cheap small-model calls at high volume

### Choose Cohere for

- Private RAG inside a customer network
- Fine-tuned enterprise models
- Reranking and embeddings for search

## At a glance

| Attribute | Groq | Cohere |
|---|---|---|
| Model access | Open weights | Closed, plus open Command A+ |
| Flagship models | GPT-OSS 120B, Qwen 3.6 27B | Command A+, Command A, Embed 4, Rerank 4 |
| Speed | 500–1,000 tok/s | 375 tok/s on Command A+ W4A4, per Cohere |
| Price | Near the floor on small models | $0.0375–$2.50 in, $0.15–$10 out per 1M |
| Customization | No fine-tuned model hosting | Enterprise fine-tuning, incl. private |
| Deployment | GroqCloud API | API, Bedrock, Azure, OCI, VPC, on-prem |
| Long context | Around 131K max | 256K on Command A; 128K on A+ |

## FAQ

### What is the difference between Groq and Cohere?

Groq serves a few open models very fast on its LPU. Cohere serves its own enterprise models for RAG, with private and on-prem installs Groq does not offer.

### When should I choose Groq over Cohere?

Voice agents that need instant replies; Strict tail-latency SLAs; Cheap small-model calls at high volume.

### When should I choose Cohere over Groq?

Private RAG inside a customer network; Fine-tuned enterprise models; Reranking and embeddings for search.

### Is Groq or Cohere cheaper?

Groq: Near the floor on small models. Cohere: $0.0375–$2.50 in, $0.15–$10 out per 1M. The cheaper choice depends on the model and workload.

### Which has more context, Groq or Cohere?

Groq: Around 131K max. Cohere: 256K on Command A; 128K on A+.

## Running long-horizon agents?

If your agents run past 200K tokens, compare both against Subconscious: [Subconscious vs Groq](https://www.subconscious.dev/compare/subconscious-vs-groq.md), [Subconscious vs Cohere](https://www.subconscious.dev/compare/subconscious-vs-cohere.md).

Full profiles: [Groq](https://www.subconscious.dev/providers/groq.md), [Cohere](https://www.subconscious.dev/providers/cohere.md).