Back to all tools
Groq
World's fastest LLM inference — 800 tokens/sec on Llama 3 with LPU chips.
0Free-Limited
LLM Platforms
Overview
Groq provides ultra-fast LLM inference using custom Language Processing Units (LPUs), delivering Llama 3, Mixtral, and Gemma at 300–800 tokens/second.
Available Models
— may change as new versions release| Model | Context | Best For | Cost Tier |
|---|---|---|---|
| Llama 3.3 70B | 128K | Best open model for general tasks, fast on Groq | Free |
| Llama 3.1 8B | 128K | Ultra-fast, lightweight, high-volume use | Free |
| DeepSeek R1 70B | 128K | Reasoning and math at open-source speeds | Free |
| Mixtral 8x7B | 32K | MoE model, instruction following | Free |
| Gemma 2 9B | 8K | Small, fast, Google-tuned model | Free |
| Whisper Large v3 | — | Speech-to-text transcription | Free |
Key Features
- 300–800 tokens/second inference speed
- OpenAI-compatible API — drop-in replacement
- Llama 3, Mixtral, Gemma, and Whisper models
- Generous free tier — 30 requests/minute
- Streaming responses with minimal latency
- Tool use and JSON mode support
Real-World Workflows
Low-latency AI for real-time applications
- 1Replace your OpenAI client with Groq (base_url change only)
- 2Select llama-3.3-70b-versatile as your model
- 3Measure response latency — typically under 1 second for medium prompts
- 4Use streaming to show tokens as they arrive in your UI
Getting Started
pip install groq
from groq import Groq
client = Groq() # uses GROQ_API_KEY env var
chat = client.chat.completions.create(
model='llama-3.3-70b-versatile',
messages=[{'role': 'user', 'content': 'What is Groq?'}]
)
print(chat.choices[0].message.content)Compare Alternatives
See how Groq stacks up against similar tools.