freellmpool › providers › Groq

Free Groq API: the fast one

Groq's free API is the one developers reach for when latency matters. It runs models on custom LPU hardware that streams tokens noticeably faster than typical GPU inference. The current freellmpool routes include OpenAI's GPT-OSS 20B/120B, Qwen 3.6 27B, Groq Compound, and a pin-only Qwen 3.8 27B Preview route. You get a key free at console.groq.com/keys and call an OpenAI-compatible endpoint at https://api.groq.com/openai/v1. The main constraint is the free daily/minute rate limit, which is where pooling Groq with other tiers via freellmpool helps.

What Groq's free tier is good for

Because generation is so fast, Groq is a great default for anything interactive: chat UIs, autocomplete, streaming agents, and tool-use loops where round-trip latency compounds. It's less suited to extremely long context windows or multimodal input — for big documents or vision, Gemini is a better free pick. Treat Groq as your low-latency workhorse and keep a higher-context provider in reserve.

Which free model to pick

Model	Use it for
`openai/gpt-oss-120b`	Strong reasoning when you can spare the budget
`openai/gpt-oss-20b`	Fast drafts, classification, and routing
`qwen/qwen3.6-27b`	General chat and multilingual work
`qwen/qwen3.8-27b`	Explicit pin for newer text/streaming work; Preview, not automatic
`groq/compound`	Groq's compound system; lower free-plan daily limit

Get a key and call it

Sign in at console.groq.com/keys, create a key, and use the OpenAI-compatible route — most OpenAI SDKs work by only changing the base URL:

curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-oss-120b","messages":[{"role":"user","content":"Hi"}]}'

Limits and gotchas

Free usage is capped by requests/minute, requests/day, and tokens/minute per model — the per-minute token cap is often what you hit first on long outputs. Check Groq's current rate-limit table; the values change.
llama-3.1-8b-instant and llama-3.3-70b-versatile were shut down for Free and Developer plans on August 16, 2026. Groq's deprecation history is authoritative.
qwen/qwen3.8-27b is a Groq Preview model on the Free plan (1,000 RPD and 2M TPD in the official table checked 2026-08-29). It passed exactly three sanitized chat completions and a separate streaming canary through freellmpool. The forced tool canary returned unsupported, so the route is enabled pin-only for explicit text use and is not advertised for automatic agent/tool routing. The sanitized admission evidence is retained in machine-readable form.
Some models are reasoning/“thinking” models and burn extra output tokens — set a generous max_tokens or you'll get truncated answers.

Pool Groq with other free tiers

Groq's speed is great until you hit the minute/day cap mid-task. freellmpool keeps Groq as a preferred provider and transparently fails over to Gemini, NVIDIA and others on a 429, so a single limit doesn't stall you:

pip install freellmpool
export GROQ_API_KEY=...                 # plus any other free keys
freellmpool ask -p groq "..."           # pin Groq when you want its speed
freellmpool ask "..."                   # or pool + fail over automatically
FREELLMPOOL_ROUTING=fast freellmpool ask "..."   # prefer the lowest-latency tier

See also Cerebras (fast inference with finite trial credit), best free LLM API gateway, and using multiple free LLM APIs together.

FAQ

Is the Groq API free?

Yes. Create a key at console.groq.com/keys and call the OpenAI-compatible endpoint at api.groq.com/openai/v1. Free usage is rate-limited per minute and per day; verify current numbers in the console.

Why is Groq so fast?

Groq runs inference on its own LPU hardware designed for sequential token generation, which gives much higher tokens-per-second than typical GPU serving — useful for interactive and streaming workloads.

What's the best free Groq model?

Use openai/gpt-oss-120b for harder reasoning, openai/gpt-oss-20b for speed, qwen/qwen3.6-27b for a general automatic Qwen route, or explicitly pin the Preview qwen/qwen3.8-27b for its newer text/streaming capability. Its observed tool canary is unsupported, so keep tool loops on a verified route. Verify Groq's current model and rate-limit pages because availability changes.

Part of freellmpool (MIT, open source). Limits and model IDs change — check Groq's docs. Updated 2026-08-29.