AMD Radeon RX 6700 XT
LLM Inference Performance
| Model | Tokens / sec | Local Fit |
|---|---|---|
| Mistral 7b Q4 | 13 tok/s | fits · single GPU |
| Llama 3 8b Q4 | 12 tok/s | fits · single GPU |
| Llama 3 13b Q4 | 8 tok/s | fits · single GPU |
| Llama 3 70b Q4 | — OOM — | OOM / offload |
Local Model Compatibility
Spec Sheet
Analysis notes
Quick Summary
AMD Radeon RX 6700 XT is a 12GB AMD card for local AI workloads. It uses RDNA 2, draws about 230W, and can run many 13B quantized models locally. For AI buyers, the main questions are VRAM ceiling, ROCm support, memory bandwidth, and used-market price.
Specs That Matter for AI
The 12GB VRAM pool sets the practical model-size limit. Below 12GB, local LLM use becomes tighter and often requires smaller quantizations, smaller context windows, or CPU offload. Memory bandwidth is listed at roughly 384 GB/s, which helps token generation when the whole model fits on card.
AI Workload Fit
ROCm is the platform note to verify first. ROCm support can be strong on Linux, but app support and version matching need more care than CUDA. The card does not have enough VRAM for comfortable 70B 4-bit inference.
Verdict
AMD Radeon RX 6700 XT starts as a 2.8/5 AI-value candidate in this seed catalog. That rating should be refined after Playwright harvest pulls rendered review pages, benchmark tables, and firsthand reports into the evidence corpus.
Frequently Asked Questions
- Can the AMD Radeon RX 6700 XT run local LLMs?
- Yes. With 12GB of VRAM, it can run 7B quantized models locally and many 13B quantized models with practical settings.
- Is the AMD Radeon RX 6700 XT good for AI inference?
- It can work well with ROCm-supported stacks, especially on Linux, but compatibility should be checked per tool.