The question sounds like it needs a complicated answer. It does not. For most people reading this — hobbyists, researchers, indie developers, and small teams — the answer is one. One GPU is enough for the vast majority of AI training workloads you will actually encounter.
Quick answer: One GPU handles LoRA fine-tuning, small model training, and most hobbyist/researcher workflows. Multiple GPUs only become necessary for training 70B+ parameter models from scratch, large-scale datasets, or production training pipelines where speed is a business constraint.
NVIDIA GeForce RTX 4090
24GB GDDR6X24GB VRAM handles LoRA fine-tuning, 7B-13B training, and most hobbyist/research workflows
Affiliate links — we may earn a commission at no extra cost to you. Amazon ships globally; Shopee SG covers Singapore & ASEAN.
The single GPU truth
Most AI training content on the internet is written for production machine learning teams or academic researchers working on frontier models. For everyone else, the scale assumptions are wrong.
Here is what a single high-end GPU actually handles:
- LoRA and QLoRA fine-tuning — the primary way most users customize LLMs, runs on 16-24GB with standard configurations
- 7B parameter model training — fits on a 24GB card with mixed precision
- 13B model fine-tuning with QLoRA — fits on 24GB with 4-bit quantization
- Diffusion model training — Dreambooth, LoRA for image models, custom checkpoint training
- Small model training from scratch — sub-1B parameter models, domain-specific models
If your work lives in this list, buying multiple GPUs adds cost and complexity without improving your actual capability.
When does one GPU become insufficient?
The ceiling for single-GPU training is real. You will outgrow one card when:
Model size exceeds VRAM: Training (not inference) requires holding the model weights, optimizer states, and gradients simultaneously. A 70B parameter model in FP16 requires roughly 140GB of VRAM for weights alone — far beyond any single consumer GPU. Even with QLoRA tricks, 70B training benefits significantly from multi-GPU setups.
Dataset size and training speed: With one GPU, a large dataset fine-tune might take days or weeks. If time-to-completion is a business constraint, adding GPUs multiplies throughput proportionally.
Context length during training: Training on very long context (100K+ tokens) increases memory usage substantially during attention computation. Multi-GPU with sequence parallelism becomes attractive here.
Production training pipelines: Teams that train and retrain models on a schedule — for products, customer data, or continuous improvement — need throughput one GPU cannot provide.
Single GPU vs multi-GPU: decision framework
| Your situation | How many GPUs |
|---|---|
| LoRA fine-tuning 7B-13B models | 1 |
| Full fine-tuning 7B models | 1 (24GB+) |
| Dreambooth / diffusion training | 1 |
| Training models under 1B params | 1 |
| QLoRA fine-tuning 70B | 2-4 |
| Full fine-tuning 70B | 8+ |
| Training 70B+ from scratch | 32+ |
| Production retraining pipelines | Depends on schedule |
For hobbyists and indie researchers: the first four rows cover most scenarios. You do not need the bottom rows.
Best single GPU for AI training
RTX 4090 (~$2,200)
The RTX 4090 is the standard recommendation for single-GPU training. 24GB VRAM handles 13B fine-tuning with QLoRA, 7B full fine-tuning, and all diffusion training workflows. Fast compute means shorter experiment cycles. This is the card that lets you run training overnight and iterate quickly.
Check NVIDIA GeForce RTX 4090 on Amazon→Buy on Shopee SG→RTX 4070 Ti Super (~$800)
The 4070 Ti Super at 16GB is the value option. You lose some VRAM headroom compared to the 4090, but LoRA and QLoRA fine-tuning for 7B-13B models still works. For users focused on image model training (Dreambooth, LoRA) rather than LLM training, 16GB is sufficient for most workflows.
NVIDIA GeForce RTX 4070 Ti Super
16GB GDDR6X16GB covers LoRA/QLoRA fine-tuning and diffusion training at $800
Affiliate links — we may earn a commission at no extra cost to you. Amazon ships globally; Shopee SG covers Singapore & ASEAN.
If you do need multiple GPUs
When your workload genuinely exceeds one GPU, there are practical options:
Consumer multi-GPU (2x RTX 4090): Costs ~$4,400, provides 48GB total VRAM over PCIe — the 4090 has no NVLink. Handles 34B model fine-tuning and aggressive 70B QLoRA. Requires a motherboard with two full PCIe x16 slots and a large enough case.
Cloud training: For one-off large training runs, renting multi-GPU cloud instances is often cheaper than buying dedicated hardware. Check the best multi-GPU setup for AI guide for the hardware side.
Gradient checkpointing and CPU offloading: Before buying more GPUs, exhaust the software options. Techniques like gradient checkpointing trade compute time for memory and can extend your single GPU surprisingly far.
The cost math
A second GPU roughly doubles your training throughput in ideal conditions — but never perfectly due to communication overhead. Before buying:
- 2x RTX 4090 at $4,400 total vs renting 2x A100 hours on cloud — run the math on how many training hours you actually do per month
- Driver and framework setup for multi-GPU (DDP, FSDP, DeepSpeed) adds complexity that single-GPU setups do not have
- Power requirements — two RTX 4090s pull ~1,000W under load, requiring a PSU upgrade and potentially dedicated circuit — use our PSU calculator for AI GPUs to size your power supply correctly before buying
For AI training at home, the single-GPU path avoids all of this complexity.
Which GPU should YOU buy?
Start with 1 GPU if:
- You are doing LoRA or QLoRA fine-tuning of models up to 13B
- You train diffusion models (Dreambooth, image LoRA)
- You are learning AI training workflows — adding GPUs later is always possible
- Your training runs complete in acceptable time with one card
Consider 2 GPUs if:
- You regularly need to fine-tune 34B-70B models
- Training time is a direct business cost (you sell the models or depend on retraining schedules)
- You have already maxed out software optimizations on one card
Use cloud instead of buying more GPUs if:
- Your large training runs are infrequent (monthly or less)
- You need 8+ GPUs — owning that hardware is almost never cost-effective for individuals
Common mistakes to avoid
- Buying multiple GPUs before exhausting one-GPU software tricks. Gradient checkpointing, QLoRA, mixed precision, and CPU offloading together can dramatically extend single-GPU capability. Try them before spending on more hardware.
- Assuming training requires the same VRAM as inference. Training holds the model, gradients, and optimizer states simultaneously. A 7B model that runs inference on 6GB needs 20GB+ for training.
- Treating “supports multi-GPU” as a reason to use it. Libraries support multi-GPU training, but single-GPU setups have less communication overhead, simpler debugging, and fewer driver issues. Complexity has a real cost.
- Scaling hardware before understanding your bottleneck. If your training loop is CPU-bound or IO-bound, more GPUs do not help. Profile first.
Final verdict
For the overwhelming majority of people asking this question: one GPU is enough. Buy the best single GPU your budget allows — the RTX 4090 for deep learning is the standard recommendation — and use software optimizations to stretch its capability as far as possible. Add more GPUs when you have a specific, quantified reason to do so, not because the training infrastructure exists to support it.
Check NVIDIA GeForce RTX 4090 on Amazon→Buy on Shopee SG→