GPU (Graphics Processing Unit)
Technical InfrastructureA processor originally designed for rendering graphics that turned out to be ideal for AI training and inference due to its ability to perform many calculations simultaneously.
GPUs are the hardware engine behind the AI revolution. Originally built for video games, their architecture — thousands of small cores working in parallel — is perfect for the matrix math that neural networks require. Training GPT-4 reportedly used tens of thousands of NVIDIA GPUs.
NVIDIA dominates the AI GPU market with its H100 and upcoming B200 chips. Each H100 costs $25,000-40,000, and the biggest AI labs buy them by the tens of thousands. This GPU scarcity is why AI compute is expensive and why NVIDIA became one of the most valuable companies in the world.
For end users, GPU specs matter when running local AI models (Stable Diffusion, Llama). You need a GPU with sufficient VRAM — 8GB minimum for basic image generation, 12-24GB for larger models. Cloud GPU rental (RunPod, Lambda) is an alternative to buying expensive hardware.
Real-World Example
Running Stable Diffusion locally requires a decent GPU — the image generation happens on your graphics card's parallel processors.
Related Terms
More in Technical Infrastructure
FAQ
What is GPU (Graphics Processing Unit)?
A processor originally designed for rendering graphics that turned out to be ideal for AI training and inference due to its ability to perform many calculations simultaneously.
How is GPU (Graphics Processing Unit) used in practice?
Running Stable Diffusion locally requires a decent GPU — the image generation happens on your graphics card's parallel processors.
What concepts are related to GPU (Graphics Processing Unit)?
Key related concepts include VRAM (Video RAM), Training, Inference. Understanding these together gives a more complete picture of how GPU (Graphics Processing Unit) fits into the AI landscape.