1
Inputs
Chips, power, memory, networking
Learn
What is AI compute?
The basic resource behind training and running AI models.
AI compute is the capacity used to train and run artificial-intelligence models. It includes the chips doing the work, the memory and networking that connect them, the data centers and power that support them, and the cloud or infrastructure access that makes them usable.
More than GPUsBasics
Compute includes chips, memory, networking, power, facilities, and access to them.
Two main jobsWorkloads
Training builds models. Inference runs them after deployment.
Example
A simple way to think about AI compute
Think of AI compute as the factory capacity behind AI. GPUs do the core work, but the factory also needs memory, networking, power, cooling, and data-center space to turn chips into usable output.
2
Work
Training or inference
3
Output
A model learned or a model response served
Market map
What AI compute includes
- Accelerators such as GPUs that perform the core calculations.
- Memory and networking that move data fast enough to keep chips useful.
- Data centers, power, and cooling that make large-scale deployments possible.
- Cloud, rental, and ownership models that determine how buyers access capacity.
Workloads
Training and inference use compute differently
Build
Training
Training uses large amounts of compute to teach a model from data. It is usually concentrated, expensive, and capacity-intensive.
Run
Inference
Inference uses compute after deployment to answer prompts, generate outputs, or serve users repeatedly at scale.
Common mistake
Compute is not just a chip count
Two companies can own or rent the same number of GPUs and still have very different usable compute. Chip generation, memory, networking, power, software, and utilization all affect how much real work the system can deliver.
- A GPU without enough power or cooling is not usable capacity.
- Weak networking can limit how well many GPUs work together.
- The same hardware can produce different value depending on the workload and operating environment.
Why it matters
Why AI compute matters
- It determines how fast models can be trained and improved.
- It affects how expensive AI products are to run after launch.
- It links chips, cloud, power, and infrastructure into one market.
Keep learning
Related lessons
Concept
Why compute matters
Why chips, power, and capacity are becoming economic constraints.
Unit
What is a GPU-hour?
The basic unit behind compute pricing.
Compare
H100 vs H200 vs B200
How accelerator generations affect performance, supply, and cost.