AI requires a complete infrastructure pipeline
Accelerators are only one layer of an AI service; data, software, networking, operations and governance are equally important.
Training and inference
| Dimension | AI training | AI inference |
|---|---|---|
| Purpose | Learn model parameters from data | Use a trained model to produce outputs |
| Workload | Large repeated computations over datasets | Requests from users or applications |
| Hardware emphasis | Many accelerators and fast interconnects | Latency, availability and efficient serving |
| Storage | Training data and checkpoints | Model weights, cache and logs |
| Operational concern | Long jobs, failure recovery and scheduling | Response time, scaling and reliability |
Responsible operation
AI infrastructure must protect data, control access, monitor models, manage bias and consider the energy and water implications of computation.