> ## Documentation Index
> Fetch the complete documentation index at: https://docs.jaylin-docs.online/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Infra 基础入门

> 系统推理架构、并行策略（DP/PP/MP/TP）与 Mooncake 架构解析

# AI Infra 基础入门

### 系统推理架构

<img src="https://mintcdn.com/jnu-e8007313/dskRMJ8NpIGQNbDD/docs/Mooncake%20AI%20Infra/images/image-20260327212858916.png?fit=max&auto=format&n=dskRMJ8NpIGQNbDD&q=85&s=f6f681dee0fae0fea36dddb46e85e160" alt="image-20260327212858916" width="588" height="456" data-path="docs/Mooncake AI Infra/images/image-20260327212858916.png" />

### SM 架构

[SM 架构解答](https://zhuanlan.zhihu.com/p/8435687930)

<img src="https://mintcdn.com/jnu-e8007313/dskRMJ8NpIGQNbDD/docs/Mooncake%20AI%20Infra/images/image-20260327213251709.png?fit=max&auto=format&n=dskRMJ8NpIGQNbDD&q=85&s=92686189a7d75d862303de06bae589cd" alt="image-20260327213251709" width="574" height="648" data-path="docs/Mooncake AI Infra/images/image-20260327213251709.png" />

### HBM 架构

<img src="https://mintcdn.com/jnu-e8007313/dskRMJ8NpIGQNbDD/docs/Mooncake%20AI%20Infra/images/image-20260327213343632.png?fit=max&auto=format&n=dskRMJ8NpIGQNbDD&q=85&s=b8e8c677501f8b358bb81aec7312fe0b" alt="image-20260327213343632" width="379" height="518" data-path="docs/Mooncake AI Infra/images/image-20260327213343632.png" />

## DP/PP/MP/TP

**DP 数据并行**:每个GPU保存完整的模型副本，处理不同的数据批次

<img src="https://mintcdn.com/jnu-e8007313/dskRMJ8NpIGQNbDD/docs/Mooncake%20AI%20Infra/images/image-20260327195737267.png?fit=max&auto=format&n=dskRMJ8NpIGQNbDD&q=85&s=49777b7c293d75a11d8a449f490ca231" alt="image-20260327195737267" width="589" height="523" data-path="docs/Mooncake AI Infra/images/image-20260327195737267.png" />

**MP 模型并行**：将模型切分到不同GPU，每个GPU只存储部分参数

<img src="https://mintcdn.com/jnu-e8007313/dskRMJ8NpIGQNbDD/docs/Mooncake%20AI%20Infra/images/image-20260327195842794.png?fit=max&auto=format&n=dskRMJ8NpIGQNbDD&q=85&s=7bce886ac3e8862c2d1a05bcd0a392aa" alt="image-20260327195842794" width="553" height="358" data-path="docs/Mooncake AI Infra/images/image-20260327195842794.png" />

GPU利用率低，存在流水线气泡

**PP 流水线并行**：结合数据并行和模型并行，将数据分批处理以减少空闲时间。

<img src="https://mintcdn.com/jnu-e8007313/dskRMJ8NpIGQNbDD/docs/Mooncake%20AI%20Infra/images/image-20260327200110691.png?fit=max&auto=format&n=dskRMJ8NpIGQNbDD&q=85&s=072e329ae8def398fb7b50be57b0a07d" alt="image-20260327200110691" width="533" height="398" data-path="docs/Mooncake AI Infra/images/image-20260327200110691.png" />

**TP 张量并行**：单层计算拆分到多个GPU

<img src="https://mintcdn.com/jnu-e8007313/dskRMJ8NpIGQNbDD/docs/Mooncake%20AI%20Infra/images/image-20260327200936942.png?fit=max&auto=format&n=dskRMJ8NpIGQNbDD&q=85&s=e7d0680833dd30b12ba360818db193f4" alt="image-20260327200936942" width="576" height="443" data-path="docs/Mooncake AI Infra/images/image-20260327200936942.png" />

### RMDA

允许机器直接访问另一台机器的内存，无需CPU介入

#### NCCL 通信库

#### Moonckae 架构

<img src="https://mintcdn.com/jnu-e8007313/dskRMJ8NpIGQNbDD/docs/Mooncake%20AI%20Infra/images/image-20260327215008621.png?fit=max&auto=format&n=dskRMJ8NpIGQNbDD&q=85&s=2441a9466b8ba44f016b0467122311cb" alt="image-20260327215008621" width="599" height="626" data-path="docs/Mooncake AI Infra/images/image-20260327215008621.png" />