The AMD Instinct MI350P is AMD’s first new PCIe Instinct GPU in four years, succeeding the 2022 MI210. It brings CDNA 4 to a standard add-in card, delivering up to 4.6 PFLOPS MXFP4 performance, 144GB HBM3E and 4TB/s memory bandwidth in a dual-slot, 600W air-cooled design. It lets users deploy Instinct hardware without adopting the larger 8-GPU OAM MI350/5X platform.

Key Takeaways

Its headline feature is 144GB of memory on a PCIe card — more than any current PCIe accelerator, 3GB above the H200 NVL and 48GB above the RTX PRO 6000 Blackwell. Ten of fourteen tested model and precision combinations ran entirely on one MI350P.

Measured compute sits between the two NVIDIA cards. At 450W, it hit 580 TFLOPS BF16, 1,120 TFLOPS FP8 and 2,228 TFLOPS block-scaled FP4, 36–38% faster than a 600W RTX PRO 6000 across all precisions. The H200 NVL outperformed it in BF16 and FP8 and lacks native FP4 support.

GPT-OSS was its standout workload. On AMDGPU 31.60, ROCm 10.1.0 and vLLM 0.32.0, GPT-OSS-120B hit 5,057 output tokens/sec at 128 chat streams, 65% ahead of the RTX PRO 6000 and 50% ahead of the H200 NVL, with throughput still climbing up to 128 streams for both prompt formats.

MI350P excels on chat-length prompts; long prompts produce mixed results. At 512-token inputs, it beat the RTX PRO 6000 by 35–65% on all FP8, BF16 and GPT-OSS runs at 128 streams, while FP4 comparisons landed within 5–21%. For 8,192-token long prompts, it led on GPT-OSS and Mistral Small but fell behind on Llama, Qwen FP8 and MiniMax-M2.7, where its throughput curve flattened early.

Dual-card results used ROCm 7.2.3 for DeepSeek-V4-Flash, MiniMax-M3, GLM-5.2 and GLM-5.3 Flash. ROCm 10.1 test runs for these models relied on CPU expert offloading, and DeepSeek also used a speculative decoder not enabled on the NVIDIA side, making those runs non-comparable.

The 600W full-power performance remains untested. Dell’s documentation lists eight 600W MI350Ps in the PowerEdge XE7745, and full-power testing is planned.

AMD also demonstrated liquid-cooled MI350Ps at IFA 2026 inside its Threadripper Halo Station prototype. This workstation design supports up to four MI350Ps paired with a 96-core Threadripper PRO CPU for 576GB total HBM3E; AMD’s updated product page targets a 2027 launch. This positions the MI350P both as a server PCIe accelerator and a building block for deskside AI systems competing with platforms like the GB300 DGX Station. This review focuses on server use cases, benchmarking primarily against the same power-class, same-slot NVIDIA RTX PRO 6000 Blackwell, while the H200 NVL is included as the highest-bandwidth NVIDIA PCIe option. The GB300 DGX Station serves as a high-end deskside reference, sitting in a separate performance and price tier.

Specifications

AMD’s rated peak performance emphasizes low-precision math: 4.6 PFLOPS MXFP4/MXFP6, 2.3 PFLOPS FP8 and 1.15 PFLOPS FP16/BF16, all dense compute figures. Its memory subsystem includes 144GB HBM3E over a 4096-bit bus at 4TB/s, backed by 128MB Infinity Cache. The card supports a 600W maximum board power and a configurable 450W mode; the review sample ran at 450W.

It shares the CDNA 4 architecture with the larger MI350/5X series and is effectively half of an MI350X. The MI350X uses TSMC CoWoS 3D-stacked chiplet packaging with eight accelerator complex dies (XCDs), two I/O dies and eight HBM3E stacks. The MI350P cuts this arrangement in half: four XCDs, one I/O die and four HBM3E stacks, for 128 compute units versus the MI350X’s 256. Memory and cache resources are also halved.

The 450W test setting put the MI350P on equal power footing with the H200 NVL in the R770, while the RTX PRO 6000 ran at its full 600W, giving it a 150W power advantage in side-by-side comparisons. Its single 12V-2×6 connector and standard double-slot PCB let it reuse risers built for RTX PRO 6000 or H200 NVL.

The comparison table below contrasts it with the three NVIDIA platforms tested: the slot-compatible RTX PRO 6000 Blackwell Server Edition, the high-bandwidth Hopper H200 NVL PCIe card, and the GB300 DGX Station combining one Blackwell Ultra GPU with a Grace CPU.

Against the RTX PRO 6000, the MI350P’s memory subsystem is its biggest differentiator, with 50% more VRAM and roughly 2.5× higher bandwidth. This extra capacity determines whether many large models fit on a single card instead of needing multi-GPU distribution. The RTX PRO 6000, however, includes graphics, ray tracing and video hardware alongside AI acceleration for broader workload coverage.

The H200 NVL is its closest high-memory PCIe competitor, with nearly identical memory capacity and 20% higher bandwidth. Unlike the MI350P, it supports NVLink for direct GPU-to-GPU communication. The MI350P’s native MXFP4/MXFP6 support is a key advantage for low-precision inference, though its lower bandwidth can limit performance on bandwidth-bound long-prompt workloads.

The GB300 DGX Station represents a much higher tier. It uses a full datacenter-class Blackwell Ultra GPU, while the MI350P is half of a datacenter chip fitted into a PCIe power envelope. On paper, two MI350Ps match or exceed its total memory and bandwidth, but cross-card traffic must traverse PCIe peer-to-peer links, limiting scaling gains.

Performance

The test bed for the MI350P was a Dell PowerEdge R7725 with two AMD EPYC 9555 CPUs, 1.5TB DDR5-6400 and two MI350Ps capped at 450W for 2U server operation. RTX PRO 6000 cards ran in a Dell PowerEdge XE7740, the H200 NVL ran inside a Dell R770, and GB300 results came from a separate DGX Station. The two MI350Ps occupied PCIe Gen5 x16 slots attached to different CPU sockets, meaning tensor-parallel traffic crosses the socket interconnect. Host memory capacity was critical for GLM-5.2, which spills some expert weights out of GPU memory and onto system RAM.

Maximum Achievable Matmul FLOPS (MAMF)

MAMF measures realistic sustained matrix multiplication throughput rather than theoretical peak FLOPS. At 450W, one MI350P delivered 580 TFLOPS BF16, 1,120 TFLOPS FP8 and 2,228 TFLOPS FP4, outperforming the 600W RTX PRO 6000 by 36–38% across all tested precisions. The H200 NVL was faster in BF16 and FP8 without FP4 support. The GB300 led all PCIe cards by a large margin.

GPU Memory Bandwidth

Raw compute only matters if data can feed the execution units, especially for LLM decode, which is often bandwidth-limited. The MI350P achieved 3,291GB/s read and 3,204GB/s write throughput, around 80% of its rated 4TB/s, giving it roughly 2.1× the usable bandwidth of the RTX PRO 6000’s GDDR7. The H200 NVL reached 4,472GB/s read and 4,156GB/s write, 30–36% higher than the MI350P, matching its 4.8TB/s specification. Dual MI350Ps reached aggregate read throughput near the GB300, but this bandwidth is split across two PCIe cards with cross-traffic overhead.

LLM Inference

Inference testing used vLLM serving with an OpenAI-compatible endpoint, tracking output tokens per second. Two prompt shapes were tested: 512-token input + 512-token output (chat-focused, decode-heavy) and 8,192-token input + 1,024-token output (document/RAG, prefill-heavy). Concurrency scaled upward until a configuration failed to complete all requests. Models ran on the minimum number of GPUs required to hold their weights. Single-card MI350P tests used ROCm 10.1.0 and vLLM 0.32.0; dual-card large-model tests used ROCm 7.2.3. NVIDIA workloads ran on CUDA 13.4.1 with vLLM 26.08 containers. AMD uses MXFP4 while NVIDIA uses NVFP4 for FP4 workloads.

Chat-Style (512-in, 512-out) Summary

At 128 streams, the MI350P beat the RTX PRO 6000 by 35–65% on all FP8, BF16 and GPT-OSS workloads. It only fell behind NVIDIA’s NVFP4 runs on the highest concurrency points of Llama 8B and MiniMax-M2.7. The H200 NVL held a small lead on dense FP8/BF16 models, while the MI350P led strongly on both GPT-OSS variants.

Long Prompt (8192-in, 1024-out) Summary

Prefill-heavy long prompts shifted the advantage toward NVIDIA in many dense models. The MI350P retained the lead for GPT-OSS and Mistral Small, split results on Qwen3-Coder, and flattened early on Llama and MiniMax-M2.7, finishing behind the RTX PRO 6000. The H200 NVL pulled ahead on most dense long-prompt runs, though the MI350P still won on GPT-OSS.

Individual model trends:

• GLM-5.3 Flash: Two MI350Ps surpassed the GB300 Station at high concurrency for chat workloads, while the Station remained faster on long prompts.
• GLM-5.2: Weight spilling to host memory limited single-stream speed. Four RTX PRO 6000s led at lower concurrency; the dual MI350P pair overtook them at 32 chat streams. Long-prompt throughput stayed low for PCIe configurations.
• MiniMax-M3: Two MI350Ps competed closely with four RTX PRO 6000s at low concurrency on chat prompts, with the NVIDIA quad pulling ahead at 16 streams. The MI350P pair led all PCIe setups for long prompts up to 8 streams.
• DeepSeek-V4-Flash: Dual RTX cards led at low chat concurrency; dual MI350Ps took over from 16 streams onward. The pattern repeated for long prompts.
• MiniMax-M2.7: One MI350P outperformed two RTX PRO 6000s at low concurrency, but the NVIDIA pair scaled better and pulled ahead at high streams.
• GPT-OSS-120B / GPT-OSS-20B: The MI350P consistently outperformed both the RTX PRO 6000 and H200 NVL across all concurrency levels and prompt types. Published MLPerf results on tuned stacks show even higher throughput for eight MI350Ps.
• Qwen3-Coder-30B-A3B: The MI350P led on chat workloads. For long prompts, it won on BF16 but fell behind on FP8 at high concurrency.
• Mistral Small 24B: One of the MI350P’s strongest dense-model wins against the RTX PRO 6000 for chat prompts, and it also led for long prompts, though the H200 NVL remained faster.
• Llama 3.1 8B: The MI350P led the RTX PRO 6000 at low concurrency on MXFP4, FP8 and BF16 for chat prompts, nearly matching the H200 on FP8 at 128 streams. Long-prefill workloads reversed this, with NVIDIA pulling ahead at higher concurrency.

Conclusion

The MI350P is AMD’s first modern Instinct add-in card built for standard servers. At 450W in a 2U chassis, it delivers competitive PCIe accelerator performance, with its 144GB HBM3E and 4TB/s bandwidth forming its core strength. It outran the RTX PRO 6000 on chat-style inference for FP8, BF16 and GPT-OSS, thanks to larger memory and higher bandwidth, and dominated GPT-OSS across all prompt lengths. The H200 NVL still held an edge on most dense FP8/BF16 workloads, aided by its higher bandwidth, while the MI350P’s native FP4 support paid off on GPT-OSS. Larger MoE models showed mixed scaling, with GPU count and total VRAM often deciding the winner.

Full 600W performance remains untested, though AMD estimates roughly 10% additional throughput at the higher power limit, and testing on the Dell XE7745 platform is planned. AMD’s software stack maturity has improved noticeably; most tested workloads ran cleanly on stock vLLM.

This card targets buyers seeking NVIDIA alternatives, motivated by supply, pricing or vendor diversification, with Dell enterprise server support bringing this option into mainstream deployments. It is best suited for models falling in the 96GB–144GB memory range, where it can keep the entire model resident on one PCIe GPU. As ROCm and supporting tooling mature further, the MI350P and future CDNA 4 add-in products have strong potential in the enterprise AI accelerator market. It represents a promising return for AMD into the server add-in GPU segment.

Beijing Qianxing Jietong Technology Co., Ltd.
Sandy Yang/Global Strategy Director
WhatsApp / WeChat: +86 13426366826
Email: yangyd@qianxingdata.com
Website: www.qianxingdata.com/www.storagesserver.com
Business Focus:
ICT Product Distribution/System Integration & Services/Infrastructure Solutions
With 20+ years of IT distribution experience, we partner with leading global brands to deliver reliable products and professional services.
“Using Technology to Build an Intelligent World”Your Trusted ICT Product Service Provider!