REFERENCE DESIGNRHODIUM-192

RHODIUM-192 — dual-GPU training rig.

Two 96 GB cards and a Threadripper with the PCIe lanes to feed them. For work that does not fit on one card and cannot go to a cloud.

2× NVIDIA RTX PRO 6000 Blackwell Max-Q 192 GB ECC

RHODIUM-192 Dual-GPU training rig

GPU

2× RTX PRO 6000

VRAM

192 GB ECC

System RAM

256 GB ECC

Interconnect

PCIe 5.0

Best for tensor-parallel serving of 70B–123B models, full-parameter fine-tuning to roughly 10–13B with DeepSpeed ZeRO-3 across two GPUs, LoRA and QLoRA well past 70B, large-scale embedding and GNN training, and multi-tenant inference partitioned with MIG or MPS. The 192 GB is two 96 GB address spaces sharded in software, not a hardware-pooled pool — there is no NVLink on these cards.

Bill of materials

GPU	2× RTX PRO 6000 Blackwell Max-Q — 192 GB GDDR7 ECC total, 300 W each, PCIe 5.0 ×16 each with ~64 GB/s peer-to-peer. RTX PRO Blackwell does not support NVLink; the blower-style Max-Q is also the SKU meant for stacking two cards.
CPU	AMD Threadripper 7960X — 24C/48T, Zen 4, 152 MB total cache, 4.2 / 5.3 GHz, 350 W TDP, sTR5, 48 PCIe 5.0 lanes
Memory	256 GB DDR5-5600 RDIMM ECC (4×64 GB), registered, quad-channel — 5600 is an EXPO profile; AMD's official rating for this CPU is 5200
Storage	2× 2 TB PCIe 5.0 NVMe for scratch, plus 2× 7.68 TB U.2 NVMe (Kioxia CM7-V) in RAID-0 for datasets — ~26 GB/s aggregate read across the U.2 pair
Power	Seasonic Prime TX-1600 — 1,600 W 80+ Titanium, single +12V rail, 3× EPS12V and 2× native 12V-2×6. ~1,150 W under load with Max-Q cards.
Networking	Onboard 10 GbE, and a spare PCIe 5.0 ×16 slot for a future 25/100 GbE NIC. IPMI is available on TRX50 only as an optional expansion card, which costs a slot.
Motherboard	ASUS Pro WS TRX50-SAGE WIFI — sTR5, quad-channel RDIMM, onboard 10 GbE
Cooling & chassis	sTR5 air or AIO cooler; full-tower or 4U rackmount with spacing for two dual-slot cards
Electrical	Draws ~10 A at 120 V under load, so it fits a standard 15 A circuit. Specced with 600 W Workstation Edition cards instead, it needs a dedicated 20 A circuit or 208/240 V.
Software	Ubuntu 24.04 LTS · CUDA 12.8 · NCCL 2.25 · DeepSpeed 0.16 · Slurm 24.05 · Enroot + Pyxis

On price. Part costs move week to week and the right spec depends on what you run, so a published number would be out of date and probably wrong for you. Tell us the workload and what you have to spend and you get an itemised quote — parts at cost, build fee listed separately, nothing hidden in the bill of materials.

The other designs

Three starting points, each sized against a model it has to hold. If none of them is the right fit we adjust the spec to what you actually plan to run.

PALLADIUM-16 PLATINUM-96 All reference designs

Get a build quoted.

Tell us the model, the data and the constraint. We will come back with a spec and an itemised price.

Start a project