There's a ladder to how GPU compute gets built, and most of the conversation skips straight to the top rung.
The bottom rung is one server. An HGX B300 is eight GPUs joined by NVLink at 1.8 TB/s between any two of them, with 2.1 TB of GPU memory in a single box. Inside it, the eight behave close to one very large accelerator.
The next rung is a few of those servers on a scale-out network. GPUs in different boxes now talk over Ethernet or InfiniBand, which is fast, and still a long way short of NVLink. Most design decisions from here on are about hiding that gap.
Then the SuperPOD, built in scalable units. In NVIDIA's B300 design one unit is 64 servers and 512 GPUs, with network, storage and management sized as a block. At the top sits rack scale. In an NVL72, 72 GPUs share one NVLink domain and the rack itself is the computer.
Every rung costs more than hardware. You take on a fabric to run, a scheduler, cooling the room may not have, and people who can keep the lot healthy at 3am.
A lot of enterprise work never leaves the first rung. Inference for an internal user base, fine-tuning a model you could sensibly run in-house: very often that's one server's worth.
You climb when something specific forces it. The model and its working memory no longer fit in one box. A training run would take months on eight GPUs. Or the queue of requests outgrows what one server can answer.
Name which of those is true before you buy the next rung. If none of them is, what you need is a second server for resilience. Not a cluster.