The case for running a big model on your own hardware is easy to like. No API bill, and nothing leaves the building.
A breakdown I watched this week did the sums on a setup people are actually buying for it. Four NVIDIA DGX Sparks at $6,950 each, so $27,800, with DeepSeek V4.1 Flash on top at its full context. One owner measured it at 87.7 tokens a second for a single user.
Run that flat out for a year, every hour of every day, and it writes about 2.8 billion tokens. DeepSeek's own API charges $0.60 per million output tokens at its cheaper rate. That year of tokens would cost around $1,660.
Nearly seventeen years to pay back the hardware. Before the electricity.
Load it properly and the picture changes. Sixteen sessions at once take the same four boxes to 342.7 tokens a second. That's about 10.8 billion tokens a year, or roughly $6,500 of API usage, and payback drops to just over four years. Keeping a machine that busy all year is hard to arrange, and four years is a long time to bet on any GPU staying current.
So two things have to be true together. The data genuinely can't leave the building, because once it can, the hosted price is the one to beat. And the work has to be steady enough to keep the whole machine busy. One person's chat window during office hours won't do it.
If it isn't private, rent the tokens. A private box that sits idle most of the day is a very expensive way to keep a secret.
Credit where it's due: the sums and the "private and busy" framing are from this breakdown of every way to get 512GB of memory for local AI. I checked the prices and benchmarks at source. Worth the 22 minutes. https://youtu.be/6tihP50zlcE