Size an AI factory before you buy one.

What the full report adds

• All six platforms sized for your workload in one table: parallelism, GPUs, servers, racks, power, owned and rented cost per year, cost per million tokens.
• Memory plan for your platform: weights, KV cache per request, cache capacity per replica, and the concurrency limits set by memory and by your speed target.
• Facility plan: rack elevation, power per rack, cooling approach, IT and facility load, yearly energy and the colocation capacity to contract.
• Build, rent or API: totals over your amortisation period, the load at which owning beats the API, and the same volume priced on six hosted APIs.
• Sensitivity of cost per million tokens to load, energy price, hardware price, amortisation period and cloud discounts.
• Every assumption with its source and date, written so it can go straight into a business case.

How the estimate works

The sizer uses a first-order roofline model: the reasoning an infrastructure architect would do on a whiteboard, applied the same way to every platform so the comparison is fair.

Memory

Weights take parameters times bytes per parameter (2 for BF16, 1 for FP8, about 0.56 for FP4 including scales). Each request holds a KV cache of 2 times layers times KV heads times head size times bytes per value, for every token in its context. Models with compressed attention, such as DeepSeek V3 and Kimi K2, use their latent width instead, and sliding-window layers only keep their window. A replica uses 90% of GPU memory, keeps 2 GB per GPU for activations, and gives the rest to the KV cache.

Speed

Each decode step reads the weights once plus the KV cache of every request in the batch, and does about 2 FLOPs per active parameter per token. Step time is the larger of memory time and compute time, divided by a parallelism efficiency, plus a fixed scheduling overhead. For mixture-of-experts models the share of experts read in a step grows with the batch. Prompt processing is compute bound and interrupts decoding, so the speed each request sees is:

Sizing

For each tensor-parallel width the sizer finds the largest batch that fits in memory and still meets your speed target, then picks the width that needs the fewest GPUs for your peak. Servers and NVL72 racks are bought in whole units, so spare GPUs show as headroom.

Cost

Owning is hardware plus networking, storage and integration, amortised straight line, plus colocation per kW of IT capacity, metered energy times PUE, support, and any staff cost you enter. Renting is the GPUs in use times the hourly rate for every hour of the year, less any commitment discount. The API figure prices your yearly input and output tokens at list price. Yearly output tokens are peak concurrency times average load times target speed times seconds in a year.

Limits

This is a planning estimate, not a benchmark. Real throughput depends on the serving stack, scheduler settings, quantisation, prompt caching and network design, and can differ from these figures by 30% or more. The three software profiles (conservative, typical, optimised) assume 50%, 65% and 75% of peak memory bandwidth. Benchmark the shortlisted platform with your own prompts, for example with NVIDIA AIPerf or GenAI-Perf, before committing capital.

Data and sources

Prices and specifications as of 29 Oct 2026. Every figure can be overridden in the assumptions panel.

About

The sizer is a free tool from Loomiks, the AI factory planning platform from Sirin Digital, a UK company. It was written by Onur Sirin, founder of Sirin Digital. Onur has more than 20 years in enterprise technology, including six years at IBM, where his roles included Transformation Chief Architect, and AI factory and data centre consulting at World Wide Technology.

To take a sizing from estimate to an AI factory design and delivery plan, see loomiks.ai. For a review of your own design, training sizing or a sovereign AI business case, email onur.sirin@sirindigital.uk.