Hardware for AI Automation

Key takeaways

• Memory capacity decides which model fits; bandwidth decides its speed. Measured 4-bit files take about 0.54–0.58 bytes per parameter, so a 120B model needs about 63 GB plus its KV cache, and a 1.6T model nearly a terabyte.
• 120B mixture-of-experts models are the sweet spot for single-box private deployments. gpt-oss-120b fits one 80 GB GPU, a 96 GB RTX PRO 6000 or a 128 GB DGX Spark. On the Spark it measured 58.7 tokens/s for one stream and 353 tokens/s across 32 streams.
• Frontier-scale open models are data-centre workloads. DeepSeek V4-Pro (1.6T), Qwen3.8 (2.4T) and Kimi K3 (2.78T) need about 0.9–1.7 TB at 4-bit (calc), which means an 8-GPU H200, B200 or B300 node, rented or reached through an API.
• An H100 hour costs ₹91.80 to about ₹1,180. That runs from IndiaAI's ₹153 list rate less its maximum 40% subsidy (₹91.80, subject to IndiaAI approval; GST extra), through Indian commercial rates of ₹217–362 before GST, E2E's $2.69 (₹255.55 on its own INR page) and Lambda's $3.99–4.29, to AWS's $6.88 ($8.26 in Mumbai) and $12.29 on an Azure 8-GPU VM.
• Hardware got dearer in 2026. DRAM contract prices rose 93–98% in Q1 2026, the DGX Spark was reported at $4,699 in August 2026 (secondary; NVIDIA shows no price), and NVIDIA's own Marketplace listed the RTX PRO 6000 at $16,000 by August 2026, against $8,565 at launch and street prices of about $11,100–12,100 in June–July (secondary).
• Self-hosting rarely wins on price. A $3.99-an-hour H100 (Lambda's largest instance) must sustain about 6,500 output tokens/s around the clock to beat hosted gpt-oss-120b at $0.17 per million output tokens. Data control is the usual reason to own hardware.
• Tiers at ₹96/USD: an SME pilot costs ₹26,000–38,000 in test rental or ₹4.5–5.4 lakh for a 128 GB desktop. A department needs about ₹13.5–20 lakh for a 96 GB workstation at mid-2026 card prices (calc: $11,100 street to $16,000 Marketplace, plus a $3,000–5,000 host), or ₹1.9–5.6 lakh a month rented. An enterprise pays ₹2.7–8.3 crore a year per rented 8-GPU node, or ₹34–43 crore for an NVL72 rack.
• Edge devices suit small models doing OCR, classification and extraction. Copilot+ PCs require an NPU of more than 40 TOPS, and Jetson AGX Thor has 128 GB at 273 GB/s. MLPerf v6.1's agentic-edge test measured Jetson AGX Thor at 1,466 ms per query (DGX Spark 3,808 ms, Strix Halo 7,059 ms), but no measured NPU LLM speed was found.

How to read the numbers in this document

All facts are as of 12 October 2026. Cloud rates, MLPerf results, checkpoint sizes, Apple prices and IndiaAI rates were re-checked against primary sources on 12 October 2026; facts read only on 11 October keep that read date. Figures carry one of four labels. Measured means a published benchmark run: community llama.cpp results for desk-side machines, and vendor submissions under MLPerf rules for servers. Vendor spec and vendor claim mean the maker's own page or announcement. Secondary or single source means a blog, aggregator or news summary that was not confirmed elsewhere. Calc means this document's own arithmetic on the stated inputs. USD is converted to INR at ₹96/USD (X-Rates, Oct 2026 average) (X-Rates monthly average). INR prices from Indian sources are shown as quoted. Indian GPU rental prices usually exclude 18% GST, while Indian reseller hardware prices usually include it. Import duties were not researched.

This page covers the hardware layer. Token prices and total cost of ownership are in Costs and economics; serving engines (vLLM, SGLang, llama.cpp, Ollama) in Tool stack and Frameworks; model profiles in DeepSeek, OpenAI and Other models; subsidised compute policy in India; chip roadmaps in Future outlook; and terms such as KV cache and MoE in the Glossary.

Memory math sizes a deployment before any benchmark does

Sizing a self-hosted model takes three numbers: weight memory, KV-cache memory (the model's per-conversation working memory), and the memory bandwidth every generated token must cross. Compute matters mainly for reading long prompts. Most sizing mistakes come from memory.

Step 1: weights. Weight memory equals parameters times bytes per parameter: 2 bytes at FP16/BF16, 1 at FP8, and about half a byte plus scaling overhead at 4-bit. Real file sizes anchor the 4-bit figure. The gpt-oss-120b MXFP4 file used in llama.cpp's DGX Spark benchmarks is 59.02 GiB for 116.8B parameters, which is 0.54 bytes per parameter (calc), and the gpt-oss-20B file is 11.27 GiB, or about 0.58 bytes per parameter on the model card's 21B count (llama.cpp DGX Spark benchmarks; gpt-oss model card). DeepSeek V4-Flash ships in mixed FP4 and FP8 (V4-Flash model card). One secondary guide puts its weights at about 158 GB for 284B parameters, or 0.56 bytes per parameter (calc; from a search summary, unverified) (BestLLMFor VRAM guide). A custom 2.88-bit quantisation of the same model is 102.3 GB (Lucebox). The 284B figures in this document refer to the V4-Flash open weights (latest build 0731). DeepSeek's current Flash API model, V4.1-Flash (552B backbone, 763B parameters in total, FP8 plus packed 4-bit), has a 510.3 GB checkpoint (Hugging Face file sizes) (Hugging Face API: V4.1-Flash; V4.1-Flash model card). One GPU-cloud vendor says it needs "a 4-GPU Blackwell tray or an 8-GPU H200 node" (vendor claim) (Yotta Labs guide). This document plans with 0.55–0.6 bytes per parameter at 4-bit, 1.0 at FP8 and 2.0 at FP16.

Step 2: total versus active parameters. In mixture-of-experts (MoE) models, total parameters set the memory needed and active parameters set the bytes each token reads. gpt-oss-120b has 117B total and 5.1B active parameters, and gpt-oss-20b has 21B total and 3.6B active (gpt-oss model card). DeepSeek V4-Flash is 284B total with 13B active, and V4-Pro is 1.6T total with 49B active (V4-Pro model card). DeepSeek V3 was 671B total with 37B active (DeepSeek-V3 report). This split is why a 120B MoE can run faster on a desktop than a 70B dense model.

Step 3: KV cache. Per token, the cache takes 2 × layers × KV heads × head dimension × bytes per element. Take an illustrative 70B-class dense configuration of 80 layers, 8 KV heads and head dimension 128 at FP16 (assumed; check the real model card). The cache is then 327,680 bytes, about 0.33 MB per token (calc). One 8k-token (8,192-token) conversation needs about 2.7 GB, and 32 such conversations at once need about 86 GB (calc). Newer attention designs shrink this considerably. DeepSeek says V4-Pro at a 1M-token context needs 10% of V3.2's KV cache (company claim) (V4-Pro model card), and a secondary guide puts a 1M-token cache plus overhead for V4-Flash at about 10 GB (BestLLMFor). This document's planning assumption is 20–30% of memory above the weights for light use, and explicit KV arithmetic for any multi-user design.

Step 4: speed ceiling. Single-stream decode speed is capped at memory bandwidth divided by the bytes of active weights read per token. On a DGX Spark (273 GB/s, vendor spec), gpt-oss-120b reads about 2.77 GB per token (5.1B × 0.54, assuming the active weights match the file's average density). The ceiling is about 99 tokens/s against 58.7 measured (59%) (calc). gpt-oss-20b reaches 63% of its ceiling (83.4 measured against about 132). On the Ryzen AI Max+ 395 ("Strix Halo"), with about 230 GB/s measured, DeepSeek V4-Flash at 2.88 bits has a ceiling of about 49 tokens/s and measured 25.3, or 52% (calc from llama.cpp Vulkan discussion and Lucebox). For planning, expect 50–65% of the bandwidth ceiling.

Step 5: concurrency. Users supported equals aggregate tokens/s at the chosen batch size divided by each user's needed speed. The Spark's 353.3 aggregate tokens/s across 32 streams on gpt-oss-120b is about 11 tokens/s per stream (calc). That suits a team of about thirty at a chat-like pace (an assumed per-user target of about 10 tokens/s). For overnight batch jobs it means about 1.27 million output tokens an hour, or roughly 2,500 documents an hour at 500 output tokens each, before prompt-reading time (calc).

Against V4-Pro's released 864.7 GB checkpoint (Hugging Face API: V4-Pro), the guides' figures of up to about 1.2 TB would need KV cache and runtime on top and are unverified (Codersera). DeepSeek's model cards state no hardware requirement, though Think Max mode needs a context of at least 384K tokens (V4-Pro model card). Size a production V4-Pro deployment from a test load, not a blog table; model detail is in DeepSeek.

Seven model-size classes map onto five hardware classes

The table below turns the memory math into a buying guide. Weight sizes are calculated at 0.55–0.6 bytes per parameter for 4-bit and 1 byte for FP8. Parameter counts come from model cards and Hugging Face listings (Qwen3.5 sizes; Qwen3.8-27B; Step-3.7-Flash; Tencent Hy3; Tencent Hy4-preview; MiniMax M3; GLM-5; Kimi K2.5; Kimi K3; Qwen3.8-2.4T).

Two patterns stand out. First, dense 70B models fit desktops badly: their 38–42 GB of 4-bit weights are read in full for every token, so a 273 GB/s unified-memory box tops out near 7 tokens/s (calc), while a 120B MoE on the same box runs at 58.7 tokens/s (measured) because each token reads only about 5B parameters. Second, the jump from one device to several comes at about 130B total parameters. Below that, one 80–128 GB device works; from 200B to about 300B, two GPUs or a 256 GB-plus unified-memory machine are needed. Above about 400B, the realistic choice is a multi-GPU server; a 512 GB Mac Studio could hold such models once Apple ships that option ("coming late October"), but its speed is untested.

Licences add a constraint that hardware cannot solve. Qwen3.8-27B is Apache 2.0, while Kimi K3 and Qwen3.8-2.4T carry custom licences that require a separate agreement once a firm running a model-as-a-service business passes US$20M and US$50M of revenue respectively, counting affiliates (Kimi K3 licence; Qwen3.8-Max licence). See Other models for model-level terms.

Measured local speeds: bandwidth sets decode, compute sets prompt reading

The most reliable desk-side speed data is llama.cpp's community scoreboard and its DGX Spark benchmark file. The scoreboard runs the same small model (Llama 2 7B at Q4_0) on every device, a clean comparison of bandwidth and compute. "pp512" is prompt processing (prefill) and "tg128" is token generation (decode).

Sources: llama.cpp CUDA scoreboard and llama.cpp Apple Silicon results, both read 11 October 2026. Rows are community submissions with no per-row date.

Three things stand out. On decode, the RTX 5090, RTX PRO 6000 and H100 are within about 8% of each other for small models; capacity (32, 96 and 80 GB) separates them, not speed. The DGX Spark decodes about 5 times slower than a 5090 (calc), in the range of an M4 Pro with the same 273 GB/s. Prompt reading follows compute, not bandwidth. The PRO 6000 ingests prompts about 4.9 times faster than the Spark and about 10 times faster than an M3 Ultra (calc), though the M3 Ultra has nearly three times the Spark's bandwidth. Agentic automation keeps re-sending long contexts, so prefill speed is what users feel: on the 7B test a 20,000-token prompt takes about 1.3 seconds on the PRO 6000, 6.5 on the Spark and 13.6 on the M3 Ultra (calc), and larger models stretch all three.

Among desk-side machines, larger models were measured only on the DGX Spark and Strix Halo (MLPerf data-centre results are in the accelerator section):

Sources: llama.cpp DGX Spark benchmarks, linked from llama.cpp Discussion #16578; Lucebox. The same Spark thread reports an AIME25 score of 93.75% for gpt-oss-120b at high reasoning (self-reported), which suggests the quantised setup keeps the model's reasoning quality.

The Strix Halo result shows the appeal and the catch of cheap unified memory: a 284B open model runs on a $3,149 mainboard, but at about 245 tokens/s of prefill a 20,000-token agent context takes about 80 seconds to read (calc). On that machine, the Vulkan backend has overtaken AMD's ROCm, according to the llama.cpp thread's author (llama.cpp Vulkan discussion).

Hosted services set the reference. Artificial Analysis measures gpt-oss-120b output at 1,721 tokens/s on Cerebras, 706 on SambaNova and 473 on Groq, so the fastest is 29 times the Spark's single stream (calc). Blended prices start at $0.04 per million tokens across 17 providers (re-read 11 October 2026) (Artificial Analysis, gpt-oss-120b providers). Closed frontier models stream at about 63–137 tokens/s (median, maximum settings, read 12 October 2026), for example Claude Sonnet 5.5 at 137, Opus 5.5 at 97, Fable 5.1 at 65 and GPT-6 Astra at 63 (Haiku 5.5: 235) (Artificial Analysis leaderboard). A $4,699 desktop serving gpt-oss-120b therefore streams about as fast per user as the slower frontier APIs, though with a far less capable model. Unverified vendor claims go higher. Cerebras reports more than 2,500 tokens/s per user on Llama 4 Maverick against about 1,000 for Blackwell (Wikipedia: Cerebras), and Etched claims about 500,000 tokens/s on Llama 70B, from weak sources only (fast.io review).

Data-centre accelerators: Blackwell Ultra is the volume part, and Rubin and custom chips come next

NVIDIA's Blackwell Ultra (B300 and the GB300 NVL72 rack) is the volume data-centre part in October 2026. NVIDIA reported Rubin in production, with racks running at CoreWeave, Google Cloud, Azure, OCI and Nebius, in its Q2 FY2027 results of 26 August 2026 (secondary: summaries of the release) (HotHardware; Energy Mag). MLPerf v6.1 includes a preview Vera Rubin NVL72 submission (DeepSeek-R1 Offline about 16,435 tokens/s per GPU, 2.26 times GB200 NVL72, calc) (MLPerf Inference v6.1 results). Shipment volumes are not public. No official 2026 list price was found for any NVIDIA, AMD or Google data-centre part, so every price below is a rental rate or an analyst estimate.

Three conclusions follow for buyers. First, memory per chip has converged at 141–288 GB: Trainium3, TPU v7 and Maia 200 sit in the same class as B200 and B300, so model fit depends on chip count, not vendor. Second, power and cooling gate the top tier. A GB300 NVL72 draws about 132 kW, needs liquid cooling and costs $48,600–62,500 per GPU at the rack estimates (calc), so it is a facility decision. Third, the custom accelerators cannot be bought as hardware by a typical business, in India or elsewhere; they come as cloud services, with India-region availability not verified. Vendor concentration matters too. Cerebras listed on Nasdaq on 14 May 2026, raising $5.55B, and Wikipedia, citing its S-1, says MBZUAI and G42 supplied 62% and 24% of its $510M revenue in 2025 (Wikipedia: Cerebras). SambaNova took investment led by Vista Equity Partners and Intel in February 2026; a reported Intel acquisition is unconfirmed (Wikipedia: SambaNova).

Measured data-centre throughput, MLPerf Inference v6.1 (vendor submissions under MLPerf rules; per GPU, Offline scenario; compare only within a benchmark):

Source: MLPerf Inference v6.1 results file, read 12 October 2026; per-GPU values are the system result divided by its accelerator count (calc). No H100 or H200 rows were submitted. On gpt-oss-120b, a B200 delivers about 5.8 times the per-GPU throughput of an RTX PRO 6000 (calc: 11,400 ÷ 1,955).

China runs a parallel track. Huawei's Ascend 910C pairs two SMIC-made 910B dies and is described as performing like an H100 (Wikipedia: Huawei Ascend). Huawei's CEO says Ascend still lags "by a generation" (Wikipedia: iFlytek Spark). A Bloomberg headline of 4 September 2026 reports that DeepSeek plans to order more than 160,000 Ascend 950DT chips. DeepSeek has also published an Ascend port of its DeepGEMM kernels (HN listing; DeepGEMM-Ascend). Cambricon has been on the US Entity List since December 2022 (Wikipedia: Cambricon). None of these is a practical purchase for a business that serves US customers. See Asia-Pacific for the wider picture.

Cloud rental: global and Indian price tables

Rental is the default route to data-centre GPUs for nearly every organisation below hyperscale, and 2026 prices are rebounding. An H100 contract index built on SemiAnalysis data rose from $1.70 to $2.35 per GPU-hour between October 2025 and March 2026, and on-demand capacity was described as effectively sold out (Crypto Briefing, single secondary source). That is about 38% (calc; the article says 40–50%).

Global rates (USD per GPU-hour; INR at ₹96 is calc)

Azure's official $12.29 per GPU-hour matches the aggregator's hyperscaler ceiling, and E2E's $2.69 sits inside the independent Indian range below. Hyperscaler on-demand costs 1.7–4.6 times the specialist rate for the same H100 (calc: $6.88–12.29 against $2.69–3.99). Google Cloud's committed-use discounts of 28% for one year and 46% for three years narrow the gap (Google Cloud VM pricing). In India, AWS Mumbai charges 1.2 times its US rate for H100 ($8.26), and Azure Central India offers only NC-series H100 at $9.77 per GPU-hour. Google Cloud A3/A4 GPU rates and an Azure B200 VM were not found.

Indian rates (as quoted; USD at ₹96 is calc)

This document uses E2E's dated price table ($2.69, about ₹258 at ₹96); its INR pages give the same price at E2E's own ₹95 per dollar (₹255.55), inside the independent ₹217–362 range. Its ₹70 spot rate is below even IndiaAI's maximum-subsidy H100 rate of ₹91.80.

The IndiaAI pool is real but gated. The portal's empanelment list counts 45,081 GPUs across 14 bidder rows (undated; H100 16,496, B200 8,192, H200 7,916, TPU v6e 5,548; largest bidder Yotta 17,408) (IndiaAI empanelled GPUs); MeitY's "more than 38,000" (25 March 2026) was an earlier announcement (TelecomTalk). A parliamentary answer in August 2026 reported 93.18 lakh GPU-hours sanctioned across 237 projects and 15 empanelled providers (Communications Today, single source). Access is by application (portal eligibility page; primary). Named categories are researchers and academia (h-index of 5 or more, an AI/ML journal with impact factor above 2.5, or 150+ citations), DPIIT-recognised startups and MSMEs with ₹50 lakh annual revenue or ₹100 lakh funding, India AI fellows, students, early-stage startups with a letter from the IndiaAI CEO or a JS-rank officer, and government entities. Other companies, large enterprises included, are not named (inference: they qualify only as DPIIT-recognised startups or MSMEs). Requests under 5,000 GPU-hours are auto-approved; approvals last 30 days. A minister's "about ₹65" per GPU-hour (Storyboard18) and a guide's "about ₹67" after subsidy (Tensordata) cannot be reproduced from the portal (₹65 would need more than 57% off, against a 40% cap) and are not used here. The government says Indian AI compute costs less than half the global rate (Outlook Business). At the ₹91.80 maximum-subsidy rate, IndiaAI H100 time costs about 36% of E2E's ₹255.55 on-demand rate, and the ₹153 list rate about 60% (calc). E2E's 3-month commitment rate of ₹155.90 is close to the IndiaAI list rate. Apply where your project qualifies, but do not make the subsidised rate the base case of your budget.

What rental adds up to (calc; monthly = 730 hours, annual = 8,760 hours)

Workstations and small servers: 96 GB and 128 GB machines, priced by the memory shortage

Below the data centre sit two kinds of machine. Discrete GPUs with fast GDDR7 (RTX 5090 at 32 GB, RTX PRO 6000 at 96 GB) read prompts and decode quickly but cap model size at the card's memory. Unified-memory machines (DGX Spark, Ryzen AI Max+ 395, Mac Studio) offer 128–256 GB of slower shared memory today (Apple lists a 512 GB M5 Ultra as "coming late October"), which holds bigger models but reads prompts slowly. The 2026 memory shortage has raised prices in both groups.

Prices have moved fast. The DGX Spark is up 17.5% from launch (the $3,999 launch price is snippet-only), the RTX PRO 6000's $16,000 NVIDIA Marketplace listing (secondary) is 87% above its $8,565 launch price, street prices of $11,100–12,100 were 30–41% above launch, and the M3 Ultra configuration rose 33% (calc). Indian resellers track US prices plus tax: without 18% GST, the ₹5,42,499 Spark listing is about $4,789, within 2% of the US price (calc). Expect US list price plus GST, and treat reseller quotes as short-lived.

For a pilot, the DGX Spark is the cheapest complete machine with measured 120B-class results, and it runs NVIDIA's software stack. An RTX PRO 6000 workstation costs about 3.0 to 4.5 times as much at mid-2026 card prices once a host is added (calc: $14,100–21,000 against $4,699). It reads prompts about five times faster and splits into four isolated GPU instances for multi-tenant work. The Ryzen AI Max+ 395 board is the cheapest route to 128 GB, but it runs on Vulkan rather than CUDA and has the slowest measured prefill. Apple's M5 Ultra offers up to 256 GB at 1.2 TB/s today, with 512 GB listed as "coming late October", but it is not a CUDA platform and has no M5 results in the research set. The DGX Station, the desk-side step up to 1T-parameter models, has no NVIDIA price, and no sourced partner price was found.

Edge devices handle small models; one MLPerf edge test is the only measured edge LLM result

Edge hardware means laptops, phones, embedded modules and on-site appliances. It matters where automation must run offline, keep data on the device or respond instantly, such as OCR at a branch or classification on a factory line. Specifications are better documented than performance.

An NPU's TOPS rating is a floor for operating-system AI features, not a measure of LLM decode, which bandwidth limits everywhere. Jetson Thor shares the Spark's 273 GB/s and 128 GB, so it could hold a 20–120B MoE at a similar decode ceiling (calc). The small open models suit edge work: Qwen3.5 at 0.8–9B (0.4–5.4 GB at 4-bit, calc), and gpt-oss-20b, whose 11.27 GiB file fits a 16 GB device. MLPerf v6.1's "agentic edge" test (qwen3.6-27b, single-stream latency per query, lower is better) measured Jetson AGX Thor at 1,466 ms, an RTX PRO 4500 at 2,616–2,758 ms, the DGX Spark at 3,808 ms and AMD Strix Halo at 7,059 ms (vendor submissions with different serving stacks) (MLPerf Inference v6.1 results). No measured speed for any laptop NPU or phone was found, so test edge LLM speed on the device before relying on it. 70B-class dense work and anything harder belongs on a workstation or in the cloud.

Supply risks: memory doubled in price, export rules reversed, and the roadmap is crowded

Memory is the binding constraint of 2026. Conventional DRAM contract prices rose about 93–98% quarter on quarter in Q1 2026 (TrendForce data relayed by IntuitionLabs). A 36 GB HBM3E module reportedly traded near $2,100 on the spot market in early September 2026, against $300–400 under long-term agreements, a 5–7× spread (calc; search summary, re-verify) (IntuitionLabs). Another tally puts the rise in server DRAM at about 90% (Algeria Tech). A TEAM Group executive expects shortages into late 2027 or beyond (Tom's Hardware; see also S&P Global). Apple and NVIDIA both cited memory costs when they raised prices.

Demand is not easing. NVIDIA's data-centre revenue was $89.0B in the quarter ended 26 July 2026, up 117%, and its guidance for the next quarter excludes China (NVIDIA Q2 FY2027 results). Over the longer run, Epoch AI measures AI-chip price-performance improving about 49% a year per dollar since 2023, in spurts ("nearly flat through mid-2024"), so capability gets cheaper year to year even while a given box gets dearer (Epoch AI trends; Epoch data insights). Epoch also estimates a 1 GW AI data centre at about $38B of up-front capex, with servers taking 60% of lifetime cost (Epoch AI trends; Epoch AI data centres). That is why rack-scale hardware is financed like infrastructure.

Export controls reversed within days. From 15 January 2026, the US moved H200 and MI325X licences for China and Macau to case-by-case review. Eligible chips must have a TPP below 21,000 and DRAM bandwidth below 6,500 GB/s, and exporters must show no US shortage (Gigazine; Introl). China then blocked H200 imports (Wikipedia, citing BBC and Guardian reporting), and customs reportedly turned the chips away (Wikipedia: Nvidia; eeNews Europe). Earlier, NVIDIA halted H20 production in August 2025 after Chinese directives, and ByteDance and Alibaba were ordered in September 2025 not to buy the RTX Pro 6000D (Wikipedia: Nvidia). Epoch reports trade data "consistent with $3B worth of chips smuggled to China via Malaysia" (Epoch data insights). US export rules for India were not researched; check them before importing data-centre GPUs.

The next generation is starting to ship. NVIDIA reported Rubin in production, with partner racks running at CoreWeave, Google Cloud, Azure, OCI and Nebius (26 Aug 2026). AMD said Helios is in production, with initial shipments later in Q3 2026 and a ramp into 2027 (both secondary, via summaries of the results releases). AMD claims up to 10× MI355X for MI400/MI450; NVIDIA's Groq 3 LPU is due in the second half of 2026, with Nebius named as the first deployer; Cerebras introduced CS-4 in August (sources in the accelerator table). Shipment volumes are not public, so plan for tight Blackwell and H100 rental supply in the near term. India's capacity is growing but full. JLL puts it at 1.6 GW in June 2026, with record-low vacancy of 2.8%, heading for 6 GW by 2029 and needing about $110B of investment (IANS, 2 Sep 2026, reporting a JLL report; secondary; JLL release not read). Announced projects are pipeline, not capacity available today: AMD and TCS deploying up to 200 MW, an L&T venture with NVIDIA, Blackstone's $600M into Neysa, and Reliance-Jio's $110B over seven years (Fortune). See Future outlook and India.

Practical tiers from a ₹26,000 test to a ₹43 crore rack

The tiers below combine the measured speeds, prices and memory arithmetic above; they show orders of magnitude, not quotes.

The owned-workstation running cost comes from Costs and economics: 36-month amortisation and 0.7 kW at $0.12 per kWh (no Indian tariff was sourced), excluding staff time. A quarter of an Indian AI engineer's time, about ₹20,833 a month (salary only, at a secondary ₹10 lakh-a-year average), on its own exceeds a Spark's amortised cost of about ₹12,531 a month (calc).

The break-even arithmetic explains why renting and APIs win most cost comparisons. One H100 at Lambda's $3.99 (its rate on the largest self-serve instance) costs $2,912.70 a month. Hosted gpt-oss-120b on OpenRouter costs $0.037 per million input tokens and $0.17 per million output tokens (OpenRouter models API; Lambda pricing). On output alone, the GPU breaks even at about 17.1 billion tokens a month, or about 6,520 tokens/s sustained 24/7 (calc). At Lambda's $4.29 rate for a smaller instance the bar rises to about 7,000 tokens/s, and at E2E's $2.69 it drops to about 4,400 tokens/s (calc). The desktop case is similar: a Spark running flat out at its measured 353 tokens/s produces about 928 million output tokens a month, worth about $158 (₹15,153) at $0.17 per million, against $131 (₹12,531) a month for its $4,699 price over 36 months, before power and staff (calc). MLPerf v6.1 has no H100 rows, so the H100 break-even is a threshold to test, not a verdict, but its vendor-submitted gpt-oss-120b results show what saturated servers reach. A B200 at about 11,400 tokens/s produces about 30 billion output tokens a month, worth about $5,093, against $5,103 to rent at E2E's $6.99 an hour, so it breaks even only at about 100% utilisation. An owned RTX PRO 6000 at about 1,950 tokens/s (about 5.1 billion tokens, worth about $871 a month) costs $453–645 a month, so it needs roughly 52–74% sustained utilisation (calc, before staff time). Owning breaks even only at high sustained utilisation, so the case for owning rests on data control, a fine-tuned model no provider serves, or offline operation, not on price.

What this means for your business

Rent first, measure, then decide whether to buy. For most Indian SMEs and US mid-market companies, the right hardware is none: hosted APIs serve production, and a few hundred rented GPU-hours cover an open-model evaluation. A bake-off of gpt-oss-120b, Qwen3.5-122B and DeepSeek V4-Flash costs about ₹2.6 lakh at E2E's rates for 200 hours per model (calc; two H200s for V4-Flash). You get measured throughput on your own prompts, which public benchmarks lack. If your project qualifies for IndiaAI compute (requests under 5,000 GPU-hours are auto-approved; approvals last 30 days), budget at the ₹153 list rate and treat the subsidy of up to 40% (subject to approval) as upside.

Buy only for a reason price cannot fix. Owning breaks even only at high sustained utilisation, so the case rests on data control, a fine-tuned model no provider serves, or offline operation. If data must stay on site, a 128 GB DGX Spark-class box (₹4.5–5.4 lakh) serves 1–30 users on a 120B-class MoE; it is the only desk-side machine with measured 120B results here. A 96 GB RTX PRO 6000 workstation (₹13.5–20.2 lakh with a host, calc) adds fast prompt reading, up to four isolated GPU instances and a path to two cards for 284B-class models. Send dense 70B models and anything above about 300B parameters to rented nodes or APIs. Count staff time: a quarter of an Indian AI engineer's salary exceeds a Spark's amortised monthly cost (calc).

Match the model to the hardware. MoE models with 3–13B active parameters (gpt-oss-20b, gpt-oss-120b, Qwen3.5-122B-A10B, DeepSeek V4-Flash) fit on-premises automation best. Prefer Apache 2.0 or MIT weights (gpt-oss, Qwen3.8-27B, DeepSeek V4) to avoid revenue-threshold licence clauses. Security issues by model family: DeepSeek, Governance.

Budget for moving prices. Workstation prices have risen about 30–87% since launch, and a TEAM Group executive expects memory shortages into late 2027 or beyond, so treat hardware quotes as short-lived. Hyperscaler on-demand H100s cost 1.7–4.6 times specialist rates ($2.69–3.99) before committed-use discounts (28–46% on Google Cloud). In India, AWS Mumbai lists H100s at 1.2 times its US rate and Azure has no ND H100 or H200 VMs, so check region coverage against where your data must stay.

Test long contexts before you commit. Agents re-send documents and tool output at every step, so prompt-reading speed sets the latency users notice. Discrete NVIDIA GPUs read prompts 5–10 times faster than unified-memory machines, gpt-oss-120b on the Spark loses 27% of its decode speed at 32k context, and a cheap 128 GB box can take over a minute to read a long prompt. Engines: Tool stack, Playbook.

Questions to ask any vendor or implementation partner:

• What tokens/s per stream and in aggregate were measured on your prompts at your context length, and on which hardware?
• Which licence covers the model weights, and does it have a revenue threshold?
• Which region runs inference, and is that GPU offered there?
• Is the quote in INR or USD, at what exchange rate, with or without GST, and valid for how long?
• Which custom-silicon figures (Cerebras, Groq, Etched, Maia, Trainium) are vendor-reported? Several are unresolved (Etched shipping, Maia FP8, Groq 3 memory). Cerebras serves gpt-oss-120b at 1,721 tokens/s (Artificial Analysis), but benchmark it on your workload before agreeing service levels.

Open questions and unverified items

Sources

1. X-Rates: USD to INR monthly average, 2026
2. llama.cpp: DGX Spark benchmark file
3. llama.cpp Discussion #16578: DGX Spark
4. llama.cpp Discussion #15013: CUDA scoreboard
5. llama.cpp Discussion #4167: Apple Silicon
6. llama.cpp Discussion #10879: Vulkan
7. Lucebox: DeepSeek V4 Flash on Strix Halo
8. Hugging Face: gpt-oss-120b model card
9. Hugging Face: DeepSeek-V4-Pro model card
10. Hugging Face: DeepSeek-V4-Flash model card
11. arXiv 2412.19437: DeepSeek-V3 Technical Report
12. BestLLMFor: DeepSeek V4 self-hosted VRAM guide
13. Codersera: DeepSeek V4 VRAM and GPU requirements
14. DeepLearning.AI: Qwen3.5 open-weight sizes
15. Hugging Face: Qwen3.8-27B
16. Hugging Face: Qwen3.8-2.4T-A95B
17. Hugging Face: Qwen3.8-Max licence
18. Hugging Face API: Step-3.7-Flash
19. Hugging Face API: Tencent Hy3
20. Hugging Face API: Tencent Hy4-preview
21. Hugging Face API: MiniMaxAI models
22. NYU Shanghai RITS: GLM-5
23. NYU Shanghai RITS: Kimi K2.5
24. Hugging Face API: Moonshot AI models (Kimi K3)
25. Hugging Face: Kimi K3 licence
26. Artificial Analysis: gpt-oss-120b providers
27. Artificial Analysis: model leaderboard
28. OpenRouter models API
29. Wikipedia: Cerebras
30. fast.io: Etched AI review 2026
31. 3DTested: NVIDIA delivers first Vera Rubin samples
32. Spheron: NVIDIA B300 Blackwell Ultra guide
33. Spheron: GB300 rental catalogue
34. io.net: GB300 NVL72 cloud rental
35. E2E Networks: GPU cloud pricing
36. Introl: BIS H200 and MI325X case-by-case review
37. Wikipedia: Nvidia
38. AMD: Instinct MI350 series and beyond
39. DCD: AMD launches MI350, unveils Helios
40. TechRadar: AMD MI400 with 432 GB
41. Google Cloud: TPU7x documentation
42. The Register: Google's Ironwood TPUs debut
43. AWS: Trainium
44. StorageReview: Microsoft Maia 200
45. Intel: Gaudi AI accelerators
46. Shopifreaks: NVIDIA licenses Groq technology, unveils Groq 3 LPU
47. SoftwareSeni: The NVIDIA–Groq deal explained
48. Wikipedia: SambaNova Systems
49. Wikipedia: Huawei Ascend (chip)
50. Wikipedia: iFlytek Spark
51. Hacker News search: Huawei Ascend 950
52. GitHub: deepseek-ai/DeepGEMM-Ascend
53. Wikipedia: Cambricon Technologies
54. Crypto Briefing: H100 rental costs surge
55. Lambda: pricing
56. Azure Retail Prices API: ND96isr H100 v5
57. IntuitionLabs: H100 rental prices compared
58. Akash Network: H100 rental price 2026
59. packet.ai: GB300 vs GB200 NVL72
60. Google Cloud: VM pricing
61. Storyboard18: AI compute at ₹65 per hour
62. Tensordata: GPU cloud pricing India 2026
63. ECorpIT: India GPU cloud rental pricing
64. E2E Networks: H100 pricing in India
65. Yotta
66. TelecomTalk: MeitY on IndiaAI GPUs onboarded
67. Communications Today: IndiaAI GPU-hours sanctioned
68. Outlook Business: AI compute in India at less than half the global rate
69. NVIDIA: DGX Spark
70. ainauten: DGX Spark now costs $4,699
71. Variety Infotech: NVIDIA listings
72. Unanswered.io: NVIDIA AI supercomputer price in India
73. NVIDIA: DGX Station
74. NVIDIA: RTX PRO 6000 Blackwell
75. Tech Insider: RTX PRO 6000 Blackwell price 2026
76. NVIDIA: GeForce RTX 5090
77. Framework: Desktop mainboard, Ryzen AI Max 300 series
78. Apple: Mac Studio technical specifications
79. Apple India: Mac Studio
80. Notebookcheck: Apple raises M3 Ultra RAM pricing, retires 512 GB
81. Ubergizmo: Mac Studio and the DRAM shortage
82. Intel: Arc Pro B60 specifications
83. Microsoft Learn: Copilot+ PCs developer guide
84. NVIDIA: Jetson Thor
85. IntuitionLabs: HBM and DRAM AI memory demand
86. Algeria Tech: memory shortage and server and PC prices
87. Tom's Hardware: TEAM Group warns the RAM crisis will worsen
88. S&P Global: AI memory boom squeezes legacy DRAM
89. NVIDIA: Q2 fiscal 2027 results
90. Epoch AI: trends
91. Epoch AI: data insights
92. Epoch AI: AI data centres
93. Gigazine: H200 and MI325X export rule
94. eeNews Europe: Beijing limits H200
95. IANS: India data-centre capacity to reach 6 GW by 2029
96. Fortune: India AI Impact Summit closes
97. Sentisense: NVIDIA RTX Spark AI PC superchip (headline)
98. Hugging Face: DeepSeek-V4.1-Flash model card
99. Yotta Labs: DeepSeek V4 Flash hardware requirements
100. Hugging Face API: DeepSeek-V4.1-Flash file sizes
101. Hugging Face API: DeepSeek-V4-Pro file sizes
102. MLPerf Inference v6.1 results (summary.csv)
103. HotHardware: NVIDIA Q2 FY2027 results
104. Energy Mag: NVIDIA Q2 revenue and Vera Rubin ramp
105. hartware.de: AMD Q2 2026 results (release reprint)
106. AMD Q2 2026 results, 8-K exhibit
107. CoreWeave: pricing
108. AWS Price List: EC2 us-east-1 bulk file
109. AWS Price List: EC2 ap-south-1 (Mumbai) bulk file
110. Azure Retail Prices API
111. Microsoft Learn: ND GB200 v6 series
112. IndiaAI compute portal: price calculator
113. IndiaAI compute portal: Ready Reckoner
114. IndiaAI compute portal: empanelled GPUs
115. IndiaAI compute portal: eligibility criteria
116. E2E Networks: NVIDIA H100 product page
117. E2E Networks: H200 price in India (blog)
118. E2E Networks: inference endpoints
119. Yotta Shakti Cloud: AI endpoints
120. Let's Data Science: NVIDIA raises RTX PRO 6000 price again
121. GPU Poet: RTX PRO 6000 prices, June 2026
122. GPU Poet: RTX PRO 6000 prices, July 2026
123. Guru3D: RTX 5090 street prices in the EU
124. TechSpot: RTX 5090 price reports
125. Apple: Mac Studio store
126. Apple India: Mac Studio store