All-In Summit talk · September 2026 · a review for the Sutro energy effort

Naveen Rao’s energy heuristics, checked

Rao’s direction is right: moving bits is what costs energy, and brains prove a far cheaper regime exists. But the three numbers that carry the business case — joules per token, Google’s gigawatts, and energy’s share of cost — are each off by 5× or more. Every claim below opens into a derivation you can rerun.

~~12 GW~~≤ 5 GW

Google’s token power: the claim, against everything Google drew in 2025

~~10 J~~0.2–4 J

energy per generated token: “the low end”, against measured fleets

~~50%~~7–13%

electricity’s share of what a token costs on bought GPUs

~~8–9~~2–3 orders

brain-to-GPU gap per operation: a synaptic release against a multiply-add

6 hold, 4 roughly, 4 are off by 5× or more

One more mixes levels of description and two cannot be checked yet. Tap a row for its derivation.

	Rao’s heuristic	Recomputed	Verdict
The energy wall
H1	Google serves 3.2 quadrillion tokens a month	3.2 quadrillion, disclosed May 2026	holds
H2	10 J per token is the low end	0.2–4 J measured	off 3–50×
H3	Those tokens draw 12 GW	all of Google drew 5.0 GW	off ~10×
H4	US data centres: 40 GW, half a world under 100 GW	40 GW is capacity; draw is 22 of 55 GW	roughly
H5	Energy runs out in ~3 years	3 years if growth doubles, 4–7 if steady	plausible
H6	Half the cost of a token is energy	7–13% electricity	off ~5×
Brains as proof
H7	The human brain runs on 20 W	16–22 W by three routes	holds
H8	A macaque brain runs on 1 W, like a phone	1.5 W; a phone averages 0.6 W	roughly
H9	A squirrel brain runs on 8 mW; rats and bats on milliwatts	squirrel 159 mW; 8 mW is a shrew	off ~20×
Moving bits
H10	Most of the energy goes into moving information	94% on a saturated A100; 35:1 in cortex	holds
H11	The cortex moves ~16 Gbit/s	16.3 B neurons × 1 Hz × 1 bit; range 3–83	roughly
H12	A GPU moves ~30 trillion bits a second through memory	H100: 3.35 TB/s = 26.8 Tbit/s	holds
H13	Digital sits 10 orders above the thermodynamic limit, brains 1–2	per operation: 9–10 vs 7; per bit: level	mixes levels
The new machine
H14	A million oscillators need a trillion couplings; 98.4% can go	N² holds; 16,000 per node ≈ a neuron’s fan-in	holds
H15	Bobcat: <500 nJ per image vs GPU millijoules	no public measurement	unverifiable
H16	1000× efficiency in 3.5 years	7.2× a year; NVIDIA claims 3.8×	a goal
H17	Jevons: cheaper intelligence, more demand	prices −10× a year, tokens +7×	holds so far

Rao’s claim Rpublished Pmeasured in sutro-problems Mderived here ESutro working number S

The energy wall is real; the 12 gigawatts is not

The token count is Google’s own. The joules per token are not: ten joules is an unbatched 100-billion-parameter decode, and multiplying it through yields more power than Google uses for everything.

Ten joules per token is the top of production serving, not the bottomJoules per generated token, log scale

NVIDIA P100, GPT-MoE-1.8TComputex 2024 slide

17,000 J

405B-class dense on 16×H100TokenPowerBench, unoptimised serving

40–236 J

Rao: “the lower end”

10 J

100B-active model, batch 1every weight crosses HBM once per token

8 J

DeepSeek-R1, long reasoningML.ENERGY, 11k output tokens

6.7 J

Ceiling: all of Google ÷ its tokens43.6 TWh in 2025 ÷ 3.2 quadrillion a month

4.1 J

Frontier model on 8×H100Microsoft Research, per output token, at the wall

2.8–3.7 J

GPT-4o, per output tokenEpoch AI estimate

2.2 J

Llama-3.1-8B on H200CMU, batch 16 → batch 1

1.4–6.0 J

NVIDIA Blackwell, GPT-MoE-1.8TComputex 2024 slide

0.4 J

gpt-oss-120B on B200InferenceMAX, 2.8 M tokens/s per MW

0.36 J

Same 100B model, batch 32weight traffic shared by the batch

0.25 J

Qwen3-32B on one B200ML.ENERGY v3

0.15–0.31 J

Measured fleets sit at 0.2–4 J. Ten joules is what a 100-billion-parameter model costs when it serves one request at a time.

Twelve gigawatts for tokens would be 2.4× everything Google drew last yearGigawatts, linear scale

World data centres, installedIT capacity, Synergy Research

122 GW

World data centres, average draw485 TWh in 2025, IEA

55 GW

US data centres, installedIT capacity, EPRI — Rao’s “40 GW”

35–44 GW

US data centres, average draw192 TWh in 2024, Berkeley Lab

22 GW

Rao: Google’s tokens alone3.2 quadrillion × 10 J

12 GW

All of Google, average draw43.6 TWh in 2025, Google environmental report

5.0 GW

Google’s tokens at measured rates0.3–1 J per token

0.4–1.2 GW

Installed capacity and average draw differ by about 2×; the talk quotes capacity for the US and draw for the world.

H1Google serves 3.2 quadrillion tokens a monthholds

A disclosed figure, not a derivation: >3.2 quadrillion tokens a month, Google I/O, May 2026 — 7× the 480 trillion of a year earlier.
3.2×10¹⁵ ÷ 2.63×10⁶ s = 1.2 billion tokens per second.
Series: 9.7 trillion (May 2024) → 480 trillion → 980 trillion → 1.3 quadrillion (Oct 2025) → 3.2 quadrillion: 330× in two years.

Inputs and where to get them

Sundar Pichai’s I/O keynotes and Alphabet earnings remarks (blog.google) carry each figure.
Model-API traffic alone was ~22 billion tokens a minute in July 2026 ≈ 0.96 quadrillion a month, about 30% of the total; the rest is first-party surfaces (Search, Workspace, the Gemini app).

Assumptions that move the answer

The count adds input, output and reasoning tokens. Google has never published the split, and input tokens are both the majority and far cheaper to process.
It is a May figure; at 7× a year it grows ~18% a month.

Verdict. Holds exactly. What it cannot carry is a single energy price per token.

H2“10 joules per token is on the lower end”off 3–50×

Decode is memory-bound: each generated token pulls every active weight across the HBM interface once. J/token ≈ active parameters × bits × J per bit.
Unconventional’s own post (May 2026): 100 B active × 39 pJ per byte = 3.9 J, plus ~2 J of KV-cache reads at 32k context.
With Dally’s ~10 pJ per HBM bit (5 access + 4.8 die crossing): 100×10⁹ × 8 × 10 pJ = 8 J. Add the KV-cache reads and that is the 10 J.
Batching divides the weight term: the fetch is shared by every request in the step. Batch 32 → 0.25 J; batch 256 → 0.031 J. Only KV-cache reads stay per-sequence.

Inputs and where to get them

Measured: ML.ENERGY leaderboard v3 (Qwen3-32B on B200, 0.15–0.31 J), InferenceMAX (gpt-oss-120B, 0.36 J on B200, 1.1 J on H100), CMU 2026 (Llama-3.1-8B on H200, 1.4–6 J), TokenPowerBench (405B-class, 40–236 J).
Modelled at the wall: Microsoft Research 2025 (0.31 Wh per query, 3.7 J per output token on 8×H100), Epoch AI (0.3 Wh per GPT-4o query ≈ 2.2 J per output token), Altman (0.34 Wh per query).
Google, all-in: 0.24 Wh = 864 J per median Gemini text prompt, down 33× in a year.
Sutro data point S: one GPU serving an 8B model idles at 50 W, decodes at ~150 W, prefills at ~390 W. If decode streams weights flat out on an A100-80GB, 150 W ÷ 16.3 Tbit/s = 9 pJ per bit: Dally’s HBM figure, seen from a power meter.

Assumptions that move the answer

Batch size: ÷32 to ÷256 on the weight term.
Context length: KV reads add ~0.1–2 J at 32k tokens and dominate at 1M.
Input tokens are compute-bound: 2 × active parameters FLOPs × 1–4 pJ ≈ 0.2–0.7 J for 100 B active, and they batch perfectly E.
Wall versus chip: server overhead and PUE roughly double GPU TDP.

Verdict. Ten joules prices an unbatched 100B-class decode. Fleets do not run that way: use 0.2–0.4 J for mid-size models on current hardware, 2–4 J per output token for frontier models, and several times less for input tokens.

H3“12 gigawatts is going into one company just for AI services”off ~10×

3.2×10¹⁵ tokens × 10 J ÷ 2.63×10⁶ s = 12.2 GW. The 12 GW is H1 × H2, not an observation.
Google’s whole 2025 electricity use: 43.59 TWh ÷ 8,760 h = 4.98 GW (data centres 4.84 GW; fleet PUE 1.09). 2024: 3.62 GW.
Invert it: if every watt went to tokens, 43.59 TWh ÷ 12 ÷ 3.2×10¹⁵ = 4.1 J per token. Extrapolating last year’s +37% into 2026 lifts the ceiling to 5.6 J.
At measured 0.3–1 J per token the tokens draw 0.4–1.2 GW.
Cross-check: 12 GW ÷ 864 J per Gemini prompt = 14 million prompts a second, 1.2 trillion a day — about 150 per human per day.

Inputs and where to get them

Google 2026 Environmental Report: total electricity 21.6 → 25.0 → 31.7 → 43.6 TWh over 2022–2025.
Google’s “Measuring the environmental impact of delivering AI at Google scale” (Aug 2025) for the 0.24 Wh prompt.

Assumptions that move the answer

Search, YouTube, Cloud, Ads and all training share the same 5 GW, so the true token ceiling is well under 4 J.
The token count is May 2026; the electricity is calendar 2025.

Verdict. Off by at least 2.4× against a hard bound, and by roughly 10–30× against measured serving energy. The scaling argument survives; this number should not be repeated.

H4US data centres take ~40 GW, half of a world total under 100 GWroughly

Two units hide in the sentence: installed IT capacity (nameplate GW) and average draw (TWh ÷ 8,760 h). They differ by about 2×.
US: 35–44 GW installed in 2024 (EPRI); 39 GW active in North America (JLL). Draw: 192 TWh = 22 GW, 4.7% of US electricity (Berkeley Lab).
World: 485 TWh in 2025 = 55 GW of draw (IEA); 122 GW installed (Synergy). US share 44–45% on either basis.
Berkeley Lab’s 2030 reference case: 649 TWh = 74 GW of draw, needing 148 GW of grid connection — +17 GW a year.

Inputs and where to get them

Berkeley Lab, United States Data Center Energy Usage Report (Dec 2024) and its 2025 update; IEA Energy and AI (2025) and Key Questions (2026); EPRI Powering Intelligence (2026); JLL and Synergy Research for capacity.

Verdict. “40 GW” matches US installed capacity and “about half the world” holds. “Under 100 GW” is true of world draw (55 GW) but not of world capacity (122 GW). Keep the two units apart.

H5“We’re going to run out of energy in about three years”plausible

Read “run out” as grid headroom ÷ growth, not as generation vanishing.
Headroom: the existing US grid can absorb 76 GW of new load that curtails 0.25% of the time, 98 GW at 0.5%, 126 GW at 1% (Duke, 2025).
Steady growth: +17 GW a year (Berkeley Lab reference) uses that up in 4–7 years.
Doubling growth: 17 + 35 + 70 = 122 GW in three years. That is the 3-year answer.

Assumptions that move the answer

Whether new AI sites accept curtailment at all; firm load has far less headroom.
Interconnection queues, transformers and turbine lead times bind before terawatt-hours do.
On-site generation and efficiency gains (H16) push the date out.

Verdict. Plausible if AI additions keep doubling; four to seven years at the published reference growth. A statement about connections, not about energy.

A token is paid for in silicon, not kilowatt-hours

Efficiency still matters, for the reason Rao gives next: sites are capped by megawatts, so a saved joule is worth the tokens it frees, about 13–16× its price.

Electricity is 7–13% of what a token costs; it nears half only when the silicon is almost freeShare of total cost of ownership; the right column is the electricity share

hardware and networkpower and cooling plantelectricity

Rao’s claimhalf energy, half “CapEx, floor space”

50%energy

1 GW GB200 campus, 2026Epoch AI: servers 60, network 14, facility 16

7%23% with plant

H100 bought from NVIDIA$40k per GPU over 4 years, 9.2¢/kWh

10%18% with plant

CPU-era data centre, 2010James Hamilton, AWS

13%31% with plant

Accelerator built at cost$10k per slot over 6 years

30%56% with plant

Fully depreciated A100only power and plant remain

55%hardware already paid

All rows but the two published ones are estimates; assumptions are in H6.

H6“About 50% of the cost of serving a token is energy”off ~5×

Per GPU-hour: hardware = price ÷ life, electricity = wall kW × PUE × $/kWh, plant = $/W ÷ life.
H100: $40k all-in ÷ 4 years = $1.14/h. DGX H100 draws 10.2 kW ÷ 8 = 1.275 kW; × PUE 1.15 × 9.17¢ = $0.134/h. Plant: $11.3/W × 1,275 W ÷ 15 years = $0.11/h.
Shares: hardware 82%, plant 8%, electricity 10%.
For electricity to reach 50% on a GB200 (≈$1.65/h of everything else, 2.0 kW at the wall) power must cost 1.65 ÷ 2.0 = $0.82/kWh, nine times the US industrial rate.
Silicon at cost — $10k per slot over 6 years: electricity 30%, plant 26%. Everything sized by the megawatt is then 56%: the only honest route to “half”.

Inputs and where to get them

Epoch AI, AI datacenter cost breakdown (May 2026): $8.5B a year per GW; servers 60%, facility 16%, network 14%, energy 7%.
James Hamilton, Overall data center costs (2010): servers 57%, power and cooling plant 18%, electricity 13%.
EIA Electric Power Monthly for $/kWh; NVIDIA DGX guides for wall power; JLL for $11.3M per MW of shell and core.
SemiAnalysis prices an H100-hour at about $1.5 all-in for a weakly financed operator.

Assumptions that move the answer

Depreciation life (4 vs 6 years) and what the buyer pays for silicon: NVIDIA’s ~75–80% gross margin is most of the hardware bar.
Electricity price: US industrial averages span 5.4¢ (New Mexico) to 19.9¢ (California).
Utilisation. Idle GPUs still depreciate, which lowers the electricity share further.

Sutro working number. “Half of GPU TCO is electricity, 80% with your own silicon” S needs the same correction: about 10%, and about 30% (56% counting the plant). “Halving electricity cuts TCO by 25%” becomes 5%.

Verdict. Off by about 5× for anyone buying GPUs. The stronger argument is the one Rao makes next: sites are capped by megawatts, so efficiency is worth the revenue of the extra tokens, not the power bill.

What a joule and a FLOP cost in 2026

9.2¢

per kWh, US industrial average, June 2026 P
commercial 14.2¢ · residential 18.3¢ · new solar PPA 6.1¢

39 MJ

what one dollar buys at that rate E
2.5×10⁻⁸ dollars per joule · 1 GW-year = $0.8B

$50–60B

to build 1 GW of AI data centre, chips included P
≈ $10–12B a year against $0.8B of electricity

$1.23–1.46

revenue each kWh must earn to pay for that campus E
13–16× the price of the kWh

$2.65

per H100-hour, rental index, 2026 P
A100 $1.57 · H200 $3.27 · B200 $5.74

13¢

of electricity in that H100-hour E
1.47 kW at the wall ≈ 2× TDP · 5% of the rental price

$1.86

per 10¹⁸ FLOPs on a rented H100 at 40% utilisation E
of which electricity 9¢ (3.7 pJ per FLOP at the wall)

1.09 · 1.54

PUE: Google fleet · industry average P
Microsoft 1.17

Brains: the existence proof holds, one animal does not

Measured glucose use and a body-mass cross-check agree within 2× for every species.

Twenty watts holds; the squirrel is twenty times hungrier than claimedBrain power, log scale. Ring = the talk; dot = measured glucose use; bar = 2–8% of resting metabolism

Human86 billion neurons

~~20 W~~ → 16–22 W

Macaque6.4 billion neurons, 87 g

~~1 W~~ → 1.5 W

Smartphone, daily averagea 15 Wh battery a day

~~1 W~~ → 0.6 W

Grey squirrel472 million neurons, 5.5 g

~~8 mW~~ → 159 mW

Rat200 million neurons

59 mW

Mouse71 million neurons

18 mW

Smoky shrew36 million neurons — the brain 8 mW fits

9.7 mW

Small bat35 million neurons, 10 g body

2–10 mW

Glucose use from Herculano-Houzel at 0.0478 W per µmol a minute; the bar is Kleiber’s law times the 2–8% that most vertebrates spend on the nervous system.

H7The human brain runs on about 20 wattsholds

Glucose: 77 mg a minute ÷ 180 g/mol × 2,870 kJ/mol = 20.4 W (Clarke & Sokoloff).
Same route, Herculano-Houzel’s human row: 467.8 µmol/min × 0.0478 = 22.4 W.
Oxygen, textbook rate: 3.5 ml O₂ per 100 g per minute × 1,400 g × 20.1 J/ml = 16.4 W.
What is usable: Levy & Calvert (2021) turn 17 W of glucose into 6.2 W of ATP; cortex gets 4.9 W, of which computation is 0.10 W and communication 3.5 W.

Inputs and where to get them

Clarke & Sokoloff (1999), Basic Neurochemistry; Raichle & Gusnard (2002); Herculano-Houzel (2011), Table 1.
Howarth, Gleeson & Attwell (2012): housekeeping is 25% of cortical grey-matter energy, up to 50% for the whole brain; the rest is signalling.

Assumptions that move the answer

Complete oxidation; about 11% of brain glucose is not oxidised, so 20 W is an upper bound.

Verdict. Holds: 16–22 W. Only about 6 W of it is ATP available for signalling.

H8A macaque brain runs on 1 watt — the phone in your pocketroughly

Measured glucose use: 31.4 µmol/min × 0.0478 = 1.50 W; cortex alone 0.94 W.
Against Kleiber’s 3.39 × 7^0.75 = 14.6 W of resting metabolism that is 10% of the body’s budget: between the 2–8% of most vertebrates and the human 20%.
Phone: a 15 Wh battery over a day is 0.6 W on average, 3–5 W at peak.

Verdict. Roughly right, a little low. The phone comparison works on average power, not peak.

H9A squirrel brain runs on 8 mW; rats and bats on milliwattsoff ~20×

Grey squirrel, measured glucose use: 3.33 µmol/min × 0.0478 = 159 mW (472 million neurons, 5.5 g).
Cross-check by body: 3.39 × 0.5^0.75 = 2.0 W of resting metabolism × 2–8% = 40–161 mW. Eight milliwatts would be 0.4%.
Eight milliwatts is a smoky shrew (9.7 mW, 36 million neurons) or a bat with 35 million neurons (~10 mW). Right number, wrong animal.
A 1 W phone budget runs about 6 squirrel brains, not “over a hundred”.
Rat 59 mW; mouse 18 mW. “Milliwatts” is fair for rats and bats.

Inputs and where to get them

Herculano-Houzel (2011), Scaling of brain metabolism with a fixed energy budget per neuron: Table 1 lists glucose use for mouse, rat, squirrel, macaque, baboon and human; about 6 kcal per billion neurons per day throughout.
Mink, Blumenschine & Adams (1981): the central nervous system takes 2–8% of resting metabolism in most vertebrates.
Kleiber (1947): 3.39 W per kg^0.75. Herculano-Houzel et al. (2020) for bat neuron counts.

Assumptions that move the answer

The source’s squirrel row pairs grey-squirrel mass and neurons with a ground-squirrel glucose rate; the body-mass route is independent of that.
No bat brain has a measured glucose rate; the bat figure applies the per-neuron budget.

Verdict. Off by about 20×. The qualitative point stands: a 160 mW brain flies a squirrel through a canopy with millisecond timing.

Moving bits: the cortex’s edge is how few it moves

Both bit rates check out. Divide each system’s watts by them and the brain pays more per bit, because a spike is delivered to thousands of weights that never travel.

One H100 moves 1,675× the bits of a human cortexBits per second, log scale

Human cortex, spikes out of neurons16.3 billion neurons × ~1 Hz × ~1 bit

16 Gbit/s ~~(3–83)~~

A100 40GB HBM

12.4 Tbit/s

A100 80GB HBM

16.3 Tbit/s

H100 HBM3.35 TB/s

26.8 Tbit/s ~~(“~30”)~~

H200 HBM

38.4 Tbit/s

B200 HBM

64.0 Tbit/s

Cortex, counted per synapse reachedeach spike lands on ~7,000 synapses

114 Tbit/s

Count each spike once and the cortex is a trickle. Count every synapse it reaches and it matches four H100s.

Per bit moved the brain is not cheaper; its weights simply never moveJoules per bit moved, log scale

Brain: 20 W ÷ 16 Gbit/sRao’s own two numbers

1.25 nJ

Cortical neuron, per bitLevy & Calvert: 2.9×10¹² bits per joule

0.34 pJ

H100: 700 W ÷ 26.8 Tbit/severy watt charged to memory traffic

26 pJ

DRAM read, per bit (45 nm)Horowitz: 1.3–2.6 nJ per 64 bits

20–40 pJ

HBM, per bitDally: 5 pJ access + 4.8 pJ die crossing

~10 pJ

Brain, per synapse reached20 W ÷ 1.1×10¹⁴ deliveries a second

0.17 pJ

On-chip wire, per bit per mmDally: an add is worth 10 µm of wire

0.1 pJ

A spike is one bit that meets 7,000 stored weights in place. A GPU fetches the weights to the arithmetic instead.

H10Most of the energy in a computer goes into moving informationholds

Horowitz (45 nm): 8-bit add 0.03 pJ, FP32 multiply 3.7 pJ, 64-bit DRAM read 1.3–2.6 nJ. One fetch = 350–700 multiplies.
Dally (5 nm-class): fetching two 32-bit words from main memory 1.3 nJ against 20 fJ to add them: 64,000×. One bit moved 1 mm costs 100 one-bit adds.
Measured on an A100 M: an FP16 multiply-add costs 2.6 pJ at the board against ~0.16 pJ of arithmetic. 6% arithmetic, 94% movement, clocks, leakage and conversion.
Brains obey the same rule: Levy & Calvert (2021) put cortical communication at 3.5 W against 0.10 W of computation, 35:1.

Verdict. Holds, in silicon and in cortex. It is the premise of the Sutro metric as well.

H11The human cortex moves only about 16 gigabits per secondroughly

16.34×10⁹ cortical neurons × 1 spike/s × 1 bit per spike = 16.3 Gbit/s. It reproduces exactly.
The rate is the weak term. Lennie’s energy budget sustains 0.16 Hz → 2.6 Gbit/s; rodent budgets assume 4 Hz.
Bits per spike: 0.6–1.5 in monkey cortex. Levy & Calvert’s codes carry 1.7–5.1 bit/s per neuron → 28–83 Gbit/s.
Per synapse reached: × ~7,000 → 1.1×10¹⁴ bit-deliveries a second.
For contrast: about 10⁹ bit/s enters through the senses and about 10 bit/s leaves as behaviour.

Inputs and where to get them

Azevedo et al. (2009): 86 billion neurons, 16.3 billion in cerebral cortex.
Lennie (2003), The cost of cortical computation; Attwell & Laughlin (2001); Buzsáki & Mizuseki (2014) on log-normal firing rates.
Borst & Theunissen (1999), Table 2, for bits per spike; Levy & Calvert (2021).
Zheng & Meister (2025), The unbearable slowness of being.

Verdict. A fair order of magnitude, ±1 order; an assumption about firing rate, not a measurement. No derivation has been published; this is the reconstruction that lands on 16.

H12A GPU moves nearly 30 trillion bits a second in and out of memoryholds

H100: 3.35 TB/s × 8 = 26.8 Tbit/s. B200: 8 TB/s = 64 Tbit/s. A100-40GB: 12.4 Tbit/s.
“10–100× more inside the chip”: FlashAttention estimates A100 on-die SRAM at ~19 TB/s against 1.5 TB/s of HBM, 12×.
Ratio of bits: 26.8×10¹² ÷ 16×10⁹ = 1,675×.
Joules per bit, all power charged to traffic: H100 26 pJ; brain 1.25 nJ — the brain is 48× dearer per bit.
Per synapse reached, the brain pays 0.17 pJ, 149× cheaper than the H100’s bit.

Inputs and where to get them

NVIDIA datasheets: A100 1,555 / 2,039 GB/s, H100 3.35 TB/s, H200 4.8 TB/s, B200 8 TB/s.
Dao et al. (2022), FlashAttention, for the SRAM estimate.

Verdict. Holds; it is the H100 datasheet. The lesson to take is architectural: the brain wins by fan-out onto weights that stay put, not by cheap wires.

The thermodynamic limit: both figures reproduce, from different levels

The honest gap between a synapse and a GPU multiply-add is 200–2,000×. That is still the whole prize.

Only molecules sit 1–2 orders above Landauer; neurons sit 6–8, GPUs 9–10Energy per event as a multiple of kT ln 2 = 2.87 zJ at 300 K, log scale

Biology

Rao: animal brains“within one to two orders”

10¹–10²

Ribosome, per bit of sequenceKempes: 26× the bound

10^1.4

One ATP hydrolysed83 zJ — where “1–2 orders” lands

10^1.5

Synapse, per bitLaughlin: 2–6×10⁴ ATP per bit

10^5.8–10^6.3

Cortical synaptic release1.64×10⁵ ATP = 14 fJ

10^6.7

Cortical neuron, per bitLevy & Calvert: 0.34 pJ

10^8.1

Brain: 20 W ÷ 16 Gbit/sRao’s own numbers

10^11.6

Silicon

Rao: digital systems“ten orders of magnitude away”

10¹⁰

Loaded CMOS gate switchIRDS, 3 nm: 0.65 fJ

10^5.4

8-bit add, 45 nmHorowitz — the Sutro “7 orders”

10^7.0

Supercomputer, per bit operationKempes: 0.53 pJ

10^8.3

A100 FP16 multiply-add, at the board2.6 pJ

10^9.0

HBM, per bitDally

10^9.5

H100: 700 W ÷ 26.8 Tbit/s26 pJ — where “10 orders” lands

10^10.0

A100 FP32 multiply-add, at the board29 pJ

10^10.0

“1–2 orders” is one ATP molecule; “10 orders” is a whole GPU’s watts per bit moved. Per operation the gap is 2–3 orders; per bit there is none.

H13Digital sits ten orders above the thermodynamic limit; animal brains one to twomixes levels

Landauer: kT ln 2 = 1.38×10⁻²³ × 300 × 0.693 = 2.87 zJ per bit erased (2.97 zJ at body temperature).
“Ten orders” reproduces from the talk’s own numbers: 700 W ÷ 26.8 Tbit/s = 26 pJ = 10^9.96. The measured A100 FP32 multiply-add (29 pJ) lands on 10^10.0 too.
The same formula for the brain: 20 W ÷ 16 Gbit/s = 1.25 nJ = 10^11.6. On this yardstick the brain is farther from the limit.
“One to two orders” is molecular: one ATP is 83 zJ = 10^1.5, and a ribosome spends 26× the bound per bit of sequence. Silicon’s counterpart, a loaded gate switch, is 0.65 fJ = 10^5.4.
Neural computation is far above that: 2–6×10⁴ ATP per bit at a synapse (“five to eight orders”, Laughlin 1998); 0.34 pJ per bit for a cortical neuron = 10^8.1 (Levy & Calvert 2021).
Per operation: a synaptic release costs 1.64×10⁵ ATP = 14 fJ; a GPU multiply-add 2.6–29 pJ. Gap: 190–2,100×. Per bit the two are level: 10^8.1 against 10^8.3.

Inputs and where to get them

Landauer (1961); Bérut et al. (2012) for the experimental confirmation.
Laughlin, de Ruyter van Steveninck & Anderson (1998); Levy & Calvert (2021); Attwell & Laughlin (2001) for ATP per vesicle.
Kempes et al. (2017): translation 26× and DNA replication 165× the bound; supercomputers 0.53 pJ per bit operation.
IRDS 2023 More Moore for 0.65 fJ per switch; Horowitz (2014); Dally (2022–23). Mead (1990) is the classic version: 10⁻¹⁶ J per brain operation against 10⁻⁷ for a microprocessor.
A100 figures measured in sutro-problems.

Assumptions that move the answer

What counts as one operation. Molecular event, gate switch, synaptic release, multiply-add and bit moved span eleven orders.
Landauer bounds irreversible bit erasure. Real chips lose energy charging wires, which reversible logic would not recover.

Sutro working number. “About 7 orders above Landauer” S is exact for an 8-bit add (0.03 pJ = 10^7.0). The three orders between that and Rao’s ten are movement and overhead: the quantity the Sutro metric prices.

Verdict. Mixes levels: a molecule for biology, a whole system for silicon. The literature puts neurons five to eight orders above the limit. Per operation the brain leads by 200–2,000×, which happens to be the size of Rao’s own goal.

The chip: plausible physics, no public measurement

Bobcat’s 500 nJ and the GPU’s millijoule are both batch-1 numbers. On the GPU side that millijoule is overhead, which a batch amortises.

A GPU’s millijoule is launch latency times idle power, not arithmeticJoules per small inference, log scale

Un-0 at 6,656 oscillators, 50 stepsA100 arithmetic only; configuration assumed

5.8 mJ

One CUDA kernel launch on an A10018 µs at 65 W of loaded idle

1.2 mJ

Rao: “a standard GPU”, per image

~1 mJ

MNIST-small record on an A100whole train-and-test run, above idle

0.59 mJ

Rao: Bobcat, per imageFID under 9

<500 nJ

What 500 nJ buys on an A100192,000 multiply-adds at the measured 2.6 pJ

500 nJ

The 2,000× gap is real at batch 1. A batched GPU amortises the launch; the comparison needs a stated batch size.

H14A million oscillators need a trillion couplings; 98.4% can goholds

All-to-all coupling: N² = (10⁶)² = 10¹² directed couplings (5×10¹¹ pairs). Holds.
Pruning 98.4% leaves 1.6×10¹⁰ ÷ 10⁶ = 16,000 per oscillator — the fan-in of a cortical pyramidal cell (5,000–10,000 synapses).
Public Un-0: 16,384 oscillators → 16,384² = 268 M couplings, the bulk of its 322 M parameters.
Cortex for scale: 1.5×10¹⁴ synapses among 1.6×10¹⁰ neurons is a density of 6×10⁻⁷. Brains are far sparser than 98.4%.

Inputs and where to get them

Unconventional’s Un-0 post and repository (June 2026): ImageNet 64×64 FID 6.74 at 16,384 oscillators, 8.04 at 10,240, 8.36 at 6,656; Kuramoto phase oscillators, explicit Euler steps.
The 7.15 sparse result and the ~1.9 gain over dense are from the talk; they are not in the public material yet.
Sutro working numbers S: cortical activity is 98–99.9% sparse and layer-to-layer connectivity ~96–97% sparse.

Verdict. The arithmetic holds and the biological parallel is apt. That sparsity also helps training is the company’s own result.

H15Bobcat: under 500 nJ per image; a GPU needs millijoulesunverifiable

Claimed ratio: 1 mJ ÷ 500 nJ = 2,000×.
A GPU millijoule is mostly overhead: one kernel launch is 18 µs × 65 W = 1.2 mJ on an A100, and the MNIST-small record is 0.59 mJ M.
As arithmetic, 500 nJ is 500 nJ ÷ 2.6 pJ ≈ 192,000 FP16 multiply-adds, or 41 pJ per value of a 64×64×3 image.
If the chip runs the 6,656-oscillator network (44 M couplings) for ~50 steps, that is 2.2×10⁹ coupling updates: 5.8 mJ of A100 arithmetic, and 0.23 fJ per update on Bobcat E.
For scale: a loaded CMOS gate switch is 0.65 fJ, and the kT/C floor for an analog value is 1.5 × 2²ᴺ × kT = 0.10 fJ at 7 bits, 0.41 fJ at 8.

Inputs and where to get them

The talk is the only source. No process node, die size, oscillator count, step count or measurement method is public; press coverage in June 2026 still described the 1000× as a projection.
A100 figures are measured in sutro-problems (NVML energy counter, loaded idle subtracted).

Questions that decide it

What batch size did the GPU baseline run at?
Does 500 nJ include the converters, I/O and host, or only the oscillator core?
Which network and dataset produced FID under 9, and how many steps?

Verdict. Unverifiable today, and physically plausible only at about 7-bit analog precision. The GPU side of the ratio is a batch-1 overhead number, the same floor the MNIST leaderboard fights.

A thousandfold in 3.5 years asks for twice NVIDIA’s own claimed paceYears to reach 1000×, at each trend’s yearly rate

Koomey’s law since 2000computations per kWh double every 2.6 years

25.9 years · 1.3× a year

Koomey’s law before 2000doubling every ~1.6 years

15.9 years · 1.5× a year

NVIDIA’s claimed joules per token17,000 J → 0.4 J, P100 to Blackwell, 8 years

5.2 years · 3.8× a year

Rao’s goal1000× in 3.5 years

3.5 years · 7.2× a year

LLM price per token, equal qualitya16z: 10× a year

3.0 years · 10.0× a year

NVIDIA’s figure bundles FP16→FP4, larger NVLink domains and software.

H161000× better power efficiency within 3.5 yearsa goal

Required rate: 1000^(1/3.5) = 7.2× a year (the earlier 5-year version: 4.0×).
NVIDIA’s slide: (17,000 ÷ 0.4)^(1/8) = 3.8× a year → 1000× every 5.2 years.
Koomey since 2000: 2^(1/2.6) = 1.31× a year → 26 years.
Sutro working decomposition of an analog “1000×” S: freezing the workload 10–30×, denser memory 5–100×, the analog arithmetic itself ≤3×.

Assumptions that move the answer

The baseline. Unconventional’s post measures against what conventional hardware is expected to reach around 2030, a moving target.
Whether the gain is per chip or per served token; the serving stack has been improving faster than silicon.

Verdict. A goal, not a finding. It needs about twice the yearly pace NVIDIA claims for itself, sustained for 3.5 years.

H17Jevons: make intelligence 1000× cheaper and demand explodesholds so far

Jevons needs price elasticity above 1: after a 1000× efficiency gain, total energy rises only if token demand grows more than 1000×.
Price: a GPT-3-class token fell from $60 to $0.06 per million in three years — 10× a year (a16z).
Volume: Google’s tokens rose 7× in a year and 330× in two (H1).
Energy: Google reports 33× less energy per Gemini prompt in a year, while its electricity use rose 37%.

Verdict. Holds so far. It also cuts against the pitch: under Jevons a 1000× chip enlarges the market, it does not shrink the power bill. H5 and H17 cannot both be the selling point.

More heuristics for the Sutro effort

Counts, what a joule buys, and the group’s own working numbers held to the same standard as Rao’s.

An H100 has a brain’s worth of transistors; the brain has 1,875× more synapsesCounts, log scale

Synapses, human cortexthe brain’s “weights”

1.5×10¹⁴

Bits of HBM on a B200180 GB

1.4×10¹²

Bits of HBM on an A100-80GB

6.4×10¹¹

Transistors, B200two dies

208 billion

Neurons, human brain

86 billion

Transistors, H100

80 billion

Transistors, A100

54 billion

Neurons, cerebral cortex

16 billion

Neurons match transistors one for one. Synapses, which hold the weights, outnumber every memory bit on the package by 100×.

S1Neurons in a brain ≈ transistors in a GPUholds for H100

Neurons to transistors: 86×10⁹ ÷ 80×10⁹ = 1.07 for an H100; 1.6 for an A100; 0.4 for a B200.
Power: 20 W against 700 W, 35×. (Sutro working ladder S: brain ~15 W; Hopper 700 W, Blackwell 1 kW, Rubin ~2 kW.)
The weights are the mismatch: 1.5×10¹⁴ synapses ÷ 80×10⁹ = 1,875 per transistor. A neuron is a small computer with ~7,000 inputs, not a switch.

Verdict. A good mnemonic for scale, a poor one for capability. Compare synapses with memory bits instead.

S2Flipping a neuron costs ~100× less than flipping a transistorper farad only

The voltage argument S: switching energy is CV². A neuron swings ~0.1 V, CMOS ~1 V, so per farad biology pays (1 ÷ 0.1)² = 100× less. Holds.
But a neuron is a far larger capacitor. One action potential with its synaptic traffic costs 4.5×10⁸–2.2×10⁹ ATP ≈ 37–183 pJ; a loaded gate switch costs 0.65 fJ.
So one spike ≈ 60,000–280,000 gate switches. Spread over ~7,000 synapses it is 5–26 fJ per synapse reached: ten to forty switches.
What biology buys with low voltage and leaky, stochastic parts is tolerance: determinism is the expensive feature of CMOS.

Inputs and where to get them

Attwell & Laughlin (2001), Lennie (2003) and Howarth, Gleeson & Attwell (2012) for ATP per action potential.
IRDS 2023 for 0.65 fJ per switch; Dally (2023) for ~1 fJ per flip-flop bit.

Verdict. The 100× is right per unit of capacitance and wrong per spike. Quote it as a statement about voltage, not about neurons versus transistors.

One joule buys 10¹⁵ transistor flips, 4×10¹¹ A100 multiply-adds, or about one tokenEvents per joule, log scale

Landauer bit erasures

3.5×10²⁰

ATP molecules

1.2×10¹⁹

Transistor flips0.65–1 fJ each, wire included

10¹⁵–1.5×10¹⁵

8-bit adds (45 nm)

3.3×10¹³

Cortical synaptic releases14 fJ each

7.3×10¹³

A100 FP16 multiply-addssaturated GEMM, above idle

3.8×10¹¹

Bits moved through HBM

10¹¹

64-bit DRAM reads (45 nm)

7.7×10⁸

8192³ matrix multiplies, FP161.4 J each on an A100

0.7

Generated tokens0.2–4 J each

0.25–5

At 9.2¢ per kWh a joule costs 2.5×10⁻⁸ dollars. The spread from flips to tokens is the overhead the Sutro metric exists to expose.

S3The number of flips: what a joule buys, and the floor of the cost modela constant

400 W ÷ 1 fJ = 4×10¹⁷ flips a second on an A100, or 7.4 million per transistor per second.
Against a 1.41 GHz clock that is an activity factor of 0.5%. Most of the chip is dark, or is memory, on any given cycle.
Flips per useful operation: 2.6 pJ ÷ 1 fJ = 2,600 per FP16 multiply-add at the board, against ~160 in the arithmetic itself.
In the idealised DMD machine every instruction loads at least one operand, so the count of flips is the floor of the cost; everything above it is distance S.

Verdict. Flips per joule (10¹⁵) is the constant worth memorising. Divide any measured joules by it and compare with the operation count: the ratio is the overhead.

S4A brain-sized network on today’s GPUs is about a megawattholds

One byte per synapse: 1.5×10¹⁴ B ÷ 180 GB = 833 B200s just to hold the weights.
At 1 kW each: 0.83 MW of GPUs, ~1.7 MW at the wall. Against 20 W: ~86,000×.
Arithmetic is not the problem: 1.5×10¹⁴ synaptic events a second × 2.6 pJ = 390 W. Holding and reaching the weights is.
The megawatt buys inference only; a brain learns on the same 20 W S.

Verdict. “A brain-sized model is a megawatt” is a sound rule of thumb, and it is a memory statement, not a compute one.

Seven working numbers, rechecked

Sutro working number	This check	Action
Half of GPU TCO is electricity; 80% with own silicon	~10%; ~30% with own silicon (56% counting the plant)	revise
We are ~7 orders above Landauer	exact for an 8-bit add; the system is at 9–10	keep, with scope
A neuron flip is 100× cheaper than a transistor flip	per farad yes; a spike is ~10⁵ gate switches	reword
An add is worth 10 µm of wire (sometimes said as 100 µm)	1 fJ ÷ 100 fJ/mm = 10 µm	say 10 µm
Fetch-to-FLOP cost went from 1:1 to 1:100,000	Dally: 64,000× for a main-memory fetch	holds
Average PUE is 1.54	Uptime 2025: 1.54; Google 1.09, Microsoft 1.17	holds
An 8K matmul is ~1 J on an A100	8192³ × 2.6 pJ = 1.4 J in FP16; 16 J in FP32	holds in FP16

Sources

Tokens and energy per token

Google I/O 2026 keynote and Alphabet Q3 2025 / Q2 2026 CEO remarks (blog.google): 480 trillion → 1.3 quadrillion → 3.2 quadrillion tokens a month; 22 billion API tokens a minute.
Google, 2026 Environmental Report: 43.59 TWh in 2025 (31.71 in 2024); fleet PUE 1.09.
Elsworth et al., Measuring the environmental impact of delivering AI at Google scale, arXiv 2508.15734: 0.24 Wh per median Gemini text prompt.
Oviedo et al. (Microsoft Research), arXiv 2509.20241: 0.31 Wh per query, 3.7 J per output token. Epoch AI, How much energy does ChatGPT use? (2025). Altman, The Gentle Singularity (2025).
ML.ENERGY leaderboard v3 (2026); SemiAnalysis InferenceMAX (2025); TokenPowerBench, arXiv 2512.03024; CMU, arXiv 2608.28044.
NVIDIA Computex 2024 keynote: 17,000 J per token on P100 to 0.4 J on Blackwell for GPT-MoE-1.8T.
Unconventional AI, How to improve AI energy efficiency by 1000× (May 2026) and the Un-0 release (June 2026).

Grid and cost

Berkeley Lab, United States Data Center Energy Usage Report (2024) and 2025 update; IEA, Energy and AI (2025) and Key Questions on Energy and AI (2026); EPRI, Powering Intelligence (2026); JLL year-end 2025; Synergy Research (2025).
Norris et al., Rethinking Load Growth, Duke Nicholas Institute (2025): 76–126 GW of flexible headroom.
Epoch AI, AI datacenter cost breakdown (2026); Hamilton, Overall data center costs (2010); SemiAnalysis, GPU cloud economics explained (2023).
EIA Electric Power Monthly, June 2026 data; LevelTen PPA price index, Q2 2026; Uptime Institute PUE survey (2025).
NVIDIA datasheets and DGX user guides for TDP, memory bandwidth, transistor counts and system power; SemiAnalysis and Silicon Data GPU rental indices (2026).
a16z, Welcome to LLMflation (2024); Koomey et al., Implications of historical trends in the electrical efficiency of computing (2011) and Koomey & Naffziger (2015).

Brains and limits

Herculano-Houzel, Scaling of brain metabolism with a fixed energy budget per neuron, PLoS ONE (2011); Azevedo et al., J Comp Neurol (2009); Mink, Blumenschine & Adams, Am J Physiol (1981).
Clarke & Sokoloff, Basic Neurochemistry (1999); Attwell & Laughlin, JCBFM (2001); Howarth, Gleeson & Attwell, JCBFM (2012); Harris, Jolivet & Attwell, Neuron (2012); Lennie, Curr Biol (2003); Raichle & Gusnard, PNAS (2002); Kleiber, Physiol Rev (1947).
Levy & Calvert, Communication consumes 35 times more energy than computation in the human cortex, PNAS (2021).
Laughlin, de Ruyter van Steveninck & Anderson, Nat Neurosci (1998); Kempes et al., Phil Trans R Soc A (2017); Borst & Theunissen, Nat Neurosci (1999); Zheng & Meister, Neuron (2025); Landauer, IBM J Res Dev (1961); Mead, Proc IEEE (1990).

Silicon

Horowitz, Computing’s energy problem (and what we can do about it), ISSCC 2014; IRDS 2023, More Moore, Table MM-9.
Dally, On the model of computation, CACM 2022; Dally, Turakhia & Han, Domain-specific hardware accelerators, CACM 2020; Dally, AHA Retreat keynote 2023.
Dao et al., FlashAttention (2022). A100 measurements: NVML energy counter, loaded idle subtracted, in sutro-problems; see the A100 numbers brief.