Measure

Hi, Tudinh here.

In 1800 a retired sake brewer named Inō Tadataka set out to walk Japan’s coast. He spent 17 years and 35,000 km on foot, and died before the work was done. His team finished the first accurate map of Japan in 1821, and it stayed in use for a century. A country drawn at the scale of one man’s stride.

One of the 214 hand-drawn sheets of Inō’s map of Japan, and his team surveying Mitarai in 1806. (Courtesy of the National Diet Library and Kure City)

Two centuries later Japan counted its islands again, this time from digital charts. The number more than doubled, from 6,852 to 14,125. No land had risen from the sea. The ruler had simply got finer, and thousands of specks that had always been there finally registered. The closer you look, the more there is to see.

What we measure

Benchmarks: Arena, OpenRouter, LiveBench and Artificial Analysis

A busy month. OpenAI shipped the 5.6 family, SpaceXAI released Grok 4.5 and Meta’s rebuilt lab followed with Muse Spark 1.1, all inside two days. Anthropic brought Fable 5 back from suspension, then closed the month with Opus 5. Which leaves the question of how to know which to use and when.

There are now more benchmarks than models. Price has become one of the key measures, partly as we move past tokenmaxxing, partly because it is the one number nobody disputes. It is also the weakest thing to build a business on. Kimi K3 arrived on 16 July at a third of Fable 5’s rate. Opus 5 landed eight days later at half Fable 5’s price and took the agentic coding benchmarks. A price advantage lasts about as long as the next release.

Kimi K3 edited its teaser, clip selection, audio processing, and rounds of revision

Moonshot’s K3 got close to frontier level across nearly every benchmark, at a third of what the frontier charges. Yet Moonshot lists, among the limitations, something no benchmark scores. It concedes a noticeable gap in user experience against leading models. Perhaps labs can learn from the world of brand and start to invest in measuring the je ne sais quoi. The race may turn out to be about character and experience rather than capability and price.

Being measured

Models are not the only things being measured. Barry wrote a Field Note on how this is already reshaping sales. Buyers now use AI to draft their own requirements, and the model arrives with a shortlist of platforms already attached, sized up and recommended before anyone picks up the phone.

The model is drawing the map of your market, and your website is the survey it works from. Leave that unattended and it fills the gaps with whatever it can find, much of it written by competitors with every reason to rank themselves first. Your website no longer explains your business. It defines it.

Read the Field Note

Measuring ourselves

FOL-IO is an operating system for creative businesses. We work with its founders and run Made by ON on it. It holds everyone’s OKRs, keeps projects moving, and what we learn each day goes back into it. Everyone works from the same map, and it redraws itself as the work changes shape. The point isn’t the measurement. It’s the space it buys us to do better work for our clients.

Learn about FOL-IO

Case studies work in the other direction. A map makes a territory legible to people who have never walked it, and that is what a case study does for the client we haven’t met, the person thinking about joining, the partner weighing us up.

Someone has to do the surveying, though. Inō’s team carried the equipment and worked the arithmetic, so his attention could stay on the coast. CaseMaker does that surveying for our case studies, so ours can stay on the work. It’s behind every one we publish, and the more we produce, the sharper our sense of what good looks like.

View recent case studies

Every shift in technology becomes a shift in how we measure the world, and that shapes what we decide to do in it. Sitting inside that cycle, learning and passing it on, is the part I enjoy most.

Until next month,

Tudinh Duong

Founding Partner & CEO

Made by ON

Ps. Every flight to Japan, I shrink my handwriting because there are seven layers to fit into the box on the arrival card. Prefecture, city, district, chōme, block, building, room, each one nested inside the last. The Wikipedia entry is an enjoyable read.