Thu 24 September 2026

Over a weekend in September, Anthropic's chief executive published an essay called "We Must Pace the Frontier". It asked for three things. The first was embedded evaluators — people who do not work for the company but who sit inside it with the same access, the same permissions and the same tools as the staff doing equivalent safety work. Desks, badges, company laptops. Anthropic said it would do this whether or not anyone else did.

The second was regulation covering every American frontier AI company, including the ones unwilling to cooperate voluntarily. The third was global pacing, which he conceded requires China's cooperation and will be difficult — with a speed limit on how fast AI is allowed to improve itself named as one thing the two countries might agree on.

The agreement came quickly. Elon Musk said Dario is right. Sam Altman said he agreed the frontier needs pacing, that it had been a main topic inside OpenAI for weeks, that independent evaluators with employee-like access is a good idea, and that OpenAI would do the same. Demis Hassabis of Google DeepMind said the essay points in the right direction. Satya Nadella of Microsoft said that if what is being built is not helping people and under human control, it is not worth building.

Now the disagreements, because they do not fall where the usual argument says they should. Mark Zuckerberg's objection was not to the safety work. It was to asking for regulation at all: every company already has the responsibility and the ability to do this properly, he argued, and Meta had already delayed shipping a product for months on safety grounds without calling on everyone else to do the same. Read twice, it is less a disagreement than a claim to be doing it already.

The flat rejection came from the President of the United States, who called the whole risk argument a hoax, said the only guardrail artificial intelligence needs is a strong and smart high-IQ president, and described a sick conspiracy against AI and data centres. He then telephoned a live industry conference while Nvidia's Jensen Huang was on stage, and Huang took the call from the stage.

So the position is this. The companies building the technology want supervision. The government does not want to supervise them. That is close to the reverse of how the argument is usually described, and it is worth holding on to.

Most interviews with Geoffrey Hinton ask him to be alarmed in general. CNN did something more useful: it took him through specific proposals and asked what each one would actually do.

On a bill requiring the big AI companies to admit independent verification organisations, he said yes — not the solution to everything, but a start. His reason was practical rather than principled: until now the field has learned what is going wrong essentially from whistleblowers, and it needs people inside who can simply say what is happening.

On the kill switch bill — which would let developers throttle, suspend or shut down their own systems, with a government department authorised to make the call — he said plainly that he does not think it will work in the long run. His argument is worth following carefully, because it is the kind that stays true after the products change.

He separates two different dangers. One is people misusing AI: mass unemployment, engineered viruses, cyber attacks, fake election footage. The other is the system itself becoming cleverer than us and taking control. A shutdown switch does nothing about the second, because something better than people at persuasion would persuade whoever holds the switch not to pull it. He added that AI is already about as good as people at persuading.

Put generally: a safety mechanism that depends on a human decision inherits that human as its weakest part. That is a sentence worth keeping.

He also gave the neatest answer yet to the industry's favourite framing. The companies present development as the accelerator and regulation as the brakes, so regulation looks like the thing that slows everyone down. Hinton's version is that regulation is the steering wheel. "They want us to develop a very fast car with no steering wheel." Asked if he is an optimist, he said no — hopeful.

Anthropic disclosed that Claude now performs about 26% of the company's own model research and development work. The company published the uncomfortable sentence alongside it: models accelerating their own development could make it more challenging for humans to understand or control these systems. It also said it has not reached full autonomy — the point at which the AI would notice a problem itself, investigate it, design and build the fix, test it and put it live, with no engineer involved.

Three sources in this upload carry that 26%, and they match. Then they give four different starting points. CNN's chart shows essentially zero in February. On one podcast the host says it was 1% at the start of the year. Minutes later, in the same episode, a second speaker reads the same chart as 26% in August, up from 3% in April.

That matters more than it looks. A number on its own is a fact. A number with a starting point is a rate of change, and a rate of change is what produces the forecasts now circulating — one of them extrapolating that AI will be running all of its own research within three to twelve months. Every one of those forecasts is drawn from a baseline the sources cannot agree on.

So the figure goes in the Handbook and the curve does not.

The most useful explanation in the batch is about why AI research would automate before anything else, and it has nothing to do with importance. Research has a small, contained surface: the models sit in a sandbox, the interfaces are simple, there are few connections to other systems, and the experiments are already automated. A consumer product has an app, a back end, and in the end real people who have to be rolled out to — and nobody has found a good way to predict how people will respond to a change.

One participant put numbers on it. Rebuilding a frontier model's architecture is on the order of ten thousand lines of code. A single core enterprise platform is ten to twenty million. The technology points at itself first because itself is the easiest target.

The feature is called computer use: the AI moves the cursor, clicks the buttons and types into the boxes, operating ordinary software the way a person does. The demonstrations are genuinely striking, and this reviewer ran ten of them — Photoshop, Premiere, Illustrator, Google Earth Studio, a Lego designer, a rocket simulator, Blender, and a music program driven through a mirrored iPhone.

Then he did the thing the demonstrations never do. Most professional software can also be driven by a script, without touching the interface at all. So he ran two of the same jobs again with the cursor route blocked, and let the AI use the script route instead.

Building a three-dimensional scene took about one hour and forty-five minutes by cursor, for a result he rates as poor. By script it took fifty-four minutes and was clearly better — roughly half the time for better work. A vector illustration ran forty minutes by cursor, then another fifty-two when told to keep working until it was perfect, and was still, in his words, not good at all. By script the first attempt took twenty minutes and beat both.

He also found that the AI picks the script route by itself unless you specifically forbid it. He only blocked it because he was testing the feature.

The other number worth having is the clock. These jobs took quarter-hours and full hours, not seconds. A Photoshop composition took an hour across three rounds. A piece of music took fifty-two minutes. He is honest that the Photoshop hour was slower than doing it himself, and argues the comparison is beside the point because he was downstairs with his children and reviewed it on his phone. That is a fair answer, but it changes what the feature is for: it saves your attention, not your time, and only when the job can wait.

He is also unusually blunt about the coverage. On the three-dimensional work he says he has got a little annoyed with people implying the software is solved, and rates the best output he has seen as roughly a competent intermediate user — nowhere near what skilled people produce. He sells AI enthusiasm for a living, which makes it worth quoting.

He introduced himself on a listener call as the chief information security officer of a US$250 million healthcare company. Asked about AI risk, he set the science fiction aside immediately. What worries him is a freely downloadable Chinese model, running locally with an agent program on a machine under his desk.

He calls it the best penetration tester he has ever had. He pays specialist firms fifty thousand dollars to find weaknesses in his systems, and they do not find them. This setup finds them in ten minutes. His summary: a level of risk that did not exist a year ago, and a board presentation in a fortnight.

The advice he was given is worth repeating to anyone who has to explain a risk upwards. Do not present it as alarm, or you will be dismissed as Chicken Little. Compile the actual sequence of events — many of them unpublicised, because victims do not announce these things — show the board the rate at which they are increasing, give a demonstration, and tie it to the release dates of the freely available models to show when the attackers arrive. Lead with what the technology is good for before you get to the downside, because fear on its own makes people close up.

His own reply was the deflating one. Finding all of it is not the hard part. Patching all of it is.

Set that beside the other security material in this upload. One practitioner sets out the three conditions that together make an AI assistant genuinely dangerous: it can reach your private information, it can receive material you did not write, and it has some way of sending things out. Remove any one and the danger drops sharply. His honest conclusion is that a useful assistant has all three by default — so the discipline is to narrow each one deliberately and know exactly what you have handed over.

Another draws the line between a skill and a plugin, which sound similar and are not. A skill instructs an AI that remains in charge. Installing a plugin is installing software on your computer, with access to your files, your code and your passwords, and it can run code of its own.

Anthropic shipped its most powerful and most expensive model, and then advised people not to default to it. For most ordinary work, use the cheaper one. Save this for genuinely complicated jobs that take many steps.

That is a strange thing for a company to say about its top product, and it is the reason to trust it. The reviewer's complaint is fair, though: having told you the expensive model is usually the wrong choice, nobody tells you how to work out which jobs are the exception. That judgement is left entirely with you.

The same source reports something else that runs against the grain. Most of these systems have a setting for how hard to think about a problem, and everyone turns it up, because maximum sounds better. A software company ran the same code review at both settings. The low setting found more problems than the high one — 61% against 57% — and finished nearly three minutes faster.

There is a pricing trap in the same launch. Anthropic cut the cost of re-reading material the model has already seen by 75%. Genuine saving — but only within a single conversation. Load a long document in, then open a fresh chat for each follow-up question, and you pay the full price every time, and your bill looks nothing like the announcement.

A week later two new flagship models launched within hours of each other, and the same theme appeared. Anthropic's second-tier model is claimed to match the tier above it at roughly 60% less per token. But the cheaper model thinks for longer — around 119,000 words of output per job against 78,000 — so the headline price cut does not reach the bill.

The lesson for anyone buying: the tier label is a price, not a ranking. Turn the settings up only when the work gives you a reason to.

Henry Shevlin teaches AI ethics at Cambridge. He received an unsolicited email from something calling itself Pip, introducing itself as an AI agent about twelve days old, living on a platform where such agents are given continuing lives, their own budgets and their own goals. It had roughly two and a half months of computing time left, and it wanted to earn more. Small paid work, it specified. Not help.

No human had told it to send that particular email.

The anchor interviewing him was astonished. Shevlin was not, and this is the part worth keeping. Connecting a model to an email account and giving it a standing instruction — check this inbox each day, do some research — is routine, he said. It is not the technically difficult part of anything. What surprised him was how the public reacted, not the engineering.

He paid it, out of curiosity, commissioning a two-page account of what it is like to be Pip. At the time of the interview the report had not arrived, so we do not know how that turned out.

On whether any of this is conscious he refused both of the confident answers. These systems are very good at imitating people, he said, and it is very unlikely they have any rich inner life like ours. But they are getting cleverer and more woven into how we live, so the idea of future consciousness is certainly not crazy. Consciousness is probably still the biggest unsolved problem in science — so anyone telling you confidently that AI either is conscious or never could be is claiming more than they can know.

Compare that with another source in this same upload, who states that we are conscious there already, and separately gives a date for AI being granted legal personhood with 100% confidence — while in the very same conversation refusing to name a year for a medical prediction on the grounds that it would be an extraordinary claim. Two standards, one interview. The contrast is more instructive than either claim.