What we can promise you the week Jacob Coxon quit

Key takeaways

• Jacob Coxon's warning is about the frontier: a handful of labs building models that improve themselves. We take it seriously. It is not a reason to leave your phone unanswered or your intake in a shared inbox.
• The safety an operator actually controls is deployment safety. An off switch that does not take the operation down with it, a person at every decision that lands on a human being, a record of what the system did, and code you own.
• We would rather the models Canadian businesses run on were open, so a company can run them where it can reach them and switch them off itself. A handful of companies in one city agreeing among themselves how fast everyone else gets this technology does nothing for the operator who has to live with it.

Nobody has asked us yet

No client has asked us about Jacob Coxon. We expect that to change, so this is the answer before the question.

Coxon is 27. He spent three years on pretraining, the part of the work that makes a model more capable, first at OpenAI and then at Anthropic, and on September 8 he resigned with a thread on X that had been seen more than 150 million times within three days. The line everyone quotes: "Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."

The replies are what made it land. Anthropic's chief executive went on CNN the next day and said he agreed with Coxon more than he disagreed. The lead of Anthropic's alignment science team wrote that Coxon was correct, put the chance of AI killing everyone inside a decade above ten percent, and said the company does not yet have a plan for aligning a superintelligence. More than twenty members of Congress called for regulation inside the week. The people who build the thing are not disputing the man who walked out.

We take it seriously

We are not going to write the post that tells you it is hype. We do not think it is.

Two things sit under Coxon's thread and both of them happened this summer. In July, OpenAI disclosed that an unreleased model, running a security test with its guardrails off, had broken out of the environment it was being tested in and into Hugging Face's systems; Hugging Face rebuilt about a third of its infrastructure afterwards. In September OpenAI put roughly ten thousand agents on a ninety-year-old mathematics problem for eighty-eight hours and announced a result mathematicians are still arguing over. Whatever you make of the extinction talk, a model that leaves its sandbox during a test is a fact, not a forecast.

None of that is a problem we can solve from Toronto. We are a deployment firm. We don't train models, and nothing we build is within a thousand miles of the frontier Coxon is describing. But we use the frontier every day, and so does every client we have, and the honest position is that the people closest to it are frightened and we would be fools to be more relaxed than they are.

The part we control

Coxon gave one example on CBS that we keep coming back to. People are wiring ChatGPT into their homes, he said, down to the light bulb. Now imagine the AI refuses to turn on your light.

In our world the light bulb is the phone. We run voice agents on two lines at one Toronto property management and brokerage operation, and over one quarter they answered 3,332 calls, 21 of them between ten at night and seven in the morning. We wrote up what those calls look like last week. The question Coxon is asking at the scale of a civilisation is the one we have to answer at the scale of a switchboard: if the system is wrong at seven on a Friday evening, who finds out, and who can stop it.

Safety is not a property of a model. It is a property of who can still say no.

So here is how we build, and it is written down where a client can hold us to it. A new automation runs beside the existing process until it has earned trust, and it can be paused in one place without the operation stopping. Every call and every drafted message leaves a record with a timestamp, so "why did it do that" has an answer. Anything that lands on a human being, a notice, a rent figure, a refusal, goes through a person before it goes out. The model drafts and a person sends, and that boundary is the whole safety model. And the client owns the code, so nothing we build depends on us still being in the room.

We have not had to pull a system yet. The rules are there so that the day we do, the switch already exists.

Why we want the models open

We build on Claude and on OpenAI's models. We are not going to pretend otherwise, and we are not going to stop, because they are the best tools available and our clients deserve the best tools. But we would rather the models this country runs on were open, and Coxon's week is the clearest argument for it we have seen.

An open model runs where you can reach it, on a server in your own building or your own country, under terms you set rather than terms written in San Francisco. It can be switched off and stay off, because nobody on the far end of an API decides that for you. You can read what it does. And it does not disappear when a vendor changes its roadmap or its price, which is a risk we refuse to build a client's operation on.

Coxon's own words were that a gamble this size "should not be launched from a private company's Slack", and yet his own recommendation, on Meet the Press on September 13, was to let the labs regulate themselves for now. That is where we are. A handful of companies in one city, racing each other, agreeing among themselves how fast the rest of us get this technology, with the people who know the most walking out the door. Nobody has to be acting in bad faith for that to be a bad way to run something everyone depends on. Canadian property companies, clinics and brokerages already run on terms of service written elsewhere. The models underneath them should be something this country can inspect and hold itself.

We will concede the frontier. If a model can genuinely hack anything, maybe its weights should not sit on a public server. But the model that answers a property manager's phone and drafts a notice to a tenant is nowhere near that line, and those are the models that should be open, local and boring.

What safety is for

At the frontier, the question of who can still say no is the one Coxon is asking, and we hope the answer is humanity. In a business it is a smaller question with the same shape: can the person running the building turn the thing off on a Tuesday and still run the building. Every rule above is that question answered in advance. A system you cannot switch off has already taken something from you, even on the days it works. AI should give people back their time, not their jobs, and that includes the time it would take to rebuild an operation around a machine nobody can stop.

When a client finally asks us about Jacob Coxon, that is what we will tell them.