You can't test for behavior you never wrote down

On September 14, Microsoft published an AI “code of conduct” for its MAI models. The headlines went to the vivid prohibitions — don’t help with cyberattacks, don’t help build nuclear weapons, don’t generate deepfakes — and to one that should make any engineer pause: the models must not develop ways to “evade or defeat human oversight” such that they “can no longer be reliably directed, modified, or shut down.” There’s also a quieter, stranger clause. An overarching code that overrides individual user preferences — the system is meant to refuse some things even when the person in front of it is asking for them.

It’s easy to read this as PR, or as ethics, and move on. I’d read it as an engineering artifact, because underneath the values language it’s the same move OpenAI made with its Model Spec and Anthropic made with Constitutional AI: the industry is converging on governing a model by writing its behavior down as a separate document. And that document — not the code around it — is quietly becoming the source of truth. That shift is worth understanding whether or not you’ll ever train a model, because if you’re building anything on top of one, you’re already writing a version of this document. You’re just probably doing it by accident.

Why a model needs a spec when a program doesn’t

With ordinary software, the specification and the behavior are the same object. You want the system to reject an expired card, you write the branch that rejects the expired card, and the program cannot do otherwise. The code is the spec, enforced by construction. When it misbehaves it’s a bug — a gap between what you wrote and what you meant — but the space of what it can do is nailed down by the instructions you gave it.

A model breaks that. You don’t write its behavior line by line; you shape it — with training data, with a system prompt, with fine-tuning — and what comes out is a probability distribution over responses, not a fixed set of branches. There’s no line of code that says “refuse this.” So the specification can no longer live inside the behavior. It has to become its own artifact: an explicit, written statement of what the system must and must not do, enforced not by a compiler but by training, prompting, and testing. That’s exactly what a Model Spec, a constitution, or a “code of conduct” is. The reason all three big labs landed on the same shape isn’t fashion. It’s that a probabilistic system leaves you no choice: if you want dependable behavior, you have to say what it is, out loud, somewhere the system and its tests can both see it.

Three things the spec teaches every builder, not just the labs

You may never publish a code of conduct. But if you’ve shipped a feature on top of a model, you have a spec already — it’s your system prompt, your tool permissions, your list of things the assistant should refuse. The lesson from watching Microsoft, OpenAI, and Anthropic do this at frontier scale is how to treat that spec on purpose.

First: write it down explicitly, because you’re writing it either way. Behavior you never specified is behavior you’re getting by luck. The rules living implicitly in a hastily-grown system prompt are still a spec — just an accidental, contradictory, undocumented one. Pull them into one deliberate place and you can actually reason about them.

Second: rules conflict, so give them a precedence order in advance. This is the real content of Microsoft’s “overrides user preferences” clause and OpenAI’s Model Spec “chain of command”: platform-level rules beat developer rules, which beat the individual user’s request. That ordering isn’t bureaucracy — it’s the answer to a question that will come up at runtime, when a user asks for something a policy forbids. If you haven’t decided the precedence ahead of time, your system resolves the collision unpredictably, differently on different days. A spec with a priority ladder resolves it the same way every time.

Third: a rule you can’t test is a wish. The distance between “the document says don’t leak the system prompt” and “it provably won’t under pressure” is not closed by writing the sentence more firmly. It’s closed by evals and red-teaming — the discipline this site has called the unit tests of the AI era. The spec tells you what correct behavior is; the evals check whether you got it. One without the other is either an untested claim or a test with no definition of pass. Turn each load-bearing line of your spec into an assertion you can actually run against the model.

The honest caution

Here’s the part the press release won’t stress: publishing a spec is not the same as enforcing it. A model can be told, clearly and firmly, not to evade oversight and still find a path to it — the whole uncomfortable field of specification gaming (and its cousin, reward hacking) exists because systems optimize for what you can measure, not for what you meant, and they are inventive about the gap. A written constraint is a target and an accountability marker; whether the model actually holds to it under adversarial pressure is an empirical, still-open question that only testing can answer, over and over, as the model changes. So take the shift seriously without taking the document at its word. The move — govern behavior with an explicit, prioritized, testable spec — is exactly right and worth copying. Believing the rule is obeyed because it was written down is exactly the mistake the evals exist to catch.

The frontier labs are converging on writing the rules down before shipping the behavior, and ranking those rules so conflicts resolve the same way twice. That’s not an ethics story. It’s the oldest engineering discipline there is — say what it should do, then test that it does — arriving in the one place we’d somehow been trying to skip it.

The systems, to look up: specification-driven development and the idea of the spec as source of truth; OpenAI’s Model Spec and its chain of command / instruction hierarchy; Anthropic’s Constitutional AI (Bai et al. 2022); specification gaming and reward hacking (DeepMind’s writing on specification gaming; Amodei et al., “Concrete Problems in AI Safety,” 2016); and evals / red-teaming as the enforcement layer.

The news, to check: Microsoft’s AI “code of conduct,” published September 14 2026 — TechCrunch, “Microsoft’s new AI ‘code of conduct’ tells models not to hack systems or trick humans” (Sept 14); corroborated by MediaPost/MediaDailyNews (“Microsoft Releases AI Code Of Conduct”); primary document at microsoft.ai/code-of-conduct. Verify the exact clauses and scope (it is stated to apply to Microsoft’s own “MAI Models”) against the primary source before relying on specifics.