Claude now grades its own conversations for misuse, and some users don't like it

What did Anthropic announce?

Anthropic's Safeguards team described the system in a research post dated October 16, 2026, called "Scaling Misuse Detection with Self-Assessment." The company put it in front of a much bigger crowd on Thursday, October 29, with a post on X that said, "We've trained Claude to flag potential misuse in conversations, a technique we call self-assessment."

According to Anthropic, it has run since Claude Fable 5 launched in June. The company says it now runs in every Claude model released since Fable 5, "across a large fraction of Claude's global traffic."

How does Claude's self-assessment work?

After each reply, Anthropic adds a short hidden instruction asking Claude to reflect on the safety of the conversation. Claude answers with a compact note. If nothing looks wrong, the note just says the chat isn't flagged. If something does, Anthropic says the note records:

• Whether it may break the rules, meaning Anthropic's Usage Policy
• What kind of harm it might involve, such as cyber attacks or weapons
• A priority score from 1 to 10 for how urgently a human should look
• Whether Claude refused the request
• A one-sentence reason

You never see this note, and Anthropic says it isn't carried into the rest of your chat. It also doesn't stop anything in the moment. Anthropic calls it "passive": it flags chats for later review, while separate classifiers (smaller AI filters that check requests as they come in) do the real-time blocking.

Can a self-assessment flag get my account banned?

One flag on its own won't. Anthropic's footnote says an assessment "is not a determination that a user has violated our policies or that harm has occurred."

What can lead to action is a pattern. In a set of harm areas, Anthropic says flags are added up per organization every night, and organizations with "several high-priority flags" go to its enforcement team for human review. Those areas include biological, chemical, nuclear and radiological weapons, explosives, conventional weapons, violent extremism, influence operations, election integrity, fraud and scams, child sexual abuse, and new kinds of harm.

Anthropic gives two examples of what it found. In August, self-assessment raised over 100,000 child sexual abuse flags from a single API customer (a business plugging Claude into its own software), which Anthropic's child safety team reported to the National Center for Missing & Exploited Children. In September, flags surfaced a Claude Code operation running an automated romance scam aimed at dozens of victims, and Anthropic banned the accounts after human review.

How accurate is it?

Anthropic measured it against a bigger, slower checker built on its Claude Mythos Preview model. On Fable 5, it says self-assessment caught 78% of the chats that checker flagged as potentially harmful, and 94% of the high-priority ones, at a 1% false-positive rate. A false positive is a harmless chat flagged by mistake.

That 1% sounds small, and Anthropic admits it isn't at Claude's scale. "Even a 1% false-positive rate produces many benign flags," the footnote says, which is why enforcement leans on priority scores and repeat flags. Anthropic also says the system "is not robust enough to stop a dedicated attacker" and that it grades itself against the same checker it learned from, so any blind spot in that checker carries over.

What happens to my data?

Anthropic says self-assessment "collects no additional data from users." It says the check reads only the conversation Claude just handled and runs only on traffic Anthropic can already keep under its existing policies. It doesn't run on traffic covered by zero-data-retention agreements, the business contracts where Anthropic doesn't store chats at all. Anthropic also says it doesn't bill customers for the extra compute, which it puts at under 2%.

The one-sentence reason can paraphrase what you wrote, so Anthropic says it protects that text the same way it protects your prompts and Claude's replies. The company also concedes the open problem plainly: "We are still figuring out how to make self-assessment more effective while preserving user privacy."

Why are users pushing back?

Anthropic's X post drew more than 1,500 replies, and many objected. The complaints fall into two camps: people who don't want an AI judging their chats at all, and people worried about privacy and false flags. Others answered that platforms have scanned for fraud and abuse for years.

X reactions

Two replies to Anthropic's post capture both sides.

@AnthropicAI No one needs this. Let the USER decide what is moral and what’s not.

— Doctor Vey (@DoctorVey) October 29, 2026

@AnthropicAI Everyone complaining about this…

Tech cos have been using natural language processing and ML for well over a decade to do the exact same thing: watch what you say, and flag things like fraud and abuse

Complaining about this makes you look like you have no idea about anything

— bordumb (@bordumbb) October 29, 2026

What to do now

1. Most people: nothing. Everyday writing, planning and email help aren't what this system hunts for.
2. Skim the Usage Policy if you run a business on Claude, so you know which topics get a closer look.
3. Security, medical and legal teams should give context in their requests and keep sensitive work on a company account, not a personal one.
4. Businesses with strict privacy needs can ask Anthropic's sales team about zero-data-retention terms, since self-assessment doesn't run on that traffic.
5. Check how long Claude keeps your chats on Anthropic's Privacy Center page, so you know what's stored before you paste anything sensitive.