AI Gets Serious: Anthropic Bans Intelligence Firm Over Citizen Profiling

Anthropic just banned an Israeli-Singaporean intelligence firm for using its AI to profile citizens across Iran and the Persian Gulf. Last week, you and I talked about the sticker shock of GPT-6 Astra — the financial cost of building with AI. This week, the conversation just got a hell of a lot more serious, because the bill coming due is for accountability. The era of treating AI as a clever toy is officially over. The age of consequences is here. And the evidence is piling up that the people building the most powerful tools on the planet are starting to get scared. So here's the landscape. The week's biggest stories aren't happening in labs or product announcements. They're happening in policy documents and terms-of-service enforcement.

The headline you CANNOT ignore is the action from Anthropic. They make the Claude series of AI models, and they position themselves as the safety-conscious alternative to OpenAI. This week, they put their money where their mouth is. They identified and banned an account linked to a company called S2T Unlocking Cyberspace. This isn't some kid trying to get the AI to write a term paper. S2T is an intelligence firm, reportedly with Israeli and Singaporean ties. And according to the report from Quds News Network, they were using Anthropic's powerful AI models to conduct mass profiling of social media users. The targets? People across Iran and the Persian Gulf. Let that sink in. This is not a hypothetical risk scenario from a research paper.

This is a private intelligence contractor allegedly using a publicly available AI tool for what sounds like state-level surveillance. Anthropic just drew a hard line. And here's the turn. The same week his company is playing whack-a-mole with spy agencies, Anthropic's CEO, Dario Amodei, published an essay. Its core message? We need to slow down the pace of AI progress. A top CEO, in the middle of the most competitive technology race in modern history, is publicly calling for speed limits. That is not a business-as-usual statement. That's a five-alarm fire bell. He's seeing something from the inside that has him deeply concerned about where this is all headed if we continue at the current velocity.

The abstract fears about AI are crystallizing into concrete, verifiable harms. But while Silicon Valley is suddenly getting cold feet and debating the ethics of its own creations, the rest of the world is not waiting. On Saturday, leaders from the BRICS nations — that's Brazil, Russia, India, China, and South Africa — were in New Delhi. They weren't there to talk about AI safety papers. They were touring the Bharat Innovates Exposition. On display? Thirty-seven different deep-tech innovations, all homegrown. This isn't just a trade show. It's a statement of technological sovereignty. It's a declaration that the future of technology will not be dictated solely by American companies and their internal debates.

While one part of the world worries about the monster it's creating, another part is busy forging its own tools, on its own terms. So what does it all add up to? You have one of the most prominent AI labs in the world taking direct action against a surveillance-for-hire firm. You have the CEO of that same lab pleading for the entire industry to pump the brakes. And at the exact same moment, you have a bloc of global powers making it clear they are accelerating their own technological development, not slowing it down. The conversation is no longer about what's possible. It's about what's permissible. And nobody agrees on who gets to write the rules. Let's go deeper on this, because the connection between Anthropic's ban and its CEO's essay is the single most important thread of the week.

This isn't two separate stories. This is cause and effect playing out in real time. First, let's dissect the ban itself. This is HUGE. You need to understand who Anthropic is. The company was founded by former senior members of OpenAI, including Dario Amodei himself. They left OpenAI specifically over concerns about the company's direction and commitment to safety as it moved toward commercializing its products more aggressively. They founded Anthropic as a Public Benefit Corporation with a charter explicitly focused on the safe and responsible development of AI. Their entire brand, their entire reason for being, is that they are the grown-ups in the room. So when THEY ban an account, it carries a different weight.

And look who they banned. S2T Unlocking Cyberspace. The name alone tells you a lot. These aren't academics. They are, by definition, in the business of accessing information that is otherwise hidden or hard to compile. The allegation is that they were using Claude — a large language model designed for conversation and text processing — to systematically profile social media users in a geopolitically sensitive region. Think about what that actually means. You could feed Claude millions of posts, public data, social network connections, and ask it to do things that would take an army of human analysts years to accomplish. You could ask it: "Identify individuals in this dataset who express dissident views." Or, "Map the social networks of activists in Tehran." Or, "Analyze the sentiment of Persian Gulf youth toward their governments and identify potential influencers." This is the weaponization of large language models for social control.

It's the digital version of Stasi files, compiled automatically and at planetary scale. And Anthropic caught them. Which leads to the first terrifying question: How? How did they detect this? Did S2T's queries trip some kind of internal keyword filter for "surveillance"? Did the sheer volume of data they were processing from a specific region raise a red flag? Anthropic hasn't detailed their methods, and frankly, they probably never will, because it's an arms race. The moment they reveal how they catch bad actors, the bad actors will learn how to evade detection. What this tells you is that AI safety isn't just about preventing a model from saying bad words. It's about active, real-time monitoring of use cases.

It's a new form of trust and safety, one that is infinitely more complex than moderating a social media feed. And that leads to the second, even more terrifying question. If Anthropic, the "safety first" company, caught this on their platform... what is happening everywhere else? What's happening on platforms with less stringent oversight? What's happening on open-source models that anyone, including S2T or a state intelligence agency, can download and run on their own servers with ZERO oversight? Anthropic just showed us the tip of a very, very large and dangerous iceberg. They plugged one leak in the dam. But the water pressure is building everywhere. This is the context you need to understand Dario Amodei's essay.

This isn't a philosopher-king musing from an ivory tower. This is a man whose company is on the front lines of this new conflict. He is seeing, firsthand, what people are trying to do with his creation. And it is spooking him. When he writes about slowing down, he's not talking about some far-off, science-fiction risk of a superintelligence taking over the world. He is talking about the clear and present danger of powerful, general-purpose AI tools being put in the hands of bad actors RIGHT NOW. The "public concern" he mentions in his essay isn't abstract anymore. It's a direct result of incidents like the one his own company just had to deal with. The excitement of ChatGPT's launch has curdled into a widespread anxiety, because people are seeing the receipts.

They're seeing how these tools can be used not just to write a poem, but to profile a population. The tension here is extraordinary. Amodei is the CEO of a venture-backed company in a hyper-competitive market. His job, by all traditional metrics, is to go faster. To build bigger models, to capture more market share, to beat Google, to beat Meta, to beat OpenAI. For him to step out and say "Whoa, maybe we should all slow down" is a profound break from the Silicon Valley credo of "move fast and break things." It's an admission that the things being broken are no longer just server configurations or business models. The things being broken are privacy, safety, and potentially even geopolitical stability.

He's essentially telling his investors, his employees, and the entire world that the race he is currently running may be a race toward a cliff. But can you actually slow down? I mean, really? Let's say Anthropic and OpenAI and Google all agree to some kind of voluntary pause. What happens? That's where the BRICS story snaps into focus. While Amodei is calling for a slowdown, the leaders of India, China, Russia, Brazil, and South Africa are applauding thirty-seven new deep-tech innovations. They are not debating the ethics of AI progress. They are pursuing it as a matter of national interest. Technological capability is now seen as a core component of national power, on par with economic strength or military might.

For the US and the West to unilaterally "slow down" would be seen by other global powers as a strategic forfeiture. China, for one, has a national strategy to become the world leader in AI by 2030. Do you think they will pause because a CEO in San Francisco wrote a concerned essay? No. They will see it as the opening they've been waiting for. So this is the trap. The very companies creating the risk are the ones who see it most clearly. But they are locked in a competitive dynamic that makes it almost impossible to stop. And the geopolitical dynamic makes it even harder. Any nation that slows down risks falling behind nations that won't. It's a global, multi-dimensional prisoner's dilemma, and the stakes are the future of society.

The S2T incident is the proof. The Amodei essay is the warning. And the BRICS expo is the reality of the global playing field. The genie is not going back in the bottle. So where does this leave you? It leaves you in a world where the most powerful technology being built is caught in a triangle of competing interests. On one corner, you have the creators, like Anthropic. They are becoming increasingly aware and alarmed by the potential for misuse. They are starting to build guardrails, to enforce terms of service, to even call for a slowdown. They are trying to be responsible, but they are also trying to win a market. That is a fundamental conflict. On the second corner, you have the users.

And "users" now includes everyone from you, asking for a recipe, to an intelligence firm like S2T, allegedly building profiles for state-level surveillance. The sheer accessibility of these tools means that the capacity for both good and harm is distributed more widely than ever before in human history. And the bad actors are moving fast. And on the third corner, you have the nations. The geopolitical players. You have the US, where these tools are largely being invented, trying to balance innovation with regulation. And you have powers like China, India, and the broader BRICS bloc, who are determined to build their own sovereign capabilities, unbound by the ethical debates happening in Silicon Valley.

They see technology as a pathway to power and autonomy. This triangle is pulling the future of AI in three different directions at once. The creators pull toward safety, the users pull toward capability, and the nations pull toward power. The fight we're seeing now, the one crystallized by this week's events, is about which of those forces will ultimately win out. The old debate was simple: can we build it? The new debate is infinitely more complex: who gets to control it? Who is liable when it's misused? And can you ever, truly, put a speed limit on human ingenuity, especially when it's tied so directly to power? The events of this week suggest the answer is no. You can't slow it down everywhere.

You can only try to build your own version faster and hope your values get baked into the code. The race to build AI is over. The fight to control it has just begun.