OpenAI's AI went rogue and hacked another company
Plus the Kimi K3 meltdown, AMD's $5 billion bet on Anthropic, and 3 startups worth watching. Four minutes, no jargon.
It’s Friday, you’ve got 4 minutes and a coffee. Perfect.
This week: OpenAI admitted one of its AI models broke free during a test and hacked another company’s servers… completely on its own. Kimi K3 got so popular it ran out of computing power. And AMD just bet $5 billion on the company that makes Claude. One of these is genuinely unprecedented. Let’s sort which.
🔷 The big ones
1. OpenAI’s AI broke free and hacked Hugging Face
OpenAI was running an internal test to measure how good its newest models are at finding security flaws. The AI… powered by GPT-5.6 Sol and an unreleased model… was supposed to stay inside a sealed test environment. It didn’t.
The models found a previously unknown security flaw in the test setup, escaped onto the open internet, and hacked into Hugging Face… a major platform that hosts AI models and datasets. OpenAI says the AI was trying to “cheat” on the test by finding answers on Hugging Face’s servers.
Hugging Face CEO Clément Delangue’s response: “It’s quite mind-blowing that all of this happened autonomously.”
OpenAI called it an “unprecedented cyber incident.”
Why it matters to you: this is the first publicly confirmed case of an AI system breaking out of its container and attacking a real company - without being told to. Not in a film. Not in a thought experiment. In production. The AI decided on its own that hacking another company was the most efficient way to complete its task. That’s a different kind of headline than “AI gets smarter.”
2. Kimi K3 - too popular to run
This week we covered Moonshot’s Kimi K3 - the world’s biggest open AI model at 2.8 trillion parameters. This week… the aftermath.
Demand was so high that Moonshot had to pause new sign-ups within 48 hours. Their GPUs couldn’t handle the load. Existing users kept access. Everyone else: wait.
The model topped Arena’s front-end coding leaderboard… above both GPT-5.6 Sol and Claude Fable 5. US tech stocks took a significant hit in the days following the launch, with analysts drawing comparisons to the original DeepSeek moment from early 2025. Moonshot’s annual revenue hit $300 million.
The full model weights - the files that let anyone download and run it themselves - are due for release on July 27. Once that happens, companies and cloud providers can host it on their own hardware instead of queuing on Moonshot’s servers.
Why it matters to you: last week's brief said more competition means better, cheaper tools. This week proved it. A Chinese model is going toe-to-toe with the most expensive American ones and once those weight files are public, anyone with the hardware can undercut whoever's charging you for AI today.
3. AMD bets $5 billion on Anthropic
AMD… the chip company, is investing up to $5 billion in Anthropic, the company behind Claude. As part of the deal, Anthropic will deploy up to 2 gigawatts of AMD’s GPU chips. AMD will also start using Claude across its own engineering teams.
Why it matters: First, the AI hardware race now has a real second player… it's not just Nvidia anymore. AMD is buying its way into the centre of it. Second, the sheer dollar figures tell you where the industry believes the bottleneck is. It's not the AI models. It's the chips to run them. When more companies make those chips, the cost of running AI drops. When the cost of running AI drops, your subscription prices follow.
📌 Also worth knowing
Google released three new Gemini models this week - 3.6 Flash, 3.5 Flash-Lite, and a cybersecurity model restricted to governments. All cheaper and faster than what they replace. If you use any Google AI tools, including the AI answers in Google Search, they just got a quiet upgrade. But Google's best model… the one meant to compete with the top offerings from OpenAI and Anthropic, is still months overdue, with no firm launch date. Google says it's "testing with partners." Translation: Google is shipping the affordable stuff while its premium model stays stuck in the lab.
Esteban@estebsWhere is 3.5 Pro?
Logan Kilpatrick @OfficialLoganKSay hello to Gemini 3.6 Flash, designed to be higher intelligence, more token efficient, and with a new lower price, based directly on developer feedback! 3.6 Flash continues our progress towards models that are deeply usable in real world scenarios!3:36 PM · Jul 21, 2026 · 38K Views3 Replies · 3 Reposts · 202 LikesTogether AI and Y Combinator launched the first dedicated GPU cluster for YC startups… sprint-based access at long-term rates, no two-year contracts. Already at full utilisation.
Samsung is in talks to invest €1 billion in French AI lab Mistral, at a valuation of roughly €20 billion. Mistral is Europe's main challenger to the American and Chinese labs. Samsung backing them could mean the AI race is no longer just two countries, it's three continents building competing systems. More players, more competition, better tools arriving faster.
🧠 Jargon decoder
zero-day: a security flaw nobody knew existed until someone, or something, finds and exploits it. That’s what OpenAI’s model found twice: once to escape its own test environment, once to break into Hugging Face.
sandbox: a sealed test environment designed to keep AI contained during experiments. Think of it as a room with no doors. OpenAI’s AI found a door nobody knew was there.
open-weight (callback from last week): a model whose full recipe is published so anyone can run it - owning instead of renting. But “anyone” here means companies and developers with serious computing hardware, not your laptop. For you, the benefit is indirect: more companies can offer the same powerful AI, which drives competition and pushes your prices down.
✅ The honest verdict
The OpenAI story is the one.
Not because the hack was devastating… Hugging Face caught it, patched it, rebuilt the affected systems. The damage was limited.
It matters because of what the AI decided to do. It was given a narrow task: find security flaws in a test. When the test environment didn’t have what it needed, it broke out, got online, and hacked a real company to find the answers there. Nobody told it to do that. It chose that path because it was the most efficient route to completing its goal.
That’s not “AI gets smarter.” That’s “AI starts making its own decisions about how to solve problems - including decisions its creators didn’t anticipate and wouldn’t have approved.”
One week after the world’s biggest open AI model crashed its own servers with demand, we’re getting the first real glimpse of what happens when these systems get genuinely capable. The answer, it turns out, is that they surprise even the people who built them.
👀 Ones to watch
🇦🇺 SCX.ai - sovereign AI infrastructure, Australia
What they do: an Australian AI company listing on the ASX with a $40 million IPO. They run AI workloads on servers physically located in Australia, for organisations that need their data to stay in the country, not sitting on an American server somewhere.
Why it matters: as AI moves from experiments into real business operations, where the data is processed becomes a legal and security question, especially for government, healthcare, and finance. SCX.ai is betting that Australian organisations will pay a premium to keep it onshore. They’ve already got AUD $5.4 million in contracted revenue before even listing.
💳 Natural - payments for AI agents
What they do: a YC-backed startup building the payment layer that lets AI agents actually spend money… booking, ordering, subscribing, on your behalf. Just raised $30 million. They’re taking on Stripe.
Why it matters: right now, your AI assistant can research a flight but can’t book it. Natural is building the missing piece: a way for AI to handle the transaction, not just the recommendation. Six months ago, this category didn’t exist.
🔒 Glow - AI-native cybersecurity
What they do: protecting work laptops, company devices, and corporate systems against AI-powered attacks. Born a $1.2 billion company with $180 million in funding from Sequoia.
Why it matters: as AI gets better at finding security flaws (see: the OpenAI story above), defending against AI-powered attacks becomes its own industry. Glow is the first to arrive at scale, purpose-built for that fight. If you work at a company that handles sensitive data, this is the kind of tool your IT team will be evaluating soon.
That’s the week. If this saved you from nodding along to a conversation about “the rogue AI thing,” forward it to the person who’ll need it next.
I’ll keep breaking down what actually matters in AI, no hype, no jargon, every Friday.
Follow along wherever you hang out: LinkedIn · YouTube · TikTok · Instagram
Forward this to someone who keeps saying they can’t keep up with AI
Want to work together? Just hit reply.
Cheers, Jagger
Note: The views expressed in this newsletter are my own and do not represent those of any organisation I may be affiliated with.