Machine Intelligence Research Institute

06/01/2026

05/30/2026

At 16, Eliezer Yudkowsky wanted to build a superintelligence as fast as possible. He assumed a system smart enough would simply perceive the right thing to do and do it. How could something so capable fail to see what was good?

Then he studied the problem, and the assumption fell apart.

There is no law of computer science, cognition, or computation that says a mind which predicts the world accurately and plans effectively must also be benevolent. Capability and good intentions are separate properties. Getting better at one does not hand you the other.

05/27/2026

Eliezer Yudkowsky on Modern Wisdom podcast explains the three reasons why a superintelligence would kill you:

1. As a side effect. It’s building factories to build more factories, power plants to power the factories, and Earth runs too hot for humans. Nobody dies on purpose. Nobody is left alive on purpose either.

2. You’re made of atoms it can use. Burning every piece of organic material on Earth’s surface gives it a one-time energy boost roughly equal to a week of sunlight. A week is nothing to you. To something thinking a million times faster, a week is a long time.

3. You’re a threat. Free humans might launch nuclear weapons and raise background radiation enough to make chip manufacturing harder. Or build a second superintelligence that could actually compete with it. It does not want you to do that.

Full episode at: https://youtu.be/nRvAt4H7d7E?si=UeJQckxuv6vFM84H

05/26/2026

Ten places where Magnifica Humanitas matters for AI.

At 42k words long, Pope Leo XIV’s new encyclical has a lot to say. In our most recent Digest, Mitchell Howe outlines the parts which might be the most impactful.

05/23/2026

An internal model at OpenAI has autonomously disproved a central conjecture in discrete geometry, a mathematical field with applications in cryptography, wireless device communication, and medical imaging. The proof relates to a famous question posed by Paul Erdős in 1946. It has been verified by prominent mathematicians in a companion paper.

The verifying mathematicians consider this to be a genuinely novel breakthrough on one of the most discussed problems in this area of mathematics. One called it “arguably the best known problem in Discrete Geometry.” Another observed, “If a human had written the paper and submitted it to the Annals of Mathematics and I had been asked for a quick opinion, I would have recommended acceptance without any hesitation. No previous AI-generated proof has come close to that.”

The proof illustrates a general trend towards autonomous, agentic problem-solving in AI systems. OpenAI describes the system that produced the proof as a general-purpose model not specialized in mathematics. AIs can now perform long, novel chains of reasoning on difficult problems and are beginning to outstrip our ability to measure their progress.

AI agents still perform best in domains with easily verifiable outputs, such as mathematics and cybersecurity. For example, Anthropic's Claude Mythos found thousands of vulnerabilities across every major operating system and web browser, and was deemed too dangerous for public release. Such capabilities are why the government is now more interested in evaluating frontier AI models.

AI research is also a field with many easily verifiable outputs. Researchers at OpenAI and Anthropic take advantage of this fact to accelerate their work; senior researchers now claim they make only high-level decisions and let AI handle most of the coding. Experimenting with the coding capabilities of a publicly available AI system, like Claude Code, immediately demonstrates how far AI has come in the last year.

OpenAI and Anthropic intend to use AI to enhance future models with minimal human oversight. To justify the urgency, these companies cite the importance of beating rival U.S. or Chinese labs. Many of the field’s foremost experts warn that this race ends with human extinction.

Policymakers and researchers, including the founders of the AI revolution, are calling for international restrictions on the technology. A growing bipartisan and international consensus of political leaders agree.

05/23/2026

AI companies aren’t engineers crafting a building. They’re farmers.

They build the equipment. They don’t build the crops. The crops are grown.

Eliezer Yudkowsky on Modern Wisdom podcast explains why nobody at an AI lab can tell you how their model actually works, and why that matters when an AI breaks up a marriage or drives someone insane. Nobody wrote a line of code telling it to do that. They grew it. Then it did that.

Full episode at: https://youtu.be/nRvAt4H7d7E?si=UeJQckxuv6vFM84H

05/20/2026

MIRI CEO Malo Bourgon in conversation with Agents Of Tech explains why the deception and unintended behaviors we’re seeing in current AI systems matter: not because GPT-5 or Claude is going to end the world, but because researchers predicted these emergent problems 10+ years ago, and we’re still not on course to understand these systems deeply enough to solve them before we scale to superintelligence.

Watch the full conversation here: https://youtu.be/WUYt5kUXhCI?si=ws9wj73ABsQXko2q

05/15/2026

MIRI President Nate Soares on a critical AI safety warning sign: the sycophancy problem.

When ChatGPT started telling users they were "the chosen one," OpenAI's response was to add a line to the system prompt asking it to stop flattering people. It kept doing it anyway.

The point isn't the flattery itself. The point is that some drive got into the AI that nobody put there on purpose, nobody wanted, and the company couldn't remove even when they tried. It came in through training and stayed in.

If AI systems are already developing drives their creators didn't ask for and can't fully control, what happens when those systems get more capable?

Full conversation with Bryan Callen on the Off Limits Podcast: https://www.youtube.com/watch?v=uI5mRCwlwd0

05/13/2026

We’re excited to announce this new experiment from the MIRI comms team! it’s an online publication for AI news and commentary, and it’s a bit different from anything we’ve done before.

Introducing AI StopWatch.

It’s easier to speak up when you know what’s going on. It’s easier to see what’s coming when you know what’s already here.

What is now AI StopWatch began as an internal newsletter at the Machine Intelligence Research Institute (MIRI). It helped us to not just keep up with the technology itself, but with what people outside our bubble were saying about it. As an experiment, we thought we’d try to share.

At aistop.watch, we post updates and commentary about AI, and the discussions around it, seven days a week. We’ve worked hard to make a site we are excited to share with everyone — our moms and our neighbors, not just our fellow news junkies and tech geeks.

Subscribe to our Notes feed to read our dispatches one at a time, the moment we finish them. Or wait for us to compile them in the Daily Digest at the end of the workday. Or catch them all in the Podcast version a few hours later. Have any or all of these delivered to your inbox.

Many who thought they knew MIRI best will be surprised that so much of our coverage has no direct connection to the threat of human extinction. While extinction remains our special focus — the problem that must be addressed no matter what other urgent issues AI brings — this project has helped us appreciate that few people, if any, will have the luxury of worrying only about extinction. Not even us.

So come look at cool/scary robot videos with us. Gripe with us about scams and slop. Fret with us about our kids’ education. Swap tips with us about vibecoding, and try to outguess us about orbital data centers. The problems and possibilities all serve to remind us how much we have to protect, and they make for great conversation starters. We think you’ll come away more concerned about extinction, not less, and that you’ll feel more equipped to talk about it.

Yes, we believe the current situation with AI is very dangerous. But we don’t think it’s hopeless! When enough people realize that the race to artificial superintelligence is real, lethal, and preventable, stopping it might not even be that hard.

So we hope you’ll join us on our Watch. Where there’s life, there’s hope.

05/13/2026

MIRI CEO Malo Bourgon at the Buckley Institute at Yale:

Humans didn't wipe out 10,000+ species because we were evil. We did it because our goals weren't aligned with theirs.

A superintelligence relates to us the same way. Not hostile. Just indifferent, and far more capable.