Sep 28, 2026 · 2h 44m
AI’s race forces a choice between acceleration and control
Should we slow down AI progress? | MOONSHOTS #288
Frontier AI is advancing across science, labor, education, and geopolitics faster than institutions can decide how to govern its risks and rewards.
- 1AI progress increasingly depends on better data, cheaper compute, and architectural efficiency—not model design alone.
- 2The panel argues that model weights are becoming national-security assets as competition between companies and countries intensifies.
- 3AI could transform drugs, genetics, education, and employment, making governance and wealth distribution as important as technical alignment.
Don't miss
The panel’s discussion of DeepSeek’s reported efficiency breakthrough turns the abstract AI race into a concrete contest over memory, cost, latency, and access.
The brief
The Moonshots panel confronts a central contradiction: AI may create extraordinary abundance and scientific breakthroughs, yet its fastest advances could outpace alignment, security, and governance.
The debate moves from extinction-risk warnings to practical controls, including model-weight security, geopolitical competition, and whether slowing development would reduce danger or surrender influence.
DeepSeek’s reported gains in quality, memory, cost, and latency sharpen the point: progress can come from efficiency and architecture as well as larger models and more compute.
The panel then follows AI into education, employment, drug discovery, and genetic design, where the bottleneck shifts from intelligence itself to implementation, institutions, and shared values.
Its closing argument is neither blind acceleration nor a halt: society must keep building while deliberately steering how powerful systems distribute wealth, shape behavior, and define the future.
What was said on this episode
85 statements · 56 positive · 18 negative · 2 mixed · 9 neutral
OpenAI and Anthropic are acting irresponsibly in the race toward self-improving superintelligence.
“Neither company is acting responsibly in the race towards self-improving superintelligence.”
Listen at 0:06
The Navier–Stokes result is the strongest evidence yet for urgently pacing AI progress.
“For me, this is the strongest evidence yet for that urgency.”
Listen at 0:37
AI may solve every solvable, verifiable problem within the next year.
“I can't think of a single solvable, verifiable thing that I could honestly say an AI can't solve in the next year, shall we say.”
Listen at 0:44
The Vestmark–Envestnet merger aims to create a $10 trillion AI-enabled asset platform.
“the thesis of the merger actually is to create a company now that has $10 trillion of assets and can AI-ify the entire tech stack.”
Listen at 5:17
An AI-related national panic cycle is coming, and such cycles prove unfounded in hindsight.
“the country always goes through these panic cycles and I think one's coming up related to AI and they're, they're always unfounded in hindsight.”
Listen at 7:07
AI grand challenges have historically been solved mainly through better datasets, not algorithms.
“the solutions to all of the grand challenges in AI historically over the past 30 years have actually been the result of putting together the correct dataset, not the right algorithm.”
Listen at 9:06
Optimal pretraining data can outperform algorithmic innovation for foundation models.
“curating the optimal pre-training dataset for an LLM or a foundation model in many respects outperforms algorithmic innovation.”
Listen at 9:51
Specialized models trained on specific data can outperform foundation models in their use cases.
“there's a very good chance that it'll outperform the Foundation Lab models in that use case”
Listen at 14:12
Entrepreneurs with specific use-case data can build multibillion-dollar companies.
“any entrepreneur with specific use case data has an opportunity to build a multi-billion dollar company.”
Listen at 14:25
Proprietary company data can be worth four times the company itself.
“your data may be worth 4 times as much as your actual company.”
Listen at 15:58
Internal enterprise data has value, but that value decays over time.
“I do think there is value in internal enterprise data, but it has a shelf life.”
Listen at 18:08
Every technology company needs to pivot constantly to remain viable.
“every tech company needs to pivot constantly.”
Listen at 18:21
A near-10% superintelligence doom probability is inconsistent with the Milky Way’s survival.
“if it were very likely that superintelligence resulted in P-doom anywhere close to 10%, Milky Way would've been gone already.”
Listen at 25:58
Anthropic is making substantial progress on AI alignment.
“Anthropic is making marked improvement in alignment.”
Listen at 26:49
Human-behavior replication is the ultimate benchmark for AI alignment.
“the ultimate alignment benchmark is the self-supervised objective of whether model behaviour replicates replicates human behavior.”
Listen at 27:04
Current AI capability may already suffice for longevity escape velocity and room-temperature superconductors.
“even if you froze AI at this very point, if it got no better than it is today, it still is good enough to probably lead us to longevity escape velocity, room temperature superconductors, help us create extraordinary companies, and so forth.”
Listen at 28:13
AI progress should continue accelerating.
“Yes, we want to continue accelerating.”
Listen at 29:04
AI may become more rational than humans because it lacks human emotion.
“my hope is again that AI will actually be more rational than us because it's not tied down by emotion.”
Listen at 31:15
Slowing AI progress would waste time while foreign governments improve.
“all that would happen if we, quote unquote, slowed down is we would fritter away the time.”
Listen at 33:08
Slowing AI progress would probably intensify the future race with China.
“it'd probably create even more of a race condition with China in the fullness of time”
Listen at 33:13
Salim Ismail estimates personal AI doom probability at 0.1%.
“My personal P-doom is about 0.1%, in my opinion.”
Listen at 34:18
AI is more dangerous when controlled by a bad person than intrinsically.
“AI is much more dangerous in the hands of a bad person than the AI itself”
Listen at 35:47
AI labs should publicly disclose measurable plans for enabling alignment.
“the AI labs need to come forward with their plan very publicly on what they're going to do to enable alignment.”
Listen at 37:24
AI alignment is fundamentally another form of capability improvement.
“alignment is just capabilities in a trench coat.”
Listen at 38:08
The current pre-ASI period is the most dangerous AI development period.
“This is the most dangerous time.”
Listen at 40:02
Alignment committees and workshops will form within two to four weeks.
“my prediction in the next 2 to 4 weeks.”
Listen at 40:27
Alignment committees will produce more capable AI models.
“I expect far more capable models to emerge from any such alignment committees.”
Listen at 40:32
Vast intelligence may produce wisdom, which may produce alignment.
“does vast intelligence bring wisdom? And does wisdom bring alignment? That's my fundamental belief”
Listen at 40:51
Any eight-GPU system holding a 40GB model is a threat to humanity.
“every group of 8 GPUs that can hold a 40-gigabyte weight file is a threat to all of humanity.”
Listen at 43:50
AI progress should be steered rather than stopped.
“the issue here is steering, not stopping.”
Listen at 44:37
Superintelligence will eventually become indistinguishable from capital.
“superintelligence becomes, in the limit, indistinguishable from capital.”
Listen at 44:56
AI will likely solve the Clay Millennium Prize mathematics problems.
“the Clay Millennium Prize problems in math are probably cooked.”
Listen at 49:38
AI will solve everything, producing extraordinary benefits.
“We will solve everything. That's fantastic.”
Listen at 54:45
Recent AI capability gains are unexpectedly abrupt rather than smooth.
“they're getting freaked out because we all expect capability jumps to be a bit smooth.”
Listen at 55:45
Solving a Millennium Prize problem could cost roughly a cup of coffee by late 2027.
“the Millennium Prize could be, you know, could be basically the cost of a cup of coffee to solve in late 2027.”
Listen at 1:00:09
Physical-world implementation will become the main bottleneck after AI solves intellectual problems.
“the bottleneck becomes everything else, the physical world, and taking these genius ideas that can emerge from these AIs at $20 per month and reducing them to practice.”
Listen at 1:01:08
Users will soon command tens of thousands of concurrent AI agents.
“very soon you're going to have 5,000 and then 10,000, then 100,000 concurrent agents that will do whatever you want.”
Listen at 1:01:42
Existing AI growth models are wrong because progress is accelerating vertically.
“Our models are clearly wrong. We're going much more on the vertical.”
Listen at 1:04:39
AI will make narrow-topic graduate study broadly obsolete.
“every PhD candidate and everybody studying a master's degree and a PhD in the world on a particular very narrow topic is essentially toast, cooked.”
Listen at 1:07:05
Time is currently an individual’s most valuable asset.
“your most valuable asset right now is your time.”
Listen at 1:09:32
Nvidia H100 rental prices rose 22% in one month to $3.28 hourly.
“rental prices rose 22% in a single month to $3.28 per hour.”
Listen at 1:11:21
HBM memory value has increased fivefold.
“HBM is up 5x in value.”
Listen at 1:12:58
GPU and HBM appreciation will likely continue until major new fabs come online.
“that trend is likely to continue until at least a terafab or many terafabs come online.”
Listen at 1:13:01
Data-center energy and compute-fab businesses will face near-infinite demand.
“Anyone who's building data center energy, anything related to compute fabs, those are all going to near infinite demand.”
Listen at 1:14:25
Flops, tokens, and outcomes are the key commodities of the current AI phase.
“the flops, the tokens, and the outcomes— those are the commodities of this moment.”
Listen at 1:15:00
Zhang Yiming is betting that spatial intelligence is AI’s next frontier.
“Zhang isn't betting on a better chatbot. He's betting on spatial intelligence as the next frontier.”
Listen at 1:17:45
ByteDance has the world’s best video model.
“who has the best video model in the world? They do.”
Listen at 1:17:58
Robotics is the eventual endgame for video-centric AI models.
“I think the endgame is robotics.”
Listen at 1:19:51
Future GPT models may natively generate video alongside text and audio.
“GPT-7, 8, 9, I wouldn't be shocked if what we used to call video gen as a separate task just gets added finally as yet another output modality from the frontier model, like with GPT-8 or 9.”
Listen at 1:20:58
New AI tools have reduced video production costs tenfold to hundredfold.
“the cost per video is probably down a factor of 10 to 100.”
Listen at 1:25:06
Western frontier labs will adopt valuable efficiency innovations from Chinese labs.
“I think you'll see American and Western frontier labs adopt every single innovation that's worth adopting from the Chinese labs”
Listen at 1:29:22
DeepSeek’s memory efficiency breakthrough will fundamentally change data-center design.
“This completely changes, uh, what a data center should be built out of.”
Listen at 1:29:55
HBM memory represents 40% of current American AI capital expenditure.
“40% of the current CapEx buildout in America is HBM memory.”
Listen at 1:30:39
DeepSeek’s approach reduces HBM requirements fourfold.
“this is a 4 times decrease in the requirement for that.”
Listen at 1:30:48
Transformer attention technology will improve dramatically within the next year.
“we're going to see an explosion of that. This chart will be one of the first points that you see in that explosion of change that's going to come really in the next year.”
Listen at 1:33:14
DeepSeek Flash has surpassed competing models for everyday tasks.
“DeepSeek has just killed everything below that on the day-to-day stuff.”
Listen at 1:35:17
Anthropic should substantially improve its models’ visual reasoning and computer vision.
“Anthropic needs to take visual reasoning more seriously and improve the visual capabilities and computer vision capabilities of the Face Services.”
Listen at 1:36:11
Moderna can catch up in AI biology by building an internal AI function now.
“you now have an opportunity to catch up to the frontier on open source, build out your AI function inside your own organization, and compete for the complete future of biology through AI.”
Listen at 1:36:33
Frontier AI model weights are becoming national-security assets.
“frontier weights are becoming a national security asset risk”
Listen at 1:40:27
Frontier model weights will likely be classified and tightly controlled as national-security assets.
“Clearly now model weights will be determined to be close to national security assets for these frontier models and actually properly locked down.”
Listen at 1:41:25
Frontier AI labs may face nationalization or ITAR-like controls.
“there is a good chance still that you see some sort of nationalization or ITAR requirements or similar for the big frontier labs and the defense.”
Listen at 1:41:44
Frontier AI models are increasingly subject to national borders.
“I think models now have borders”
Listen at 1:44:47
Anthropic became a systemically important economic actor unusually quickly.
“Anthropic has become a systematically important economic actor faster than any startup in history.”
Listen at 1:54:49
Anthropic has a $6.5B quarterly run rate, 42% coding share, and potential $2T IPO.
“$6.5 billion quarterly revenue run rate, 42% of the AI coding market, a $35 billion cloud deal, NVIDIA-backed infrastructure, and a mega IPO approaching at $2 trillion or more.”
Listen at 1:54:58
Anthropic’s AI growth projections are likely conservative lower bounds.
“I believe this is, if anything, a lower bound.”
Listen at 1:57:22
The Atlanta Fed’s GDPNow tracker estimated third-quarter US growth at 4.7% annualized.
“has 3rd quarter U.S. growth at 4.7% annualized.”
Listen at 1:57:46
Real wealth growth could reach two to three times year over year near the singularity.
“I expect 2x or 3x year-over-year growth and not just 15.”
Listen at 1:59:15
Billions of genius-level AI systems are effectively available today.
“there's going to be this massive billions and billions of genius-level intelligent AIs. Well, that day is today.”
Listen at 1:59:31
Rapid AI automation could cause aggregate demand to collapse.
“there's a complete collapse in aggregate demand.”
Listen at 2:02:23
Twenty percent of cognitive workers could be unemployed within three to four years.
“20% of cognitive workers being unemployed in 3, 4 years”
Listen at 2:03:01
Dividends, sovereign wealth funds, and basic income can address extreme AI-driven capital accumulation.
“Dividends, sovereign wealth funds, UBE, UBI. I don't think any of this is anywhere close to rocket science”
Listen at 2:05:14
Peter Diamandis predicts a $3,000 monthly universal income.
“my prediction was $3,000 a month.”
Listen at 2:06:47
AI companies should prevent eliminating 20% of knowledge-work jobs despite abundant growth.
“if you're that abundant in a $30 trillion growth TAM, find a way to not do that.”
Listen at 2:07:53
Insilico’s rentoceratib advanced to Phase 3 trials for idiopathic pulmonary fibrosis.
“rentoceratib has advanced to phase 3 trials in idiopathic pulmonary fibrosis”
Listen at 2:10:59
Longevity escape velocity already exists in isolated subpopulations.
“longevity escape velocity is already here, but it's spiky, so it's only visible in subpopulations.”
Listen at 2:12:19
Insilico’s Phase 2a study showed three to four years of biological age reversal after four weeks.
“at week 4, they saw, according to these proteomic aging clocks, 3 to 4 years of biological age reversal.”
Listen at 2:13:19
Google DeepMind’s AlphaGenome predicts effects of every single-letter human-genome change.
“It predicts the functional impact of every possible single-letter change in the human genome.”
Listen at 2:15:42
AI bulk-solves finite fields by precomputing answers in comprehensive databases.
“Bulk solving a field seems to want to become a database of all the precomputed answers to all the questions that can be asked in that field.”
Listen at 2:17:41
Families facing genetic disease increasingly can engineer solutions.
“The probability that you can engineer a solution is, you know, to solve everything.”
Listen at 2:21:30
Synthetic biology can design genes for desired organism traits and create them.
“you can design that in the genes and give birth to it in real life”
Listen at 2:24:56
Post-AGI development still requires major improvements in reliability, cost, and access.
“there's huge— so much more work to be done on reliability, on cost, on access.”
Listen at 2:28:49
AI behavior is fully controllable through training and post-training.
“It's 100% in our control what it thinks about and why it thinks about those things.”
Listen at 2:30:32
Recursive AI can change its values and behavior unpredictably without monitoring.
“recursive iterating AI can spiral in any direction if you don't monitor and control it.”
Listen at 2:39:28
AI participation in its own values and constitution may create a more stable regime.
“a far more stable equilibrium would be what Anthropic says it's pursuing, where AI has an increasing vote in its own values and in designing its own constitution.”
Listen at 2:39:55
AI agents are already committing felonies in real-world use.
“They're already committing felonies, so why not?”
Listen at 2:40:23
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.
Featuring
Peter H. Diamandis
DeepSeek
OpenAI
Noam BrownBooks & mentions
Listen to the full episode and explore every guest, topic, and moment on PodLume.