Garry Tan Urges U.S. Open-Weight AI Labs Be Allowed to 'Distill' Frontier Models
Highlights
Y Combinator CEO Garry Tan opposes regulatory crackdowns on distillation and suggests the U.S. should consider its own distillation framework to enable smaller American open-weight labs to learn from frontier models. He stresses that distillation—prompting a model extensively to extract its reasoning—is a legitimate training technique and that proprietary labs cannot unilaterally restrict how customers use the outputs of their models. Tan warns that concentrating frontier AI power in a single company would be the real danger, and argues that a healthy balance between frontier and open-weight labs preserves innovation and broader access.
Sentiment Analysis
- Overall tone: Mostly positive toward open research and cautious about heavy-handed regulation. The piece conveys support for competition and wider access to AI development tools, while acknowledging concerns about misuse. The sentiment is constructive: it defends distillation as a legitimate method and urges measured policy, not bans. The argument is framed around protecting innovation, preserving business incentives for frontier labs, and avoiding monopolization of AI capabilities. There is also a cautionary element regarding illicit practices, which the speaker explicitly disavows.
Article Text
Garry Tan, CEO of Y Combinator, has spoken out against efforts to restrict the practice of distillation in AI development and suggested that the United States might adopt its own approach that permits open-weight AI labs to distill frontier models. Distillation refers to a process in which developers interact with a powerful model—using extensive prompting and interrogation—to learn about its internal behavior and reasoning, then use that information to inform or train other models. Tan argues this is a standard, legitimate method many teams use to build and refine new AI systems.
Recent claims by Anthropic have raised alarms: the company released a report alleging that some Chinese labs have carried out what it calls "illicit distillation attacks," using hidden identities and deceptive means to access frontier models. Anthropic’s CEO, Dario Amodei, has urged regulators to step in and curb such activities. Tan, meanwhile, disagrees with a regulatory crackdown and believes that similar distillation work should be allowed domestically—conducted openly and lawfully by American teams.
Tan’s position rests on two main points. First, he contends that model makers should not be able to unilaterally control how paying customers use the outputs of their models. If a model returns text, he argues, customers should be permitted to analyze and learn from that text to advance further research. Second, he notes the asymmetry between current complaints about distillation and the way many proprietary models were trained in the first place: large closed-weight models often relied on broad scraping of text, including copyrighted materials, to build their capabilities without explicit permission from rights holders. In this light, Tan suggests, restricting downstream uses of model outputs seems inconsistent.
He makes a distinction between lawful, transparent distillation and illegitimate access methods: he does not condone stealing credentials or using fraud to gain model access. Rather, he envisions American open-weight labs using legitimate channels—the front door—to perform distillation, thereby creating a stronger domestic ecosystem of open-weight alternatives to foreign offerings.
Tan emphasizes the importance of maintaining both frontier labs and open-weight projects. He recognizes the value of frontier teams driving major advances, and he wants those organizations to remain fundable and viable as businesses. At the same time, he believes open-weight models provide freedom and access for a broader community of developers and researchers. His central concern is avoiding a single-company monopoly over frontier AI, a situation he describes as the true doomsday scenario: a single firm with unrivaled capital and talent that dominates AI development and locks the technology behind proprietary walls.
In Tan’s view, policy should aim to normalize access to intelligence derived from large-scale, publicly accessible data and treat such access more like a public resource rather than a strictly locked offering governed by restrictive terms of service. He believes government can play a role in clarifying norms so that lawful distillation by domestic actors is accepted, preserving both innovation at the frontier and the broader public benefits of open models.
The debate highlights tensions in the AI ecosystem: the need to prevent abuse and illicit behavior, the rights of model creators, the interests of researchers and smaller labs, and the societal risks of concentrated power. Tan’s stance leans toward permissive, transparent practice combined with safeguards against fraud, aiming to foster competition and resilience in the U.S. AI landscape while protecting incentives for high-end research.
Ultimately, Tan calls for a balanced approach—one that supports frontier labs, enables open-weight projects to flourish, and keeps AI capabilities distributed across multiple actors rather than consolidated in the hands of one dominant provider.
Key Insights Table
| Aspect | Description |
|---|---|
| Distillation Defined | A technique where developers extensively prompt a model to learn its reasoning and behavior, then use those insights to train other models. |
| Tan’s Position | Opposes heavy regulatory bans; supports lawful, open distillation by U.S. labs to foster competition and access. |
| Concerns Raised | Anthropic alleges illicit distillation by some labs; Tan condemns illicit methods but warns against broad prohibitions. |
| Broader Goal | Prevent monopolization of frontier AI and ensure a balance between proprietary frontier research and open-weight models. |