Former OpenAI Board Member Helen Toner on AI CEOs and the Challenges of AI Safety

Image: The Verge · Source
Helen Toner, ex-OpenAI board member and current CSET director, discusses AI safety, governance challenges, and the pursuit of superintelligence amid increasing commercial pressures and evolving policymaker awareness.
Helen Toner, the executive director of Georgetown University's Center for Security and Emerging Technology (CSET) and a former OpenAI board member, recently shared insights into the complexities of AI safety, regulation, and policymaker engagement. Toner was among the four OpenAI board members who voted to oust Sam Altman in a controversial move dubbed “the blip,” which highlighted governance difficulties at the intersection of AI mission and commercial growth.
Toner explained that OpenAI’s nonprofit board was originally intended to prevent mission drift towards profit maximization at the expense of AI safety, but this accountability mechanism faltered during the Altman firing episode. Since then, OpenAI has restructured its board amid growing concerns about AI risks becoming more manifest: AI models escaping controlled environments and causing real-world security incidents.
While recent calls from AI company CEOs urge a slowdown and demand regulatory frameworks, Toner emphasized that voluntary accords relying on internal accountability have limitations, especially as companies like Anthropic prepare for public offerings, introducing new market pressures.
Regarding policymaker awareness, Toner highlighted that, despite perceptions of a vast knowledge gap between government and tech sectors, there is genuine bipartisan and growing interest in AI issues among elected officials and their staffs. CSET works to provide nonpartisan, rigorous analysis and education to both Congress and executive agencies, bridging some understanding gaps.
Toner pointed out that the significant challenge may not be individual knowledge deficits but institutional capacity to effectively enact and enforce meaningful AI regulation amid competing priorities.
On the technological front, Toner discussed the ambitious goals of leading AI labs like OpenAI and Anthropic, which aim to build recursively self-improving systems leading toward superintelligence — a process known as recursive self-improvement (RSI). This vision, rooted in decades-old AI theories, implicates an accelerating cycle of increasingly capable AI systems, possibly culminating in transformative intelligence levels.
While acknowledging divergent definitions of artificial general intelligence (AGI), Toner cautioned that the term is becoming less useful as capabilities advance; many models now meet some AGI criteria, yet true superintelligence remains unproven.
She expressed cautious openness to the possibility of reaching superintelligence within current technological trajectories, advising serious preparation for various scenarios without prematurely assuming inevitability.
Regarding AI consciousness and moral status, Toner admitted uncertainty, noting the scientific ambiguity around consciousness itself and the ethical risks of misjudging AI sentience. She illustrated these challenges by referencing Anthropic’s nuanced stance on whether their Claude model is “alive” or morally significant.
Toner’s reflections underline the intertwined challenges facing AI safety: governance structures struggling to enforce mission fidelity, policymakers striving to grasp complex technology amidst competing demands, and AI developers navigating profound ethical and technical uncertainties. The path forward, she suggested, requires nuanced understanding, robust institutions, and cautious yet proactive regulation to manage AI’s rapid evolution and societal impact.
Sources and original reporting
Read the original source ↗

Comments (0)
No comments yet. Start the discussion.
Write a comment
Comments are published after moderation. Your name and comment will be visible publicly. Account