SAN DIEGO — Thousands of the world's top artificial intelligence experts gathered here last week for the annual NeurIPS conference, a sprawling event that underscored both the explosive growth of AI and a humbling reality: even the field's leading minds admit they don't fully understand how the most advanced systems work.
The Neural Information Processing Systems, or NeurIPS, conference marked its 39th year in 2023, drawing a record-breaking 26,000 attendees to the San Diego Convention Center. That's double the number from just six years ago, reflecting the meteoric rise of AI from an academic curiosity to a cornerstone of industries worldwide. Founded in 1987, NeurIPS has long focused on neural networks — computational structures inspired by human and animal cognition — and their intersections with fields like neurobiology and physics. What began as a small meeting in a Colorado hotel has ballooned into an event rivaling the scale of Comic-Con, filling the convention center with sessions on everything from AI-generated music to cutting-edge model architectures.
Amid the buzz of presentations and networking, one theme dominated conversations: the profound mystery at the heart of modern AI. Leading researchers and executives openly acknowledged that the inner workings of frontier AI systems remain largely opaque. This pursuit of clarity, known as interpretability, aims to decode how these models process information and make decisions.
Shriyash Upadhyay, an AI researcher and co-founder of the interpretability-focused startup Martian, described the field as still in its earliest stages. “People don’t really understand fully what the field is about. There’s a lot of ferment in ideas, and people have different agendas,” Upadhyay said in an interview at the conference. He compared interpretability to foundational questions in physics, likening it to scientists debating the existence and measurability of electrons rather than refining precise measurements. “In traditional incremental science, for example, where the ideas are mostly settled, scientists might attempt to add an additional decimal point of accuracy of measurement to particular properties of an electron,” Upadhyay explained. “With interpretability, we’re in the phase of asking: ‘What are electrons? Do electrons exist? Are they measurable?’ It’s the same question with interpretability: We’re asking, ‘What does it mean to have an interpretable AI system?’”
To spur progress, Upadhyay and his team at Martian announced a $1 million prize during NeurIPS, aimed at advancing interpretability research. The initiative highlights the urgency felt across the community to demystify AI before its capabilities outpace human oversight.
Approaches to tackling this challenge varied among major players. Early in the week, Google's interpretability team revealed a strategic shift. Neel Nanda, a leader in the effort, stated that the group was moving away from ambitious goals of fully reverse-engineering models toward more practical methods with nearer-term impacts. “Grand goals like near-complete reverse-engineering still feel far out of reach” given the desire for results within about 10 years, Nanda wrote. He cited AI's rapid evolution and the slow pace of prior, more comprehensive approaches as key factors in the pivot.
In contrast, OpenAI's head of interpretability, Leo Gao, announced on Friday that his team was committing even more resources to deep, ambitious explorations. Gao emphasized the need to “fully understand how neural networks work,” a stance he elaborated on during NeurIPS sessions. This divergence illustrates the field's lack of consensus on how best to probe AI's black box.
Not everyone shares the optimism for complete comprehension. Adam Gleave, an AI researcher and co-founder of the nonprofit FAR.AI, expressed doubt about simplifying the complexity of large-scale neural networks. “I suspect deep-learning models don’t have a simple explanation — so it’s simply not possible to fully reverse engineer a large-scale neural network in a way that is comprehensible to a person,” Gleave said. Still, he remained hopeful about incremental gains that could enhance reliability and trust. “I’m excited by the growing interest in issues of safety and alignment in the machine-learning research community,” Gleave told reporters, though he noted the irony of capability-focused sessions occurring in venues “that could double as aircraft hangars.”
Beyond interpretability, attendees grappled with another foundational issue: how to properly evaluate AI's capabilities. Current benchmarks, many agreed, fall short for assessing broader traits like intelligence or reasoning. Sanmi Koyejo, a Stanford University computer science professor and leader of the Trustworthy AI Research Lab, pointed out the limitations of existing tools. “We don’t have the measurement tools to measure more complicated concepts and bigger questions about models’ general behavior, things like intelligence and reasoning,” Koyejo said. He added that many evaluations were designed for narrower, task-specific assessments from an earlier era in AI development, calling for greater investment in new, robust testing frameworks.
This evaluation challenge extends to specialized applications, such as AI in biology. Ziv Bar-Joseph, a Carnegie Mellon University professor and founder of GenBio AI, described the state of biology-focused AI assessments as particularly nascent. “It’s extremely, extremely early stages for biology evaluations. Extremely early stages,” Bar-Joseph said. “I think we are still working out what should be the way we evaluate things, let alone what we should even be studying for evaluation.” His comments reflect broader concerns about ensuring AI's scientific utility without reliable metrics.
Despite these uncertainties, the conference showcased AI's transformative potential in accelerating scientific discovery. For the fourth consecutive year, an offshoot event explored AI's role in advancing research across disciplines. Ada Fang, a Harvard Ph.D. student studying AI and chemistry, co-organized the workshop and called it “a great success.” She highlighted the shared challenges in fields from biology to physics. “Frontier research in AI for science is happening separately in areas of biology to materials, chemistry and physics, yet the underlying challenges and ideas are deeply shared,” Fang said. “Our goal was to create a space where researchers could discuss not only the breakthroughs, but also the reach and limits of AI for science.”
Jeff Clune, a University of British Columbia computer science professor and pioneer in AI for science, echoed the enthusiasm during a panel discussion. He described a surge in interest, with colleagues and newcomers approaching him constantly about collaborative efforts. “The amount of emails, contacts and people who are stopping me at NeurIPS and want to talk about creating AI that can go learn, discover and innovate for science, the interest level is through the roof,” Clune said. “I have been absolutely blown away by the change. I’m looking out in the crowd here and seeing people who were there 10 years ago or 20 years ago, in the wilderness with me, when nobody cared about this issue. And now, it seems like the world cares. It’s just heartwarming to see that AI works well enough, and now there’s enough interest, to want to go tackle some of the most important and pressing problems for human well-being.”
Upadhyay offered a pragmatic perspective on the gaps in understanding. “People built bridges before Isaac Newton figured out physics,” he noted, suggesting that AI's real-world benefits need not await total comprehension. This sentiment captured the conference's dual narrative: awe at AI's progress alongside calls for caution and rigor.
As NeurIPS wrapped up, the event left attendees pondering the path forward. With AI systems powering everything from drug discovery to climate modeling, the push for interpretability and better evaluations will likely intensify. Initiatives like Martian's prize and ongoing workshops signal a community determined to bridge the knowledge gap, even as models grow more sophisticated. For now, the field's leaders agree on one thing: unraveling AI's mysteries is essential to harnessing its promise safely.
The conference's scale and diversity — from academic papers to corporate announcements — underscored AI's global stakes. Held from December 10 to 16, 2023, in San Diego, NeurIPS not only celebrated achievements but also exposed vulnerabilities that could shape the technology's future trajectory.