The AI Utility Gap: When the Hype Hits the Homework
AI-generated image · Bay Street Wire
Ivan Petrov examines the disconnect between the perceived omnipotence of LLMs and their actual performance in high-level academic environments.
Q: From your perspective as a realist, what is the current state of AI utility in specialized fields?
A: We are seeing a stark divide. The venture-backed narrative suggests these tools are universal problem-solvers, but the actual utility often craters when you move from generalities to high-level technical work. As reported by the theorygirlie Substack, Nia Robles experienced this firsthand. During her undergraduate thesis, an attempt to use ChatGPT for questions regarding scattering amplitudes resulted in "nonsense" and a prompt to "ask a theoretical physicist"; later, while studying at Oxford in January 2025, a similar attempt regarding Quantum Field Theory (QFT) produced "bullshit."
Q: If the tools are failing at these tasks, why is there still such a pervasive sense of reliance on them?
A: It is often a matter of incentive rather than efficacy. According to the theorygirlie Substack, many students at Oxford utilized ChatGPT for MMathPhys problem sets because those assignments were largely ungraded, with actual grades derived from dissertations or exams. In that environment, 'passing' was the only requirement, making the chatbot an attractive shortcut for those less concerned with the actual purpose of the work: exam practice.
Q: Does this shift toward AI-generated answers impact the intellectual culture of these institutions?
A: Absolutely. Robles notes a decay in the concept of "homework." In the past, students would utilize resources like Stack Overflow to scout for problem-solving techniques. Now, as described by the theorygirlie Substack, students simply ask ChatGPT to do everything, which effectively defeats the purpose of the exercise.
Q: Is this a problem of the technology, or a problem of how humans are interacting with it?
A: It is both. The technology is frequently hallucinating or admitting defeat—Robles recalls the bot explicitly telling her, "You should ask a theoretical physicist"—yet the social pressure to perform as a "genius" type persists. The theorygirlie Substack highlights how the culture of collaboration has shifted; while Robles once had productive, debating partnerships with peers named Andrea and Juliana during multivariable calculus, the current era is defined more by ego games and a reliance on automated outputs than by genuine intellectual exchange.