What is reasoning anyway? A closer look at reasoning in LLMs
Ulrike Hahn
2026-01-13
There is a remarkable degree of polarisation in current debate about the capacities of Large Language Models (LLMs). One example of this is the debate about reasoning. Some researchers see ample evidence of reasoning in these systems, while others maintain that these systems do not reason at all. This paper seeks to shed light on this debate by examining the divergent uses of the term reasoning across different disciplines. It provides a simple clarificatory framework for talking about behaviour that highlights key dimensions of variation in how ‘reasoning’ is used across psychology, philosophy and AI. This highlights not just the extent to which researchers are talking past each other, but also that common inferences about model capability that accompany classification decisions are, in fact, far less compelling than they might seem.
URL: https://zenodo.org/doi/10.5281/zenodo.18231171
DOI: 10.5281/ZENODO.18231171
![Screengrab from a thesis. Key parts: Da Silva Neves, Bonnefon, and Raufaste (2002) write, of a set of properties they hypothesised that reasoners’ inferences ought to satisfy:
"...even if we expect human inference to corroborate these properties, we know
of no sufficient reason to think that lay reasoners would recognize any ratio-
nality postulate as valid, neither that they would conscientiously use them to
guide their reasoning. [...] we assume that human inference is constrained by
knowledge organisation in memory and that its formal properties emerge from
a spreading activation process operating directly on knowledge structures.
Politzer et al. (2006, Footnote 2) write that they "do not test and are not committed to the notion that people have internal representations in the form of Gergonne (1817) diagrams."
Stenning and van Lambalgen (2008, p. 39) write: "Of course we agree that subjects don’t “know these logics” just in the sense that they don’t know the grammar of English, but they do know these logics just in the sense that they do know the grammar of English.”](https://cdn.masto.host/sciencessocial/media_attachments/files/116/006/868/110/720/285/original/073c1054ad75aa0c.png)
