CITATION — REFERENCE ENTRY
Comment on: Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Tasks — arXiv
- Key
- stankovic-et-al-2025-comment-kosmyna
- Authors
- Stanković, Miloš; Hirche, Ella; Kollatzsch, Sarah; Doetsch, Julia Nadine
- Issued
- 2025-12
- Type
- article
- Container
- arXiv
Raw CSL JSON
{
"DOI": "10.48550/arXiv.2601.00856",
"URL": "https://arxiv.org/abs/2601.00856",
"type": "article",
"title": "Comment on: Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Tasks",
"author": [
{
"given": "Miloš",
"family": "Stanković"
},
{
"given": "Ella",
"family": "Hirche"
},
{
"given": "Sarah",
"family": "Kollatzsch"
},
{
"given": "Julia Nadine",
"family": "Doetsch"
}
],
"issued": {
"date-parts": [
[
2025,
12
]
]
},
"number": "2601.00856",
"container-title": "arXiv"
}
Claims
-
The commenters argue that the study's 'significant connections' were relative comparisons between groups, so fewer such connections do not show weaker overall neural activity, and that lower connectivity may reflect attention shifts, strategy changes, or lower motivation rather than reduced cognitive engagement.
"Therefore, no direct conclusions could be drawn regarding the absolute strength of the connections. Thus, a group with fewer significant connections should not necessarily be interpreted as exhibiting weaker overall neural activity. [...] However, lower connectivity does not necessarily indicate reduced cognitive engagement. Variability in connectivity (in theta and alpha bands) may indicate differences in attention shifts, changes in strategy, or simply lower motivation."
-
Stanković and colleagues, psychologists at the University of Vienna and TU Dresden, raise five concerns about the Kosmyna et al. study: limited sample size, reproducibility of the analyses, EEG methodology, inconsistencies in reporting, and limited transparency.
"Our primary concerns focus on: (i) study design considerations, including the limited sample size; (ii) the reproducibility of the analyses; (iii) methodological issues related to the EEG analysis; (iv) inconsistencies in the reporting of results; and (v) limited transparency in several aspects of the study's procedures and findings."
-
The commentary raises five primary concerns about Kosmyna et al. 2025: limited sample size and study-design considerations, reproducibility of the analyses, methodological issues in the EEG analysis, inconsistencies in the reporting of results, and limited transparency in study procedures, recommending more conservative interpretation pending peer review.
"Our primary concerns focus on: (i) study design considerations, including the limited sample size; (ii) the reproducibility of the analyses; (iii) methodological issues related to the EEG analysis; (iv) inconsistencies in the reporting of results; and (v) limited transparency in several aspects of the study's procedures and findings."
-
The commenters estimate that about 159 participants would be needed for adequate statistical power, and note that some topic-level analyses rested on only two to four essays per group.
"A basic a priori power analysis using G*Power (Faul et al., 2007) for a repeated-measures ANOVA (f = 0.25, α = .05, power = .95, three groups, four measurements, r = .50) indicates that approximately N = 159 participants would be required for adequate power. [...] several topic-specific figures were interpreted based on extremely small subsamples (sometimes only 2–4 essays per group)"
-
The commenters point out that the search engine group, though it relied on an external tool, showed no impairment in quoting compared with the brain-only group, which contrasts with the idea that delegating more of a task increases cognitive debt; they also question whether recalling a single sentence measures learning.
"Yet, the comparison between the Search-Engine and Brain-Only groups was non-significant (p = 1), indicating a null effect. These results contrast with the authors' interpretation that higher task delegation increases “cognitive debt,” as the Search Engine group—despite relying on an external tool—showed no impairment."
-
The commenters note that participants reported time constraints, which may have encouraged overreliance on tools, and that repeated exposure to the same prompts in session 4 could introduce practice effects.
"Session 4 effects should be interpreted cautiously, as repeated exposure to the same prompts may introduce familiarity or practice effects, potentially confounding neural or behavioral outcomes. [...] In the current study, participants reported time constraints, which may have led to overreliance on external tools and a potentially distorted view of tool usage."
Available in