Home › Research › Brain tissue and pathology
Context Matters in LLM-Assisted Qualitative Data Analysis: Workflow Development and Multidimensional Evaluation in Health Research
Dai, H.; Li, Y.; Villalobos-Quesada, M.; Ng, Y. M.; Aardoom, J.; Versluis, A.; Shen, H.; Spruit et al.. medRxiv (preprint), 2026-09-11
Topic: Risk, prevention and care in population studies
Large language models (LLMs) are increasingly used for qualitative analysis, but strong language performance does not guarantee strong interpretation when meaning depends on method and context. We developed a context-specific workflow that translated framework analysis into bounded, sequential LLM-supported tasks and applied it to 20 Chinese-language interviews on fear of dementia (approximately 150,000 characters). Four multilingual LLM-generated outputs and a previous researcher-only analysis were blindly evaluated by 16 relevant experts across different analytical domains and qualitative quality dimensions (496 expert-item records). All LLMs completed the workflow, but rankings varied markedly. Qwen-output showed the most consistently favourable overall profile, while Claude-output was most often ranked first in individual domain-dimension evaluations. The researcher analysis received both highly favourable and highly unfavourable rankings; no output was consistently preferred. LLM-assisted qualitative analysis should be evaluated within its research contexts, with transparent workflows, multidimensional assessment, and continued researcher oversight.
Key findings
- Qwen-output showed the most consistently favourable overall profile, while Claude-output was most often ranked first in individual domain-dimension evaluations.
Why it matters
This study developed and evaluated a workflow for using large language models to analyze interviews about fear of dementia. It is about research methods for qualitative data analysis, not about treating, curing, or preventing Alzheimer's disease.
Study details
- Evidence strengthLow
How this study scored 16/100
Preprint — not yet peer-reviewed. It scored 23 on the same rubric, but non-peer-reviewed sources are capped at 16, so it shows as 16. Scored zero for study design, sample size, effect reported, studied in people and randomised / blinded because the record does not state it. We have no data on impact for its field and year, so that was left out: it earned 22 of the 94 points we could judge it on, which is 23 out of 100.
| Component | Points | Why |
|---|---|---|
| Journal tier | 2 / 10 | Tier 4 — not a peer-reviewed journal (medRxiv (preprint)) |
| Journal's citation record | 0 / 4 | Not a journal article, so the journal's record does not count |
| Study design | 0 / 12 | No study design recorded — scores zero, not guessed (not stated in the record — scored 0) |
| Strength of evidence | 1 / 5 | Rated “low” in our summary |
| Peer-review validation | 4 / 10 | Preprint — not yet peer-reviewed. Hosted on a preprint server. |
| How recent | 5 / 5 | 2026 — within 1 year(s) of the newest study here |
| Sample size | 0 / 4 | Participant count not stated — scores zero (not stated in the record — scored 0) |
| Effect reported | 0 / 3 | No effect size reported — scores zero (not stated in the record — scored 0) |
| Studied in people | 0 / 3 | Not stated whether this was studied in people — scores zero (not stated in the record — scored 0) |
| Randomised / blinded | 0 / 2 | No randomisation or blinding stated — scores zero (not stated in the record — scored 0) |
| Times cited | 1 / 7 | Cited 1 times (OpenAlex, 2026-10-10) |
| Citations per year | 0 / 5 | 0.5 citations a year over 2 year(s) since 2026 |
| Impact for its field and year | — / 6 | No field-weighted citation impact (OpenAlex has none yet) — left out of this score |
| Cited by studies here | 0 / 6 | No other study in this library cites it |
| Still being cited | 0 / 4 | Cited 1 times in 2024–2026 (2026 still in progress; OpenAlex) |
| Senior author's track record | 3 / 4 | Last-listed author Niels Henrik Chavannes: h-index 67 (OpenAlex, whole career) |
| Institution's track record | 1 / 3 | University of Groningen: institution h-index 1,383 (OpenAlex) |
| Multi-centre collaboration | 2 / 3 | OpenAlex lists 6 institution(s) in 2 countries |
| Indexed in PubMed | 1 / 2 | Not in PubMed, but has a DOI |
| Free to read | 2 / 2 | Free to read |
| Points | 22 / 94 | Scored on 19 of 20 parts. We have no data for the rest, so they are left out: 22 of the 94 points we could judge it on is 23 out of 100. |
| Total | 23 / 100 | Capped to 16 because this is not a peer-reviewed article. |
What each of these 20 components measures, and what this score is not →
Read the original
Full text: free to read — OpenAlex lists this copy as free to read: the link above (checked 6 Feb 1020).
Cite this study APA, Vancouver, BibTeX · download .bib or .ris
Built from the record alzhub holds for this paper, not looked up. Volume, issue and page numbers are not held in the record. The record's author list ends in “et al.”, so not every author is named. Check the citation against the original before you use it.
APA 7th
Dai, H., Li, Y., Villalobos-Quesada, M., Ng, Y. M., Aardoom, J., Versluis, A., et al. (2026). Context Matters in LLM-Assisted Qualitative Data Analysis: Workflow Development and Multidimensional Evaluation in Health Research. medRxiv (preprint). https://doi.org/10.64898/2026.09.07.26362410
Vancouver
Dai H, Li Y, Villalobos-Quesada M, Ng YM, Aardoom J, Versluis A, et al. Context Matters in LLM-Assisted Qualitative Data Analysis: Workflow Development and Multidimensional Evaluation in Health Research. medRxiv (preprint). 2026. doi:10.64898/2026.09.07.26362410.
BibTeX
@article{dai2026-095cf1dc,
author = {Dai, H. and Li, Y. and Villalobos-Quesada, M. and Ng, Y. M. and Aardoom, J. and Versluis, A. and Shen, H. and Spruit and others},
title = {{Context Matters in LLM-Assisted Qualitative Data Analysis: Workflow Development and Multidimensional Evaluation in Health Research}},
journal = {medRxiv (preprint)},
year = {2026},
doi = {10.64898/2026.09.07.26362410},
url = {https://doi.org/10.64898/2026.09.07.26362410}
}