Simon Eickhoff: Diagnosis of major depression from structural MRI by machine-learning. Sobering result: 61% accuracy with N=1868
Mimicked smaller studies by subsampling reveals strong effect of training AND test size. Even with N~100, estimated accuracies vary by >20%
15 replies, 233 likes
Tim Dalgleish: Large scale (N=1868) machine-learning study reveals very poor performance – 61% – in a binary discrimination of depression/no depression, from MRI data. Suggests that the many smaller N studies with much higher predictive power are likely inflated. Not surprised ...
4 replies, 99 likes
Eiko Fried: After 2 "decades of the brain":
1. Accuracy far too low for clinical implementation.
2. Studies test healthy vs depressed groups. Since results likely generalize to other disorders, unclear what we learn about depression in absence of comparing to e.g. anxiety disorders.
3 replies, 89 likes
David Mehler: Latest preprint, showing that test sets require large N to avoid overstimating classification accuraries. Fantastic work by Claas Flint @cl445 & team @UK_Muenster in collabo @INM7_ISN + @TheFlorey #MachineLearning cc @KordingLab @talyarkoni @danilobzdok
6 replies, 59 likes
Allen Frances: Lots of "coolest" psychiatry research is useless because it amounts to mindless playing with technical toys (eg MRI & Machine Learning)
By combining them & torturing the data, you can publish whole bunch of papers & get grants & promotions.
But findings never seem to replicate.
3 replies, 50 likes
Rick Adams: Great study. Time to give up doing case v control classification studies of psychiatric diagnoses using sMRI. All elements of this are flawed: the diagnoses themselves, case v control isn’t clinically useful*, and structural MRI seems so unlikely to hold answers...
1 replies, 42 likes
Allen Frances: Machines will soon beat humans at psych diagnosis:
3) much bigger samples
4)can detect new patterns
But current studies:
1) have tiny n's
2)small data base
3)ignore complexity of comorbidity
14 replies, 25 likes
Michael P. Hengartner, PhD: Excellent study with very important implications: 1) MRI has no diagnostic validity in depression, 2) Small studies generate unreliable and severely overestimated effects (as explained and predicted by Button et al https://www.nature.com/articles/nrn3475)
1 replies, 21 likes
KordingLab 𝛁∈🧠: Why do so many small sample fMRI classification studies show such high % correct? Great new analysis. Relates to yesterday's decision on biomarkers.
0 replies, 19 likes
Rebecca Todd: Important reality check.
0 replies, 8 likes
James C.Coyne: Why is this kind of thing still undertaken? Results still inferior to 2 question depression screening, which sucks.
0 replies, 3 likes
Chadi Abdallah: Low but realistic accuracy. It is unlikely that crude estimate of brain structure would explain high variance of complex behavioral syndrome like depression.
As a field, it's about time we let go of "diagnosis" and start focusing on what clinically matter -- treatment biomarkers
0 replies, 2 likes
Found on Mar 06 2020 at https://arxiv.org/pdf/1912.06686.pdf