Accessibility navigation


Stimulus-specific random effects inflate false-positive classification accuracy in multivariate-voxel-pattern-analysis: a solution with generalized mixed-effects modelling

Kajimura, S., Hoshino, T. and Murayama, K. (2023) Stimulus-specific random effects inflate false-positive classification accuracy in multivariate-voxel-pattern-analysis: a solution with generalized mixed-effects modelling. NeuroImage, 269. 119901. ISSN 1053-8119

[img]
Preview
Text (Open Access) - Published Version
· Available under License Creative Commons Attribution Non-commercial No Derivatives.
· Please see our End User Agreement before downloading.

1MB
[img] Text - Accepted Version
· Restricted to Repository staff only
· Available under License Creative Commons Attribution Non-commercial No Derivatives.

860kB

It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

To link to this item DOI: 10.1016/j.neuroimage.2023.119901

Abstract/Summary

When conducting multivariate-voxel pattern analysis (MVPA), researchers typically compute the average accuracy for each subject and statistically test if the average accuracy is different from the chance level across subjects (by-subject analysis). We argue that this traditional by-subject analysis leads to inflated Type-1 error rates, regardless of the type of machine learning method used (e.g., support vector machine). This is because by-subject analysis does not consider the variance attributed to the idiosyncratic features of the stimuli that have a common influence on all subjects (i.e., the random stimulus effect). As a solution, we proposed the use of generalized linear mixed-effects modelling to evaluate average accuracy. This method only requires post-classification data (i.e., it does not consider the type of classification methods used) and is easily implemented in the analysis pipeline with common statistical software (SPSS, R, Python, etc.). Using both statistical simulation and real fMRI data analysis, we demonstrated that the traditional by-subject method indeed increases Type-1 error rates to a considerable degree, while generalized mixed-effects modelling that incorporates random stimulus effects can indeed maintain the nominal Type-1 error rates.

Item Type:Article
Refereed:Yes
Divisions:Life Sciences > School of Psychology and Clinical Language Sciences > Department of Psychology
ID Code:110298
Publisher:Elsevier

Downloads

Downloads per month over past year

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation