ERIC Number: EJ1057418
Record Type: Journal
Publication Date: 2014-Sep
Abstractor: As Provided
Reference Count: 111
Type I Error Inflation in the Traditional By-Participant Analysis to Metamemory Accuracy: A Generalized Mixed-Effects Model Perspective
Murayama, Kou; Sakaki, Michiko; Yan, Veronica X.; Smith, Garry M.
Journal of Experimental Psychology: Learning, Memory, and Cognition, v40 n5 p1287-1306 Sep 2014
In order to examine metacognitive accuracy (i.e., the relationship between metacognitive judgment and memory performance), researchers often rely on by-participant analysis, where metacognitive accuracy (e.g., resolution, as measured by the gamma coefficient or signal detection measures) is computed for each participant and the computed values are entered into group-level statistical tests such as the "t" test. In the current work, we argue that the by-participant analysis, regardless of the accuracy measurements used, would produce a substantial inflation of Type I error rates when a random item effect is present. A mixed-effects model is proposed as a way to effectively address the issue, and our simulation studies examining Type I error rates indeed showed superior performance of mixed-effects model analysis as compared to the conventional by-participant analysis. We also present real data applications to illustrate further strengths of mixed-effects model analysis. Our findings imply that caution is needed when using the by-participant analysis, and recommend the mixed-effects model analysis.
Descriptors: Metacognition, Memory, Accuracy, Statistical Analysis, Evaluation Methods, Error Patterns, Hierarchical Linear Modeling, Correlation, Perception, Bias, Inferences, Monte Carlo Methods, Simulation, Comparative Analysis, Regression (Statistics), Equations (Mathematics), Association (Psychology), Word Lists, Prediction, Cues, Visual Stimuli, English, African Languages, Recall (Psychology), Experimental Psychology
American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail: email@example.com; Web site: http://www.apa.org
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A