Research methods and statistics are core courses in the undergraduate psychology major. To assess learning outcomes, it would be useful to have a measure that assesses research methods and statistical literacy beyond course grades. In two studies, we developed and provided initial validation results for a research methods and statistical knowledge concept inventory for eventual use in further scholarship of teaching and learning. In Study 1, we created vignettes and administered open-ended questions to psychology subject pool students. In Study 2, we refined the vignettes and created multiple-choice items using participant responses from Study 1. After administering the measure to psychology subject pool students and a community-based sample of Amazon Mechanical Turk workers, we used item response theory to select 20 items to compose the final Psychological Research Inventory of Concepts.