This study investigated the effects of two different accountability scoring mechanisms (ASMs), which were used during the peer instruction (PI) process, on preservice middle school mathematics teachers' (PSTs) achievements in statistics and probability. In the spring semester of 2016-2017 academic year, 46 third-year PSTs, who had been attending a statistics and probability course, participated in the study. Based on their pre-test scores, the PSTs were randomly divided into two equally achieving groups (Group 1 and Group 2). The data of this study were collected using an academic achievement test and PI and course evaluation forms. A learning management system (LMS), which was a web-based application designed by the first author, was used in collecting the PSTs' responses to the given conceptual questions. Two different ASMs were used in calculating the PSTs' PI scores. The findings showed that the PSTs in Group 1 obtained significantly higher overall academic, PI, and post-test scores than the PSTs in Group 2. Therefore, the PSTs' PI and overall academic achievement scores differed based on the ASM used. Thus, using an ASM during a PI process found to be effective in increasing the PSTs' engagement in the peer discussion. Finally, the analysis of the PSTs' opinions regarding to the PI, learning process, and learning environment indicated their overall satisfaction.