The research on the second language (L2) speaking assessment has been predominantly concerned with formal proficiency tests. However, with a growing interest in learning-oriented assessment (LOA), more researchers are interested in learner involvement in speaking assessment (Blanche & Merino, 1989; Luoma, 2004; Ocarson, 1989). Self- and peer-assessment of speaking, where learners evaluate the performance of their own and their peers, can be an exemplar of assessment with its primary purpose on learning (Chen, 2006, 2008; Ibersson, 2012; Saito, 2008; Topping & Ehly, 1998). Depending on the purpose, self- and peer-assessment can take various forms: a questionnaire on general speaking ability, a learning log for metacognitive reflection, or a classroom activity where learners use the same rating criteria as their teachers (Bachman & Palmer, 1989; Chen, 2008, Rivers, 2001). This article aims to address two questions: (1) do learners have the ability to assess the oral performances of their own and their peers; and (2) what factors in self- and peer-assessment affect the enhancement of L2 speaking ability. To do this, it reviews the theoretical framework of LOA, and uses the guiding questions from the LOA framework to review the empirical studies on both planned and unplanned assessment. While these empirical studies focused on varying factors influencing feedback perception and incorporation, their results commonly indicated that effective feedback should put the learner and their perception into the center of consideration.