In an investigation with advanced language learners, Brantmeier [Brantmeier, C., 2006. "Advanced L2 learners and reading placement: self-assessment, computer based testing, and subsequent performance." "System" 34 (1), 15-35.] reports that self-assessment (SA) of second language (L2) reading ability, when measured with self-rated scales, is not an accurate predictor of subsequent reading performance as measured via multiple choice items. In another experiment with advanced learners that utilizes criterion-referenced SA items, Brantmeier and Vanderplank [Brantmeier, C., Vanderplank, R., 2008. "Descriptive and criterion-referenced self assessment with L2 readers." "System" 36 (3), 456-477] reveal that learners accurately estimate their reading comprehension when it is measured via multiple choice items. For the present study, an SA instrument of language learning achievement was designed according to specific course content to take into consideration the direct experience learners have had in practicing reading, listening, speaking, and writing [Ross, S., 1998. "Self-assessment in second language testing: a meta-analysis of experimental factors." "Language Testing" 15, 1-20.]. With 276 participants, the study examines skill-based SA across beginning, intermediate and advanced levels of language instruction, and it offers evidence to validate the relationship between the SA instrument and achievement on an online abilities test with advanced learners. Findings hold important implications for language learner assessment, especially in terms of advanced students' ability to rate themselves when given specific criteria. A discussion about the value of SA as a complement to other traditional approaches for language program assessment is offered. (Contains 1 figure and 4 tables.)