This paper outlines the development of a performance assessment measure of language speaking ability, the Language Ability Assessment System (LAAS), which is highly reliable and can be examined for reliability through modern measurement theories, such as generalizability theory (G-theory) and the many-facet Rasch theory. LAAS was developed to determine which University of California students were ready for full academic immersion in a foreign country whose language of instruction is not English. Test takers read passages and view recorded lectures in the target language, answering questions in writing and orally, and summarizing the lectures orally. LAAS was tested on 218 University of California students planning to spend an academic year abroad in a Spanish-speaking country. An analysis of the scores, raters, and the test itself found that, according to G-theory measures, the reliability of all the scales, with the possible exception of pronunciation, were well within acceptable limits. Many-facet Rasch measurements demonstrated how individual raters and tasks could affect the estimation of a test-taker's language ability, finding a large range of severity in the raters' judgments of performance. (MDM)