This study investigated some issues regarding the validity of the Scholastic Achievement Test (SAT) Subject Test: Korean with Listening. The SAT Korean has been administered just once a year since its inception in 1997. As of March 2006, it had been administered nine times. However, SAT foreign language tests are not as rigorously researched as other high-stakes tests such as SAT or the Test of English as a Foreign Language (TOEFL). Neither Educational Testing Service (ETS) nor the College Board provides information on validity even though the College Board states that the test is highly reliable. The SAT Korean deserves more research and attention in order to be established as an appropriate, meaningful, and useful standard of Korean ability measurement. In the present study, the SAT Korean Sample Test was administered in 2005 to 12 non-native Defense Language Institute Foreign Language Center (DLI) Korean students and seven Korean heritage students living in Los Angeles and Monterey, California. Based on their test results, quantitative item analyses were shown to lead to important comments on content and construct validity of the SAT Korean. The results suggest that several fundamental considerations should be kept in mind for future modifications in the SAT Korean. (Contains 10 tables and 4 notes.)