Finding, accessing, and determining the credibility of information are skills most people would deem necessary for the college educated person, if not the average citizen, to possess today. At the same time, educators, as well as constituents of educational institutions are asking for better and more sophisticated assessment instruments of learning to gauge the progress of the students. This paper discusses the Information Seeking Skills Test (ISST), a computer-based test of information literacy that purports to measure these skills. The ISST is a competence test required of all freshmen at James Madison University (JMU) in Virginia. The test development process and the lessons learned are described to assist other institutions wishing to adopt and assess the Association of College and Research Libraries (ACRL, 2000) conceptual standards of information literacy. (Contains 2 tables and 1 figure.)