Following Messick's definition of validity as a multi-faceted construct that includes contextual, substantive, structural, generalizable, external, and consequential dimensions, this study examined an established directed self-placement (DSP) system that had been functioning for ten years at a large university. The goal was to determine the extent to which this manifestation of DSP could be described as a valid assessment system for students choosing between a developmental and a first-year writing course. Analysis of data, including details of students' academic records, course materials, DSP questions, surveys, and interviews, led to the conclusion that DSP at this university does not have strong validity. Because validity is always embedded in a local context, the profession needs further investigations of the validity of DSP in a variety of other college and university settings, and this study includes an analytical framework that can be used in such work. (Contains 4 figures.)