New testing theories, concepts, and psychometric methods (e.g., item response theory, test equating, and item bank) developed during the past several decades have many advantages over previous theories and methods. In spite of their introduction to the field, they have not been fully accepted by physical educators. Further, the manner in which many assessments are developed and used in physical education has limitations, including isolated test development, weak or poor psychometric quality control, lack of evaluation frameworks, and failure to measure change or growth. To eliminate these shortcomings and meet the needs of standard-based assessment, a major national effort was undertaken to develop an item or assessment bank, called "PE Metrics," for assessing the national content standards for physical education. After providing a brief introduction to the background of PE Metrics, this article will describe the nature of the testing theory, psychometric methods, and how they were used in the construction of PE Metrics. Constraints of developing such a system are acknowledged, and future directions in physical education assessments are outlined. (Contains 1 figure and 1 table.)