The purpose of this study was to evaluate technical adequacy features of an online adaptation of vocabulary matching known as critical content monitoring. Validity and reliability studies were conducted with a sample of 106 students from one school in fifth-grade science content. Participants were administered 20 parallel forms of the general outcome measure over a 2-week period. Criterion-related validity correlations with a statewide accountability test ranged from 0.36 to 0.55 for the 20 parallel forms. A pooled estimate of a common correlation between the state test and the probes was found to be 0.45. While correlation differences were not found, statistically significant differences in probe mean scores were identified across the body of parallel forms. Student commentary regarding the online assessment process was largely positive. Alternate-form reliability correlations ranged from 0.21 to 0.73, with a median correlation of 0.56. Limitations and implications are addressed. (Contains 2 tables.)