Nearly two dozen states now administer online exams to deliver testing to K-12 students. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students' learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the state administering the same exam (PARCC) in online and offline formats each year during the transitional period. We find an online test penalty of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA), which partially but not fully fades out in the second year of online testing.