Previous large-scale education technology interventions have shown only modest impacts on student achievement. Building on results from an earlier randomized controlled trial of three different applications of information and communication technologies (ICTs) on primary education in Kenya, the Tusome Early Grade Reading Activity developed the National Tablets Program. The National Tablets Program is integrated into the Tusome activity by providing tablets to each of more than 1,200 instructional coaches in the country to use when they visit teachers. This enables a national database of classroom instructional quality, which is used by the education system to monitor overall education quality. The tools provided on the tablets are designed to help coaches increase the quality of their instructional support to teachers, and deepen the shallow accountability structures in Kenya's education system. Using results of a national survey, we investigated the ability of the National Tablets Program to increase the number of classroom observations done by coaches and to improve student learning outcomes. Survey results showed high levels of tablet program utilization, increased accountability, and improvements in learning outcomes. We share recommendations regarding large-scale ICT interventions and literacy programs.