Recent experience of test development in Ireland, which had no tradition of formal standardized testing, is described, in order to highlight a number of practical and psychometric problems. It is pointed out that some conventional procedures for developing norm-referenced attainment tests have limitations, especially in countries with centralized curricula. (Author/SJL)