Describes an expert panel member's experiences in working with Educational Testing Service to redesign the scoring for the National Board for Professional Teaching Standards Early Adolescence/English Language Arts Assessment. Maintains that a less mechanistic and more valid scoring system could have been developed if criticism and open discussion had been part of the process. (SR)