A Web-based tool that allows students to generate multiple-choice questions in a collaborative, distributed setting was evaluated through several comparisons. Students first completed a Web-based tutorial on writing effective multiple-choice questions and then authored questions on a given topic. Next, using the Web-based tool, groups of students reviewed and critiqued questions written by others within their group on the same topic. Based on these critiques, students were permitted to modify their original questions. They then were tested on questions prepared by other groups, either on the same or on other topics. Students who collaborated within a topic scored approximately 7% higher on the test within that topic than students who either collaborated on other topics or did not use the collaboration tool. Of the 336 questions developed, 77% were considered acceptable by instructors, indicating that the questions could be repurposed for inclusion in future tests. A majority of the critiques were constructive, indicating that the collaborative process was supportive of learning.