Researchers undertake a number of different research evaluation tasks, taking up a substantial part of their research time--estimated to about one work month per year for a professor. This paper addresses the various evaluator roles and tasks researchers take on, and the tensions they involve. How the research evaluator role may conflict with the researcher role and with societal expectations is discussed, as well as the intrinsic tensions in peer review; including expertise vs. impartiality, evaluators as neutral judges vs. exercise of power and influence, divergent peer assessments vs. the need for unanimous conclusions in peer panels, peer review vs. increase in quantitative indicators, and accountability to society vs. peer review as preserving the autonomy of science. The examination of these tensions provides insight in the political aspects of peer review, and a basis for discussing an agenda for future studies on the role of peer evaluators. Major future challenges for peer review concern how to meet demands for transparency and public accountability, and maintain academic autonomy.