This paper reports findings from a project called "The National Panel of Raters" (NPR) that took place within a writing test programme in Norway (2010-2016). A recent research project found individual differences between the raters in the NPR. This paper reports results from an explorative follow up-study where 63 NPR members were surveyed with 23 items that were dilemma-like in the sense that deviating from the NPR rules would follow another--but socially acceptable--rationale. Four NPR members participated in a follow-up interview in which they motivated why they had agreed or disagreed with certain items. The results indicate two distinctly different stances toward rating work, with one stance threatening the validity of the scoring process.