Scientific argumentation is a key practice in the construction, confirmation, and legalization of scientific knowledge. Although studies on scientific argumentation have been relatively plentiful, there are few quantitative studies on how students mentally perceive scientific argumentation, given that traditional quantitative tools such as Likert scales are usually decontextualized or applied in unrealistic situations. In light of this limitation in educational research, the following research question was formulated: how do contextual factors, added in to quantitative scales, influence the large-scale assessment (assessment involving complex statistical methods) of students' epistemic cognition of scientific argument (ECSA)? The current study engaged with this research question by developing a large-scale assessment tool for students' ECSA. After establishing the ECSA framework based on Toulmin's argument pattern, and designing the questionnaire by combining two authentic contexts ("the selection of cable material" and "the necessity of adding tax on petrol to reduce global warming"), the tool was examined and modified through interviews and a Rasch model analysis. The instrument was then applied in a large-scale assessment of high school students' ECSA. The results indicate two possible ways in which context influences students' ECSA--creating differences directly and amplifying possible pre-existing differences. The results pointed to the complexity of context in the development and application of large-scale epistemic cognition assessments, with the key implications that context affects the building of epistemic cognition framework and the interpretation of the results.