A physical theory is said to be covariant with respect to a certain class of transformations when its basic equations retain their "form" under those transformations. It is one of the basic notions encountered in physics, particularly in the domain of relativity. In this paper we study in some detail how students deal with this notion in different contexts (rotational, Galilean, Lorentz and general covariance). The tool of the study is a diagnostic questionnaire developed through a pilot study and then administered on a sample of 27 senior undergraduates undergoing a course on relativity. The analysis helps us identify a number of conceptual vulnerabilities of students in the domain. The most important of these concerns the non-covariant manifestations of a covariant theory, and the connection between Galilean and Lorentz invariants. The study should be useful in the teaching of relativity at the senior undergraduate and graduate levels. (Contains 3 figures, 22 tables and 5 footnotes.)