This study assesses the psychometric properties of a newly tested self-report assessment tool for media literacy, based on the twelve new media literacy skills (NMLs) developed by Jenkins et al. (2006). The sample (N = 327) consisted of normal volunteers who completed a comprehensive online survey that measured their NML skills, media exposure, digital participation, and civic engagement. A factor analysis performed on the survey items yielded 10 NMLs that emerged as significant subscales. A series of multivariate analyses of variance indicate a strong relationship between these NMLs and respondents' exposure to new media forms, their participation in Web 2.0 platforms, and their civic engagement. Specifically, individuals who consume and produce new media extensively had the highest NML levels; Facebook, Twitter, YouTube and blogging emerged as the most significant factors in this analysis. The reliability and scalability of this assessment tool are discussed, in the context of current challenges facing media literacy evaluation.