In 2004 national educational standards for chemistry were implemented in Germany. While the standards describe different competencies to be reached after grade 10, no compulsory contents are defined. The contents are defined by the different German states individually. This means that there are no defined common topics taught in all states that can be used for test items for the evaluation of the standards. As a result, two different types of test items are developed based on a model of competence. One type of items contains chemical content information that can be used to solve the task. The other type of items does not contain this information. The two types of items have different advantages and disadvantages; the first type are more dependent on the reading ability of the students, but their use leads to a fairer test across the different states, whereas the second type are more related to chemical content knowledge but are less fair. The two types of items are compared according to the item difficulty, correlations to reading skills, intelligence and chemical content knowledge. The results of this study can be generalized to other assessments using a similar design. (Contains 10 tables and 8 figures.)