The assessment of learning in large online courses requires tools that are valid, reliable, easy to administer, and can be automatically scored. We have evaluated an online assessment and learning tool called Knowledge Assembly, or Knowla. Knowla measures a student's knowledge in a particular subject by having the student assemble a set of arbitrarily sized scrambled fragments into a logical order using a web-based interface. Instructors can use Knowla to create assessments and grading criteria. Initial testing of the tool indicates that the measure is reliable and that students access critical thinking skills as they engage with the material. Additionally, testing demonstrates that Knowla can be used to assess learning gains in a subject area. Knowla also shows promise as a learning tool and has already been integrated into university courses.