We introduce a cognitive framework for measuring reading comprehension that includes the use of novel summary-writing tasks. We derive NLP features from the holistic rubric used to score the summaries written by students for such tasks and use them to design a preliminary, automated scoring system. Our results show that the automated approach performs very well on summaries written by students for two different passages. [This manuscript is an early draft of a paper published in: "Proceedings of the 8th Workshop on Innovative Use of Natural Language Processing for Building Educational Applications" (pp. 163-168). Atlanta, GA: Association for Computational Linguistics.]