In the Netherlands, as in many other countries, the algebraic expertise of students graduating from secondary education is an issue. The use of digital tools for algebra education is expected to change epistemologies, activity structures and student achievement. Therefore, a study was set up to investigate in what way the use of ICT in upper secondary education might enhance the algebraic expertise of students. One of the first decisions to be made concerned the choice of appropriate digital tools. This paper describes the process of designing and using an instrument for evaluating digital tools. The conceptual framework guiding this process includes notions on symbol sense, instrumental genesis and formative assessment. The evaluation instrument is designed through a Delphi method and provides a blueprint of tool features that are relevant for the purpose of this study. The results show that such an evaluation instrument is valuable both for choosing appropriate digital tools and for making concrete the aims and expectations that researchers have on the issue of integrating technology in algebra education. The final instrument is presented and illustrated through examples implemented in different digital algebra tools. (Contains 5 tables, 8 figures, and 3 footnotes.)