This paper introduces Ontoolsearch, a new search system that can be employed by educators in order to find suitable tools for supporting collaborative learning settings. Current tool search facilities commonly allow simple keyword searches, limiting the accuracy of obtained results. In contrast, Ontoolsearch supports semantic querying of tool knowledge bases annotated with the Ontoolcole ontology, specifically designed to fit educators' questions. Moreover, Ontoolsearch offers an innovative direct manipulation interface to educators, intended to facilitate query formulation as well as the analysis of obtained results. To evaluate this proposal, a group of educators was engaged in a formal comparison study of Ontoolsearch with a keyword search facility based on Lucene. Six search tasks were proposed, each responding to the learning tool needs of a real CSCL setting. Participants had to find tools for these search tasks using both systems alternatively. Evaluation results showed that retrieval performance was significantly better with Ontoolsearch, despite educators' previous experience with keyword searches. Further, educators rated very positively the user interface of Ontoolsearch and considered this system very useful to find tools for their own learning situations. (Contains 7 tables and 7 figures.)