Introduction: Critical appraisal is a crucial aspect of evidence-based practice. In order to determine whether research is valid, reliable and applicable, the evidence-based practice process advocates that published research be critically appraised. Between 2006 and 2008, the journal Evidence Based Library and Information Practice published 101 evidence summaries, critically appraising research in library and information studies. These evidence summaries can be examined in order to determine common strengths and weaknesses of research relevant to library and information studies and identify commonalities in existing evidence summary commentaries. Method: We undertook a directed qualitative content analysis of the commentary portion of all 101 evidence summaries published in the journal, Evidence Based Library and Information Practice from 2006-2008. Findings: Evidence summaries reveal more weaknesses than strengths in the library and information studies research. In general, evidence summary writers tend to remark on weaknesses relating to validity and reliability, yet paradoxically point out strengths with respect to research's applicability to practice. Conclusions: Further research is required to understand why evidence summary writers note more weaknesses than strengths in library and information studies research and whether this reflects the actual quality of the research in general. (Contains 2 tables.)