This paper presents a comprehensive picture of what has been investigated in terms of CALL effectiveness over the period 2000-2004 and determines why this question is still such a difficult one to answer. The study looks at both strengths and weaknesses in this body of work and singles out models of good design practice. While a perfect research design remains elusive, a positive trend towards combining quantitative and qualitative methods and including learning processes in the investigation can be observed. There is also a move away from simple comparison studies (teaching with or without computers) in favor of complex designs including a variety of variables under investigation and comparing different types of CALL. In this context, the usefulness of meta-analyses to determine clear cause and effect results, relying solely on effect sizes, appears questionable. The author argues that a series of systematic syntheses of findings related to one particular variable such as learning strategy or writing quality might produce more valuable insights into the potential impact of technologies on learning processes and outcomes and proposes a research agenda along those lines.