This work investigates the effect a Web 2.0 learning environment may have in higher education in adding value to the students' existing competencies. The major issues that this work examines are whether the incorporation of a Web 2.0 environment in higher education has an effect on the students' performance and what are the significant factors that should be taken into account in the deployment of these technologies to achieve the maximum possible benefits and whether and how they correlate to each other. These factors are derived from the students' views on the use of technology in a university course deployment and from the students' personal opinions about a pilot course in a Web 2.0 learning environment. Although the results indicate no direct effect on the students' performance, significant factors have been revealed via a thorough assessment, which has been performed in the context of a semester-long course, utilizing statistical process control techniques. The derived factors are namely: "Technology as an educational reinforcement", "Technology as a tool to enhance comprehension" and "Enhancement of student interest and experience", while in the second one "Completeness of the educational approach", "Satisfaction from the educational approach" and "Course demands". These factors can then form the basis for a feedback processes and a monitoring mechanism for a continuously updated educational process.