Aim/Purpose: There is a huge array of educational technology tools that are now in use today. These tools have changed the way teachers teach and the way students learn. Among the many educational technology tools that are gaining popularity are the online collaboration tools. Online collaboration tools are web-based tools that allow individuals to do things together online like messaging, file sharing, and assessment. However, when new educational technology is integrated in the classroom, its effects must be determined as this is an essential component for evaluation. Having the tool is not enough, there has to be an evaluation of its quality to make it more effective. It is on this premise that the customized online collaboration tool of one university in the Philippines was assessed. The study specifically aimed to identify the perceived effectiveness of the customized online collaboration tool; and identify the effectiveness of using Usability Metrics for Effectiveness. Background: Most studies provide evaluation of newly developed software using a set of quality standards such as functionality and usability. However, there is limited evidence where online collaboration tools are evaluated on their effectiveness using user's perception as well as assessing their usability using a set of effectiveness metrics. Methodology: The study employed questionnaire-based (n=127) and metrics-based (n=81) approaches. Data gathered were analyzed using simple and relative frequencies, weighted mean, ANOVA, and Tukey HSD (Honestly Significant Difference). Contribution: This study supplements the literature regarding the effectiveness of online collaboration tools that are used for teaching and learning particularly those that are custom-made for an institution. It provides additional information on other ways to evaluate the effectiveness of customized online collaboration tools. It likewise provides information on the difference between what the users report about the tool's effectiveness and what the metrics show. Findings: The customized online collaboration tool was perceived to be very effective for collaboration, teaching and learning. But there is a significant difference on the students' perception based from their courses and year levels. The customized online collaboration tool's effectiveness as regards its application performance is poor because of design interface faults. However, the customized online collaboration tool has good completion rate. Recommendations for Practitioners: Findings suggest that the customized online collaboration tool can be an alternative tool to open-source learning platforms which faculty members may use for teaching and learning as students find it very effective not only for collaboration but for teaching and learning as well. However, errors caused by interface design problems must be addressed to yield a higher application performance rating and higher completion rate. Recommendation for Researchers: Further assessment on the tool's effectiveness using an empirical study is recommended to provide additional definitive evidence which may support the benefits of using online collaboration tools in the classroom.