While competency-based education is growing, standardized tools for evaluating the unique characteristics of course design in this domain are still under development. This preliminary research study evaluated the effectiveness of a rubric developed for assessing course design of competency-based courses in an undergraduate Information Technology and Administrative Management program. The rubric, which consisted of twenty-six individual measures, was used to evaluate twelve new courses. Additionally, the final assessment scores of nine students who completed nine courses in the program were evaluated to determine if a correlation exists between student success and specific indicators of quality in the course design. The results indicate a correlation exists between measures that rated high and low on the evaluation rubric and final assessment scores of students completing courses in the program. Recommendations from this study suggest that quality competency-based courses need to evaluate the importance and relevance of resources for active student learning, provide increased support and ongoing feedback from mentors, and offer opportunities for students to practice what they have learned.