Binary operation is one of the main topics of undergraduate mathematics. A binary operation is also used as a foundation for other disciplines such as physics, chemistry and biology. This study aims to describe the conceptual knowledge level of pre-service mathematics teachers regarding binary operation. In order to achieve this, a test consisting of open-ended and true-false questions on binary operation was administered to a total of 77 pre-service teachers; 32 primary school pre-service mathematics teachers, 30 high school pre-service mathematics teachers and 15 pedagogical formation teacher candidates. The data collected was analyzed using descriptive analysis in the context of the framework developed by the researchers. The findings of the study indicated that the performance of the pre-service teachers was insufficient with regards to the underlying conceptual knowledge that the questions sought. The study also indicates that pre-service teachers have difficulty in forming their subject knowledge successfully and they need to re-structure this as soon as possible.