Academicians, experts, and other stakeholders have contributed extensively to the literature on university rankings also known as "league tables". Often the tone is critical usually focused on the subjective aspects of the process; e.g., the list of the universities' attributes used in the rankings, their respective weights, and the size and composition of the comparison group. These aspects of a ranking are an easy target since, after all, they are based on someone's opinion even if this person is considered an expert. There are other, purely technical, reasons why ranking schemes are problematic. In this paper we discuss these aspects of rankings by studying the handling of the data, exposing logical mistakes, and raising interpretation issues. We present these as a list of four "pitfalls" invoking in each case an example from an actual rankings. Each case also results in recommendations that address the technical issues involved.