Shanghai Jiao Tong University has published on the Internet an Academic Ranking of World Universities that has attracted worldwide attention. Institutions are ranked according to academic or research performance and ranking indicators include major international awards, highly cited researchers in important fields, articles published in selected top journals and/or indexed by major citation indexes, and performance per capita. Methodological problems discussed here include quantitative versus qualitative evaluation, assessing research versus education, the variety of institutions, the language of publications, selection of awards, etc. Technical problems such as the definition and naming of institutions, the merging and splitting of institutions, and the search for and attribution of publications are discussed. (Contains 5 tables and 1 figure.)