To examine trends in educational practice research, this article conducted a survey and analysis of factors affecting the review of research papers in the field of the educational technology in Japan. Two factors, namely, practical orientation and theoretical orientation, were extracted from 63 survey responses, and scores from members of a Japanese academic society were compared with the scores of its editorial board. Results show that the two factor scores were not correlated to each other, and they are independent measures for any evaluation or review of research articles. Also, there was no significant difference between society members and editorial board members, i.e., their preferences when reviewing research papers were identical. For the second part of this study, a group composed of anonymous members of an editorial board conducted a paper review of 12 published "practical papers" and 11 published "general papers." Results indicated that there was no significant difference between the two categories. The final rating scores of these 23 papers were analyzed using a decision-tree rating model. Results also indicated that for publication, papers need to contribute to theoretical research, in addition to contributing to basic educational practice research in educational technology.