It has been repeatedly argued among vocabulary researchers that semantically related words should not be taught simultaneously because they can interfere with each other. However, the question of what types of relatedness cause interference has rarely been examined carefully. In addition, there are disagreements among the past studies that have examined this issue, with some providing results showing that semantic clustering does not cause interference and confusion (e.g. Papathanasiou, 2009) and some seeming to suggest that different types of semantic similarity affect memory in different ways (Tinkham, 1997). Replicating a study by Ishii (2013), this article reports the results of a study conducted with the hypothesis that the shared visual feature of the referents of the words learned simultaneously is a significant causal factor in interference. The study compared the learning of (1) unrelated, (2) semantically related, and (3) physically related sets of words; the results confirmed that physically related sets were harder to learn than the other two sets and exhibited more cases of confusion in a test conducted after an interval of 20 minutes. The semantically related sets, which were controlled for physical relatedness, were neither harder nor easier to learn than the unrelated sets, although there were some signs of confusion in the process of learning them. This suggests the possibility that the impeding effect of semantic clustering repeatedly reported in the past (e.g. Tinkham, 1993, 1997; Waring 1997) could be partly due to the visual features sometimes shared between the referents of semantically similar words.