Background: Previous studies have suggested an association between reasoning strategies and diagnostic success, but the influence on this relationship of variables such as question format and task difficulty, has not been studied. Our objective was to study the association between question format, task difficulty, reasoning strategies and diagnostic success. Methods: Study participants were 13 Internal Medicine residents at the University of Calgary. Each was given eight problem-solving questions in four clinical presentations and were randomized to groups that differed only in the question format, such that a question presented as short answer (SA) to the first group was presented as extended matching (EM) to the second group. There were equal numbers of SA/EM questions and straightforward/difficult tasks. Participants performed think-aloud during diagnostic reasoning. Data were analyzed using multiple logistic regression. Results: Question format was associated with reasoning strategies; hypothetico-deductive reasoning being used more frequently on EM questions and scheme-inductive reasoning on SA questions. For SA question, non-analytic reasoning alone was used more frequently to answer straightforward cases than difficult cases, whereas for EM questions no such association was observed. EM format and straightforward task increased the odds of diagnostic success, whereas hypothetico-deductive reasoning was associated with reduced odds of success. Conclusions: Question format and task difficulty both influence diagnostic reasoning strategies and studies that examine the effect of reasoning strategies on diagnostic success should control for these effects. Further studies are needed to investigate the effect of reasoning strategies on performance of different groups of learners.