Digital learning increasingly makes use of realistic visualizations, although realism can be demanding for learners. Color coding is a popular way of helping learners understand visualizations and has been found to aid in learning with detailed visualizations. However, previous research has shown that color coding must not always be an effective aid, and that it even may reduce retention when used with simple visualizations. This study assessed whether the presence of color coding in learning tests has an effect after having learned using a detailed visualization that either featured color cues or one that did not. The results indicate that color coding helps learners the most if the learning tests also feature color coding. Importantly, learning with color-coded visualizations and being tested without color cues leads to the worst results in retention and transfer tests. Regarding transfer, color coding in the testing visualization boosts performance regardless of the presence of color cues in the learning phase. The results of this study challenge popular perspectives aiming at optimizing learning by removing potential sources of difficulty. Depending on the learning test, it may be more effective to keep a certain level of difficulty in the learning task when learning with digital media.