The web is a rich and diverse information source with incredible amounts of information about all kinds of subjects in various forms. This information source affords great opportunity to build systems that support users in their work and everyday lives. To help users explore information on the web, web search systems should find information that is not only "relevant to," but also strategically "different from," users' current context. In exploratory tasks, users are learning about topics while they are searching for information about them online. They would like to discover information that is inspiring and enriches their current knowledge. The diverse information available on the web about various topics related to the user's task, with distinct characteristics, such as different authors, publication times, and locations, can provide a big picture for the user to support sense making and problem solving. This dissertation presents three diversified information systems that support users in three different exploratory tasks. "Spectrum" is an innovative blog search system that helps users to find different points of view in the blogosphere. The system diversifies a user's blog search query to retrieve opinions from different domain experts with different epistemic points of view. "LocalSavvy" creates an interesting news reading experience by aggregating news articles published at different locations about the same news event, enabling users to read and compare the various local perspectives. "Compare&Contrast" supports situation analysis by discovering comparable cases for the issues that the user is investigating. These three systems explore some of the different dimensions of diversity in web documents relevant to users' tasks, namely point of view based on expertise or interest, point of view based on location, and analogous situations involving different entities. Information retrieval and machine learning techniques are developed to identify the salient characteristics of web documents necessary to diversify the search results along these dimensions. A general framework for diversified information systems is derived based on the development of these three systems. The framework utilizes existing search engines as information sources and creates diversified search results through query modification, source selection and result post-processing. The goal is to delight and inspire users with the rich and diverse information available on the web. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]