It is well established in studies of two-party conversation that conversational partners jointly establish brief labels for repeatedly mentioned entities. When speaking to a new partner who is unfamiliar with the labels, speakers use longer expressions to facilitate understanding. How this process of audience design scales up to multiparty conversation, where individuals differ in mutual knowledge, is unknown. Here we propose, and test, three potential hypotheses regarding how speakers design referring expressions in three-party conversation. Participants completed a referential communication task in groups of three in which one participant, the Director, gave instructions to two Matchers who differed in their knowledge of referential labels. Directors flexibly alternated between partner-specific representations of common ground (CG), producing longer descriptions for low-CG than for high-CG partners. When addressing multiple parties at once, speakers tailored descriptions for the least knowledgeable person. These findings shed light on the mechanisms that support audience design in multiparty conversation: Audience design begins with access to distinct representations of common ground held with the intended addressee or addressees. These distinct representations support an audience design process in which utterances are tailored to accommodate the least knowledgeable addressee in a group.