This article discusses an experimental evaluation of an adaptive, multimodal user interface which generates context-sensitive help for guiding a user's interaction with an information retrieval system. Test users were studied during their interaction with three versions of the SPEAK! system. Two versions offeredactive help, either in the form ofspoken or writtenoutput, whereas a control version provided standard passive help texts only. The study was explorative in nature and mainly relied on qualitative data analyses. In order to evaluate the dialogue guidance/active help approach and the influence of the output modes, evaluation criteria focused on usability and subjective satisfaction of users with the interaction. The study was designed to be open for the exploration of typical usage patterns and the users own evaluation criteria. Data from the observations, users*think-aloud comments, and an open-ended questionnaire were combined and further qualified by differential analyses considering various user characteristics. We discuss a number of specific results, some of which may productively be used to design confirmatory, larger-scaled experiments with similar types of multimodal retrieval dialogue systems.