Sample, Nudge and Rank: Exploiting Interpretable GAN Controls for Exploratory Search

Exploratory search is characterized by open-ended search tasks and uncertainty with respect to the clarity of users’ information needs. In the context of image retrieval, generative adversarial networks (GANs) present numerous opportunities for satisfying the information needs of users engaged in exploratory search compared to a collection of images. In this article, we present a novel approach for performing exploratory search on a GAN’s image space using interpretable GAN controls that can be summarized as sample, nudge, and rank. At each search iteration, we sample images from the GAN’s latent space. We implement faceted search by nudging the sampled images towards regions of the latent space containing the attributes associated with selected facets. Lastly, we rank the nudged images using reinforcement learning with relevance feedback. We present a comprehensive evaluation of the proposed approach, incorporating results from simulations and a user study. In simulation, we show that our approach efficiently adapts to user preferences, while preserving a high-level of image diversity. In the user study (N=30), a majority of participants (23/30) preferred our system to the baseline. Concordant with simulation results, users reported both higher perceived search efficiency and image diversity compared to the baseline. Indeed, due to the baseline system’s dependence on a warm-start procedure, users of our system examined significantly fewer images while achieving task outcomes of similar subjective quality.