Generating Images Instead of Retrieving Them: Relevance Feedback on Generative Adversarial Networks

Antti Ukkonen, Pyry Olavi Joona, Tuukka Ruotsalo

Forskningsoutput: Kapitel i bok/rapport/konferenshandlingKonferensbidragVetenskapligPeer review


Finding images matching a user’s intention has been largely basedon matching a representation of the user’s information needs withan existing collection of images. For example, using an exampleimage or a written query to express the information need and re-trieving images that share similarities with the query or exampleimage. However, such an approach is limited to retrieving onlyimages that already exist in the underlying collection. Here, wepresent a methodology for generating images matching the userintention instead of retrieving them. The methodology utilizes arelevance feedback loop between a user and generative adversarialneural networks (GANs). GANs can generate novel photorealisticimages which are initially not present in the underlying collection,but generated in response to user feedback. We report experiments(N=29) where participants generate images using four differentdomains and various search goals with textual and image targets.The results show that the generated images match the tasks andoutperform images selected as baselines from a fixed image col-lection. Our results demonstrate that generating new informationcan be more useful for users than retrieving it from a collection ofexisting information.

Titel på värdpublikationSIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval
Antal sidor10
UtgivningsortNew York, USA
FörlagACM, Association for Computing Machinery
ISBN (tryckt)9781450380164
StatusPublicerad - 2020
MoE-publikationstypA4 Artikel i en konferenspublikation
EvenemangInternational ACM SIGIR conference on research and development in Information Retrieval - Virtual, Kina
Varaktighet: 25 juli 202030 juli 2020
Konferensnummer: 43


NamnProceedings of ACM SIGIR


  • 113 Data- och informationsvetenskap

Citera det här