Integrating neurophysiologic relevance feedback in intent modeling for information retrieval

Giulio Jacucci, Oswald Barral, Pedram Daee, Markus Wenzel, Baris Serim, Tuukka Ruotsalo, Patrik Pluchino, Jonathan Freeman, Luciano Gamberini, Samuel Kaski, Benjamin Blankertz

Research output: Contribution to journalArticleScientificpeer-review

Abstract

The use of implicit relevance feedback from neurophysiology could deliver effortless information retrieval. However, both computing neurophysiologic responses and retrieving documents are characterized by uncertainty because of noisy signals and incomplete or inconsistent representations of the data. We present the first-of-its-kind, fully integrated information retrieval system that makes use of online implicit relevance feedback generated from brain activity as measured through electroencephalography (EEG), and eye movements. The findings of the evaluation experiment (N = 16) show that we are able to compute online neurophysiology-based relevance feedback with performance significantly better than chance in complex data domains and realistic search tasks. We contribute by demonstrating how to integrate in interactive intent modeling this inherently noisy implicit relevance feedback combined with scarce explicit feedback. Although experimental measures of task performance did not allow us to demonstrate how the classification outcomes translated into search task performance, the experiment proved that our approach is able to generate relevance feedback from brain signals and eye movements in a realistic scenario, thus providing promising implications for future work in neuroadaptive information retrieval (IR).
Original languageEnglish
JournalJournal of the Association for Information Science and Technology
ISSN2330-1635
DOIs
Publication statusPublished - 12 Mar 2019
MoE publication typeA1 Journal article-refereed

Fields of Science

  • 113 Computer and information sciences
  • 3112 Neurosciences

Cite this

Jacucci, Giulio ; Barral, Oswald ; Daee, Pedram ; Wenzel, Markus ; Serim, Baris ; Ruotsalo, Tuukka ; Pluchino, Patrik ; Freeman, Jonathan ; Gamberini, Luciano ; Kaski, Samuel ; Blankertz, Benjamin. / Integrating neurophysiologic relevance feedback in intent modeling for information retrieval. In: Journal of the Association for Information Science and Technology. 2019.
@article{3cc365b8807f4eceaee70f3d60375a45,
title = "Integrating neurophysiologic relevance feedback in intent modeling for information retrieval",
abstract = "The use of implicit relevance feedback from neurophysiology could deliver effortless information retrieval. However, both computing neurophysiologic responses and retrieving documents are characterized by uncertainty because of noisy signals and incomplete or inconsistent representations of the data. We present the first-of-its-kind, fully integrated information retrieval system that makes use of online implicit relevance feedback generated from brain activity as measured through electroencephalography (EEG), and eye movements. The findings of the evaluation experiment (N = 16) show that we are able to compute online neurophysiology-based relevance feedback with performance significantly better than chance in complex data domains and realistic search tasks. We contribute by demonstrating how to integrate in interactive intent modeling this inherently noisy implicit relevance feedback combined with scarce explicit feedback. Although experimental measures of task performance did not allow us to demonstrate how the classification outcomes translated into search task performance, the experiment proved that our approach is able to generate relevance feedback from brain signals and eye movements in a realistic scenario, thus providing promising implications for future work in neuroadaptive information retrieval (IR).",
keywords = "113 Computer and information sciences, 3112 Neurosciences",
author = "Giulio Jacucci and Oswald Barral and Pedram Daee and Markus Wenzel and Baris Serim and Tuukka Ruotsalo and Patrik Pluchino and Jonathan Freeman and Luciano Gamberini and Samuel Kaski and Benjamin Blankertz",
year = "2019",
month = "3",
day = "12",
doi = "10.1002/asi.24161",
language = "English",
journal = "Journal of the Association for Information Science and Technology",
issn = "2330-1635",
publisher = "John Wiley & Sons, Ltd",

}

Integrating neurophysiologic relevance feedback in intent modeling for information retrieval. / Jacucci, Giulio; Barral, Oswald; Daee, Pedram; Wenzel, Markus; Serim, Baris; Ruotsalo, Tuukka; Pluchino, Patrik; Freeman, Jonathan; Gamberini, Luciano; Kaski, Samuel; Blankertz, Benjamin.

In: Journal of the Association for Information Science and Technology, 12.03.2019.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Integrating neurophysiologic relevance feedback in intent modeling for information retrieval

AU - Jacucci, Giulio

AU - Barral, Oswald

AU - Daee, Pedram

AU - Wenzel, Markus

AU - Serim, Baris

AU - Ruotsalo, Tuukka

AU - Pluchino, Patrik

AU - Freeman, Jonathan

AU - Gamberini, Luciano

AU - Kaski, Samuel

AU - Blankertz, Benjamin

PY - 2019/3/12

Y1 - 2019/3/12

N2 - The use of implicit relevance feedback from neurophysiology could deliver effortless information retrieval. However, both computing neurophysiologic responses and retrieving documents are characterized by uncertainty because of noisy signals and incomplete or inconsistent representations of the data. We present the first-of-its-kind, fully integrated information retrieval system that makes use of online implicit relevance feedback generated from brain activity as measured through electroencephalography (EEG), and eye movements. The findings of the evaluation experiment (N = 16) show that we are able to compute online neurophysiology-based relevance feedback with performance significantly better than chance in complex data domains and realistic search tasks. We contribute by demonstrating how to integrate in interactive intent modeling this inherently noisy implicit relevance feedback combined with scarce explicit feedback. Although experimental measures of task performance did not allow us to demonstrate how the classification outcomes translated into search task performance, the experiment proved that our approach is able to generate relevance feedback from brain signals and eye movements in a realistic scenario, thus providing promising implications for future work in neuroadaptive information retrieval (IR).

AB - The use of implicit relevance feedback from neurophysiology could deliver effortless information retrieval. However, both computing neurophysiologic responses and retrieving documents are characterized by uncertainty because of noisy signals and incomplete or inconsistent representations of the data. We present the first-of-its-kind, fully integrated information retrieval system that makes use of online implicit relevance feedback generated from brain activity as measured through electroencephalography (EEG), and eye movements. The findings of the evaluation experiment (N = 16) show that we are able to compute online neurophysiology-based relevance feedback with performance significantly better than chance in complex data domains and realistic search tasks. We contribute by demonstrating how to integrate in interactive intent modeling this inherently noisy implicit relevance feedback combined with scarce explicit feedback. Although experimental measures of task performance did not allow us to demonstrate how the classification outcomes translated into search task performance, the experiment proved that our approach is able to generate relevance feedback from brain signals and eye movements in a realistic scenario, thus providing promising implications for future work in neuroadaptive information retrieval (IR).

KW - 113 Computer and information sciences

KW - 3112 Neurosciences

U2 - 10.1002/asi.24161

DO - 10.1002/asi.24161

M3 - Article

JO - Journal of the Association for Information Science and Technology

JF - Journal of the Association for Information Science and Technology

SN - 2330-1635

ER -