Brain-mimetic Kernel: A Kernel Constructed from Human fMRI Signals Enabling a Brain-mimetic Visual Recognition Algorithm

Although the present-day machine learning algorithm sometimes beats humans in visual recognition, we still find significant differences between the brain’s and the machine’s visual processing. Thus, it is not guaranteed that the information sampled by the machines is the one used in the human brain. To overcome this situation, we propose a novel method for extracting the building blocks of our brain information processing and utilize them in machine learning algorithms. The visual features used in our brain are identifiable by applying kernel canonical correlation analysis (KCCA) to paired data of visual stimuli (images) and evoked functional magnetic resonance imaging (fMRI) activity. A machine learning algorithm incorporating the identified visual features represented in the brain is expected to inherit the characteristics of brain information processing. In the proposed method, the features are incorporated into kernel-based algorithms as a positive-definite kernel. Applying the method to fMRI data measured from a participant seeing natural and object images, we constructed a support vector machine (SVM) working on the visual features presumably used in the brain. We showed that our model outperforms the SVM equipped with a conventional kernel, especially when the size of the training data is small. Moreover, we found that the performance of our model was consistent with physiological observations in the brain, suggesting its neurophysiological validity.