Laplace approximation and natural gradient for Gaussian process regression with heteroscedastic Student-t model

Forskningsoutput: TidskriftsbidragArtikelVetenskapligPeer review

Sammanfattning

We propose the Laplace method to derive approximate inference for Gaussian process (GP) regression in the location and scale parameters of the student-t probabilistic model. This allows both mean and variance of data to vary as a function of covariates with the attractive feature that the student-t model has been widely used as a useful tool for robustifying data analysis. The challenge in the approximate inference for the model, lies in the analytical intractability of the posterior distribution and the lack of concavity of the log-likelihood function. We present the natural gradient adaptation for the estimation process which primarily relies on the property that the student-t model naturally has orthogonal parametrization. Due to this particular property of the model the Laplace approximation becomes significantly more robust than the traditional approach using Newton’s methods. We also introduce an alternative Laplace approximation by using model’s Fisher information matrix. According to experiments this alternative approximation provides very similar posterior approximations and predictive performance to the traditional Laplace approximation with model’s Hessian matrix. However, the proposed Laplace–Fisher approximation is faster and more stable to calculate compared to the traditional Laplace approximation. We also compare both of these Laplace approximations with the Markov chain Monte Carlo (MCMC) method. We discuss how our approach can, in general, improve the inference algorithm in cases where the probabilistic model assumed for the data is not log-concave.
Originalspråkengelska
TidskriftStatistics and Computing
Volym29
Utgåva4
Sidor (från-till)753–773
Antal sidor21
ISSN0960-3174
DOI
StatusPublicerad - jul 2019
MoE-publikationstypA1 Tidskriftsartikel-refererad

Vetenskapsgrenar

  • 112 Statistik
  • 113 Data- och informationsvetenskap
  • 111 Matematik

Citera det här