Re-using gradient computation in automatic variational inference

Automatic variational inference has recently become feasible as a scalable inference tool for probabilistic programming. The state-of-the-art algorithms are stochastic in two respects: they use stochastic gradient descent to optimize an expectation that is estimated with stochastic approximation. The core computation of such algorithms involves evaluating the loss and its automatically differentiated gradient for random parameters sampled from the approximation. We study ways of re-using some of the gradient computations during optimization, to speed up learning for large-scale applications. We present a stochastic average gradient algorithm that uses gradients computed for past mini-batches to reduce noise, and explain how importance sampling allows re-using gradients computed for data samples visited again during the optimization loop.