University of Sussex
Browse
119.pdf (415.86 kB)

Gray-box inference for structured Gaussian process models

Download (415.86 kB)
conference contribution
posted on 2023-06-09, 05:30 authored by Pietro Galliani, Amir Dezfouli, Edwin Bonilla, Novi QuadriantoNovi Quadrianto
We develop an automated variational infer- ence method for Bayesian structured prediction problems with Gaussian process (gp) priors and linear-chain likelihoods. Our approach does not need to know the details of the structured likelihood model and can scale up to a large number of observations. Furthermore, we show that the required expected likelihood term and its gradients in the variational objective (ELBO) can be estimated efficiently by using expectations over very low-dimensional Gaussian distributions. Optimization of the ELBO is fully parallelizable over sequences and amenable to stochastic optimization, which we use along with control variate techniques to make our framework useful in practice. Results on a set of natural language processing tasks show that our method can be as good as (and sometimes better than, in particular with respect to expected log-likelihood) hard-coded approaches including svm-struct and crfs, and overcomes the scalability limitations of previous inference algorithms based on sampling. Overall, this is a fundamental step to developing automated inference methods for Bayesian structured prediction.

History

Publication status

  • Published

File Version

  • Published version

Journal

Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS); Fort Lauderdale, Florida, USA; 20-22 April 2017

ISSN

1938-7228

Publisher

JMLR

Volume

54

Page range

353-361

Department affiliated with

  • Informatics Publications

Research groups affiliated with

  • Data Science Research Group Publications

Full text available

  • Yes

Peer reviewed?

  • Yes

Editors

Jerry Zhu, Aarti Singh

Legacy Posted Date

2017-03-17

First Open Access (FOA) Date

2017-06-16

First Compliant Deposit (FCD) Date

2017-03-17

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC