Learning to Infer Fast by Attending to Sparse Temporal Observations

摘要

Many scientific datasets contain sparse, irregular observations of dynamic processes. Fast inference techniques are required to infer the unseen evolution of physical systems underlying growing datasets, and estimate the parameters that govern them. We first accelerate the convergence of Gauss-Markov Variational Inference via efficient natural gradient computation and automatic step-size adaptation. We then propose two general-purpose schemes for initialization from sparse data, one using junction tree representations of Gauss-Markov distributions, and another attention-based neural network architecture that learns to predict correlated temporal posteriors. We show that these initializers dramatically improve performance for ecological models of soil decomposition and predator-prey dynamics. We further extend our methods to a semiparametric epidemiological model of SARS-CoV-2 RNA wastewater measurements, which uses Gaussian processes to model uncertainty in latent reproduction numbers. Our methods allow for quick inference of both local states and global dynamical parameters, and unlike prior work, can be applied even when measurements are sparse and the generative process is not fully known.