Sun 15 Jan 2023 15:00 - 15:05 at Scollay - Third Session Chair(s): Steven Holtzen, Christine Tasson

We study the foundations of variational inference, which frames posterior inference as an optimisation problem, for probabilistic programming. In practice, the optimisation problem is addressed using gradient based algorithms. In particular, the reparameterisation gradient estimator, which typically reduces variance, is applied very successfully. Unfortunately, non-differentiable models, which are readily expressible in programming languages, can compromise the correctness of this approach. We study the continuous but possibly non-differentiable setting: we provide categorical models, prove unbiasedness of the reparameterisation gradient estimator and demonstrate how to establish continuity in a language with conditionals compositionally. Abstractly, this provides a foundation for fast yet correct inference for non-differentiable continuous models.

On the Reparameterisation Gradient for Non-Differentiable but Continuous Models (lafi23-final22.pdf)456KiB

Sun 15 Jan

Displayed time zone: Eastern Time (US & Canada) change

14:00 - 15:30
Third SessionLAFI at Scollay
Chair(s): Steven Holtzen Northeastern University, Christine Tasson Sorbonne Université — LIP6
14:00
20m
Talk
The Variable Elimination Algorithm as a Let-Term RewritingParis
LAFI
Thomas Ehrhard CNRS and University Paris Diderot, Claudia Faggian Université de Paris & CNRS, A: Michele Pagani IRIF - Université de Paris Cité
14:20
20m
Talk
Contextual source code AD transformations for sum typesOnline
LAFI
Adam Paszke Google Research, A: Gordon Plotkin Google
File Attached
14:45
5m
Talk
Pitfalls of Full Bayesian Inference in Universal Probabilistic ProgrammingOnline
LAFI
A: Tim Reichelt University of Oxford, C.-H. Luke Ong University of Oxford, Tom Rainforth Department of Statistics, University of Oxford
File Attached
14:50
5m
Talk
∂ is for Dialectica: typing differentiable programmingOnline
LAFI
A: Marie Kerjean CNRS, Université Sorbonne Paris Nord, Pierre-Marie Pédrot INRIA
15:00
5m
Talk
On the Reparameterisation Gradient for Non-Differentiable but Continuous ModelsBoston
LAFI
C.-H. Luke Ong NTU, A: Dominik Wagner University of Oxford
File Attached
15:05
5m
Talk
Partial Evaluation of Forward-Mode Automatic DifferentiationBoston
LAFI
A: Oscar Eriksson KTH Royal Institute of Technology, Viktor Palmkvist KTH Royal Institute of Technology, David Broman KTH Royal Institute of Technology
15:10
5m
Talk
Distribution Theoretic Semantics for Non-Smooth Differentiable ProgrammingBoston
LAFI
Pedro Henrique Azevedo de Amorim Cornell University, A: Christopher Lam University of Illinois at Urbana-Champaign
15:15
5m
Talk
New foundations for probabilistic separation logicBoston
LAFI
A: John Li Northeastern University, Amal Ahmed Northeastern University, USA, Steven Holtzen Northeastern University
File Attached
15:20
5m
Talk
Verified Reversible Programming for Verified Lossless CompressionBoston
LAFI
A: James Townsend University of Amsterdam, Jan-Willem Van De Meent University of Amsterdam
15:25
5m
Talk
Towards type-driven data-science in Idris
LAFI
Ohad Kammar University of Edinburgh, Katarzyna Marek University of Edinburgh, Minh Nguyen University of Bristol, Michel Steuwer University of Edinburgh, Jacob Walters University of Edinburgh, Robert Wright The University of Edinburgh, UK