Towards Denotational Semantics of AD for Higher-Order, Recursive,
Probabilistic Languages
Towards Denotational Semantics of AD for Higher-Order, Recursive,
Probabilistic Languages
Automatic differentiation (AD) aims to compute derivatives of user-defined functions, but in Turing-complete languages, this simple specification does not fully capture AD's behavior: AD sometimes disagrees with the true derivative of a differentiable program, and when AD is applied to non-differentiable or effectful programs, it is unclear what guarantees (if …